ML19134A273

Interim State Agreements (SA) Procedure SA-102, Reviewing the Common Performance Indicator, Technical Quality of Inspections

ML19134A273
Person / Time
Issue date:	12/18/2019
From:	Kathy Modes NRC/NMSS/DMSST
To:
K MODES MSST
Shared Package
ML19134A271	List: ML19134A273
References
SA-102
Download: ML19134A273 (28)

Text

Office of Nuclear Material Safety and Safeguards Procedure Approval Reviewing the Common Performance Indicator, Technical Quality of Inspections Interim State Agreements (SA) Procedure SA-102 Issue Date: December 18, 2019 Review Date: December 18, 2022 Michael C. Layton Original signed by Director, NMSS/MSST Kevin Williams Date: 12/09/19 Paul Michalak Original signed by Branch Chief, NMSS/MSST/SALPB Paul Michalak Date: 12/06/19 Kathy Modes Original signed by Procedure Contact, MSST/SALPB Kathy Modes Date: 12/06/19 Terry Derstine Original signed by Chair, Organization of Agreement States Terry Derstine Date: 12/10/19 ML19134A271 NOTE Any changes to the procedure will be the responsibility of the NMSS Procedure Contact. Copies of NMSS procedures will be available through the NRC Web site at https://scp.nrc.gov.

Interim Procedure SA-102: Reviewing the Common Page: 1 of 13 Performance Indicator, Technical Quality of Inspections Issue Date:

12/18/2019 I. INTRODUCTION This document describes the procedure for conducting reviews of Agreement States and U.S. Nuclear Regulatory Commission (NRC) radiation control programs (Programs) for the common performance indicator, Technical Quality of Inspections, as specified in the (NRC) Management Directive (MD) 5.6, Integrated Materials Performance Evaluation Program (IMPEP).

II. OBJECTIVES A. To verify, through inspector accompaniments and casework review, that the Agreement State and NRC inspections of licensed activities focus on health, safety, and security issues in accordance with NRC Inspection Manual Chapter (IMC) 2800, Materials Inspection Program and the applicable Inspection Procedure (IP) (IPs 86740, and 87102 through 87250), or Agreement State equivalent procedure.

B. To verify that processes or procedures are established and followed to capture and address inspection-related findings that indicate the need to modify, correct, or amend licenses.

C. For Agreement States, to determine that inspection policies, procedures, and guidance are consistent with NRC policies, procedures, and guidance, and are being implemented by the Program.

D. To verify that the inspection findings are well-founded and well-documented in inspection records and lead to appropriate and prompt regulatory action.

E. To verify that the inspections address the necessary focus elements and that inspection records and findings are reviewed promptly by supervisors or management.

F. To verify that established procedures are used to identify root causes of findings and poor licensee performance. [NOTE: if there is a compatibility issue with the Programs procedures, see the Non-Common Program Performance Indicator Legislation, Regulations, and Other Program Elements (LROPE).]

G. To confirm that inspections address previously identified performance issues.

H. To confirm that supervisors ensure accompaniments of each inspector are conducted annually to evaluate the inspectors performance; and to assess whether the methods utilized for conducting accompaniments are effective in identifying performance issues.

Interim Procedure SA-102: Reviewing the Common Page: 2 of 13 Performance Indicator, Technical Quality of Inspections Issue Date:

12/18/2019 III. BACKGROUND This performance indicator is a companion to the common performance indicator, Status of Materials Inspection Program, and is meant to elicit information about the quality of inspections. IMPEP team member(s) will accompany a sample of the inspectors performing different types of licensed activities to directly evaluate the performance of the inspectors. IMPEP team member(s) will also conduct in-depth, on-site reviews of a representative sample of completed inspection files. These reviews will focus on the scope, completeness, and technical accuracy of completed inspections and related documentation.

IV. ROLES AND RESPONSIBILITIES A. IMPEP Review Team Leader (Team Leader):

1. In coordination with the IMPEP Program Manager, the Team Leader determines which team member is assigned lead review responsibility.

2. Assigns team member(s) to perform inspector accompaniments.

3. Gathers the information from the Program necessary to select the inspectors to be accompanied and licensed activities to be inspected.

4. Selects inspections based upon radiological safety and security significance of the licensed activity, from the Programs pending inspections. The Team Leader may delegate these actions to the principal reviewer.

5. Communicates the teams findings to Program management and ensures that the teams findings are in alignment with MD 5.6.

B. Principal Reviewer:

1. Reviews and evaluates selected inspection casework files, conducts staff discussions, and maintains a reference summary document of all inspection casework reviewed.

2. Conducts inspector accompaniments (unless they are conducted by an alternate team member(s)) and maintains a reference summary document related to the accompaniments.

3. Informs the Team Leader of the teams findings throughout the onsite review.

4. Presents the teams findings to the Program at the staff exit meeting.

5. Completes their portion of the IMPEP report for the Technical Quality of Inspections performance indicator.

Interim Procedure SA-102: Reviewing the Common Page: 3 of 13 Performance Indicator, Technical Quality of Inspections Issue Date:

12/18/2019

6. Attends the Management Review Board meeting for the IMPEP review; presents and discusses the teams findings for the Technical Quality of Inspections performance indicator (this can be done either in person or remotely).

V. GUIDANCE A. Scope

1. This procedure applies to the review of the technical quality of completed radioactive materials inspection actions performed in the period since the last review. The principal reviewer for this indicator may find it necessary to perform a limited review of earlier inspection actions to ensure that, for example:

recommendations identified during a previous IMPEP review have been addressed; findings from previous inspections have been addressed; or to verify that inspections conducted during the review period were performed in accordance with the time frames established in procedures and commensurate with the risk.

2. This procedure includes inspection accompaniments to observe the inspectors performance and demonstration of proper inspection techniques. The number of accompaniments to be performed depends on the size of the Program. In most cases, the goal is to accompany at least one-half of the Programs inspectors. Inspectors hired and qualified since the last IMPEP review should be accompanied. For a Program with a small number of inspectors, consideration should be given to accompanying all the inspectors.

3. This procedure specifically excludes the Agreement State and NRC inspections of licensees that are not authorized for the possession, use, distribution, or storage of byproduct material, source material, or special nuclear material less than critical mass quantities (i.e., agreement materials), inspections of licensees with non-Atomic Energy Act material (e.g., naturally occurring radioactive material (NORM)), and generally licensed radioactive material.

B. Preparation

1. Prior to the inspector accompaniments, the assigned reviewer(s) should:

a. Remind the Program that all materials inspectors are candidates for inspector accompaniments.

b. Identify and select for accompaniments, those inspectors that are newly qualified and were not accompanied during the previous review.

c. Confirm the inspectors qualifications prior to the accompaniments to ensure the inspector is qualified to perform the particular inspection assigned.

Interim Procedure SA-102: Reviewing the Common Page: 4 of 13 Performance Indicator, Technical Quality of Inspections Issue Date:

12/18/2019

d. Schedule inspector accompaniments in advance of the on-site portion of the IMPEP review. For example, risk-significant/complex licensees are inspected every 1-3 years. For a smaller Program, with fewer risk-significant licensees, it may be necessary to conduct accompaniments up to a year in advance of the on-site IMPEP review to ensure inspection accompaniments of such licensees.

e. Consider performing an increased number of inspector accompaniments involving a particular type of licensed activity to cover previously identified weaknesses.

f. Ensure that inspector accompaniments do not cause any inspection to become overdue.

g. Consider selecting inspections that typically take one work day to perform.

The reviewer should give priority to one work day inspections in order to observe the inspector conduct the entire inspection process from beginning to end. This will also allow for the reviewer to maximize efficiency and conduct additional accompaniments during the week. However, inspections of complex licensees may take longer than one day and are acceptable candidates for inspector accompaniments.

h. Consider including licensees implementing security requirements for Category 1 and Category 2 quantities of radioactive material as one of the risk-significant licensees.

i. Coordinate with the Programs inspector(s) to be accompanied as to the logistics of the accompaniment, such as when and where to meet.

Information should be gathered as to any specific security requirements, attire, or personal protective equipment that may be required for access to the licensed facilities being inspected (e.g., safety shoes, safety glasses).

j. Obtain a copy of the license, the previous inspection documentation, and any licensee submitted corrective actions from the previous inspection, if applicable for inspection accompaniments. The purposes of these documents would be limited to evaluating the inspector and not be used to review a licensees radiation safety program.

k. Discuss with each inspector the extent of the reviewers role in the inspection. It is not the role of the reviewer to help with the inspection effort or participate in the inspection, but rather to observe the inspectors performance during the conduct of the inspection. Observation of the inspector may include discreetly interacting with or asking questions of the inspector. IMPEP inspector accompaniments are performance-based evaluations of inspector effectiveness.

Interim Procedure SA-102: Reviewing the Common Page: 5 of 13 Performance Indicator, Technical Quality of Inspections Issue Date:

12/18/2019

l. Discuss with each inspector the methods (see Appendix A Inspector Accompaniment Summary Sheet of this procedure) that will be used in evaluating the inspectors performance and how feedback will be provided to the inspector and his or her management.

2. If either the accompaniments or inspection case files reveal weaknesses, the reviewer(s) should attempt to determine whether the issue is isolated or represents a programmatic weakness and should inform the Team Leader as soon as possible. The reviewer(s) should attempt to determine the root cause(s) of any identified weaknesses.

C. Review Guidelines

1. For each inspector accompanied, the reviewer should evaluate the inspectors performance:

a. Inspections should be of sufficient scope to determine whether the health, safety, and security of licensed activities were adequately addressed.

b. Inspectors should gather sufficient information to substantiate any identified violations or non-compliances; inspection findings and expectations regarding corrective actions should be clearly communicated at the conclusion of the inspection.

c. Any violations, non-compliances, and/or unresolved items identified during previous related inspections should be reviewed during the inspection to assure that they were appropriately addressed by the licensee.

d. Inspectors should use appropriate and calibrated instrumentation for performing independent and/or confirmatory measurements for the type of licensed activity inspected. Inspectors should utilize proper techniques when using instrumentation.

e. Inspectors should observe licensee activities including activities at temporary job sites, field stations, or satellite facilities, and ensure the activities observed are appropriately described.

f. Inspectors should demonstrate proper evaluation of radiation safety conditions as well as the security of licensed materials.

g. Any concerns should be discussed with the Team Leader as soon as possible. If the inspector accompaniments indicate a potential weakness on the part of one inspector or with respect to inspections of certain types of licensed activities, the Team Leader should assess whether additional inspection accompaniments are necessary and discuss this matter with the Program management.

Interim Procedure SA-102: Reviewing the Common Page: 6 of 13 Performance Indicator, Technical Quality of Inspections Issue Date:

12/18/2019

2. Prior to the full team on-site review, the Principal Reviewer(s) should:

a. Evaluate the response generated by the Program to relevant questions in the IMPEP questionnaire. Depending on the level of detail of the information provided, the response to the questionnaire relative to this indicator may be useful to focus the review.

b. Obtain a list containing the inspector name, inspection completion dates, name of licensee, and type of licensed activity from the Program for the review period. The Principal Reviewer can make their casework selection from this list.

c. Select a representative risk-informed inspection casework sample considering the following:

i. Select 15-25 casework files from a list of all material inspections started and/or completed by the Program since its last performance review. For NRC and applicable Agreement State programs, the reviewer may use the Web-Based Licensing (WBL) system as part of the review of case files.

ii. Select files that cover the qualified inspectors for the review period; focus on a variety of priority 1, 2, and 3 licensed activities (e.g., risk-significant radioactive material licensees). The casework may include the inspection documentation from the inspection accompaniments and initial inspections. The use of risk-informed sampling, rather than random sampling, maximizes the effectiveness of the review of inspection casework files. Focusing on safety and security risk-significant inspection activities, assures the identification of programmatic weaknesses that may impact public health, safety, and security of licensed materials.

iii. Inspection casework review should represent a cross section of the Programs inspectors.

iv. Select a representative sample of inspections of licensees implementing security requirements for Category 1 and Category 2 quantities of radioactive materials.

v. Select a cross-section of licensed activities, including medical, industrial, and academic uses. Casework selected for review should focus on higher risk-significant activities, such as medical activities requiring written directives, emerging medical technologies, panoramic and underwater irradiators, industrial radiography, radiopharmacy, isotope production/cyclotron, manufacturers/distributors, broad scope

Interim Procedure SA-102: Reviewing the Common Page: 7 of 13 Performance Indicator, Technical Quality of Inspections Issue Date:

12/18/2019 licensees, complex decommissioning, service providers approved for Category 2 and higher radioactive materials, and other appropriate activities.

vi. Select reciprocity inspections, temporary job site inspections, inspections related to license termination, bankruptcy, and decommissioning activities, as appropriate.

vii. Include additional casework if a previous programmatic weakness was identified in the last IMPEP review to assure that the weakness has been addressed.

d. Obtain a copy of IMC 2800 or equivalent Agreement State Program procedure.

e. Request the Program have available relevant inspection procedures for review when the team arrives on-site.

3. The Principal Reviewer should evaluate the following during the on-site review:

a. The review of inspection casework files and the inspector accompaniments should be used to demonstrate that the program is consistently implementing an established inspection program. Through observations and interviews of inspectors, determine if the inspection staff is familiar with the inspection policies, procedures, and guidance and their implementation.

Verify that Agreement State inspection policies, procedures, and guidance are in place and are consistent with NRC inspection policies, procedures, and guidance. These should include procedures to help identify root causes and other causal factors related to identified findings and poor licensee performance.

b. Verify that processes have been established by the Program to capture inspection-related findings that indicate the need to modify, correct, or amend licenses. If the Program has identified any such inspection-related findings, confirm independently that those actions have been completed as necessary. For Programs with separate inspection and licensing staff, determine how inspection-related matters are communicated to licensing staff and how licensing actions are initiated and completed as necessary.

For each inspection casework file selected, the reviewer should evaluate that the file adequately documents or contains (as appropriate):

i. All relevant documents, letters, file notes, email correspondence, and telephone conversations related to the inspections. These documents should be complete and, in the file, or are otherwise easily retrievable.

Interim Procedure SA-102: Reviewing the Common Page: 8 of 13 Performance Indicator, Technical Quality of Inspections Issue Date:

12/18/2019 ii. Sufficient detail to demonstrate that each inspection was adequate to assess the health, safety, and security of licensed activities.

iii. A description of the scope of each inspection such that a future inspector will understand which items or aspects of the licensed activities were reviewed, and which were not and may warrant review during future inspections.

iv. Sufficient information to substantiate any identified violations or non-compliances; that regulatory actions issued to licensees are appropriate for the safety and/or security significance of the identified violations; that violations are clearly communicated to licensees and dispatched in accordance with the established procedures; and any violations, non-compliances, and/or unresolved items identified during previous inspections were appropriately addressed by the licensee.

v. Program management review of inspection documentation has been performed and is sufficient to ensure that deficiencies, if present, were identified (e.g., unsupported conclusions and opinions in the inspection documentation, violations not properly substantiated, and apparent violations not cited). This documentation should confirm that these deficiencies were brought to the attention of the inspector for resolution.

vi. Review of licensee responses have been evaluated by the Program for adequacy and that any subsequent follow-up actions taken were appropriate.

vii. Instrumentation used by inspectors for independent or confirmatory measurements were calibrated at appropriate intervals and were appropriate for the types of licensed activities that were inspected.

viii. Licensee activities observed by the inspector(s), including activities at temporary job sites, field stations, or satellite facilities, were appropriately described.

ix. If the initial review indicates a performance weakness in the technical quality of inspections, but limited to a specific inspection case file on the part of one inspector or problems with respect to one or more type(s) of inspections, additional files of a similar nature should be identified, selected and reviewed to determine whether this is a programmatic weakness. The reviewer should seek to determine the extent of condition of the issue, and the root cause(s). If previous reviews indicate a programmatic weakness in a particular area, additional casework in that area should be reviewed to assure that the weakness has been addressed.

Interim Procedure SA-102: Reviewing the Common Page: 9 of 13 Performance Indicator, Technical Quality of Inspections Issue Date:

12/18/2019

c. For supervisory accompaniments of inspectors, the reviewer should:

i. Verify that the Program supervisor ensures all inspectors are accompanied on at least one inspection per year to evaluate the inspectors performance.

ii. Assess whether the methods utilized for conducting accompaniments are effective in identifying performance issues that need to be corrected.

iii. Confirm that the individual who accompanies inspectors to evaluate their performance is familiar with the types of inspection on which they are accompanying their inspectors (See Appendix B Frequently Asked Questions of this procedure, Question 3).

iv. Ensure the Program documents the annual inspector accompaniments.

D. Review Information Summary

1. At a minimum, the inspection casework summary reviewed by the Principal Reviewer will include:

a. Licensee name.

b. License number.

c. Location(s) inspected (city, state).

d. Inspection priority (for consistency, the reviewer should document the appropriate NRC inspection priority).

e. Description of licensed activity;

f. Inspector(s) name;

g. Type of inspection (e.g., routine/initial/special/reciprocity, announced/unannounced, office/temporary job site).

h. Date(s) of inspection.

i. Inspection findings.

j. Date inspection findings were issued.

k. Reviewers comments related to identified performance issues.

Interim Procedure SA-102: Reviewing the Common Page: 10 of 13 Performance Indicator, Technical Quality of Inspections Issue Date:

12/18/2019

2. At a minimum, the information maintained by the assigned reviewer for the inspector accompaniments will include:

a. The name of the inspector accompanied.

b. Licensee name.

c. License number.

d. Location(s) inspected (city, state).

e. Inspection priority (for consistency, the reviewer should document the appropriate NRC inspection priority).

f. Description of licensed activity inspected.

g. Type of inspection (e.g., routine/initial/special/reciprocity.

announced/unannounced. office/temporary job site).

h. Date(s) of inspection.

i. Reviewers comments related to observed performance issues and discussed with the inspector.

3. Appendix C Inspection Casework Review Summary Sheet of this procedure, Inspection Casework Review Summary Sheet, provides a template for recording the necessary information that should be maintained by the principal reviewer.

The reviewer is not required to use Appendix C but may find it to be a useful tool for recording the necessary information.

4. Appendix A Inspector Accompaniment Summary Sheet of this procedure, Inspector Accompaniment Summary Sheet, was developed to assist the reviewer in performing and documenting the inspector accompaniments. The reviewer is not required to use Appendix A but may find it to be a useful tool.

5. Not all the information maintained in the reviewers summary of the inspection casework files reviewed or inspection accompaniments performed will be included in the IMPEP report. The Team Leader can provide guidance as to what information is necessary to include in the report. Any information that is included in the IMPEP report must be factual, should be concise, and should concentrate on identified or observed performance deficiencies and their root cause(s).

Interim Procedure SA-102: Reviewing the Common Page: 11 of 13 Performance Indicator, Technical Quality of Inspections Issue Date:

12/18/2019 E. Evaluation Process

1. The Principal Reviewer should refer to Part III, Evaluation Criteria, of MD 5.6 for specific evaluation criteria. As noted in MD 5.6, the criteria for a satisfactory program is as follows:

a. IMPEP inspector accompaniments indicate that inspectors are knowledgeable of the requirements for license types being inspected; are able to identify potential health, safety, and security concerns; and demonstrate proper inspection technique.

b. An evaluation of inspection casework (e.g., 15-25 casework files, depending on the size of the Program) indicates that inspections are complete, inspection findings are well founded, and inspection results are reviewed promptly by Program management.

c. The Programs inspection procedures are compatible with the criteria in IMC 2800, the applicable Inspection Procedure (IP) (IPs 86740, and 87102 through 87250 series), or Agreement State equivalent procedures.

d. The Programs inspection procedures are implemented by the inspectors.

e. Inspection findings lead to appropriate and prompt regulatory action.

f. Supervisors or designated senior staff accompany all inspectors on an annual basis.

g. Followup actions regarding inspection findings are performed in accordance with the criteria in IMC 2800 and this procedure, or compatible Agreement State procedure.

Appendix D Examples of Less than Satisfactory Findings of Program Performance of this procedure contains examples to assist the reviewer in identifying less than fully satisfactory findings of a Programs performance.

F. Discussion of Findings with the Radiation Control Program

1. The IMPEP team should follow the guidance in State Agreements (SA)

Procedure SA-100, Implementation of the Integrated Materials Performance Evaluation Program (IMPEP), for discussions of technical findings with inspectors, supervisors, and management. If performance issues are identified by the reviewer(s) that lead to programmatic weaknesses, the reviewer(s) should seek to identify the root cause(s) of the issues which can be used as the basis for developing recommendations for corrective actions. Appendix D of SA-100 contains criteria regarding the development of recommendations by the IMPEP team.

Interim Procedure SA-102: Reviewing the Common Page: 12 of 13 Performance Indicator, Technical Quality of Inspections Issue Date:

12/18/2019

2. In terms of general guidance for the IMPEP review team, a finding of "satisfactory" should be considered when none or only a few or small number of the cases or areas reviewed involve performance issues/deficiencies (e.g.,

inspection, licensing, staffing, etc.); an "unsatisfactory" finding should be considered when a majority or a large number of cases or areas reviewed involve performance issues/deficiencies, especially if they are chronic, programmatic, and/or of high risk-significance; and a finding of "satisfactory, but needs improvement" should be considered when more than a few or a small number of the cases or areas reviewed involve performance issues/deficiencies in high risk-significant regulatory areas, but not to such an extent that the finding would be considered unsatisfactory.

VI. APPENDICES A. Inspector Accompaniment Summary Sheet B. Frequently Asked Questions C. Inspection Casework Review Summary Sheet D. Examples of Less than Satisfactory Findings of Program Performance VII. REFERENCES Management Directives (MD) available at https://scp.nrc.gov.

NMSS SA Procedures available at https://scp.nrc.gov.

NRC Inspection Manual Chapters available at https://www.nrc.gov/reading-rm/doc-collections/insp-manual/manual-chapter/.

VIII. ADAMS REFERENCE DOCUMENTS For knowledge management purposes, listed below are all previous revisions of this procedure, as well as associated correspondence with stakeholders, that have been entered into the NRCs Agencywide Document Access Management System (ADAMS).

Interim Procedure SA-102: Reviewing the Common Page: 13 of 13 Performance Indicator, Technical Quality of Inspections Issue Date:

12/18/2019 No. Date Document Title/Description Accession N b 1 6/28/04 STP-04-045, Opportunity to Comment on Draft ML041800434 Revisions to STP Procedure SA-102 2 3/28/05 Summary of Comments on SA-102 ML052250018 3 4/12/05 STP-05-030, Final STP Procedure SA-102 ML051080398 4 4/12/05 STP Procedure SA-102 ML052250016 5 5/17/07 FSME-07-048, Opportunity to Comment on Draft ML071400011 Revision to FSME Procedure SA-102 6 6/25/07 FSME Procedure SA-102, Resolution of Comments ML072160007 7 7/23/07 FSME Procedure SA-102 ML072160005 8 1/6/16 NMSS Procedure SA-102 ML15090A159 9 1/6/16 Resolution of Comments on scp website 10 12/18/19 NMSS Procedure SA-102 ML19134A273

APPENDIX A INSPECTOR ACCOMPANIMENT

SUMMARY

SHEET A/S or NRC Office:____________ Reviewer:________________ Accompaniment No._______

PRELIMINARY DISCUSSION WITH INSPECTOR DONE

1. Explain the reviewers role in the inspection

2. Discuss introducing reviewer to licensee and explaining his/her role in inspection

3. Explain methods to be used in evaluating inspectors performance and providing feedback GENERAL INFORMATION:

Licensee: License #:

License Type: Priority:

Location(s) Inspected:

Inspection Dates: Inspector:

ADDITIONAL INFORMATION:

Procedure used:

Inspection Type:

Unannounced Announced Routine Initial Special Reciprocity Office Security Temporary Job Site/Field Station PERFORMANCE COMMENTS AND INSPECTION FINDINGS A-1

ITEM OK or COMMENTS/QUESTIONS N/A INSPECTORS PREPARATION Adequate review of license, tie-down conditions, compliance history, & NMED search Inspection procedure(s), guidance, plan or field form Appropriate radiation detection and measurement instruments for activities inspected Calibrated Instrument response check, if appropriate Supplemental materials:

Identification Regulations Forms Dosimetry OPENING Entrance briefing conducted at appropriate level Explanation of inspection purpose, scope, method INSPECTION Use of appropriate inspection form or checklist "Walk through" of licensee facility Observation of licensee performance, licensee operations, licensed activities in progress Independent and/or confirmatory measurements performed including validation of public dose limits A-2

ITEM OK or COMMENTS/QUESTIONS N/A Facilities checked for proper posting, materials checked for proper labeling Security verified security of less than category 2 security of category 2 or higher o verification of NSTS o access authorization program o physical protection of materials in use o physical protection of materials in transit o review of selected records related to security requirements Workers checked for proper dosimetry Interviews and discussions conducted with:

RAM users Ancillary workers Verification of shielding of materials Adherence to ALARA evaluated Inspection conducted in sufficient scope & depth Verification of corrections to:

previous violations open or unresolved items A-3

ITEM OK or COMMENTS/QUESTIONS N/A Review of management oversight of licensed activities Review of incidents, medical events, equipment failures, overexposures, etc.

Document the specific activity(ies) the inspector reviewed:

A-4

ITEM OK or COMMENTS/QUESTIONS N/A RECORDS REVIEWED Inspectors should review records to supplement the performance-based inspection, as appropriate.

For example:

procurement & inventory receipt & transfer of material internal (safety & security) audits radiation safety committee meeting minutes qualification and training of personnel; refresher training authorized users instrument calibration surveys & monitoring personnel dosimetry, bioassay, declarations of pregnancy operating & emergency procedures utilization logs leak tests written directives & patient release decommissioning equipment & maintenance release of air & sewer effluents waste management, storage &

disposal hazmat refresher training transportation of materials any new licensee procedure A-5

ITEM OK or COMMENTS/QUESTIONS N/A INSPECTORS PROFESSIONALISM Use of proper health physics techniques (self monitoring, time, distance, shielding, use of survey instrument, etc.)

Accurate evaluation of radiation safety Knowledge of health physics &

regulations Appropriate appearance for license type, including proper use of PPE and safety equipment as appropriate Skill in wording questions Suitable rapport with management and workers CLOSING Preparation for exit meeting; assembly of supporting material Exit conducted at appropriate management level Violations fully explained; license condition or regulation cited Recommendations clearly distinguished from violations, as applicable Impending enforcement actions explained Licensee advised of expected response and need for corrective actions A-6

SUMMARY

OF EVALUATION The purposes of the inspector accompaniment(s) are to: (a) observe the inspector to ensure the inspector is familiar with the inspection process and procedures; (b) evaluate the adequacy of inspection tools and equipment used; and (c) evaluate the completeness of the on-site inspection.

1. Inspectors performance Satisfactory Needs Improvement

2. Performance comments:

3. Specific areas of improvement:

4. The inspector might benefit from additional training in:

(Specify type of training, e.g., formal course, mentoring, on-the-job, webinar, etc.)

5. Evaluation discussed with inspector at the end of the inspection on:

6 Evaluation discussed with Team Leader on:

7. Evaluation discussed during conference call with inspectors supervisor /

management and Team Leader on:

A-7

APPENDIX B FREQUENTLY ASKED QUESTIONS Q1: How often should an inspector be accompanied?

A1: Inspectors should be accompanied at least annually. U.S. Nuclear Regulatory Commissions (NRC) Inspection Manual Chapter 2800, Materials Inspection Program notes that the performance of each inspector be evaluated during actual inspections at least once annually. In the event that an inspector is not accompanied annually, it should be documented by the supervisor. The documentation should include an explanation and a proposed schedule for performing the accompaniment.

Q2: What if an inspector only performs inspections infrequently or just a few times per year?

A2: For qualified inspectors who no longer routinely perform inspections, the individual should be evaluated at an appropriate frequency, as established by management, relative to their other assigned duties.

Q3: Should the individual who performs the accompaniments be a qualified inspector?

A3: It is not required that the individual who performs accompaniments be a qualified inspector.

However, individuals that perform accompaniments should be familiar with the programs inspection practices and procedures. The individual should also be familiar with the type of licensed activity and specific requirements related to the type of licensed activity being inspected during the accompaniment. A supervisor that may not be familiar with inspections of licensed materials may have inspection experience from other program areas and can apply that experience to the accompaniment evaluation. Familiarity with the programs practices and procedures, the licensed activities inspected, and proper inspection techniques will enable the supervisor to provide more constructive feedback regarding the inspectors performance.

Q4: In Agreement States, can senior staff conduct inspector accompaniments rather than the supervisor?

A4: It is expected that supervisors generally conduct the accompaniments; however, for Agreement States, it is acceptable for senior program staff to perform inspector accompaniments when necessary and justified. For example, in an Agreement State where there is a vacancy in a supervisory position, the accompaniments may be performed by qualified, experienced senior staff during the time the vacancy is unfilled rather than not perform accompaniments at all due to the vacancy.

Q5: What should the IMPEP reviewer do if, during the accompaniment, it is discovered that the inspector has not been qualified to perform that particular type of inspection?

A5: This should not be the case, if the IMPEP Team Leader/reviewer confirms the inspectors qualifications prior to the accompaniment. However, if the inspector informs the reviewer or if the reviewer identifies that the inspector is not qualified to perform the inspection, the inspection should not continue. If an inspector is not fully qualified for all inspection types, but according to the program, is qualified for inspections of certain types of licensed B-1

activities, the reviewer can observe the inspector perform inspections of the types of licensed activities that the inspector is qualified to inspect.

Q6: If all of the inspectors were accompanied during the last IMPEP review, does the IMPEP review team need to conduct any accompaniments during the current review?

A6: Yes, even if all the programs inspectors were accompanied during the last IMPEP review, the review team should still conduct inspector accompaniments. The review team member(s) responsible for performing the accompaniments must still ensure that the technical quality of inspections and the technical competency and performance of the inspectors for the review period. In this case, consideration should be given to performing accompaniments of the inspectors for different types of licensed activities than were accompanied during the previous review. For example, if an inspector was accompanied on a medical inspection during the previous review, the inspector can be accompanied on an industrial inspection during the current review.

Q7: What if the Agreement States inspection policy is to perform compliance-based inspections rather than performance-based inspections?

A7: MD 5.6 specifies that the Agreement State procedure for inspection performance should be compatible to IMC 2800. IMC 2800 contains criteria for inspections to include the observation of licensed activities and review of documents for verification that operations are in compliance. The essential objective of an inspection procedure is to ensure that inspections are performed to identify real or potential health, safety or security issues. The method(s) used to achieve the objective should not be strictly performance or compliance based, rather the inspector should use their judgement in determining the best balance of methods to assess compliance.

Q8: If an Agreement State supervisor performs inspections, should the supervisor also be accompanied annually and by whom?

A8: Inspectors should be accompanied by their supervisor at least annually. Therefore, supervisors who perform inspections should also be accompanied annually.

In Agreement States, sometimes Program Directors perform inspections and it is not practical to have the Program Directors supervisor perform an accompaniment.

During previous IMPEP reviews, NRC has found it acceptable for a senior or more experienced inspector to accompany a supervisor that performs inspections.

However, in some cases, State labor or personnel practices would prohibit or discourage this approach. Because every possible scenario cannot be described here, Agreement States that have supervisors that perform inspections should develop and implement a policy that describes its approach to performing and documenting accompaniments of supervisors.

Q9: What if there are concerns regarding an inspectors performance during an inspector accompaniment?

A9: Under no circumstance should an IMPEP review team member conducting an inspector accompaniment allow an item that is of immediate health and safety or security concern to continue to be unidentified during an inspection. If this occurs, it is the responsibility of the reviewer to bring the concerns to the attention of the inspector during the inspection and B-2

inform the inspectors supervisor as well as the Team Leader as soon as is practicable. If the concerns are not of immediate health and safety or security significance, or are of unknown health and safety or security significance, the reviewer should allow the accompaniment to continue, but document the concerns in the accompaniment report and discuss the issues with the inspector at the conclusion of the inspection, and subsequently (e.g. end of the day, end of the week, or the following week) with the inspectors supervisor as well as the Team Leader. In all cases, after the inspector has concluded the onsite inspection, the IMPEP review team member should take a few moments with the inspector and discuss any observed or identified performance issues with the inspector.

This will allow for meaningful dialogue between the review team member and the inspector to clarify any issues prior to the review team member briefing the inspectors supervisor and/or program management.

B-3

APPENDIX C INSPECTOR CASEWORK REVIEW

SUMMARY

SHEET A/S or NRC Office:____________ Reviewer:________________ Casework File No._________

GENERAL INFORMATION:

Licensee: License #:

License Type: Priority:

Location(s) Inspected:

Inspection Dates: Inspector(s):

ADDITIONAL INFORMATION:

Inspection Type:

Unannounced Announced Routine Initial Special Reciprocity Office Security Temporary Job Site/Field Station Date of previous inspection or date of license issuance for initial Date inspections For routine inspections conducted in the timeframe established within the Yes No procedure For initial inspections conducted in the timeframe established within the Yes No procedure Supervisory review of inspection findings by Date Date inspection findings issued Within 30 days of inspection?

Yes No PERFORMANCE COMMENTS TO INCLUDE INSPECTION FINDINGS Comments Discussed With: Date:

C-1

APPENDIX D EXAMPLES OF LESS THAN SATISFACTORY FINDINGS OF PROGRAM PERFORMANCE The effectiveness of a Program is assessed through the evaluation of the criteria listed in Section III, Evaluation Criteria, of Management Directive (MD) 5.6. These criteria are NOT intended to be exhaustive but provide a starting point for the IMPEP review team to evaluate this indicator. The review team should also take into consideration other relevant mitigating factors that may have an impact on the Programs performance under this performance indicator. The review team should consider a less than fully satisfactory finding when the identified performance issue(s) is/are programmatic in nature, and not isolated to one aspect, case, individual, etc. as applicable.

This list is not all inclusive and will be maintained and updated in the IMPEP Toolbox on the state communications portal website.

A finding of "satisfactory, but needs improvement" should be considered when more than a few or a small number of the cases or areas reviewed involve performance issues/deficiencies in high risk-significant regulatory areas, but not to such an extent that the finding would be considered unsatisfactory.

The following are examples of review findings that resulted (or could result) in a program being found satisfactory, but needs improvement for this indicator:

1. During accompaniments, inspectors missed identifying violations and were not knowledgeable of the regulations or license conditions regarding these violations. The violations had potential health, safety, and/or security consequences, such as the following:

• Failure to perform an annual program review

• Transporting the shipping paper in the glove box

• Performing leak tests annually instead of every six months

• Failure to perform a linearity check for a dose calibrator

• Failure to perform a physical inventory of well logging sources

• Failure to secure a portable gauge with two independent physical controls when not under the direct control of the licensee.

2. Inspection casework files indicated that there were previously cited violations which dealt with health, safety, and/or security concerns. However, the inspection documentation did not address those previously cited violations, and interviews with inspectors indicated that the previously cited items were not reviewed during the most recent inspections.

3. Casework files indicated that the documentation was not complete or was lacking required elements (e.g. security was not reviewed during a blood irradiator inspection; a high dose rate (HDR) system was not inspected at a broad scope medical inspection, transportation was not reviewed at a pharmacy). Interviews with inspectors confirmed that those elements were not reviewed during the inspections. Program management review of inspection documentation did not identify this gap.

D-1

4. In more than a few of the medical inspection casework reviewed, the Programs inspectors did not examine written directives for therapeutic doses to determine whether a medical event occurred for licensees who are authorized for activities under 10 CFR 35.300, 35.400, 35.600, and 35.1000.

5. During accompanied inspections, the inspectors did not directly observe work activities or interview workers present on site. The inspectors relied solely on a review of records and speaking with the licensees radiation safety officer, contrary to the Programs inspection procedures which state that the inspectors evaluation of a licensees program shall be based on direct observation of work activities, interviews with workers, demonstrations by workers performing licensed activities, and independent measurements of radiation conditions at the facility, rather than exclusive reliance on a review of records. As a result, the inspectors did not identify violations.

6. Supervisory accompaniments were not performed by qualified individuals in two of the four years during the review period, and inspector accompaniments by the IMPEP team member identified performance concerns.

The following are examples of review findings that resulted (or could result) in a program being found unsatisfactory for this indicator:

1. During most of the inspection accompaniments, inspectors missed identifying violations and were not knowledgeable of the regulations or license conditions regarding these violations.

The violations had potential significant health, safety, and/or security consequences, such as the following:

• Operating a permanent radiography cell without an operating audible/visible alarm

• Failure to perform a spot-check for a gamma stereotactic radiosurgery unit before the first use of the day

• Failure to secure a radiographic exposure device at a temporary jobsite with two tangible barriers when not under the direct control of the licensee

• Failure to perform thyroid bioassays for a scientist who weekly performed 20 mCi iodinations with I-125

• Failure to assess a skin contamination event at a Positron Emission Tomography (PET) hot cell.

2. A majority of the casework files reviewed were inspections of medical facilities with administrations requiring written directives. For most casework files reviewed, the documentation indicated that administrations of radioactive materials were in excess of the written directive by more than 20%; however, this was not identified as a violation in any of the files. Interviews with Program staff indicated that they were unaware of the relevant regulation and the management review was inadequate to identify the discrepancy.

3. The reviewer determined that the Program was not evaluating compliance with 10 CFR Part 37 during the initial inspections for most of their new licensees possessing Category 1 and 2 quantities of radioactive material. The Programs procedures did not address initial security inspections and management review of the inspection reports failed to identify this issue.

4. During the accompaniments, the Programs inspectors did not review written directives, the determination of the prescribed dose, the validation of the prescribed dose compared to the administered dose, or the determination of a medical event. A review of selected medical D-2

inspection documentation showed a lack of review of written directives and medical events.

Program management review of inspection documentation and supervisory accompaniments also did not identify this gap. As a result, the Program did not identify a medical event.

5. During most inspection accompaniments for licensees who possess Category 1 and 2 quantities of radioactive materials, the inspectors failed to verify that select alarm systems were functioning as designed. The inspectors did not have licensee personnel verify the security systems functionality. The inspector did not verify that the licensee regularly tested the alarms. As a result, the Program did not identify a non-functioning security system.

6. Most of the casework reviewed for licensees who possess Category 1 and 2 quantities of radioactive material demonstrated that the inspectors did not review the licensees access authorization program. The inspectors noted in their report that new personnel were hired since the last inspection and were granted unescorted access to the security zone. During interviews, the inspectors confirmed that they did not review the access authorization process and relied solely on the licensees conclusion for trustworthiness and reliability.

7. Based on interviews with the Programs inspectors and a review of inspection casework, most of the inspections with licensees who had previously identified non-compliances were not evaluated by the inspectors. Inspectors did not evaluate the effectiveness of the licensees corrective actions to prevent recurrence. The Programs procedure was missing verification of previous violations. As a result, the review team identified multiple occurrences of repeat violations and the occurrence of an overexposure directly related to the licensees failure to implement effective corrective actions.

8. Supervisory accompaniments were not performed by qualified individuals for most of the inspectors during the review period and inspector accompaniments by the IMPEP team member identified performance concerns during most of the inspector accompaniments that were determined to be safety significant.

D-3