ML19317E286

Interim State Agreements (SA) Procedure SA-103, Reviewing the Common Performance Indicator, Technical Staffing and Training

ML19317E286
Person / Time
Issue date:	12/18/2019
From:	O'Hara J Office of Nuclear Material Safety and Safeguards
To:
O'Hara J
References
SA-103
Download: ML19317E286 (11)

Text

Office of Nuclear Material Safety and Safeguards Procedure Approval Reviewing the Common Performance Indicator, Technical Staffing and Training Interim State Agreements (SA) Procedure SA-103 Issue Date: December 18, 2019 Expiration Date: December 18, 2022 Michael C. Layton Original signed by Date: 12/17/19 Director, NMSS/MSST Michael Layton Paul Michalak Original signed by Date: 12/10/19 Branch Chief, NMSS/MSST/SALPB Paul Michalak Joe OHara Original signed by Date: 12/10/19 Procedure Contact, MSST/SALPB Joe OHara Terry Derstine Original signed by Date: 12/18/19 Chair, Organization of Agreement States Terry Derstine ML19317E286 NOTE Any changes to the procedure will be the responsibility of the NMSS Procedure Contact. Copies of NMSS procedures are available through the NRC Web site at https://scp.nrc.gov

Interim Procedure SA-103: Reviewing the Common Page:2 of 5 Performance Indicator, Technical Staffing and Training Issue Date:

12/18/19 I. INTRODUCTION This document describes the objectives and procedure for conducting reviews of the Agreement State and U.S. Nuclear Regulatory Commission (NRC) radiation control programs (Program) under the common performance indicator, Technical Staffing and Training and specified in the U.S. Nuclear Regulatory Commission (NRC) Management Directive (MD) 5.6, Integrated Materials Performance Evaluation Program (IMPEP).

II. OBJECTIVES A. To confirm that staffing levels throughout the review period were sufficient to support radioactive materials licensing and inspection activities such that a backlog did not exist impacting the indicator(s) Status of Materials Inspection Program, Technical Quality of Inspections, Technical Quality of Licensing Actions, and/or Technical Quality of Incident and Allegation Activities.

B. To determine the rate of staff turnover, especially senior-level positions, and the underlying causes and the length of time positions are vacant.

C. To determine whether staffing issues are a chronic or a short-term issue.

D. To determine the level of management commitment to training for initial staff qualification and continuing education.

E. To evaluate whether the inspector and license reviewer training and qualification program is being implemented effectively and is compatible with the NRCs Inspection Manual Chapter (IMC) 1248, Formal Qualifications Program for Federal and State Material and Environmental Management Programs.

III. BACKGROUND The ability to conduct effective licensing, inspection, and incident and allegation response programs is largely dependent on having a sufficient number of experienced, knowledgeable, qualified, and well-trained technical personnel. For this performance indicator, IMPEP review team (team) members will determine the ability to recruit and retain qualified staff and maintain staffing levels sufficient for the number and types of licensees. In addition, the reviewer will conduct interviews with both staff and management and examine the training and qualification documentation to ensure that license reviewers and inspectors are properly trained and qualified for the type(s) of licensed programs for which they are given inspection and licensing authority.

Interim Procedure SA-103: Reviewing the Common Page:3 of 5 Performance Indicator, Technical Staffing and Training Issue Date:

12/18/19 IV. ROLES AND RESPONSIBILITIES A. IMPEP Review Team Leader (Team Leader)

1. In coordination with the IMPEP Program Manager, the Team Leader determines which team member is assigned lead review responsibility and also assigns other team members to provide support, as necessary.

2. Communicates the teams findings to Program Management and ensures that the teams findings are in alignment with MD 5.6.

B. Principal Reviewer:

1. Reviews and evaluates the level of staffing and turnover and their impacts on the overall program, the training and qualification of new staff, and the continuing training of qualified staff against the criteria as established in MD 5.6.

2. Informs the Team Leader of any findings throughout the on-site review.

3. Presents the teams findings to the Program at the staff exit meeting.

4. Completes the portion of the IMPEP report for this indicator.

5. Participates in the Management Review Board meeting for the review and discusses the teams findings for this indicator (this can be done either in person or remotely).

V. GUIDANCE A. Scope

1. The team should follow the guidance provided in SA-100, Implementation of the Integrated Materials Performance Evaluation Program (IMPEP),

regarding discussions related to this indicator with inspectors, supervisors, and managers. If performance issues are identified by the reviewer(s) that lead to programmatic weaknesses, the reviewer(s) should seek to identify the root cause(s) of the issues which can be used as the basis for developing recommendations for corrective actions. Appendix D of SA-100 contains criteria regarding the development of recommendations by the team.

In terms of general guidance for the team, a finding of "satisfactory" should be considered when none or only a few or small number of the cases or

Interim Procedure SA-103: Reviewing the Common Page:4 of 5 Performance Indicator, Technical Staffing and Training Issue Date:

12/18/19 areas reviewed involve performance issues/deficiencies (e.g., inspection, licensing, staffing, etc.); an "unsatisfactory" finding should be considered when a majority or a large number of cases or areas reviewed involve performance issues/deficiencies, especially if they are chronic, programmatic, and/or of high-risk significance; and a finding of "satisfactory, but needs improvement" should be considered when more than a few or a small number of the cases or areas reviewed involve performance issues/deficiencies in high-risk-significant regulatory areas, but not to such an extent that the finding would be considered unsatisfactory.

This procedure applies only to personnel performing work that supports the Agreement State or NRC Program being reviewed.

2. If performance deficiencies are identified, the team should consider whether the root causes of these deficiencies affect more than the Technical Staffing and Training Performance Indicator. Issues impacting one performance indicator could also have a negative impact on performance with respect to other indicators. As a general matter, a performance deficiency, and associated root causes, should be assigned to only the most appropriate indicator and not counted against multiple indicators.

B. Evaluation Process The Principal Reviewer should refer to Part III, Evaluation Criteria, of MD 5.6, for specific evaluation criteria. As noted in MD 5.6, the criteria for a satisfactory program are as follows:

1. There are an adequate number of qualified technical and administrative staff to implement the regulatory program with few, if any, staffing vacancies.

2. Management commitment to training is clear.

3. Staffing trends that could have an adverse impact on the quality of the program are tracked, analyzed, and addressed by program management.

4. The program has compatible training and qualification procedures in accordance with the criteria specified in IMC 1248.

Interim Procedure SA-103: Reviewing the Common Page:5 of 5 Performance Indicator, Technical Staffing and Training Issue Date:

12/18/19

5. Staff is completing the training and qualification requirements according to the timelines specified in IMC 1248 or compatible Agreement State requirement.

6. New staff members are hired with the scientific or technical backgrounds that would equip them to receive specific technical training.

7. The programs training and qualification standards meet personnel needs.

Appendix B contains examples to assist the team in identifying circumstances that could warrant a finding of less than fully satisfactory for this indicator.

C. Review Guidelines

1. The Principal Reviewer should review the responses provided by the Agreement State or NRC Program to the questions in the IMPEP questionnaire. The reviewer can identify potential issues and generate questions to focus the review.

2. During the on-site review, training and qualification records and job descriptions should be reviewed and evaluated.

3. Staff members hired since the last IMPEP review are candidates for evaluation of the effectiveness of the programs training and qualification process. The Principal Reviewer is encouraged to interview staff member(s) hired since the last review.

4. Evaluate and document the following:

a. Number of full-time equivalent (FTE) staff dedicated to the radiation control program. (Include in the FTE both the number and type of full-time and part-time positions allocated to the program.)

b. Adequacy of the number of FTE to properly implement the radiation control program with a proper balance among FTE assigned to licensing, inspection, incident and allegation response, and regulation/guidance development activities.

c. Impact of vacancies during the review period.

d. Whether minimum qualification and training program requirements for personnel in the program are documented. (See Appendix A for what constitutes an acceptable written training and qualification program.)

Interim Procedure SA-103: Reviewing the Common Page:6 of 5 Performance Indicator, Technical Staffing and Training Issue Date:

12/18/19

e. Whether the status of each technical staff members training and qualification record is complete and current.

f. The reviewer should discuss their findings with the Team Leader in order to provide feedback to the Program.

D. Discussion of Findings with the Radiation Control Program The team should follow the guidance given in SA-100, for discussion of technical findings with staff, supervisors, and management. If performance issues are identified that lead to programmatic weaknesses, the team should seek to identify the root cause(s) of the issues which can be used as the basis for developing recommendations for corrective actions.

VI. APPENDICES A. Frequently Asked Questions B. Examples of Less than Satisfactory Findings of Program Performance VII. REFERENCES Management Directives (MD) available at https://scp.nrc.gov.

NMSS SA Procedures available at https://scp.nrc.gov.

NRC Inspection Manual Chapters available at https://www.nrc.gov/reading-rm/doc-collections/insp-manual/manual-chapter/.

Appendix A FREQUENTLY ASKED QUESTIONS Note: This is not an all-inclusive list but represents some commonly asked questions over the course of an IMPEP review. More examples will be maintained and updated in the IMPEP Toolbox on the state communications portal website (https://scp.nrc.gov/).

Q1: What constitutes an acceptable written training and qualification program?

A: An acceptable written training and qualification program could consist of a policy statement, description of the basic essential and training elements to become a qualified materials inspector or license reviewer based on the types of licensees regulated by the radiation control program, and a training qualification form for each individual. Additional information as to the details of the basic essential and training elements for specialized training can be found in the Inspection Manual Chapter (IMC) 1248, Formal Qualifications Program for Federal and State Material and Environmental Management Programs.

Q2: Does the documented training program description need to be as extensive as NRCs IMC 1248?

A: No, the documented training program description does not need to be as extensive as NRCs IMC 1248. The essential objective of the training plan should be a pathway for an individual to become qualified as a materials inspector and/or license reviewer.

Q3: Is it necessary to have a documented training and qualification program if the Program has not hired any new staff in 10 years and the program manager states that all staff members are qualified?

A: No, it is not necessary to have a documented training and qualification program as described in this scenario. The documented training and qualification program is just one piece of information the IMPEP review team (team) should use in determining the rating for this indicator. MD 5.6, Integrated Materials Performance Evaluation Program (IMPEP) indicates that the Program should have established qualification criteria for hiring technical staff and should have additional training and experience requirements based on the types of licenses the program issues or inspects. As noted in this procedure, the team should review the documented training and qualification program description including qualification requirements for personnel in the program.

Management approval of a staff members qualification should be in writing. If there is no documented qualifications for staff (including management sign-off), the team should make a recommendation that the Program documents its training and qualification program.

If there is no documented training and qualification program, the team should examine the overall performance of the Program in conducting the program activities to determine if the lack of a documented training and qualifications program has impacted the program performance. Although the team may not identify any performance issues because of the expertise of the current staff, at some point in the life cycle of the Program, it will become necessary to bring on new professionals. It is at that point that a A-1

documented training and qualifications program will help to assure that the Program is staffed with well-qualified personnel that provide adequate protection of the publics health, safety, and security related to radioactive materials.

Q4: Is there a staffing formula for Programs to use? Does the NRC still recommend the 1.0-1.5 technical full-time equivalents (FTE) per 100 licenses?

A: No. Prior to the implementation of IMPEP, NRC used prescriptive indicators to evaluate radioactive materials programs, including the formula 1.0-1.5 technical FTE per 100 licenses for staffing levels. Use of this formula was discontinued because it did not adequately account for licensee complexity and was not a reliable indicator of performance. NRC does not recommend a specific staffing formula. Instead, NRC recommends that each program examine their individual workloads, types of licensees and licensing actions (numbers and complexity), and inspection activities necessary to protect public health and safety in order to determine necessary staffing levels.

Additional staff efforts for regulation promulgation should be considered in the programs evaluation. Although there is no explicit criterion for new Programs, NMSS Procedure SA-700, Processing an Agreement, states that there must be at least two qualified technical staff in the Program.

A Program may find Appendix B of SA-700 helpful in evaluating staffing levels in their Program. This is a worksheet traditionally used in the initial implementation of a new Program; however, the same worksheet may be used by an existing Program to evaluate the adequacy of the number of FTE in their program.

Q5: Does a Program need more than one individual trained for a particular technical area or modality?

A: Although it is not necessary to have more than one individual trained for a particular technical area or modality, we consider that it is prudent to have at least two individuals with expertise in each technical area or modality. For Programs with smaller numbers of licensees, the potential exists for the Program to lose the capability to conduct certain aspects of their program with a single staff members departure. For larger Programs it will depend on the workload in a particular technical area or modality and whether it is more efficient and effective for the Program to train several or all individuals for the particular technical area or modality under review.

Q6: If an individual has taken a specific training course, such as radiography, is that individual qualified to conduct radiography inspections?

A: Attendance at a given training course is not the sole requirement for competency in a given area, whether licensing or inspection. The Programs training and qualification program should define what the Program considers to be a demonstration of competency applicable to the licensing or inspection of a specific activity. In many cases, mentoring by more experienced staff or completion of a specific number of licensing actions or inspections with senior staff members may be part of the necessary training to establish competency. The Program Director or designee should approve, in writing, individual staff qualifications.

Q7: If the Program has hired a qualified nuclear medicine technologist as an inspector, does that individual need to take the nuclear medicine course to become qualified to conduct A-2

nuclear medicine inspections?

A: As noted above, attendance at a given training course is not the sole requirement for competency. In this situation, the Program management may find the individual is qualified in the elements of the nuclear medicine programs, but as a new employee, needs additional training in the essentials of inspection techniques. Management may sign the individuals training documentation as complete for nuclear medicine based on the individuals previous work experience and after successfully completing training in inspection techniques. If the individuals work experience was limited to diagnostic nuclear medicine, additional training in therapeutic nuclear medicine and brachytherapy may be needed to be a fully qualified inspector for all medical applications.

Q8: Does a license reviewer or inspector need to be qualified in all areas, before they can perform work independently?

A: No. If a license reviewer or inspector has established competency in a given area, such as gauge licensing/inspection, the supervisor can approve independent work in that one area. The license reviewer/inspector may work independently while continuing to pursue competency in additional areas. NRC staff members are often referred to as having interim qualifications, which allows independent work in a limited area of demonstrated competency.

Q9: During difficult economic periods Programs may be forced to issue a freeze on funding for the hiring of program staff and travel expenses for staff training. What should the team member review and consider in determining whether the Programs performance for this indicator is satisfactory?

A: The reviewer can determine whether the implementation of the Programs plan for filling vacancies has been impacted by examining the results of the other indicators such as Status of Materials Inspection Program, Technical Quality of Inspections and Technical Quality of Licensing Actions to assess whether the number of overdue core (Priority 1, 2, and 3 and initial) inspections is satisfactory and whether inspection reports and licensing actions are being completed and issued within the required timeframe. For training sufficiency, in addition to examining training records, the reviewer should interview inspection and licensing staff to determine depth of knowledge. The reviewer should also consult with the team member(s) who performed inspection accompaniments to get feedback on the inspectors performances. In any case, the Program should have a plan in place to address this issue (e.g., providing in-house training, requesting to host NRC training, using managers or trained staff from another Agreement State or NRC Division to perform inspections during these periods, etc.)

Q10: If a Program uses a train-the-trainer approach to staff training to minimize staff time out of the office, what documentation should be available for the team to review?

A: A train-the-trainer approach, where one individual attends a training class and then presents the information to the staff through an in-house training session, is acceptable given the restrictions on out-of-State travel that some Programs may face. If a Program chooses to use a train-the-trainer approach, the Program should document the date(s) that the in-house training was offered and retain a summary of the scope and objectives of the training or a copy of the agenda. The effectiveness of the training will be evaluated through the review of quality of casework and interviews with staff.

A-3

Appendix B EXAMPLES OF LESS THAN SATISFACTORY FINDINGS OF A PROGRAM PERFORMANCE The effectiveness of a program is assessed through the evaluation of the criteria listed in Section III, Evaluation Criteria, of MD 5.6, Integrated Materials Performance Evaluation Program (IMPEP). These criteria are NOT intended to be exhaustive but provide a starting point for the IMPEP review team (team) to evaluate this indicator. The team should also take into consideration other relevant mitigating factors that may have an impact on the programs performance under this performance indicator. The team should consider a less than satisfactory finding when the identified performance issue(s) is/are programmatic in nature, and not isolated to one aspect, case, individual, etc. as applicable.

This list is not all inclusive and will be maintained and updated in the IMPEP Toolbox on the state communications portal website (https://scp.nrc.gov/).

The following are examples of review findings that resulted (or could result) in a program being found satisfactory, but needs improvement for this indicator:

1. The Program performed 494 Priority 1, 2, 3, and initial inspections during the review period. The Program conducted 13.8 percent of these inspections overdue (62 of 456 Priority 1, 2, or 3, and 1 out of 38 initial inspections). This is an increase compared to the Programs previous performance of less than one percent overdue inspections during the previous review period. The team identified an increasing trend in overdue inspections that is directly related to staffing. At the time of the review, there were six vacancies in the Program.

During the review period, 13 staff members left the Program and five staff members were hired. The positions were vacant from a few months to 2 years.

Two new technical staff positions were added to the Program during the review period in anticipation of future retirements. The Program management believes that low salaries are the reason for not attracting qualified applicants so to attract more qualified candidates they increased the salary being offered.

2. The Inspector and/or License Reviewer training and qualification manual is not equivalent to the NRCs Inspection Manual Chapter 1248, Formal Qualifications Program for Federal and State Material and Environmental Management Programs. The deficiencies in the training and qualification program manual are the root cause of a finding of satisfactory but needs improvement in the indicator(s) Technical Quality of Inspections and/or Technical Quality of Licensing Actions. The performance issue(s) are directly related to how the individual was trained. The team identified several instances of missing licensing and inspection documents as well as errors in the inspection tracking system which were determined to be attributed to the long-standing staffing vacancies.

B-1

The following are examples of review findings that resulted (or could result) in a program being found unsatisfactory for this indicator:

1. The Programs inspection frequency is the same for similar license types in IMC 2800. The Program performed 41 Priority 1, 2, 3, and initial inspections during the review period. The Program conducted 27 percent of Priority 1, 2, 3, and initial inspections overdue. Eleven of 37 Priority 1, 2, and 3 inspections were conducted overdue during the review period. The four initial inspections of new licenses were performed within 12 months of license issuance. In 3 of the 4 years of the review period, the Program performed reciprocity inspections.

However, The Program could not find any records supporting the completion of reciprocity inspections conducted during the review period. Additionally, poor tracking of reciprocity inspections contributed to the Program being unable to produce any measurable data for the review period. The team finds that greater than 25% of inspections were completed overdue under the indicator Status of Materials Inspection Program and determined the root cause to be lack of staff due to vacant positions going unfilled.

2. The Inspector and/or License Reviewer training and qualification manual is not equivalent to the NRCs IMC 1248. The deficiencies in the training and qualification program are the root cause of a finding of unsatisfactory in the indicator(s) Technical Quality of Inspections and/or Technical Quality of Licensing Actions (i.e. identified performance issue(s) are directly related to how the individual was trained).

B-2