ML17187A100

Focused Self-Assessment of Impep Report July 2017

ML17187A100
Person / Time
Issue date:	06/19/2018
From:	NRC/NMSS/DMSST
To:
O'Hara J
References
Download: ML17187A100 (25)

Text

FOCUSED SELF-ASSESSMENT OF INTEGRATED MATERIALS PERFORMANCE EVALUATION PROGRAM (IMPEP)

U.S. Nuclear Regulatory Commission Office of Nuclear Material Safety and Safeguards Division of Material Safety, State, Tribal, and Rulemaking Programs June 2018

EXECUTIVE

SUMMARY

Based on a Charter approved by the Director of the Division of Materials Safety, State, and Tribal Programs (MSST) and the Chairman of the Organization of Agreement States (OAS), a five-person team, comprised of staff from the U.S. Nuclear Regulatory Commission (NRC)

Headquarters and Region III, and two Agreement States, was assembled to perform a focused, self-assessment of the NRC Integrated Materials Performance Evaluation Program (IMPEP).

The 2017 self-assessment team focused on two common performance indicators, (1) Technical Staffing and Training and (2) Status of Materials Inspection Program, to determine if changes or enhancements are warranted. The 2017 focused, self-assessment team also reviewed the recommendations proposed by a 2010 IMPEP self-assessment team and the status of the adoption and implementation of those recommendations.

In conducting its review, the 2017 self-assessment team reviewed IMPEP reports from 2011-2016 including any recommendations made for the performance indicators Technical Staffing and Training and Status of Materials Inspection Program, and associated Management Review Board (MRB) meeting minutes. The 2017 self-assessment team reviewed potential factors that may lead to inconsistencies in evaluating the indicators. The 2017 self-assessment team attended the IMPEP team member training, monitored MRB discussions, and attended several IMPEP MRB meetings.

Overall, the team found the IMPEP to be effective in fulfilling its basic objective of evaluating the adequacy and compatibility of Agreement State and NRC materials activities under the performance indicators Technical Staffing and Training and Status of Materials Inspection Program. The team does not recommend any new changes to either performance indicator.

However, the Team did develop six recommendations, listed below, that could, if implemented, result in an enhanced IMPEP.

Recommendation 1: Adopt the 2010 IMPEP self-assessment recommendation to change the target for reciprocity inspections from 20 percent to 5 percent and issue a Temporary Instruction to clarify the Inspection Manual Chapter (IMC) 1220 requirements on reciprocity.

Recommendation 2: Conduct an IMPEP team leader workshop annually in conjunction with the annual IMPEP team member training.

Recommendation 3: Include in the IMPEP Team Leader training the importance of monitoring IMPEP MRB meetings.

Recommendation 4: Evaluate whether the Draft IMPEP report can be made not-publicly available until reviewed by the Agreement State program or the Region and if so, revise the process to designate the Draft IMPEP report as not-publicly available.

Recommendation 5: Expand the scope of IMPEP to include additional NRC Headquarters functions, including exempt distribution licensing, general licensing, uranium recovery program licensing, and complex decommissioning licensing.

Recommendation 6: Evaluate the advantages and disadvantages of the NRC conducting IMPEP reviews of its programs as a single entity in a similar fashion to states with multiple agencies.

2017 Focused Self-Assessment Report Page 1 of 9

1.0 INTRODUCTION

On March 16, 2009, the NRCs Office of the Inspector General (OIG) issued an audit report titled, Audit of the NRCs Agreement State Program (OIG-09-A-08). The objective of the audit was to assess NRCs oversight of adequacy and compatibility of Agreement State programs.

The OIG made several recommendations including Develop a mechanism for conducting self-assessments and capturing lessons learned for IMPEP on a regular basis. In June 2010, the former Federal and State Materials and Environmental Programs Office (FSME) issued a self-assessment of the entire IMPEP (ML102030228) which included 15 recommendations.

One substantive recommendation from the 2010 IMPEP self-assessment was that future self-assessments be conducted on a frequency of 4 to 6 years. The IMPEP has been and continues to be recognized by the Agreement States and the NRC as an effective means of evaluating performance.

2.0 PURPOSE AND OBJECTIVES The purpose of the 2017 IMPEP self-assessment was to conduct a focused review of IMPEP by evaluating the implementation of the IMPEP as it relates to two common performance indicators, Technical Staffing and Training and Status of Materials Inspection Program. The teams review evaluated the effectiveness of the implementation of IMPEP related to both performance indicators to determine if changes or enhancements to the performance indicators are warranted. These performance indicators were chosen for evaluation based upon the challenges they have presented to Agreement State programs Hiring freezes, reduced state budgets, and staff turnover are commonplace and can adversely affect the Technical Staffing and Training performance indicator. Short notice times and long distances to remote work locations can adversely affect the ability of personnel to reach the site in time to perform the reciprocity inspections as part of the Status of Materials Inspection Program performance indicator. Due to these challenges, both performance indicators were selected for the self-assessment and included in the Charter approved by Director of the Division of Material Safety, State, and Tribal Programs and the Chairman of the Organization of Agreement States.

Based on the purpose of 2017 IMPEP self-assessment, three objectives were developed.

Objective 1: Determine if any changes are needed to the indicators (or if a new indicator is needed) and identify any patterns or issues that need to be addressed in these indicators. For each indicator, the 2017 self-assessment team used the questions below as guidance in performing a review and assessment of the performance indicators.

(a.) Is the performance indicator complete and sufficiently focused?

(b.) Are changes needed in the indicator criteria and metrics?

(c.) Are there any patterns or issues identified from MRB review of the IMPEP report and discussion of Agreement State programs during MRB meetings that need to be addressed?

(d.) Are IMPEP recommendations applied consistently across the program and are those recommendations effective?

(e.) Is there any additional criteria for evaluating the indicators which should be developed based on the 2017 self-assessment team findings?

Objective 2: Determine if there are any factors which may contribute to inconsistent IMPEP findings and if so, propose solutions.

2017 Focused Self-Assessment Report Page 2 of 9 Objective 3: Determine the status (completed/on-going/deferred) of the recommendations of the 2010 IMPEP self-assessment and report the effectiveness of actions taken in response to the recommendations. The status of those recommendations is included in Appendix C.

3.0 SCOPE AND METHODOLOGY The 2017 self-assessment team was comprised of three NRC Headquarters and Regional staff members and two Agreement State representatives. (See Appendix A; Membership for a list of team members). The team conducted its self-assessment work from January 2017 until June 2017 and in accordance with an approved Charter and presented the draft recommendations to the Organization of Agreement States annual meeting in August 2017.

It is the policy of the NRC to evaluate the NRC Regional and Agreement State radioactive materials programs in an integrated manner using common and non-common performance indicators, as specified in Management Directive (MD) 5.6, Integrated Materials Performance Evaluation Program (IMPEP). The 2017 self-assessment team focused, in part, on reviewing the publicly available reports of IMPEP reviews of NRC Regional and Agreement State radioactive materials programs and by reviewing the consistency of these reports which were written by different IMPEP teams over a period of 6 years (i.e., 2011 to 2016).

There were 55 IMPEP reports and corresponding MRB minute reports issued between 2011 and 2016. In order to ensure both internal consistency and quality assurance in their reviews, some of the same reports were reviewed independently by two team members and their results were shared with the group. The Team Leader reviewed all 55 IMPEP reports and one self-assessment team member focused solely on reviewing the 2010 IMPEP self-assessment report and documenting the implementation of the 2010 recommendations.

In its evaluation of IMPEP consistency, the 2017 self-assessment team identified a set of consistency factors for each performance indicator. These consistency factors included the IMPEP review Team Leader and MRB Chairman and their experience level, whether the team made a recommendation, and the age and size of the Agreement State program. These consistency factors were compared to the results of the IMPEP teams analysis or the associated MRB decisions documented in the 2011 to 2016 IMPEP reports identified above to determine if these factors had an effect on IMPEP results or MRB decisions. The criteria in the comparisons included whether the consistency factor(s) exhibited significantly different IMPEP results or MRB decisions when compared to the overall historical results (i.e., 2011-2016) for the indicator. In the case of IMPEP team recommendations, the comparison was within the consistency factors themselves (i.e., did experience level of the IMPEP Team leader or MRB chairman, or the age or size of the program) lead to more recommendations when compared to the overall historical results. For the Status of Materials Inspection Program performance indicator, the team added several inspection-focused factors including the total number of inspections conducted, the percentage of overdue inspections, and the number of inspection reports the IMPEP team reviewed.

The team conducted bi-weekly team meetings and prepared individual reviews and summaries, and circulated them via e-mail. The result of this focused, self-assessment of IMPEP is a collaborative view of the entire team and does not necessarily reflect a unanimous recommendation on a particular course of action.

2017 Focused Self-Assessment Report Page 3 of 9 4.0 RESULTS AND RECOMMENDATIONS The results and recommendations of the 2017 self-assessment team are discussed below and organized according to the objectives outlined above. These recommendations reflect a collaborative view of the 2017 self-assessment team based upon: document reviews, bi-weekly teleconference discussions, an audit of the 2017 IMPEP team member and team leader training conducted at NRC Headquarters, and monitoring of MRB discussions. The numerical order of the recommendations does not correspond to their importance to the IMPEP program.

Objective 1: Determine if any changes are needed to the indicators (or if a new indicator is needed) and identify any patterns or issues that need to be addressed related to these indicators.

To meet Objective 1, the team compared the results, from the 55 IMPEP reports issued between 2011 and 2016, for the Technical Staffing and Training and the Status of Materials Inspection Program performance indicators against the IMPEP criteria of MD 5.6. The 2017 self-assessment team reviewed all recommendations the IMPEP teams proposed for the two performance indicators and evaluated them for consistency. Finally, the team reviewed the MRB decisions for IMPEP reports that identified similar or identical issues.

Based on this review, the 2017 self-assessment team did not find any adverse patterns or factors that would lead the team to believe that there were any significant issues requiring a change to the indicators. The team found IMPEP recommendations and MRB decisions related to the two indicators consistent across the entire review period (2011 through 2016). It is also the consensus of the self-assessment team that the performance indicators Technical Staffing and Training, and the Status of Materials Inspection Program are complete and sufficiently focused. The 2017 self-assessment team has no recommendations for changing or adding to any of the criteria or metrics associated with the performance indicators evaluated.

In the 2017 self-assessment teams analysis of the Technical Staffing and Training performance indicator, the team observed that many Agreement States, particularly those programs with relatively few materials licensees, and hence few staff dedicated to materials license review and inspection, have significant staff losses and turnover. In some cases, it has been observed that some Agreement States may lose 50 to 100 percent of their qualified staff to retirement or resignation between IMPEP reviews. It is understood that staff loss has the potential to adversely impact both timely license application review and licensee inspections. The 2017 focused self-assessment team found that many Agreement States have addressed this issue by cross-training their personnel and that the creation and maintenance of a fungible staff lessens the impact of the loss of one or more license reviewers (or inspectors) in Agreement States, particularly in small materials programs.

The 2017 self-assessment team found that IMPEP teams made few recommendations for Technical Staffing and Training, and the Status of Materials Inspection Program performance indicators over the time period evaluated. For those cases where a recommendation was made, the self-assessment team found it was appropriate and applied consistently across the program.

2017 Focused Self-Assessment Report Page 4 of 9 The Status of Materials Inspection Program performance indicator is of interest to the Agreement States, particularly with regard to the completion of reciprocity inspections and recommendations related to the performance indicator. The 2017 self-assessment team observed that reciprocity inspections as a function of meeting the 20 percent target for candidates continues to be a challenge for both the NRC and Agreement States. Of specific importance to the Agreement States were the impact of failing to meet the reciprocity inspection metric (20 percent of candidate licenses inspected) on the overall IMPEP review finding for the performance indicator and the consistency of recommendations for the performance indicator.

Although the performance of reciprocity inspections continues to be a challenge for many Agreement State programs, since 2011, there have been no IMPEP outcomes where the Status of Materials Inspection Program indicator was found less than satisfactory due solely to the reciprocity metric not being met.

Only two programs, New York in 2011 and Georgia in 2016, have been issued a recommendation about reciprocity metrics since 2011. For New Yorks 2011 IMPEP review, it was determined that the New York Department of Health did not track the reciprocity inspections performed by its staff and the program staff could not provide any data for the IMPEP team to review. For the Status of Materials Inspection Program indicator, New York was found Satisfactory with one recommendation that the New York Department of Health develop and implement a process to track reciprocity inspections to ensure at least 20 percent of candidate licensees for reciprocity are inspected. For Georgias 2016 IMPEP review, it was determined that Georgia lost sight of reciprocity in its overall effort to reduce overdue inspections (Note: Georgia was unsatisfactory in 2014 for the Status of Materials Program indicator due to overdue inspections). Because Georgia implemented corrective actions related to the reciprocity issue during the 2016 IMPEP review, the team did not make a recommendation. However, because Georgia had been placed on probation at the MRB for their 2014 IMPEP review, the 2016 MRB decided Georgia needed to demonstrate its implementation of the reciprocity corrective actions and added a recommendation to that effect.

No recommendations were issued to two programs, NRC Region IV in 2014 and Kansas 2014, that did not meet the reciprocity metric for the majority of the years of the review period. Both programs had reciprocity tracking in place, self-identified the weakness, and implemented corrective actions prior to the onsite IMPEP review. The 2017 self-assessment team did not identify any issues with the consistency of recommendations for the Status of Materials Inspection Program performance indicator during the self-assessment review period.

Objective 1 Recommendations:

Change Reciprocity Inspections Target Frequency In reviewing the Status of Materials Inspection Program performance indicator, the 2017 self-assessment team did not find any issues of programmatic inconsistency in the application of the criteria used to evaluate the indicator. However, as discussed above, the team did observe that the 20 percent target for completing reciprocity inspections continues to be a challenge for both the NRC and Agreement States.

SA-101, Reviewing the Common Performance Indicator on Status of Materials Inspection Program, calls for IMPEP reviewers to .verify that candidate reciprocity licensees are inspected in accordance with the frequencies in IMC 1220. IMC 1220 calls for inspecting 20 percent of candidate licensees from the candidate pool each year. The 2010 IMPEP self-assessment team also observed that the IMPEP criterion - that the State or Region inspects

2017 Focused Self-Assessment Report Page 5 of 9 20 percent of the candidate licensees each year - is often difficult to meet, resource-intensive, and not performance-based. In its review of the 2011 to 2016 IMPEP documents, the 2017

-assessment team re-affirmed the 2010 observation. As stated in SA-101, Agreement State programs can develop an alternative policy for reciprocity inspections in lieu of IMC 1220, using a similar risk-informed, performance based approach for determining reciprocity licensees that are candidates for inspection. (Note: This alternative method does not apply to the NRC. The NRC is required to follow IMC 1220.)

The 2010 self-assessment team recommended that the treatment of reciprocity inspections be made more performance-based and that the current 20 percent requirement for reciprocity inspections be removed from the IMPEP program along with corresponding changes to Inspection Manual Chapter 2800 and 1220. In 2010, The MRB approved that recommendation to reduce the target for reciprocity inspections from 20 percent to 5 percent. This recommendation was based in part on a 2007 Continuous Improvement Initiative Working Group Report. In that report, NRC staff recommended that IMC 1220 could be revised to require that only 5 percent of the core reciprocity licensees be inspected each year as opposed to the current 20 percent criteria. The 2007 working group observed that reciprocity licensees operating at temporary job sites are no more likely (in fact, they are less likely) to have significant violations leading to escalated enforcement than NRC licensees operating at temporary job sites and therefore a target that aligns more closely with the inspection requirements for NRC licensees is warranted. In addition, the 2007 working group also suggested that a specific target for the percentage of reciprocity inspections to complete be eliminated and replaced with a requirement that when a licensing authority receives a request for reciprocity, that the authority would make contact with the home Agreement State or Region for the reciprocity licensee to get information on recent events, significant violations, or escalated enforcement involving the licensee. This information would be used to make a performance-based determination on whether or not a temporary job site inspection should be conducted. This would allow Agreement States or Regions to inspect the more safety-significant reciprocity licensees rather than meeting a percentage of reciprocity inspections.

The 2010 self-assessment team concluded that either of these options-the 5 percent reciprocity inspection option or the performance-based inspection option-would be preferable to the current approach and would save inspector resources. Specifically, either option would maintain adequate oversight while saving resources. To date, the 2010 reciprocity recommendation has not been implemented because neither IMC 2800 nor 220 had not been scheduled for an update. The 2017 self-assessment team believes that the 5 percent reciprocity inspection target focusing on safety-significant licensees, which was approved by the MRB in 2010, should be adopted because it provides a quantitative target that is easy to measure, focuses on safety-significant licensees (similar to the performance-based option), and makes inspector resources available for more safety-significant activities.

Recommendation 1: Adopt the 2010 IMPEP self-assessment recommendation and the MRBs decision to approve changing the criteria for the completion of reciprocity inspections from 20 percent to 5 percent.

Objective 2: Determine if there are any factors which may contribute to inconsistent IMPEP findings.

The 2017 self-assessment team reviewed IMPEP reports and MRB minutes and compared the results to the standards set forth in MD 5.6 to evaluate whether that IMPEP reviews, findings, recommendations, and assessments have been conducted consistently between IMPEP

2017 Focused Self-Assessment Report Page 6 of 9 reviews and determine whether there are factors which may contribute to inconsistent IMPEP findings. The consensus of the 2017 self-assessment team is that the experience level of the IMPEP review team leader or reviewer for the two performance indicators was not a factor which contributed to inconsistent IMPEP review findings. The experience level, including training, of the IMPEP review team leaders and reviewers was found sufficient to provide sound IMPEP review findings. The 2017 self-assessment team also found that the IMPEP review teams findings were not influenced by IMPEP team composition (NRC staff or Agreement State staff). Other factors such as age and size of the Agreement State material program and variability in the composition of the MRB (i.e., the frequency individual NRC and Agreement State managers participated in each MRB review of the IMPEP reports) did not result in any apparent trends in MRB decisions. Overall, the 2017 self-assessment team did not identify any factors that contributed to inconsistent findings for either the Technical Staffing and Training or Status of Materials Inspection Program performance indicators.

Through a review of the MRB minutes, the 2017 self-assessment team found that good communications among team members was essential to a good review of a program.

Successful teams voiced their concerns, were heard, and had healthy constructive deliberations which led to accurate recommendations in accordance with existing guidance.

The consensus of the 2017 IMPEP self-assessment team is that the MRB is functioning in a way that ensures consistent results. It continues to be a strong and vital part of the IMPEP process and its decisions are fair and evenly applied. The 2017 self-assessment team found the results of the MRB reviews have been consistent based upon the standards set forth in MD 5.6.

Objective 2 Recommendations:

Annual IMPEP Team Leader Workshop Self-assessment team members attended the 2017 IMPEP Team Member and Team Leader training and found both sessions to be very informative. Based on the positive experience team members had in both training sessions, the team concluded that there may be a benefit in conducting an annual IMPEP team leader workshop to promote greater understanding of how common and non-common indicators should be reviewed and the criteria to be emphasized in IMPEP. An annual IMPEP team leader workshop would also help ensure that consistent IMPEP review outcomes are sustained in the future.

Recommendation 2: The 2017 self-assessment team recommends that an IMPEP team leader workshop should be held annually in conjunction with the annual IMPEP team member training and the workshop should include topics such as indicator criteria.

IMPEP Team Leaders Should Monitor MRB Meetings The 2017 self-assessment team reviewed 6 years of MRB minutes and found that the content of the minutes were generally consistent. In addition, the team took the opportunity to observe MRB meetings during its assessment and found that listening to the actual deliberations provided ample information as to how the MRB reached its conclusions based upon the input provided by the IMPEP team. The 2017 self-assessment team noted that it is important that the IMPEP Project Manager ask clarifying questions of either the MRB or IMPEP team member participants to explain details of pertinent issues, so that the discussion and subsequent decisions can be accurately recorded. The 2017 self-assessment team also concluded that

2017 Focused Self-Assessment Report Page 7 of 9 NMSS Procedure AD-800 contains sufficient guidance for documenting MRB evaluation of IMPEP reviews and periodic meetings.

Monitoring MRB discussions provides a unique perspective as to how the MRB operates and reaches its conclusions. Based upon the unique detail provided in MRB deliberations, the team concluded that all IMPEP Team members and Team Leaders would benefit from hearing these discussions. The team concluded that monitoring or attending these meetings is an effective way to better understand the issues that the Agreement States are facing and for understanding the methodology and rationale that underlies the MRBs conclusions and recommendations.

Monitoring MRB discussions and deliberations provides the team with the context of MRB discussions and deliberations and may better prepare team members and leaders in developing MRB presentations. Hence, the 2017 self-assessment team recommends that the importance of monitoring MRB meetings be included in the IMPEP Team Leader training. The team noted that this monitoring can be facilitated by ensuring that all qualified IMPEP Team Leaders are invited to all IMPEP MRB meetings.

Recommendation 3: Include in the IMPEP Team Leader training the importance of monitoring IMPEP MRB meetings.

Draft IMPEP Reports Should not be made Publicly Available During the course of the review, the team found no instances where MRB minutes or IMPEP reports included incorrect or inaccurate information. However, the team acknowledged that the potential for incorrect information being made available to the public is increased if the report is made publically available prior to the Agreement State program having the opportunity to review and comment on the report.

There are three review stages to an Agreement State and Regional IMPEP report: Draft, Proposed Final, and Final. Currently, the draft IMPEP report is publicly available on the Agencywide Documents Access and Management Systems (ADAMS) and contains pre-decisional information. This draft IMPEP report is subject to both Agreement State program review (for technical accuracy and completeness) and modification by the IMPEP team. The 2017 self-assessment team believes that the draft version of the IMPEP report should not be made publicly available because it increases the potential for inaccurate information to be released. Per Management Directive 3.4, Release of Information to the Public, Handbook 3.4, Part II, Draft Agreement State Documents, the NMSS Office Director has the authority to make that change and can approve the release to Agreement States of documents that are not publicly available.

Recommendation 4: The 2017 self-assessment team recommends NMSS, in coordination with OGC, evaluate whether the Draft IMPEP report can be made not-publicly available until reviewed by the Agreement State program or the Region and if so, revise the process to designate the Draft IMPEP report as not-publicly available.

Objective 3: Determine the status (completed/on-going/deferred) of the recommendations of the 2010 IMPEP self-assessment and report the effectiveness of actions taken in response to the recommendations.

In June 2010, the former FSME issued a self-assessment of the entire IMPEP program which included 15 recommendations. This self-assessment was in response to the 2009 OIG Audit of the NRCs Agreement State Program. The 2017 self-assessment team reviewed the 2010

2017 Focused Self-Assessment Report Page 8 of 9 IMPEP self-assessment report and recommendations. The 2010 IMPEP self-assessment included the status of the planned actions, estimated timelines, and resources needed for implementation for each recommendation. Of the 15 recommendations from the 2010 report, thirteen are closed and two recommendations are open. The two open recommendations are (1) Adopt the 2010 IMPEP self-assessment recommendation to change reciprocity inspections from 20 percent to 5 percent and (2) Expand the scope of IMPEP to include additional functions.

These are recommendations No. 1 (discussed above) and No. 5 (discussed below) in this report. The 2010 self-assessment recommendations, and the 2017 self-assessment team observations and recommendations for each issue can be found in Appendix C.

Objective 3 Recommendations:

Expand IMPEP to Include NRC Headquarters Materials Licensing Activities While reviewing the third objective of the 2017 focused self-assessment, the status of recommendations from the 2010 IMPEP review, the team learned that expanding a Headquarters IMPEP review to other programs was a recommendation of a 2002 working group report and a recommendation of the 2010 IMPEP self-assessment review. Headquarters licensing programs suggested for inclusion in an IMPEP review include the uranium recovery licensing program, the complex decommissioning program, and the exempt distribution licensing and general licensing programs. In 2010, the inclusion of uranium recovery licensing was considered particularly important given the substantial increase in new applications for uranium recovery licenses and the potential equity issues that could result from reviewing this program in Agreement States under IMPEP, but not within NRC. The 2017 self-assessment team discussed the issue of including additional NRC Headquarters functions into the IMPEP review process. The 2017 self-assessment team members concluded that additional Headquarters programs would benefit from an IMPEP review in the same way the NRC Regional radioactive materials licensing and inspection programs benefit from IMPEP reviews.

Where applicable, these areas are already included in IMPEP reviews of Agreement State Programs.

Recommendation 5: The 2017 self-assessment team recommends that the scope of IMPEP should be expanded to include additional Headquarters functions, including exempt distribution licensing, general licensing, uranium recovery program licensing, and complex decommissioning licensing. This will require modification of MD 5.6 and other Management Directives as well as corresponding State Agreement (SA) procedures and development of new SA procedures.

Consider Performing NRCs IMPEP Review as One Program Although not clearly part of the self-assessment Charter, the self-assessment team observed that one difference in the review of NRCs materials licensing and inspection program, compared with the Agreement States, is that the Regional and Headquarters programs are reviewed separately and not as part of an integrated system. For example, an IMPEP review of the New York State program currently involves an onsite review at each of three separate agencies/locations (prior to 2006, New York IMPEP reviews involved four agencies). The individual agency reviews are combined into a single IMPEP report and a single MRB meeting is conducted to review the IMPEP report. The single report addresses the entire New York State program. If one indicator for an agency program is assessed as less than satisfactory, the entire program is assessed as less than satisfactory for that indicator. This isnt the case with the review and evaluation of the NRC materials program. The NRC reviews its three Regional

2017 Focused Self-Assessment Report Page 9 of 9 Offices (and the Headquarters sealed source and device program) as totally separate and independent organizations, each receiving a separate IMPEP report and MRB review meeting.

During the review, the 2017 self-assessment team evaluated whether the NRC is conducting IMPEP reviews of its own programs in a similar fashion as the IMPEP and MRB reviews of Agreement State programs. There isnt a consensus on the 2017 self-assessment team regarding the benefits of treating the NRC as a single entity given the geography of NRC Regional Offices being spread across the nation rather than located within a single state. The self-assessment team did not evaluate the scale of the geographic travel requirements, nor was a benefit-cost analysis conducted to support that effort. Although the team did not reach a consensus on this matter, the team did agree that further evaluation may be may be warranted to more clearly understand the potential benefits and disadvantages.

Recommendation 6: The 2017 self-assessment team recommends that the NRC evaluate the advantages and disadvantages of conducting IMPEP reviews of its programs in a similar fashion as states with multiple agencies. The NRC should consider whether the Agency should be evaluated as a single entity or as separate Regional and Headquarters entities.

5.0 APPENDICES Appendix A - Charter Appendix B - Project Plan Appendix C - Status of 2010 recommendations

APPENDIX A - CHARTER FOR THE FOCUSED SELF-ASSESSMENT OF THE INTEGRATED MATERIALS PERFORMANCE EVALUATION PROGRAM PURPOSE The purpose of this review is to conduct a focused self-assessment of the Integrated Materials Performance Evaluation Program (IMPEP) with appropriate findings/recommendations to improve its efficiency and effectiveness.

BACKGROUND On March 16, 2009, the U.S. Nuclear Regulatory Commissions (NRC) Office of Inspector General (OIG) issued an audit report titled, Audit of the NRCs Agreement State Program (OIG-09-A-08). The objective of the audit was to assess NRCs oversight of adequacy and compatibility of Agreement State programs. The OIG made several recommendations including Develop a mechanism for conducting self-assessments and capturing lessons learned for IMPEP on a regular basis. In June 2010, the former Federal and State Materials and Environmental Programs Office issued a Self-assessment of IMPEP (ML102030228) which included 15 recommendations. One substantive recommendation was that future self-assessments be conducted on a frequency of 4 to 6 years.

The IMPEP has been and continues to be recognized by the Agreement States and the NRC as an effective means of evaluating performance, and self-assessments are an integral part of how high-performing organizations establish and maintain those levels of performance.

OBJECTIVES The work shall be undertaken by a 2017 self-assessment team comprised of NRC and Agreement State representatives. The 2017 self-assessment team shall address the following tasks at a minimum and make appropriate findings and recommendations. Recommendations should be evaluated and prioritized against the NRCs strategic goals of safety and security.

1. Review the recommendations of the 2010 IMPEP self-assessment to determine the status (completed/on-going/deferred) and effectiveness of the actions taken in response to the recommendations.

2. Evaluate the programs performance in the indicators (1) Technical Staffing and Training and (2) Status of the Materials Inspection Program and determine if changes or enhancements are warranted. For each indicator, the 2017 self-assessment team should perform an assessment using the following questions as guidance:

(a.) Is the performance indicator complete and sufficiently focused?

(b.) Are changes needed in the indicator criteria and metrics?

(c.) Are there any patterns or issues identified from MRB review of the IMPEP report and discussion of Agreement State programs during MRB meetings that need to be addressed?

(d.) Are IMPEP recommendations applied consistently across the program and are those recommendations effective?

(e.) Is the performance indicator appropriate, adequate, and applied consistently by different IMPEP teams?

(f.) Are there any additional criteria which may be developed by the 2017 self-assessment team?

3. Assess whether the experience level of the IMPEP Team Leader or other factors (e.g., program size) contributed to inconsistent IMPEP review findings.

4. Provide a documented, self-assessment, draft report to the Director, Division of Materials Safety, State, Tribal, and Rulemaking Programs no later than June 5, 2017.

5. Present the results of the IMPEP self-assessment to the MRB, if necessary.

MEMBERSHIP Robert Dansereau - New York Vincent Holahan - NMSS/MSSAT Jim Lynch - Region III/DNMS (retired)

Joe OHara - NMSS/MSST/ASPB - Team Leader Charma Waring - Rhode Island LEVEL OF EFFORT EXPECTED OF PARTICIPANTS Given the importance of this IMPEP review, the 2017 self-assessment team should anticipate spending a substantive amount of their time, up to 10 percent, on team activities. The amount of time per week will vary, but it is estimated to average approximately 3 to 5 hours5.787037e-5 days <br />0.00139 hours <br />8.267196e-6 weeks <br />1.9025e-6 months <br /> per week.

NRC employees should charge their time to code A34018.

The 2017 self-assessment team will develop a detailed work plan and will conduct the review in the allocated time, developing appropriate written products. The plan will include reviews of program documentation including IMPEP reports and MRB minutes for those reports from 2011 through 2016; MD 5.6, Integrated Materials Performance Evaluation Program (IMPEP),

appropriate State Agreements (SA) procedures including SA-101, Reviewing the Common Performance Indicator, Status of Materials Inspection Program and SA-103, Reviewing the Common Performance Indicator, Technical Staffing and Training. All documents will be provided by the NRC to each participant on a compact disc to facilitate the work being conducted at each members work location. There is no scheduled travel related to the focused IMPEP self-assessment.

MEETING SCHEDULE The 2017 self-assessment teams first meeting will be held on November 15, 2016. Available technology will be used to facilitate interaction with the working group members, (e.g., conference calls, GoToMeeting and electronic mail) and will be provided and paid for by the NRC.

A-2

The 2017 self-assessment Team Leader will determine the frequency and timing of meetings based upon the input and progress of the team. A meeting schedule will be developed and provided to the self-assessment team by the first meeting. Meetings are pre-decisional and will be closed to the public. Self-assessment team members may not delegate alternates to attend meetings.

A-3

APPENDIX B - PROJECT PLAN FOR 2016 FOCUSED IMPEP SELF ASSESSMENT Personnel Project Lead and team leader - Joe OHara Agreement State representatives - Robert Dansereau - NY, and Charma Waring - RI U.S. Nuclear Regulatory Commission (NRC) representatives - Vincent Holahan and Jim Lynch (RIII) (retired)

Key Steps/Leads to Accomplish

1) Identify/ Confirm Personnel (October 2016): Coordinate with OAS leadership and Regional/HQ managers to identify and secure commitments of team members.

(Daniel Collins - Regions/Paul Michalak - OAS/Joe OHara)

2) Prepare for Initial Meeting (October 2016): Send email to participants welcoming them and providing project plan, and expectations for initial meeting. (OHara)

3) Hold Initial Meeting (November 2016): Discuss and align on project plan, approach, deliverables, schedule key milestones. (Team)

4) Individual Work Period (November 2016 - February 2017): Team members work from home locations to review documentation, consider team discussions and begin to identify on gaps, areas for improvement, and recommendations. (Team)

5) Brief OAS Leadership (March 2017): Brief OAS leadership on status and get additional thoughts as part of a routine monthly OAS Executive Board meeting. (Collins/OHara)

6) Hold Substantive Meeting (April 2017): Team teleconference. Systematically discuss work results and formulate recommendations. Prepare annotated outline of review findings and recommendations. Brief MSTR management on nature and substance at conclusion of week. (Team)

7) Prepare and Submit Draft Report Sections (April - May 2017): (Team)

8) Assemble and Integrate Report Sections into Draft Report (May 2017): (OHara)

9) Circulate Draft Report to Team for Comment (June 2017): (OHara)

10) Incorporate Comments and Prepare Final Report for MSTR Director (June 2017):

(OHara)

11) Brief MRB, as Appropriate (TBD):

Resources Needed:

The following information will be provided in electronic format to all 2017 self-assessment team members and is also available on the NRC Web site:

1) Management Directive 5.6

2) Appropriate SA Procedures

3) 2009 OIG report

4) June 2010 Working Group (WG) on IMPEP Lessons Learned

5) ASPBs actions on 2010 WG recommendations

6) MRB Discussions B-2

APPENDIX C - STATUS OF 2010 RECOMMENDATIONS Overall, the 2010 Integrated Materials Performance Evaluation Program (IMPEP) self-assessment team found the IMPEP program to be highly effective in fulfilling its basic objective (evaluating the adequacy and compatibility of Agreement State and U.S. Nuclear Regulatory Commission (NRC) materials activities using a common set of performance review criteria and a similar review methodology). The team developed a set of recommendations and enhancements (listed below) that could, if implemented, result in an even stronger and more efficient IMPEP for the future. These recommendations were reviewed by the Management Review Board (MRB) and the members agreed with the recommendations with little or no modification.

The 2017 IMPEP focused self-assessment team reviewed the recommendations of the 2010 IMPEP self-assessment and the former Division of Materials Safety and State Agreements plan to address the recommendations of the 2010 self-assessment. The review included the status of the planned actions (completed/on-going/deferred), estimated timelines, resources needed for implementation for each recommendation, and effectiveness of the actions taken in response to the recommendations. The 2010 self-assessment recommendations and enhancements, and the 2017 focused self-assessment team comments on the status of each recommendation are listed below:

Recommendation 1 (substantive): Recommends that IMPEP review of Increased Controls, National Source Tracking System (NSTS), and other aspects of security currently addressed under TI-002 be integrated into the existing common performance indicators through modification of Management Directive (MD) 5.6 and the associated SA procedures for implementation of IMPEP. The existing set of common performance indicators remains effective and no new indicators are needed at this time.

On March 28, 2006, the former NRC Office of State and Tribal Programs issued temporary instruction TI-002, Integration of Increased Controls into the Integrated Performance Evaluation Program (IMPEP), which evaluated the implementation of increased controls in NRC and Agreement State Programs through IMPEP. On March 19, 2013, the Commission decided, through the publication of the Title10 of Code of Federal Regulations (10 CFR) Part 37, to formalize additional security controls associated with Category 1 and Category 2 licensees.

Three State Agreements procedures have been revised that incorporate all of the Part 37 requirements into the IMPEP process. On April 13, 2012, State Agreement Procedure (SA)-104, Reviewing the Common Performance Indicator, Technical Quality of Licensing Actions, was issued. On July 14, 2015, SA-101, Reviewing the Common Performance Indicator, Status of Materials Inspection Program and on January 6, 2016, SA-102, Reviewing the Common Performance Indicator, Technical Quality of Inspections were issued. The issuance of these three procedures, in part, obviates the need for TI-002 which is no longer required or used in the inspection process. TI-002 was rescinded on March 28, 2016. MD 5.6, Integrated Materials Performance Evaluation Program (IMPEP), is being revised.

Comment: This recommendation is closed. As of May 2013, 100 percent of licensees possessing nationally tracked sources had reconciled their inventory against the NSTS, and the use of the NSTS and other web based applications is fully recognized in SA-101 and SA-102.

MD 5.6, Integrated Materials Performance Evaluation Program fully recognizes the specific evaluation criteria in Part III, Evaluation Criteria. The issuance of SA-102 and 10 CFR Part 37 obviates the need for TI-002 since it incorporates Part 37 requirements which supersedes

TI-002. In March 2016, an STC letter was issued to the Agreement States to inform them that TI-002 (and TI-001) are rescinded.

Recommendation 2 (substantive): Recommends that treatment of reciprocity inspections be made more performance-based and that the current 20 percent requirement for reciprocity inspections be removed from the IMPEP program along with corresponding changes to Inspection Manual Chapters 2800 and 1220.

According to Inspection Manual Chapter (IMC) 2800, Materials Inspection Program, which was last updated on November 25, 2003, the inspecting Region shall follow the policy and guidelines found in IMC 1220, Appendix III, for performing inspections of reciprocity licensees. According to IMC 1220, Processing of NRC Form 241 and Inspection of Agreement State Licensees Operating Under 10 CFR 150.20, which has not been revised since May 1, 2003, the percentage of inspections of reciprocity licensees to be inspected each year are based on the number of candidates for inspection. For priority 1, 2, and 3 program codes - 20 percent of the candidate licensees from the candidate pool are to be inspected each year. If NRC has inspected a reciprocity licensee in the last calendar year, and the licensee has not had escalated enforcement or a significant event in the past 2 years, then the reciprocity licensee is NOT to be considered a candidate for inspection. No new guidance has been published since 2003 regarding the frequency of reciprocity inspections.

Similarly, a performance-based option as an alternative to reciprocity inspections has not been developed and adopted. While information on recent events, significant violations, or escalated enforcement involving the licensee might be considered if the licensee had been previously inspected by the NRC within the last calendar year, licensees who are outside the one year window are exempt from this performance-based option.

Comment: The 2017 focused, self-assessment team recommends adoption of the 2010 IMPEP self-assessment recommendation to change reciprocity inspections from 20 percent to 5 percent, issue a Temporary Instruction clarify how candidates for reciprocity are to be calculated and change IMC 1220 to remove the 20 percent requirement on reciprocity inspections.

Recommendation 3 (substantive): Recommend that a mechanism be put into place to recognize consistently high-performing organizations. Building on the language already contained in SA-106, two consecutive IMPEP findings of satisfactory on all indicators would lead to a letter of recognition and an extension in IMPEP interval from 4 years to 5 years, along with a press release acknowledging the organizations high level of performance.

Recommendation 3 was adopted as part of a revision of SA-106, The Management Review Board, in March 2011. During the 2010 IMPEP review, the assessment team investigated ways in which high-performing organizations could be recognized. As part of the review and revision of SA-106, if an Agreement State has been found satisfactory for all performance indicators and received no recommendations regarding program performance during two consecutive IMPEP reviews, a letter for transmitting the final IMPEP review could include language commending the State for consistently meeting the standards of performance in all program areas and for the States continued support in protecting public health and safety. If a State meets the above criteria, the MRB may exercise the option to extend an Agreement States performance review period from 4 to 5 years, which has occurred for several Agreement States. NRC Region performance review periods may be extended in a similar manner.

C-2

Comment: This recommendation should be considered closed effective March 4, 2011, with the publication of SA-106 which contains special recognitions if a State has been found satisfactory for all performance indicators during two consecutive IMPEP reviews. In such an instance, the MRB may exercise the option to extend a States, or NRC Regions performance review period from 4 to 5 years.

Recommendation 4 (enhancement): Recommends that the IMPEP reports add an executive summary at the front of the report which will provide at a glance the key findings from the review, any recommendations made by the team as well as any good practices identified. The executive summary would also include a table showing the status of all applicable indicators.

On January 24, 2011, the Nebraska 2010 IMPEP report contains an Executive Summary.

Beginning in 2011, IMPEP reports contain an Executive Summary which provides key findings from the review and recommendations by the IMPEP team. A written description of the status of all applicable indicators is included in the Executive Summary, but not a tabular description.

Team recommendations for program improvements are included. Good practices are also identified in the Executive Summary, see IMPEP review reports for Ohio (December 2013),

Colorado (April 2014), and Wisconsin (July 2014).

Comment: This recommendation is closed.

Recommendation 5 (enhancement): Recommends that the NRC encourage States to increase use of electronic documents to the extent possible to increase efficiency of the onsite review.

This could include, for example, providing as much information about the program as possible in electronic format in advance of the review to allow team members to better prepare. For IMPEP reviews of NRC Regions, this would mean expanded use of the NRCs Agencywide Documents Access and Management System (ADAMS) documents in advance of the review.

In its review of the 2010 self-assessment, the MRB agreed with this recommendation to encourage Agreement States to increase their use of electronic documents and provide as much information about their program as possible in electronic format. Initial correspondence between the IMPEP team leader and the Agreement State encourages the use of electronic documents and requests that these documents be provided to the IMPEP team in advance of the review to allow team members to better prepare for the onsite review.

Several Agreement State programs rely on electronic documents, including Web Based Licensing and in-house systems. IMPEP questionnaire responses are routinely received from Agreement States in electronic form. With the advance of electronic data systems, IMPEP teams need to be given access to the system before the review. Instructions were provided to IMPEP team leaders to discuss this issue with program management before a review so that team members can review relevant documents. Note: The use of electronic documents (ADAMS, IMPEP toolbox, etc.) was extensively used during the focused self-assessment of the IMPEP program by the 2017 self-assessment team.

Comment: This recommendation is closed.

C-3

Recommendation 6 (enhancement): Recommends that NRC take appropriate steps to ensure a consistent high level of management participation at Management Review Board meetings (for formal IMPEP reviews as well as mid-cycle reviews involving performance issues) both in terms of NRC attendees and Agreement State program and liaison attendees.

The MRB provides a senior-level review of the IMPEP team's findings and recommendations and issues the final NRC findings to the Region or Agreement State. Upon review of the 2010 IMPEP self-assessment, the MRB agreed that a consistent, high level of management participation at the MRB meetings was warranted. In order to ensure a high level of management participation, the MRB suggested considering a hard, fast requirement that if a member cannot attend, then a MRB will not be held.

SA-106 provides the guidelines that will be followed by the MRB when conducting MRB meetings for IMPEP reviews and was revised and re-issued in March 2011. The MRB is composed of the Deputy Executive Director for Materials, Waste, Research, State, Tribal, Compliance, Administration, and Human Capital Programs (DEDM); the General Counsel; the Director, Nuclear Material Safety and Safeguards (NMSS); and a Regional Administrator from an NRC Regional Office. Each of these members can designate an alternate representative in instances when attendance is not possible. An Agreement State Liaison (non-voting member) participates on the MRB to provide an objective perspective on any matter that is discussed or voted on by the MRB. In order to begin the proceedings, a quorum must be present. A quorum is established if three voting members are present. Designees count toward reaching a quorum.

The MRB minutes and attendee lists from 2010 through 2016 were reviewed by the 2017 focused self-assessment team. Given the flexibility of MRB members to designate an alternate representative in instances when attendance is not possible, MRB meeting attendance varied considerably. The DEDM or their acting generally chaired the MRB meetings. The NMSS/FSME Director or their acting generally was in attendance. NMSS/FSME attendance varied due to a re-organization that occurred in 2014. Sixteen different managers represented NMSS/FSME during a five year span. The General Counsel did not attend any MRB meetings, rather, the Assistant General Counsel for Licensing and Regulation (or their acting) generally attended. Regional Administrators attended 8 MRB meetings and deputy Regional Administrators (or their alternates) attended 47 meetings over a five year span.

Scheduling nine or ten MRB meetings each year and ensuring that all principal members attend is challenging. However, the flexibility to designate alternate MRB representatives to attend MRB meetings has allowed for the timely review of IMPEP reports without adversely impacting the quality of the overall review. This is due, in part, to the IMPEP Coordinator providing MRB training to all designated participants.

Comment: This recommendation is closed. The 2017 self-assessment team recommends that composition of the MRB not change and strongly encourages regular participation by the principal MRB members which will promote consistency in their reviews and demonstrate the highest level of senior level management commitment to the IMPEP program.

Recommendation 7 (enhancement): Recommends that new MRB members as well as Organization of Agreement States (OAS) Liaisons to the MRB receive an orientation to MRB functions, roles and responsibilities.

C-4

Between 2011 and 2016, there have been 7 different MRB chairs, 16 representatives for NMSS/FSME, 9 representatives for the General Counsel, 16 representatives for the Regional Administrator, and 27 Agreement State liaisons. Given the amount of turnover of representatives and the number of alternates asked to participate in MRB meetings, the IMPEP Coordinator began providing briefings on MRB Roles and Responsibilities. Briefings have been provided to attendees at the Organization of Agreement State meeting, or to other identified MRB participants via GoToMeeting or briefings. Refresher training also has been offered to MRB participants that have not participated in MRB meetings for several years.

Comment: This recommendation is closed.

Recommendation 8 (substantive): Recommends that, to better address issues of continuing degraded performance in a program, that a more explicit trigger point for MRB consideration of placing a program on probation be included in SA-113.

During the 2010 IMPEP self-assessment, the team observed that significant performance issues in a radiation control program stem from the lack of adequate budget resources, inadequate fees to support the program, low salary levels, personnel turnover and lack of support for the program at the management level. In cases where these issues continue unabated for an extended period of time, the NRC has been challenged to deal effectively with them. Several interviewees suggested that the NRC was reluctant to take the next step from Heightened Oversight to placing a program on Probation and that the NRC needs to take this step when circumstances are warranted. The 2010 self-assessment team concluded that one way to address this issue would be to modify the IMPEP procedures to denote specific conditions under which Probation will be considered by the MRB. At the present time those procedures, specified in SA-113, only list conditions under which Probation may be considered. In its review of the 2010 self-assessment, the MRB agreed with the recommendation and noted that the lack of evaluation factors (i.e., explicit trigger points) may have caused some states to not be considered for probation and that placing a State on probation could put State decision makers on notice of issues, resulting in necessary changes. SA-113, Placing an Agreement State Program on Probation, was published by the Office of State and Tribal Programs in 2000 and subsequently revised and issued on February 24, 2010 and again on June 5, 2017. This State Agreement procedure has been under review and revision with the emphasis on procedures for placing a program on probation or procedures for discontinuing the probationary status. No changes have been proposed that will denote specific conditions under which probation will be considered by the MRB.

The 2017 self-assessment team believes the revision contains adequate guidance that the MRB can review to determine if a probation recommendation is warranted along with changes to the administrative processes which streamline the decision making process gives the 2017 self-assessment team confidence that the State Agreement procedure is adequate for implementing probation, if necessary.

Comment: This recommendation should be considered closed effective June 5, 2017 with the publication of SA-113, Placing an Agreement State Program on Probation.

Recommendation 9 (substantive): Recommends that NRC employ a more performance-based approach (e.g., allowing the use of legally-enforceable license conditions while rulemaking is underway) to findings of compatibility in Agreement State programs utilizing the flexibility contained in MD 5.6.

C-5

The 2010 self-assessment team was concerned that individual Agreement States might have difficulty implementing compatible Agreement State regulations within 3 years after the effective date of NRCs final rules. The team believed that MD 5.6 provided ample flexibility for compatibility with statements that regulations should be adopted and the timeframe for their adoption should be 3 years after the effective date.

Agreement States need not adopt a specific regulation if the State has no licensees that would be subject to that regulation. In such cases, however, the State would need to commit to adopting the regulation, or to impose the regulation through license conditions or other legally binding means, if an application were to be received by the State.

This approach is well known by Agreement States. Communications through State and Tribal Communication letters and during monthly OAS/CRCPD calls have emphasized the fact that legally binding requirements may be used in lieu of regulations. A good example of this occurred with the adoption of 10 CFR Part 37 equivalent requirements. Several Agreement States were unable to promulgate regulations before the March 2016 deadline. Instead, they used approved license conditions to require the appropriate security requirements. A model license condition developed by Wisconsin and approved by the NRC Division of Material Safety, State, Tribal and Rulemaking Programs (MSTR), was shared with the other Agreement States.

Legally binding requirements are required to be reviewed by NRC if used in lieu of regulations.

The documents are sent to the MSTR State Regulation Review Coordinator. An Agreement States State Regulation Data Sheet is updated to identify if a regulation is adopted or a legally binding requirement is being instituted instead. IMPEP teams have this information available during their reviews.

Despite Agreement States continuing to have different processes for adopting NRC-promulgated rules, the majority of Agreement States are judged to have legally binding requirements (regulations or other necessary program elements) that are compatible with the NRC. As of March 24, 2017, only four States are currently judged to be not compatible.

Comment: In its review of the 2010 self-assessment, the MRB agreed that a performance-based approach was already being employed by the Agreement States and that no further action was recommended and the recommendation should be considered closed.

Recommendation 10 (substantive): Recommends that greater focus be placed on inspector accompaniments under the common performance indicator for Technical Quality of Inspection.

Specifically, the IMPEP review team should attempt to accompany 50 percent of a States or Regions inspectors in each IMPEP review. To minimize the resource impacts of these additional accompaniments, they could be scheduled over the 4 year period between formal reviews.

In its review of the 2010 self-assessment, the MRB generally agreed with the recommendation with some modifications (i.e. accompaniments should be more weighted towards those closer to the date of the IMPEP review). If accompaniments are scheduled over a four year period, the State may not be able to implement recommended changes before the first inspector accompaniment and staff turnover may adversely impact the effectiveness of the IMPEP review.

Rather, a two year period was discussed by the MRB.

C-6

Greater focus on inspector accompaniments was incorporated into a revised SA-102 dated January 6, 2016. The NRC staff revised the guidance so that the number of inspector accompaniments to be performed before the IMPEP review is dependent on a variety of factors, including the results of previous IMPEP reviews, as well as the size of the program being reviewed. In most cases, the new goal is to accompany one-half of the programs inspectors.

However, guidance in SA-102 also suggests that for a program with few inspectors (5 or less),

consideration should be given to accompanying all of the inspectors.

Not all accompaniments can be scheduled immediately before the on-site IMPEP review.

Depending on the size of the radioactive materials program being reviewed, it may be necessary to schedule some inspector accompaniments well in advance of the on-site portion of the IMPEP review. For example, risk-significant/complex licensees are inspected every 1-2 years. For a small materials program, with fewer risk-significant licensees, it may be necessary to conduct accompaniments up to a year in advance of the on-site IMPEP to ensure inspection accompaniments of such licensees.

Comment: SA-102 was revised to place greater emphasis on inspector accompaniments. As such, this recommendation was closed by memorandum dated February 22, 2016.

Recommendation 11 (enhancement): Recommends that in the interest of program effectiveness and succession planning, that the pool of Team Leaders for IMPEP reviews be expanded and that Team Leader training be added to the agenda for routine IMPEP training.

The training and qualification process described in SA-111, Implementation of Management Directive 5.10, Formal Qualifications for Integrated Materials Performance Evaluation Program Team Members and Team Leaders, is intended to provide IMPEP team members and team leaders with sufficient knowledge to conduct Agreement State and NRC radioactive materials program reviews that are technically sound and in accordance with NRC policies and procedures. The NRC and Agreement State employees designated as IMPEP team members and team leaders must successfully complete the requirements for individually assigned areas of this procedure. In addition to the requirements of this procedure, other training may be necessary to supplement or enhance team member development. SA-111 was initially published in December 2000 and subsequently revised and re-issued on March 22, 2017. The revised SA-111 lists the recommended specific training, supplemental training, self-study, and professional experience required for selection as a team leader.

The number of qualified IMPEP team leaders is sufficient to meet the current and projected program review schedule. There were nine scheduled IMPEP reviews for both fiscal year (FY) 16 and FY17. Ten NRC staff from Headquarters and the Regions are qualified to lead an IMPEP team. The size of the current pool of team leaders is sufficient to allow each team leader to participate in at least one IMPEP review annually. There is one projected team leader retirement at the end of 2017 and there is one team leader in training who will complete their team leader training in FY17. The IMPEP Coordinator maintains the list of qualified team members and team leaders.

An IMPEP team leader workshop was conducted during the week of February 13, 2017.

Attendees at the workshop included the 2017 IMPEP review team leaders and Regional State Agreements Officers, and several team leaders in training. The topics considered by the team leaders included current and proposed guidance for common and non-common performance indicators as well as a roundtable discussion with the NMSS Director.

C-7

Subject to the availability of travel funds, team leader training or a workshop should be offered in conjunction with the IMPEP team member training each year. Team leaders identified to lead IMPEP reviews in the next fiscal year should be highly encouraged to attend the annual team leader workshop.

Comment: No further action is recommended and the recommendation should be considered closed.

Recommendation 12 (enhancement): Recommends that FSME continue to work with Regional and Agreement State managers to assure that individuals selected for IMPEP training possess the appropriate technical skills, assessment skills, interpersonal skills and political savvy to serve as effective IMPEP team members.

State Agreement procedure SA-111, was revised and re-issued on March 22, 2017. The training and qualification process is intended to provide future IMPEP team members and team leaders with sufficient knowledge to conduct Agreement State and NRC radioactive materials program reviews that are technically sound and in accordance with NRC policies and procedures. The NRC and Agreement State employees designated as IMPEP team members and team leaders must successfully complete the requirements for individually assigned areas, as listed in Section V. of the procedure. In addition to the requirements of the procedure, other training may be necessary to supplement or enhance team member development.

Periodically, a letter of solicitation is sent to Agreement State Program Directors and NRC Regional managers to recruit new IMPEP members. The NRC offers the IMPEP team member training to the NRC and Agreement State radiation control program employees with an interest in performing IMPEP reviews. Any NRC and Agreement State staff member is eligible to participate on an IMPEP review team as long as they can meet the criteria to be a principal reviewer for at least one IMPEP performance indicator, as described in the NRC Management Directive 5.10, Formal Qualifications for Integrated Materials Performance Evaluation Program (IMPEP) Team Members and have the recommendation of their program director. Additional training and experience requirements are specified for IMPEP team leaders.

It is the responsibility of Agreement State Program Directors and NRC Division Directors to ensure that nominees possess the appropriate technical skills, assessment skills, interpersonal skills and political savvy to serve as effective IMPEP team members.

Comment: This recommendation should be considered closed effective March 22, 2017 with the publication of SA-111.

Recommendation 13 (enhancement): Recommends that MD 5.6 be revised to reflect current organizational structure, roles and responsibilities, and references to current appropriate SA procedures.

The 2010 self-assessment team recommended, and the Management Review Board agreed, that MD 5.6 be revised to reflect current NRC organizational structures and SA procedures with a timeline/due date of October 2012. In addition to revising MD 5.6 to reflect current NRC organizational roles and structures and state agreement procedures, the management directive is being revised to include the increased physical protection requirements and access control programs associated with the protection of Category 1 and Category 2 quantities of radioactive material as addressed under 10 CFR Part 37. A draft revision of MD 5.6 was forwarded to the MSTR Division Director for review and concurrence in June 2017.

C-8

Comment: Since MD 5.6 is being revised, this recommendation is on-going. Once MD 5.6 is approved, this recommendation will be closed.

Recommendation 14 (substantive): Recommends that the scope of IMPEP be expanded to include additional FSME functions, including Exempt Distribution licensing, General Licensing; Uranium Recovery licensing and complex decommissioning cases. This will require modifications to MD 5.6 and other Management Directives as well as corresponding SA procedures and the development of new SA procedures.

In its review of the 2010 self-assessment, the MRB agreed with the recommendation to expand the scope of IMPEP to include additional FSME functions and suggested that additional program areas might be added. The only Headquarters materials program reviewed was the Sealed Source and Device Evaluation Program, evaluated four times since 2001. These reviews did not include any additional functions that currently reside in NMSS.

The 2017 self-assessment team discussed the issue of including additional NRC Headquarters functions within the purview of IMPEP. Functions that might be considered include exempt distribution licensing, general licensing, uranium recovery program licensing, complex decommissioning licensing, fuel cycle inspection programs, and import/export licensing. The latter function, while part of NRCs mission, is beyond the scope of the IMPEP program and was not considered further.

Comment: While action on this recommendation has been deferred since 2010, the 2017 self-assessment team sees merit in recommending its implementation, pending the availability of resources. Hence, the scope of the IMPEP review of NRC Headquarters activities should be expanded to include additional NMSS functions, including exempt distribution licensing, general licensing, uranium recovery program licensing, and complex decommissioning licensing. This will require modifications to MD 5.6 and other Management Directives as well as corresponding SA procedures and development of new SA procedures.

Recommendation 15 (substantive): Recommends that self-assessments of the IMPEP program be conducted in the future on a frequency of 4-6 years. A draft procedure to guide the conduct of these assessments has been prepared for FSME management consideration as Appendix C to this (2010) report.

A procedure to guide the conduct of self-assessments of the IMPEP programs was published as SA-123, Conducting Self-Assessments of the Integrated Materials Performance Evaluation Program (IMPEP), dated June 24, 2016. The guidance suggests that a MD 5.3 working group should be formed to perform an IMPEP self-assessment every five years and ensure that Agreement State viewpoints are sought during the assessment and reflected in working group recommendations. The frequency of the self-assessment can be adjusted if (1) NMSS management determines that the IMPEP is strong enough to sustain a longer interval; (2) IMPEP weaknesses are identified that would suggest the need for a more frequent review interval; or (3) if workload or resource restraints suggest to NMSS management that resources and efforts involved to conduct the IMPEP self-assessment are needed to perform a different task.

In addition, MD 5.3. Agreement State Participation in Working Groups and SA-801, Agreement State Participation in NRC Working Groups, were also updated and finalized in 2016.

Comment: This recommendation is closed.

C-9