ML22305A688

Charter - Working Group on Assessing NMP Performance (20221031) - OAS-RI-OGC Comments Addressed

ML22305A688
Person / Time
Issue date:	11/07/2022
From:	Flaherty S NRC/NMSS/DMSST
To:
References
Download: ML22305A688 (8)

Text

Working Group Charter

Assessing the Ability to Monitor NMP Performance

PURPOSE

The Working Group (WG) to Assess the Ability to Monitor National Materials Program (NMP)

Performance has been established as a Management Directive (MD) 5.3 working group. The purpose of the WG is to evaluate whether the current Integrated Materials Performance Evaluation Program (IMPEP) process provides for the proactive assessment of the NMP radiation control programs (RCPs) performance. The WGs evaluations and recommendations will be briefed to senior U.S. Nuclear Regulatory Program (NRC) managers and the Organization of Agreement States (OAS) board and documented in a standalone memorandum to NRC management in July 2023.

BACKGROUND

The NMP RCPs continue to demonstrate exceptional performance. As seen in the table below, over the last five calendar years (CYs) the RCPs have maintained an overall adequacy rate above 90 percent, and an overall compatibility rate of nearly 90 percent. To date in CY 2022, the overall NMP RCP performance, as measured by performance indicator results also continues to demonstrate exceptional performance. Over the last 5 CYs Agreement State and NRC staff have conducted more than 40 Integrated Materials Performance Evaluation Program (IMPEP) reviews and evaluated more than 281 individual performance indicators.

NMP Adequacy and Compatibility

CY 2022* CY 2021 CY 2020 CY2019 CY 2018 CY 2017 A 36 37 37 39 38 36 ABNI 4 3 3 4 4 5 C 34 37 37 36 36 34 NC 5 2 2 3 2 3 CY A % 90.0% 92.5% 92.5% 90.7% 90.5% 87.8%

CY C % 87.2% 94.9% 94.9% 92.3% 94.7% 91.9%

• - As of October 31, 2022.

A - Adequate, ABNI - Adequate, but needs improvement, C - Compatible, NC - Not Compatible

Of the 281 performance indicators reviewed during that time frame, 257 were found satisfactory, 14 satisfactory but needs improvement, and 10 were found unsatisfactory. This equates to Working Group Charter - Assessing the Ability to Monitor NMP Performance

approximately 91 percent satisfactory, 5 percent satisfactory but needs improvement, and 4 percent unsatisfactory, respectively.

Overall Program Performance - CY 2022 As Measured by Performance Indicator Results During the last IMPEP Review for All Programs

SAT 257 % SAT 91.5%

SBNI 14 % SBNI 5.0%

UNSAT 10 % UNSAT 3.6%

Total PIs 281

However, recent IMPEP reviews have resulted in an increase in the number of unsatisfactory performance indicators ratings. The table below summarizes the unsatisfactory ratings by RCP, review year, and performance indicator. The 10 unsatisfactory performance indicator ratings were found across 6 RCPs and 5 different performance indicators. Of note, 8 of the 10 unsatisfactory performance indicator ratings were found in the last 2 years. Consequently, it appears that there may be an emergent trend in the number of unsatisfactory ratings recently.

PERFORMANCE INDICATORS

LAST PROGRAM IMPEP SMIP TQI TQLA TQIAA LROPE REVIEW

New York 2018 SAT SAT SAT SAT UNSAT

Florida 2019 SAT SBNI SAT SAT UNSAT

Rhode Island 2021 SAT SAT UNSAT SAT SAT

Mississippi 2022 UNSAT UNSAT UNSAT UNSAT SBNI

North Carolina 2022 SAT SAT SAT SAT UNSAT

Washington 2022 SAT SBNI UNSAT UNSAT SBNI

Additionally, there have been seven unsatisfactory performance indicator ratings so far in 2022.

As a result, the NRC will not meet the 2022 CBJ-NM-23 performance metric of zero (0) percent of the RCPs having more than one unsatisfactory performance indicator rating. The recent increase in unsatisfactory performance indicator ratings also has the potential to appear as though there has been a decline in the overall NMP performance. The NRC wants to evaluate the effectiveness of the IMPEP program to predict, identify, and reverse declines in performance indicators before a RCP's performance would result in an UNSAT finding by the NRC. Because of the recent declines in performance, the NRC wants to identify potential root causes common to declines identified in recent reviews and identify leading indicators to help identify RCPs experiencing challenges to achieving SAT findings for each indicator.

Working Group Charter - Assessing the Ability to Monitor NMP Performance

WORKING GROUP MEMBERSHIP

The working group will operate as a joint NRC/Agreement State working group as described in MD 5.3, Agreement State Participation in Working Groups. The working group will be co-chaired by an NRC staff member and an Agreement State representative from the OAS.

Potential working group candidates are listed below; however, working group membership will be finalized prior to issuing the working group charter.

Organization Working Group (WG)

Office of Nuclear Material Safety Sherrie Flaherty, NMSS, WG Co-Chair and Safeguards (NMSS) Robert Johnson, NMSS Huda Akhavannik, NMSS

Advisor Duncan White, NMSS

Agreement States Santiago Rodriguez, New Mexico, WG Co-Chair Keisha Cornelius, Oklahoma David Crowley, North Carolina Beth Shelton, Tennessee

Office of the General Counsel Brian Harris, OGC

Regional Offices Farrah Gaskins, Region I Lizette Roldan-Otero, Region IV Tammy Bloomer, Region I

Other NRC and Agreement State staff may serve as resources to the Working Group at the request of the Co-Chairs and with the support of their management. Administrative support for the working group will be provided by the Division of Materials Safety, Security, State, and Tribal Programs (MSST) in NMSS.

ACTIVITIES AND SCHEDULE

The Working Group will:

Build on the 2010 and 2017 IMPEP self-assessments recommendations, as appropriate.

Evaluate the results of recent IMPEP reviews and assess whether there are connections between unsatisfactory performance indicators.

Recommend changes to enhance the effectiveness of IMPEP processes (periodic meetings, legislation and regulation compatibility reviews, and IMPEP reviews, including the consistency of review team findings and recommendations).

Evaluate the need for changes, if any, in how Regional State Agreement Officers (RSAOs) participate in the IMPEP process.

Assess if the 2019 revision of MD 5.6 led to lower performance ratings.

Determine if the pandemic impacted the RCPs performance.

3 Working Group Charter - Assessing the Ability to Monitor NMP Performance

The Working Groups report should address the following questions:

Considering NMP performance over approximately the last 5 years, have recent IMPEP reviews identified a trend toward more unsatisfactory performance?

Are RCPs reviewed consistently through IMPEP?

Are procedures and processes, including roles and responsibilities, sufficient to ensure effective and consistent IMPEP reviews?

Does the Annual Report to the Commission provide sufficient insights on performance trends?

Considering the IMPEP process, can it support the identification of leading indicators for performance trends?

What process changes can improve awareness of performance challenges within the NMP?

What tools are available (or needed) to detect a downward performance trend?

Can we develop leading indicators to identify declining program performance early?

Can we develop performance metrics to measure NMP performance, as allowed in SA-100?

The level of effort will be consistent with the 2010 IMPEP self-assessment and is expected to take approximately 9 months.

The recommendations should consider programmatic changes depending on the scenario and appropriate end points that lead to greater effectiveness and provide a consistent application of IMPEP across the NMP. While developing these options, the WG should solicit feedback from a broad range of Headquarters, Regions, and RCP managers and staff. The table below describes the activities to be conducted:

Activity Lead Completion Date

Activity 1: Charter Working Group NMSS Nov. 4, 2022

Charter a joint NRC/Agreement State working group of approximately 10 people, with membership composed of MSST/State Agreement and Liaison Programs Branch, RSAOs, IMPEP team leaders, Agreement State representatives, and at least one manager with IMPEP/Management Review Board experience.

Activity 2: Assess and Develop Recommendations WG Mar. 17, 2022

Perform an assessment to:

Review the Rhode Island, Mississippi, Washington, and North Carolina final IMPEP reports (and any other program with one or more unsatisfactory performance indicator) to identify common themes among the unsatisfactory performance indicators. This may include interviews with IMPEP team leaders and members.

Examine IMPEP review findings from programs on monitoring or heightened oversight. This may include

4 Working Group Charter - Assessing the Ability to Monitor NMP Performance

Activity Lead Completion Date

reviewing the Program Improvement Plans and interviewing the respective RSAOs.

Understand how individual IMPEP teams consistently develop recommendations and arrive at unsatisfactory findings.

Determine whether IMPEP training requirements are appropriate.

Identify potential IMPEP performance trends, using data analytics (e.g., frequency of less-than-satisfactory performance indicator findings, numbers of recommendations).

Host a G2G meeting to gather input from Agreement State and NRC materials staff.

Consider if insights from periodic meeting process provides a proactive assessment of NMP performance.

Review MD 5.6 and IMPEP-related State Agreement procedures to ensure that guidance is appropriate.

Identify potential IMPEP process enhancements, as appropriate, to address any common themes of unsatisfactory performance.

Recommend, as appropriate, any changes that could benefit the NMP community, procedural revisions for existing processes and the implementation of new activities/processes.

ACTIVITY 3: Communicate Initial Results

Conduct an informational briefing for NMSS senior WG Apr. 14, 2023 leadership to present the WGs initial results and recommendations and gather feedback.

Host a G2G meeting to present the WGs initial recommendations and gather feedback.

Activity 4: Communicate Final Recommendations WG Jun. 30, 2023 Consolidate comments from working group members and comments received during outreach activities into the proposed recommendations.

Conduct an informational briefing for NMSS senior leadership to share the results of its review and present its recommendations.

Conduct an informational briefing for the Deputy Executive Director for Materials, Waste, Research, State, Tribal, Compliance, Administration and Human Capital Programs (DEDM) to share the results of its review and present its recommendations.

5 Working Group Charter - Assessing the Ability to Monitor NMP Performance

Activity Lead Completion Date

Conduct an informational briefing for the OAS Board to share the results of its review and present its recommendations.

Host a G2G meeting to share the results of its review and present its recommendations.

Draft memorandum documenting the WGs findings and incorporating feedback from NRC and OAS leadership, as appropriate.

ACTIVITY 5: Issue Memorandum WG July 21, 2023

The WGs recommendations will be documented in a memo to the NMSS/MSST Director.

The products from each of the activities by the working group will be used as input for briefings and a memorandum documenting the WGs findings and recommendations to NRC and OAS Board leadership. Upon completion of each activity, representatives of the working group will meet with appropriate NRC leadership to obtain their endorsement of the recommendations.

Activities associated with the development and review of certain work products, such as performing assessments of the NRC and Agreement State radiation control programs, may be limited to those with a need-to-know, consistent with NRC information security requirements.

The Division of Materials Safety, Security, State, and Tribal Programs (MSST), with assistance from WG members, will be responsible for preparation of briefings and a memorandum documenting the WGs findings and recommendations to NRC and OAS Board leadership that will be reviewed and concurred on by NRC management. The following are the major milestones and tentative dates for the development and submittal of the report or briefings:

Milestone Date (Tentative)

Develop Memorandum Apr. 14, 2023

Memorandum Concurrence - NRC and OAS Board June 30, 2023

Issue Memorandum to the NMSS/MSST Director July 21, 2023

LEVEL OF EFFORT EXPECTED OF PARTICIPANTS

To support the schedule and activities listed above, the following level of effort is expected from the working group participants:

1. Attendance at weekly meetings (1 to 2 hours2.314815e-5 days <br />5.555556e-4 hours <br />3.306878e-6 weeks <br />7.61e-7 months <br /> per week);

2. Attendance at stakeholder outreach events (approximately 15 hours1.736111e-4 days <br />0.00417 hours <br />2.480159e-5 weeks <br />5.7075e-6 months <br /> total);

6 Working Group Charter - Assessing the Ability to Monitor NMP Performance

73. Development and/or review of working group products, such as analyses (10 to 15 hours1.736111e-4 days <br />0.00417 hours <br />2.480159e-5 weeks <br />5.7075e-6 months <br /> per week);

4. Periodic briefings with interested managers on the working group activities to solicit feedback and comments (1 hour1.157407e-5 days <br />2.777778e-4 hours <br />1.653439e-6 weeks <br />3.805e-7 months <br /> per briefing); and

5. Final briefings to NMSS and DEDM (1 hour1.157407e-5 days <br />2.777778e-4 hours <br />1.653439e-6 weeks <br />3.805e-7 months <br /> per briefing).

NRC members should charge time associated with working group activities identified in this charter to Charge Accounting Code: A34018 EPID M-2022-OTH-0000; Agreement State members should charge to their respective codes.

MEETINGS Meetings are pre-decisional and will be closed to the public.

Working group members may delegate an alternative representative for a specific meeting. The working group may also invite individual(s) to a meeting to participate as a resource to assist the working group with a particular issue. However, at least one of the named Co-Chairs must be present during any working group meetings.

Available technology will be used to facilitate interaction with the working group members.

Face-to-face meetings, if necessary, will generally be held in the Washington, D.C., area unless alternate locations are agreed upon by working group members. If travel is necessary, travel and per diem expenses for Agreement State members of the working group will be covered by MSST. Regions are responsible for the travel expenses of their staff.

APPROVED Kevin Williams U.S. Nuclear Regulatory Commission Steve Seeger Chair, Organization of Agreement States Signed by Williams, Kevin on 11/03/22

ML22305A688 OFFICE NMSS/MSST NMSS/MSST NMSS/MSST NAME RJohnson BAnderson KWilliams DATE 10/01/2022 11/02/2022 11/03/2022