ML21048A320
| ML21048A320 | |
| Person / Time | |
|---|---|
| Issue date: | 03/09/2021 |
| From: | Nicole Fields NRC/NRR/DRO/IRAB |
| To: | Chris Miller NRC/NRR/DRO |
| Nicole Fields, NRR/DRO/IRAB, 630-8299570 | |
| Shared Package | |
| ML21048A262 | List: |
| References | |
| Download: ML21048A320 (13) | |
Text
Calendar Year 2020 Reactor Oversight Process Self-Assessment Metric Overview and Report
CY 2020 ROP Self-Assessment Metrics Overview Independence Metrics I-1 I-2 I-3 I-4 I-5 Completion of the Resident Inspector Fully Qualified Inspector Objectivity and Continuity of RI/SRI Site Baseline Inspection Objectivity Through Inspectors, Examiners, Performance Reviews Staffing Program Diverse Experience and Senior Risk Analysts Red Red Red Red Green Openness Metrics O-1 O-2 O-3 O-4 O-5 Conduct of Annual Issuance of ROP Public Issuance of Inspection Issuance of Assessment Assessment Meetings or Reporting and Meeting Notices and Reports Letters Other Engagement Dissemination of PI Data Summaries Activities Green Green Green Green Green Efficiency Metrics E-1 E-2 E-3 SDP Completion Completion of Completion of Temporary Timeliness for Potentially Supplemental Instructions Greater-than-Green Inspections Findings Green Green Yellow Clarity Metrics C-1 C-2 Maintenance of ROP Maintenance of ROP Web Pages Governance Documents Green Green Reliability Metrics R-1 R-2 R-3 Predictability and Repeatability of Predictability of Agency Supportability of Significance Actions and Response Inspection Findings Determination Results Green Green Green
CALENDAR YEAR 2020 ROP SELF-ASSESSMENT METRICS REPORT REFERENCED TO INSPECTION MANUAL CHAPTER 0307, APPENDIX A This metrics report follows Inspection Manual Chapter (IMC) 0307, Appendix A (Agencywide Documents Access and Management System (ADAMS) Accession No. ML19274C401) to report on the overall implementation of the Reactor Oversight Process (ROP) as measured by the following 18 ROP performance metrics for calendar year (CY) 2020. For more details on each individual metric, including the specific criteria for green, yellow, or red status, the basis, the ROP program area, the data sources, the related ROP goals, and the related ROP intended outcomes, refer to IMC 0307, Appendix A.
0307A-01 INDEPENDENCE PERFORMANCE METRICS (I)
I-1 Completion of Baseline Inspection Program Metric Status: Red Definition: The baseline inspection program is completed annually in accordance with program requirements.
Data and Analysis: In CY 2020, due to the ongoing COVID-19 public health emergency (PHE) not all regions were able to complete the baseline inspection program. IMC 2515, Section 04.07 states that completion, is defined to be not more than four (4) inspection procedures not completed, per Region (but not more than one procedure not done per plant). Each region and the Office of Nuclear Security and Incident Response (NSIR) documented in detail their implementation of the baseline inspection program for CY 2020 via memorandum (ADAMS Accession Nos. ML21040A367 for Region I, ML21056A249 for Region II, ML21050A236 for Region III, ML21054A269 for Region IV, and ML21068A145 for NSIR (nonpublic)). Region IV did complete the baseline inspection program; Region I, Region II and Region III did not complete the baseline inspection program, as defined in IMC 2515, Section 04.07.
The vast majority of the scheduled baseline inspection procedures (IPs) were completed as scheduled in CY 2020 or were rescheduled for CY 2021 or CY 2022, in particular those with biennial or triennial inspection frequencies. However, some IPs, specifically IP 71111.11 Licensed Operator Requalification Program and Licensed Operator Performance, IP 71111.20, Refueling and Other Outage Activities, IP 71124.01, Radiological Hazard Assessment and Exposure Controls, IP 71130.02, Access Control, and IP 71130.03, Contingency ResponseForce-on-Force Testing were not completed at all reactor sites during CY 2020. These IPs were not completed due to the required onsite components of the IP, the timing of the activities to be inspected, and the decisions made to protect the health and safety of both NRC and licensee staff, which restricted both onsite time and inspector travel, due to the ongoing COVID-19 PHE. While most elements of these IPs were completed, some inspectors were not able to complete the required on-site walkdowns, verifications, or observations; thus, the IPs were incomplete.
Since the baseline inspection program was not completed for all regions, this metric is red, and Regions II, III, and NSIR have met the individual threshold for evaluation for this metric. Since the cause of this red metric is very clearly the PHE, the staff does not plan to take immediate action while the COVID-19 PHE is ongoing, or to perform individual evaluations of any region or office.
For each metric that was evaluated as red due to the ongoing PHE, the staff reviewed any mitigating factors associated with achieving the intent of the metric. The baseline inspection program, although not fully completed, was still implemented by the regions and NSIR in CY 2020. The NRC completed nearly 150,000 baseline inspection hours in CY 2020 for all operating nuclear plants in the United States, with a two-unit site averaging about 2,700 hours0.0081 days <br />0.194 hours <br />0.00116 weeks <br />2.6635e-4 months <br />. The ROP is a robust and mature program, and although the baseline inspection program was not fully completed in CY 2020, this is a relatively short-term impact to the inspection program, which is expected to resolve once the ongoing PHE impacts are no longer as significant.
For each incomplete inspection procedure, inspectors focused on performing as much of the inspection as possible while maintaining personal safety, as well as maintaining awareness and communication of any issues that might arise with on-site licensee and NRC personnel. On-site walkdowns, verifications, and observations that were not performed in CY 2020, will be performed at the earliest appropriate opportunity for that on-site activity.
I-2 Resident Inspector Objectivity through Diverse Experience Metric Status: Red Definition: Permanently-staffed Senior Resident Inspectors (SRIs) and Resident Inspectors (RIs) spend a minimum of one week each year inspecting at another site.
Data and Analysis: In CY 2020, 35 of the 118 SRI and RI positions in the agency were not able to complete their required objectivity visits by spending a minimum of one week inspecting at another plant. This was due to the travel restrictions imposed due to the ongoing COVID-19 PHE which began in early 2020. A few residents were still able to perform these objectivity visits by remotely participating in team inspections, but that did not significantly impact the results of this metric. All four regions had at least six resident staff members that were unable to complete the objectivity visits.
For the agency overall this metric is red, and each region has met the individual regional threshold for evaluation for this metric. Since the cause of this red metric is very clearly the PHE, the staff does not plan to take immediate action while the COVID-19 PHE is ongoing, or to perform individual evaluations of any region.
For each metric that was evaluated as red due to the ongoing PHE, the staff reviewed any mitigating factors associated with achieving the intent of the metric. Approximately 70 percent of resident inspection staff were still able to complete objectivity visits in CY 2020.
Additionally, all resident inspectors are still subject to the objectivity requirements and restrictions in IMC 0102, Oversight and Objectivity of Inspectors and Examiners at Reactor Facilities, (ADAMS Accession No. ML12012A053) and IMC 1201, Conduct of Employees, (ADAMS Accession No. ML16211A030) which includes a 7-year maximum length of tour at a given reactor site. The resident inspector program is a robust and mature program, and although not all resident inspector objectivity visits were completed as required in CY 2020, this is a relatively short-term impact to the resident inspector program, which is expected to resolve once the ongoing PHE impacts are no longer as significant.
I-3 Inspector Objectivity and Performance Reviews Metric Status: Red Definition: Line managers perform annual on-site objectivity and performance reviews of each fully qualified inspector assigned to an inspection branch.
Data and Analysis: In CY 2020, 91 of 329 qualified inspectors did not have an annual objectivity review, primarily because of travel restrictions due to the COVID-19 PHE. Three of the regions and NSIR reported whether completed objectivity reviews were conducted virtually, with a total of 82 qualified inspectors in those organizations receiving a virtual objectivity review while performing virtual interactions with a licensee, to meet the intent of IMC 0102 requirements.
For the agency overall this metric is red, and each region has met the individual regional threshold for evaluation for this metric. Since the cause of this red metric is very clearly the PHE, the staff does not plan to take immediate action while the COVID-19 PHE is ongoing, or to perform individual evaluations of any region.
For each metric that was evaluated as red due to the ongoing PHE, the staff reviewed any mitigating factors associated with achieving the intent of the metric. Approximately 70 percent of qualified inspectors had an annual objectivity review by a manager in CY 2020.
Additionally, all inspectors are still subject to the objectivity requirements and restrictions in IMC 0102 and IMC 1201. All inspectors continued to have routine inspection related interactions with NRC management in CY 2020. All NRC staff also participate in the NRC performance appraisal process described in Management Directive 10.67, General Grade Performance Management System, (ADAMS Accession No. ML19119A071), which includes an annual performance rating and discussion. The inspection program is a robust and mature program, and although not all inspectors received an annual objectivity review as required in CY 2020, this is a relatively short-term impact to the inspection program, which is expected to resolve once the ongoing PHE impacts are no longer as significant.
I-4 Fully Qualified Inspectors, Examiners, and Senior Reactor Analysts Metric Status: Red Definition: Inspectors, operator licensing examiners, and senior reactor analysts (SRAs) remain fully qualified in accordance with qualification requirements. Training beyond the 3-year cycle is considered noncompliant regardless of the status of a deviation memo.
Data and Analysis: In CY 2020, 44 of 329 inspectors, operator licensing examiners, and senior reactor analysts did not remain fully qualified in accordance with IMC 1245 and its appendices. All four regions had staff members who did not maintain their qualifications in accordance with IMC 1245, the vast majority due to COVID-19-related course cancellations and travel restrictions. In recognition of the ongoing PHE, the Division of Reactor Oversight (DRO) has revised IMC 1245, Appendix D1, Maintaining Qualifications, dated December 16, 2020 (ADAMS Accession No. ML20246G611), to authorize a one-time blanket deviation for certain IMC 1245 refresher training requirements during the COVID-19 PHE. However, regardless of this deviation, those staff members still are considered noncompliant with regards to this metric.
For the agency overall this metric is red, and three of the regions have met the individual regional threshold for evaluation for this metric.
Since the cause of this red metric is very clearly the PHE, the staff does not plan to take immediate action while the COVID-19 PHE is ongoing, or to perform individual evaluations of any region.
For each metric that was evaluated as red due to the ongoing PHE, the staff reviewed any mitigating factors associated with achieving the intent of the metric. Approximately 85 percent of inspectors, operating license examiners and SRAs maintained their qualifications in accordance with IMC 1245 in CY 2020. Staff that were not able to maintain their qualifications by taking the appropriate refresher training in CY 2020 are aware of these training requirements and are awaiting the next available training opportunity. Additionally, there were many general training opportunities held virtually during CY 2020, which were well attended by NRC staff and inspectors. These included weekly agency-wide knowledge management presentations on a variety of technical topics and biannual inspector seminars held by all four regions. In addition, inspectors were actively performing technical inspection activities in CY 2020, which also provides ongoing proficiency training. The inspection training program is a robust and mature program, and although not all inspectors attended required refresher training in CY 2020, this is a relatively short-term impact to the inspection program, which is expected to resolve once the ongoing PHE impacts are no longer as significant.
I-5 Continuity of RI/SRI Site Staffing Metric Status: Green Definition: Permanent inspector staffing levels for both SRIs and RIs are maintained to provide continuity of regulatory oversight at each reactor site.
Data and Analysis: In CY 2020, the overall permanent resident inspector staffing percentage for the agency was 95.5%. The permanent resident inspector staffing percentages for each region ranged from 91.7% to 98.3%. In CY 2020, there were nine reactor sites which fell below the 90% metric threshold, and the responsible regions provided a detailed explanation of the specific staffing circumstances at each site.
Generally, these were due to staff promotions (temporary or permanent), staff lateral moves, staff rotations, staff leaving the agency, or in one specific case due to the anticipated permanent reactor shutdown. Site coverage was maintained by either assigning temporary resident staff to these sites in less than 6-week increments, by providing inspection support from the region, or by utilizing the resident staff still onsite to provide coverage and complete inspection activities. Since the agency exceeded 95% permanent resident inspector staffing and each region exceeded 90%, this metric is green, and no region will be individually evaluated.
0307A-02 OPENNESS PERFORMANCE METRICS (O)
O-1 Issuance of Inspection Reports Metric Status: Green Definition: ROP inspection reports are issued within applicable timeliness goals.
Data and Analysis: In CY 2020, the agency issued 485 inspection reports. Eight of those inspection reports were not issued in accordance the timeliness requirements of IMC 0611. No single region or NSIR had more than four late inspection reports. Since the agency had fewer than 11 late inspection reports, and no region or office had more than five late inspection reports, this metric is green, and no region or office will be individually evaluated.
O-2 Issuance of Assessment Letters Metric Status: Green Definition: Annual and follow-up assessment letters are issued within the applicable timeliness goals.
Data and Analysis: In CY 2020, the agency issued 57 annual assessment letters, five follow-up assessment letters, and 13 mid-cycle assessment letters to reactor licensees. All of these assessment letters were issued on time, in accordance with IMC 0305 requirements. Since the agency had fewer than three late assessment letters, and no region or office had more than one late assessment letter, this metric is green, and no region or office will be individually evaluated.
O-3 Conduct of Annual Assessment Meetings or Other Engagement Activities Metric Status: Green Definition: Public assessment meetings or other engagement activities that discuss the results of the NRCs annual assessment of the licensees performance, are conducted annually for all sites.
Data and Analysis: In CY 2020, the agency held public assessment meetings or other engagement activities for 57 reactor sites, in accordance with IMC 0305. Due to the ongoing COVID-19 PHE many of these meetings or activities were held virtually by NRC staff to interact with public stakeholders, and the public engagement activities for several sites was often held as a single combined meeting. Since all reactor sites had appropriate public engagement in CY 2020, this metric is green.
O-4 Reporting and Dissemination of Performance Indicator (PI) Data Metric Status: Green Definition: PI data submittals by the licensees are posted to the NRCs external web site within the applicable timeliness goals.
Data and Analysis: In CY 2020, all PI data submittals by the licensees were posted on-time to the NRCs external website for PI data. Since no licensee submitted PI data late to the NRC, and there were zero late web postings for PIs, this metric is green.
O-5 Issuance of ROP Public Meeting Notices and Summaries Metric Status: Green Definition: ROP-related public meetings are noticed prior to the meeting and meeting summaries are posted after the meeting within the applicable timeliness requirements.
Data and Analysis: In CY 2020, the Office of Nuclear Reactor Regulation (NRR), NSIR, and the regional staff held 38 ROP-related public meetings, many of them held virtually due to the COVID-19 PHE. The staff noticed and summarized these public meetings within the established timeliness goals for all of the public meetings held, with the exception of one meeting which had a late meeting summary. This corresponds to 97%
of meetings having been noticed and summarized on time. Since the percentage of meetings with timely notices and summaries is greater than 95%, and no region or office had more than two untimely notices or summaries, this metric is green, and no region or office will be individually evaluated.
0307A-03 EFFICIENCY PERFORMANCE METRICS (E)
E-1 Completion of Supplemental Inspections Metric Status: Green Definition: Exit meetings for supplemental inspections are completed within 180 days from licensee notification of readiness.
Data and Analysis: In CY 2020, the agency completed three 95001 supplemental inspections at Vogtle Electric Generating Plant, Browns Ferry Nuclear Plant, and Surry Power Station. The exit meetings for these supplemental inspections were all completed within the timeliness goal.
Since no more than one exit meeting did not meet the timeliness goal for the agency and no more than one exit meeting did not meet the timeliness goal for any region or office, this metric is green, and no region or office will be individually evaluated.
E-2 Completion of Temporary Instructions Metric Status: Green Definition: Temporary Instruction (TI) inspections associated with IMC 2201 and IMC 2515 are completed within the required TI completion time.
Data and Analysis: In CY 2020, the TIs in effect for IMC 2515 and IMC 2201 were TI 2515/193, Revision 1, Inspection of the Implementation of EA-13-109:
Order Modifying Licenses with Regard to Reliable Hardened Containment Vents Capable of Operation Under Severe Accident Conditions, and TI 2515/194, Revision 2, Inspection of the Licensees' Implementation of Industry Initiative Associated with the Open Phase Condition Design Vulnerabilities in Electric Power Systems (NRC Bulletin 2012 01).
The agency completed 16 TI inspections in CY 2020, and all TIs completed in CY 2020 were documented within the required TI completion time specified by the TI. Since the percentage of documented timely completions of TIs was greater than 95% for the agency and greater than 90% for each region, this metric is green, and no region will be individually evaluated.
E-3 SDP Completion Timeliness for Potentially Greater-than-Green Findings Metric Status: Yellow Definition: The time from the identification date (i.e., the date the issue of concern was brought to the licensees attention by the NRC, the date the performance deficiency was self-revealed, or the date the licensee documented the condition resulting from the performance deficiency in the corrective action program) to the date a final significance determination is issued for all potentially greater-than-green findings is within 255 days.
Data and Analysis: In CY 2020, the agency issued four final significance determinations for issues that were initially transmitted to the licensees as potentially greater-than-green (GTG). Of these four findings, three of them exceeded the 255-day timeliness goal between identification date and final issuance: Browns Ferry Nuclear Plant EA-19-130 (ADAMS Accession No. ML20076A950), Vogtle Electric Generating Plant EA-19-112 (ADAMS Accession No. ML20091L428), and Clinton Power Station EA-20-004 (ADAMS Accession No. ML20307A569). Since three issues exceeded the timeliness goal this metric is yellow. Region II issued two of these potentially GTG issues and meets the threshold for an individual evaluation. That evaluation is below.
DRO reviewed each finding that exceeded the timeliness goal to determine any key contributing causes, and whether program or implementation changes were warranted. For two of these findings, the primary contribution to missing the timeliness goal was other office review processes outside of the bounds of the ROP. The staff does not plan to make any program changes due to these findings but may consider whether to revise this metric to account for these types of circumstances. For the third finding, extensive internal discussions were had by staff to ensure adequate resolution of the finding. The staff does not see a trend related to this finding and is not planning any specific program changes as a result. DRO staff will continue to engage with potentially GTG issues in process to determine whether process improvements can be made to ensure timely decision-making and communication.
0307A-04 CLARITY PERFORMANCE METRICS (C)
C-1 Maintenance of ROP Web Pages Metric Status: Green Definition: ROP-related internal and external NRC Web pages are reviewed at least quarterly and discrepancies are corrected as necessary to ensure that ROP information is communicated accurately and effectively.
Data and Analysis: In CY 2020, beginning in the third quarter, each region verified that the data available on the NRC public website for their reactor sites was accurate, up-to-date, and had working links. Any discrepancies or errors discovered by the regions were submitted for correction as appropriate. Currently, this webpage lists the issued inspection reports by reactor site with links to the individual reports, and this webpage shows an overview of ROP findings by cornerstone on a per unit basis with links to the overall performance summary per unit, as well as links to the list of findings summaries by cornerstone per unit. All of the applicable ROP-related webpages were reviewed by the regions on a quarterly basis.
NSIR staff are responsible for reviewing five ROP-related webpages, and NRR staff are responsible for reviewing 99 ROP-related webpages to verify that the data available is accurate, up-to-date, and had working links. All of these webpages were also reviewed at least quarterly and any discrepancies or errors discovered by the staff were submitted for correction as appropriate. Since the percentage of ROP-related webpages reviewed at least quarterly by the staff is greater than 90%,
this metric is green.
C-2 Maintenance of ROP Governance Documents Metric Status: Green Definition: Baseline inspection procedures (BIPs) and other ROP-related Inspection Procedures and Manual Chapters are reviewed at least once every 5 years.
Data and Analysis: In the beginning of CY 2020, DRO staff identified a total of 82 ROP-related IMCs and IPs that were due to be reviewed in CY 2020. This population of documents did not include IPs considered to be reference procedures only and not subject to the periodic 5-year review cycle.
Reference IPs are designated as such on the NRC public website. Of the 82 documents identified, 78 were reviewed by staff in CY 2020, although 38 of those were still in the process of being revised and issued as of February 10, 2021. The remaining 40 documents were either issued in CY 2020, issued early in CY 2021, reclassified as reference procedures, or deleted. Since 95.1% of the IMCs and IPs due for review in CY 2020 were reviewed, this metric is green.
In preparation for monitoring this metric for CY 2021, as of February 10, 2021, there were a total of 306 ROP-related IMCs and IPs, with 68 of those IPs currently designated as reference IPs. At the end of CY 2020, DRO staff identified 3 IMCs and 12 IPs due for review in CY 2021 (last issued in 2016), in addition to the IMCs and IPs designated for review as discussed above.
0307A-05 RELIABILITY PERFORMANCE METRICS (R)
R-1 Predictability and Repeatability of Significance Determination Results Metric Status: Green Definition: Greater-than-Green inspection findings and the associated degraded conditions contain adequate detail to enable an independent auditor to trace through the available documentation and conclude that the significance characterization is reasonably justifiable from both programmatic and technical positions. This audit should be documented in a memo that is internally available to the NRC and referenced in the annual metric report.
Data and Analysis: The staff determined that all four of the GTG findings issued by the NRC in CY 2020 contained adequate detail to enable an independent auditor to trace through the available documentation and conclude that the significance characterization was reasonably justifiable from both programmatic and technical positions. This internal audit was documented in a memorandum dated February 2, 2021 (ADAMS Accession No. ML21019A353, nonpublic). Since zero discrepancies in the significance determination were identified, this metric is green.
R-2 Predictability of Agency Actions and Response Metric Status: Green Definition: Deviations from the Action Matrix are expected to be infrequent to ensure reliable and predictable oversight.
Data and Analysis: In CY 2020, there were zero ROP Action Matrix deviations, so this metric is green.
R-3 Supportability of Inspection Findings Metric Status: Green Definition: Inspection findings are adequately supported and documented such that contested violations by licensees that are overturned should be infrequent.
Data and Analysis: In CY 2020, determinations were made on two violations that were contested by licensees. The contested non-cited violation at Grand Gulf Nuclear Station was upheld (ADAMS Accession No. ML20345A154),
and the contested notice of violation at R. E. Ginna Nuclear Power Plant was overturned and withdrawn (ADAMS Accession No. ML20107F834).
Since no more than three contested violations were overturned and no more than two contested violations per region/office were overturned, this metric is green, and no region or office will be individually evaluated.