ML19044A573
| ML19044A573 | |
| Person / Time | |
|---|---|
| Issue date: | 03/19/2019 |
| From: | Tara Inverso NRC/NRR/DIRS/IRGB |
| To: | Chris Miller Division of Inspection and Regional Support |
| Mayer A, 415-1081, NRR/DIRS | |
| References | |
| Download: ML19044A573 (18) | |
Text
Calendar Year 2018 Reactor Oversight Process Self-Assessment Metric Report (Enclosures 1 and 2)
Metric I-1 Completion of the Baseline Inspection Program I-2 Resident Inspector Objectivity Through Rotation Policy I-3 Resident Inspector Objectivity Through Diverse Experience I-4 Inspector Objectivity Reviews I-5 Fully Qualified Inspectors and Operator Licensing Examiners I-6 Analysis of Permanent Site Staffing Result Green Green Green Green Green Green Metric O-1 Issuance of Inspection Reports O-2 Issuance of Assessment Letters O-3 Conduct of Annual Assessment Meetings or Other Engagement Activities O-4 Reporting and Dissemination of PI Data O-5 Issuance of ROP Public Meeting Notices and Summaries O-6 Responsiveness to ROP Contact Us Forms Result Green Green Green Green Green Green Metric E-1 Completion of Supplemental Inspections E-2 Initiation of Reactive Inspections E-3 Completion of Temporary Instructions E-4 Completion of Performance Deficiency Determinations E-5 Completion of Final Significance Determinations E-6 Responsiveness to ROP Feedback Forms Result Green Green Green Green (CY17: Red)
Green (CY17: Yellow)
Red Metric C-1 Maintenance of ROP Web Pages C-2 Corrections to ROP Web Pages C-3 Traceability of Greater-than-Green Inspection Findings C-4 Maintenance of ROP Governance Documents Result Green Green Green Green Metric R-1 Performance of Lessons Learned Evaluations R-2 Predictability and Repeatability of Significance Determination Results R-3 Predictability of Agency Actions and
Response
R-4 Consideration of Operating Experience Insights Result Green (CY17: Red)
Green Green Green Self-Assessment Metrics Overview/Dashboard Openness Metrics Efficiency Metrics Clarity Metrics Reliability Metrics Independence Metrics
CALENDAR YEAR 2018 ROP SELF-ASSESSMENT METRICS REPORT REFERENCED TO INSPECTION MANUAL CHAPTER 0307, APPENDIX A 0307A-01 INDEPENDENCE PERFORMANCE METRICS (I)
I-1 Completion of Baseline Inspection Program Metric Criterion Met: Yes (Green)
Definition:
The baseline inspection program is completed annually in accordance with program requirements.
Criteria:
Green Yellow Red All regions and the Office of Nuclear Security and Incident Response (NSIR) met completion requirements N/A 1 or more regions or NSIR did not meet completion requirements
- Note: No less than 100% compliance for any region or NSIR.
Baseline inspection program completion is defined in Section 04.07 of Inspection Manual Chapter (IMC) 2515, Light-Water Reactor Inspection Program - Operations Phase (ADAMS Accession No. ML17079A202.
Any region or office that does not complete the baseline inspection program per IMC 2515 is individually evaluated.
Basis:
IMC 2515 Program Area:
Inspection Lead/Data Source:
Regions, NSIR Related Principles:
Efficiency Analysis:
For CY 2018, all regions and the Office of Nuclear Security and Incident Response completed 100 percent of their baseline inspections within the allocated resources. Each region documented completion of the baseline inspection program in memoranda available at Agencywide Documents Access and Management System (ADAMS)
Accession Nos. ML19046A075 for Region I, ML19052A388 for Region II, ML19051A048 for Region III, and ML19065A122 for Region IV. NSIR documented completion of all its security baseline inspections in a memorandum at ADAMS Accession No. ML19007A122 (non-public).
I-2 Resident Inspector Objectivity Through Rotation Policy Metric Criterion Met: Yes (Green)
Definition:
Senior resident inspectors (SRIs) and resident inspectors (RIs) are stationed for a 7-year maximum tour length, unless specifically granted an extension per IMC 2515.
Criteria:
Green Yellow Red 0 inspectors exceeded 7 years without an extension N/A 1 or more inspectors exceeded 7 years without an extension
- Note: No less than 100% compliance for any region. Any region that has 1 or more inspectors exceed 7 years at a site without an extension is individually evaluated.
Basis:
IMC 0102, Oversight and Objectivity of Inspectors and Examiners at Reactor Facilities, and IMC 2515 Program Area:
Inspection Lead/Data Source:
Regions Related Principles:
Reliability Analysis:
For calendar year (CY) 2018, there was 100% compliance in all regions with no SRIs or RIs stationed at a plant beyond the maximum tour length without an extension.
I-3 Resident Inspector Objectivity Through Diverse Experience Metric Criterion Met: Yes (Green)
Definition:
Permanently-staffed SRIs and RIs spend a minimum of one week each year inspecting at another site.
Criteria:
Green Yellow Red 3 noncompliant
(~97.5%)
4-5 noncompliant 6 noncompliant
(~95%)
- Note: No more than two non-compliances in any region. Any region that has more than two non-compliances is individually evaluated.
Basis:
IMC 0102 Program Area:
Inspection Lead/Data Source:
Regions Related Principles:
Reliability
Analysis:
For CY 2018, 100% of all permanently-staffed resident inspectors completed their objectivity visits by spending a minimum of one week inspecting at another plant.
I-4 Inspector Objectivity Reviews Metric Criterion Met: Yes (Green)
Definition:
Line managers perform annual on-site employee performance and objectivity reviews of fully qualified inspectors assigned to an inspection branch.
Criteria:
Green Yellow Red 3 noncompliant
(~97.5%)
4-5 noncompliant 6 noncompliant
(~95%)
- Note: No more than two non-compliances in any region or NSIR. Any region or office that has more than two non-compliances is individually evaluated.
Basis:
IMC 0102 Program Area:
Inspection Lead/Data Source:
Regions, NSIR Related Principles:
Reliability Analysis:
For CY 2018, all line managers in the regions and NSIR performed annual on-site employee performance and objectivity reviews of their assigned inspectors.
I-5 Fully Qualified Inspectors and Operator Licensing Examiners Metric Criterion Met: Yes (Green)
Definition:
Inspectors and operator licensing examiners remain fully qualified in accordance with qualification requirements.
Criteria:
Green Yellow Red 5 noncompliant 6-9 noncompliant 10 noncompliant
- Note: No more than three non-compliances in any region or NSIR.
Any region or office that has more than three non-compliances is individually evaluated.
Basis:
IMC 1245, Qualification Program for New and Operating Reactor Programs, and Davis-Besse Reactor Vessel Head Degradation Lessons-Learned Task Force Report (ML022760172)
Program Area:
Inspection Lead/Data Source:
Regions, NSIR Related Principles:
Reliability Analysis:
For CY 2018, all inspectors and operator licensing examiners remained fully qualified in accordance with IMC 1245 and its appendices.
I-6 Analysis of Permanent Site Staffing Metric Criterion Met: Yes (Green)
Definition:
Permanent inspector staffing levels at each of the reactor sites for both SRIs and RIs are maintained to provide continuity of regulatory oversight.
Criteria:
Green Yellow Red 95%
< 95% AND 90%
< 90%
- Note: No less than 90% compliance for any region. Any single site that falls below 90% is individually evaluated.
Inspectors assigned to the site permanently or through a rotation with a minimum duration of 6 weeks shall be counted. Inspectors on 6 week or longer rotational assignments will be identified as such. Inspectors assigned to the site for less than six weeks will not be counted, but should be indicated as such. Additionally, the regions shall indicate sites where permanently assigned resident or senior resident inspectors are away from the site for greater than 6 continuous weeks. Only inspectors who have attained at least a basic inspector certification status, as defined by Appendix A to Inspection Manual Chapter 1245, shall be counted.
Data will indicate number of days a qualified resident and senior resident inspector are permanently assigned to the site during the year divided by the number of days in the year. Number of days spent on training; meetings away from the site; participation in team inspections; leave; or other temporary duties (e.g. acting for branch chiefs in his/her absence) will not be counted against the metric unless the absence exceed 6 continuous weeks.
Basis:
Davis-Besse Reactor Vessel Head Degradation Lessons-Learned Task Force Report (ML022760172) and IMC 2515 Program Area:
Inspection Lead/Data Source:
Regions Related Principles:
Reliability Analysis:
For CY 2018, the average permanent inspector staffing was 98.6% for all regions with no single region falling below 97%. One site did fall
below the 90% threshold. That site had a resident inspector transfer to a U.S. Nuclear Regulatory Commission (NRC) Headquarters position, and the resident inspector was covered by qualified inspectors for several periods amounting to less than 6 weeks before the position could be permanently filled. No further evaluation or action is required for this site since the events were not reasonably within the regions ability to foresee and prevent.
0307A-02 OPENNESS PERFORMANCE METRICS (O)
O-1 Issuance of Inspection Reports Metric Criterion Met: Yes (Green)
Definition:
Reactor Oversight Process (ROP) inspection reports are issued within applicable timeliness goals.
Criteria:
Green Yellow Red 25 late
(~ 95% timely)
> 25 AND 50 late
> 50 late
(~ 90% timely)
- Note: No more than 15 late inspection reports in any region or 5 late inspection reports in NSIR. Any region that has more than 15 late inspection reports or office that has more than 5 is individually evaluated.
Basis:
IMC 0612 and IMC 2515 Program Area:
Inspection Lead/Data Source:
Regions, NSIR Related Principles:
Reliability, Efficiency Analysis:
For CY 2018, there were five inspection reports not issued on time. No single region or NSIR had more than fifteen late inspection reports.
O-2 Issuance of Assessment Letters Metric Criterion Met: Yes (Green)
Definition:
Annual and follow-up assessment letters are issued within the applicable timeliness goals.
Criteria:
Green Yellow Red 2 late 3 late 4 late
- Note: No more than one late letter in any region. Any region that has more than one late letter is individually evaluated.
Basis:
IMC 0305, Operating Reactor Assessment Program Program Area:
Assessment Lead/Data Source:
Regions Related Principles:
Reliability, Efficiency Analysis:
For CY 2018, 100% of annual and follow-up assessment letters were issued on time.
O-3 Conduct of Annual Assessment Meetings or Other Engagement Activities Metric Criterion Met: Yes (Green)
Definition:
Public assessment meetings or other engagement activities that discuss the results of the NRCs annual assessment of the licensees performance, are conducted annually for all sites within the applicable timeliness goals.
Criteria:
Green Yellow Red 2 late 3 late 4 late
- Note: The level of public engagement and timeliness goals are determined by plant performance, as described in IMC 0305. No more than one late meeting/activity in any region. Any region that has more than one late meeting/activity is individually evaluated.
Basis:
IMC 0305 Program Area:
Assessment Lead/Data Source:
Regions Related Principles:
Reliability, Efficiency Analysis:
For CY 2018, 100% of the public assessment meetings and other engagement activities met applicable timeliness goals.
O-4 Reporting and Dissemination of Performance Indicator (PI) Data Metric Criterion Met: Yes (Green)
Definition:
Performance indicator (PI) data submittals by the licensees are posted to the NRCs external web site within the applicable timeliness requirements.
Criteria:
Green Yellow Red 0 late web posting 1-3 late web postings > 3 late web postings
- Note: Any licensee submittals that did not meet the timely reporting requirements will also be evaluated by NRC staff and discussed with
industry to address corrective actions to prevent recurrence.
Basis:
IMC 0306, Information Technology Support for the Reactor Oversight Process, and Nuclear Energy Institute 99-02, Performance Indicator Data Collection Program Area:
Performance Indicators Lead/Data Source:
NRR/DIRS Related Principles:
Reliability, Efficiency Analysis:
For CY 2018, 100% of the licensee PI data submitted to the NRC was posted to the external ROP website on time. No licensee submitted data late to the NRC.
O-5 Issuance of ROP Public Meeting Notices and Summaries Metric Criterion Met: Yes (Green)
Definition:
ROP-related public meetings are noticed prior to the meeting and meeting summaries are posted after the meeting within the applicable timeliness requirements.
Criteria:
Green Yellow Red 95%
< 95% AND 90%
< 90%
- Note: ROP-related public meetings include ROP working group meetings, annual assessment meetings, and other ROP-related meetings conducted by NRC staff. No more than two late notices or summaries for any region or office. Any region or office that has more than two late notices or summaries is individually evaluated.
Basis:
Management Directive (MD) 3.5, Attendance at NRC Staff-Sponsored Meetings, and COM-202, Meetings With Applicants, Licensees, Vendors or Other Members of the Public Program Area:
All Lead/Data Source:
NRR/DIRS, Regions, NSIR Related Principles:
Efficiency, Clarity Analysis:
For CY 2018, 100% of the public meetings held by the Regions and NSIR, as well as the ROP monthly public meetings conducted by the Division of Inspection and Regional Support (DIRS), were noticed and summarized within the applicable timeliness goals.
O-6 Responsiveness to ROP Contact Us Forms Metric Criterion Met: Yes (Green)
Definition:
ROP Contact Us forms received through the public or internal website regarding the ROP are responded to within 45 days upon receipt.
Criteria:
Green Yellow Red 95%
< 95% AND 90%
< 90%
Basis:
IMC 0307 Program Area:
All Lead/Data Source:
NRR/DIRS Related Principles:
Efficiency, Clarity Analysis:
For CY 2018, 1 of 1 (100%) feedback form received from the NRCs external ROP Contact Us button was responded to within the timeliness goal. In CY 2018, 4 of 4 (100%) of all the completed NRCs internal ROP Contact Us feedback forms were responded to within the timeliness goals.
0307A-03 EFFICIENCY PERFORMANCE METRICS (E)
E-1 Completion of Supplemental Inspections Metric Criterion Met: Yes (Green)
Definition:
Exit meetings for supplemental inspections are completed within 180 days from licensee notification of readiness.
Criteria:
Green Yellow Red 1 late 2 late 3 late
- Note: No more than one late exit meeting in any region. Any region that has more than one late exit meeting is individually evaluated.
Basis:
Management Direction Program Area:
Inspection, Assessment Lead/Data Source:
Regions, NRR/DIRS Related Principles:
Reliability, Clarity Analysis:
For CY 2018, all exit meetings for supplemental inspections were completed within the timeliness goal.
E-2 Initiation of Reactive Inspections Metric Criterion Met: Yes (Green)
Definition:
Entrance meetings for reactive inspections are conducted within 30 days of a determination that an event or specific circumstances require a reactive inspection.
Criteria:
Green Yellow Red 2 late 3 late 4 late
- Note: No more than one late entrance meeting in any region. Any region that has more than one entrance meeting is individually evaluated.
Basis:
MD 8.3, NRC Incident Investigation Program Program Area:
Inspection Lead/Data Source:
Regions, NRR/DIRS Related Principles:
Reliability, Clarity Analysis:
For CY 2018, 100% of entrance meetings for reactive inspections were completed within the timeliness goal.
E-3 Completion of Temporary Instructions Metric Criterion Met: Yes (Green)
Definition:
Temporary Instruction (TI) inspections associated with IMC 2201, Security Inspection Program for Operating Commercial Nuclear Power Reactors (ADAMS Accession No. ML18031B047), and IMC 2515 are completed within the required TI completion time.
Criteria:
Green Yellow Red 97.5% of documented completions are timely
< 97.5% AND 95% < 95% of documented completions are timely
- Note: No less than 95% completion for any region. Any region that falls below 95% is individually evaluated. The term documented completions refers to cases where TI results are documented in an inspection report.
Basis:
Applicable TI Program Area:
Inspection Lead/Data Source:
Regions, NRR/DIRS Related Principles:
Reliability, Clarity Analysis:
In CY 2018, 100% of TIs were completed within the timeliness goals.
E-4 Completion of Performance Deficiency Determinations Metric Criterion Met: Yes (Green)
Definition:
The time from the identification date (i.e., the date the issue of concern was brought to the licensees attention by the NRC, the date the performance deficiency was self-revealed, or the date the licensee documented the condition resulting from the performance deficiency in the corrective action program) to the start date used for consideration of inspection findings in the assessment process (as defined by IMC 0305) is within 120 days.
Criteria:
Green Yellow Red 90% timely
< 90% AND 75%
< 75% timely
- Note: No more than two untimely occurrences for any region or NSIR.
Any region or office that has more than two untimely occurrences is individually evaluated.
Basis:
Business Process Improvement, Process Improvement Review of the Significance Determination Process (ADAMS Accession No. ML14318A512), and IMC 0305 Program Area:
Significance Determination Process Lead/Data Source:
Regions, NSIR, NRR/DIRS Related Principles:
Reliability, Clarity Analysis:
For CY 2018, 100% of the performance deficiency determinations were completed within the 120-day timeframe. There were 2 performance deficiency determinations completed in CY 2018 for findings that were finalized as Greater-than-Green (GTG). This remained a pilot metric for CY 2018, to provide for additional data to be evaluated as part of a CY 2018 effectiveness review of the Inspection Finding Resolution Management (IFRM) process pilot effort.
E-5 Completion of Final Significance Determinations Metric Criterion Met: Yes (Green)
Definition:
Inspection items are finalized as greater-than-Green within 90 days since: (1) the date of initial licensee notification of the preliminary significance in an inspection report, or (2) the date the item was otherwise documented in an inspection report as an apparent violation or finding pending completion of a significance determination.
Criteria:
Green Yellow Red 95% timely
< 95% AND 90%
< 90% timely
- Note: No more than one late finalized significance determination for
any region or NSIR. Any region or office that has more than one late finalized significance determination is individually evaluated. NSIR will also meet the 90% timeliness expectation for security-related findings.
Basis:
IMC 0609, Significance Determination Process.
Program Area:
Significance Determination Process Lead/Data Source:
Regions, NSIR, NRR/DIRS Related Principles:
Clarity, Reliability Analysis:
For CY 2018, 100% of GTG findings were finalized within the timeliness goals. There were 3 findings finalized as GTG in CY 2018.
E-6 Responsiveness to ROP Feedback Forms Metric Criterion Met: No (Red)
Definition:
ROP feedback forms are completed within applicable timeliness goals.
Criteria:
Green Yellow Red 90% timely
< 90% AND 80%
< 80% timely Basis:
IMC 0801, Reactor Oversight Process Feedback Program Program Area:
All Lead/Data Source:
NRR/DIRS Related Principles:
Clarity, Openness Analysis:
For CY 2018, the staff did not meet the timeliness goals delineated in IMC 0801. The staff received 59 new feedback forms in CY 2018 and dispositioned 13 of them within 12 months (22 percent, Green metric is greater than 90%). The staff closed 57 total feedback forms in CY 2018, and as of February 1, 2019, there are 229 feedback forms in the backlog.
The staff continues to take action to address timely resolution of ROP feedback forms, such as prioritizing resolution of all feedback forms tied to specific IP and IMC documents undergoing revisions, and providing increased engagement with monthly ROP feedback form disposition status update meetings with Division management. DIRS staff is currently performing a detailed review of the feedback form backlog to identify feedback forms which may be obsolete due to other ROP changes, or which may be able to be dispositioned in the near term.
Additionally, DIRS leadership intends to issue a memorandum detailing expectations for IP and IMC leads, which would include timely disposition of feedback forms. As part of a holistic review of the ROP self-assessment program, the staff is evaluating more meaningful metrics with which to measure the efficiency of the feedback form process.
Additionally, the staff expects that while ROP enhancement initiative efforts in the area of inspection will likely trigger resolution of backlogged ROP feedback forms in the areas of problem identification and resolution, supplemental inspections, radiation health, and others; the ROP enhancement effort will also likely delay disposition of these feedback forms through CY 2019 due to reviews associated with the initiative. These delays may impact the performance against the E-6 metric for CY 2019, however, as noted above, the staff will continue efforts to improve performance in this area.
0307A-04 CLARITY PERFORMANCE METRICS (C)
C-1 Maintenance of ROP Web Pages Metric Criterion Met: Yes (Green)
Definition:
ROP pages on the public website are reviewed at least quarterly to ensure that the content on the page is up-to-date with accurate information.
Criteria:
Green Yellow Red 90% Web pages reviewed
< 90% AND 80%
< 80% Web pages reviewed
- Note: All ROP-related Web pages will be reviewed for general content on a quarterly basis. This review will also include a sampling of hyperlinks for accuracy.
Basis:
Management Directive Program Area:
All Lead/Data Source:
NRR/DIRS Related Principles:
Openness Analysis:
For CY 2018, 100% of ROP Web pages on the public website were reviewed within timeliness goals.
C-2 Corrections to ROP Web Pages Metric Criterion Met: Yes (Green)
Definition:
Broken hyperlinks or out-of-date content on the ROP internal or external Website are corrected within 30 days upon discovery.
Criteria:
Green Yellow Red 95% corrected within 30 days
< 95% AND 90%
< 90% corrected within 30 days
Basis:
Management Directive Program Area:
All Lead/Data Source:
NRR/DIRS Related Principles:
Efficiency Analysis:
For CY 2018, 100% of the reported broken hyperlinks were corrected within the timeless goals. An average of three to five broken hyperlinks per quarter were identified and corrected.
C-3 Traceability of Greater-than-Green Inspection Findings Metric Criterion Met: Yes (Green)
Definition:
Inspection findings are updated in the Reactor Program System (RPS) and posted to the ROP web page to ensure traceability of a greater-than-Green inspection finding from discovery to final resolution. When a report or letter follows up on an existing item (i.e., final significance determination letters and supplemental inspection reports), the RPS entry is updated to reflect the new information.
Criteria:
Green Yellow Red 95% GTG findings traceable
< 95% AND 90%
< 90% GTG findings traceable
- Note: No more than one greater-than-Green inspection finding found to be untraceable for any region or NSIR. Any region or office that has more than one untraceable issue is individually evaluated.
Basis:
IMC 0306 Program Area:
Significance Determination Process, Inspection Lead/Data Source:
Regions, NSIR, NRR/DIRS Related Principles:
Openness Analysis:
For CY 2018, 100% of the GTG inspection findings were found to be traceable from discovery to final resolution on both the internal and external websites.
C-4 Maintenance of ROP Governance Documents Metric Criterion Met: Green Definition:
Baseline inspection procedures and other ROP-related inspection procedures and manual chapters are reviewed at least once every 4 years.
Criteria:
Green Yellow Red 95% reviewed within past 4 years
< 95% AND 90% < 90% reviewed within past 4 years Basis:
IMC 0307 and Appendix B to IMC 0307 Program Area:
Inspection Lead/Data Source:
NRR/DIRS Related Principles:
Reliability Analysis:
This expectation was added to IMC 0307 when it was revised in November 2015. As noted in the CY 2015 metric report, the revised metric would be phased in beginning with the CY 2016 self-assessment, with the expectation that at least one-half of the ROP-related IMCs and ROP-related IPs have been reviewed within the 4-year requirement.
The expectation is that at least three-quarters will have been reviewed within the 4-year requirement by CY 2018, and the metric will be fully evaluated for the CY 2019 self-assessment at which time all ROP-related governance documents are expected to be reviewed and verified for relevancy and up-to-date. As of December 31, 2018, the staff has reviewed and revised 176 of the 246 (or 76%) of ROP-related IMCs.
0307A-05 RELIABILITY PERFORMANCE METRICS (R)
R-1 Performance of Lessons Learned Evaluations Metric Criterion Met: Yes (Green)
Definition:
Lessons learned evaluations are performed, reports are issued, and recommendations are considered and entered into the tracking system for significant NRC activities to ensure their completion in accordance with program expectations.
Criteria:
Green Yellow Red All required evaluations completed and documented N/A 1 or more evaluations not completed
- Note: All supplemental inspections conducted in accordance with inspection procedure (IP) 95003, Supplemental Inspection For Repetitive Degraded Cornerstones, Multiple Degraded Cornerstones, Multiple Yellow Inputs Or One Red Input, implementations of IMC 0350, Oversight Of Reactor Facilities In A Shutdown Condition Due To Significant Performance and/or Operational Concerns, Incident Investigation Team (IIT) responses, and Augmented Inspection Team (AIT) responses are individually evaluated for potential program improvements.
Timeliness expectations will be determined by senior management on a case-by-case basis.
Basis:
IP 95003, IMC 0350, and MD 8.3 Program Area:
All Lead/Data Source:
Regions, NRR/DIRS Related Principles:
Efficiency Analysis:
For CY 2018, there were no IP 95003, IMC 0350, AIT, or IIT inspections completed. During CY 2018, the staff fully integrated the ROP Lessons Learned program, completing data entry into the SharePoint-based ROP Lessons Learned tracker. The purpose of this tracker is to enable effective monitoring of more complex ROP feedback and lessons learned from significant NRC activities in a clear, transparent, and readily accessible manner. The staff provided quarterly updates to DIRS management on the status of the lessons learned tracker, including the status of new lessons learned and timeliness of disposition for all lessons learned. Accordingly, this metric is Green for CY 2018.
R-2 Predictability and Repeatability of Significance Determination Results Metric Criterion Met: Yes (Green)
Definition:
Greater-than-Green inspection findings and the associated degraded conditions contain adequate detail to enable an independent auditor to trace through the available documentation and conclude that the significance characterization is reasonably justifiable from both programmatic and technical positions. This audit should be documented in a memo that is internally available to the NRC and referenced in the annual metric report.
Criteria:
Green Yellow Red 0 deemed unpredictable 1 deemed unpredictable 2 deemed unpredictable
- Note: Any significance determination documentation determined by the auditor to be inadequate will be evaluated and appropriate programmatic changes will be considered.
Basis:
IMC 0609, Significance Determination Process Program Area:
Significance Determination Process Lead/Data Source:
NRR/DIRS Related Principles:
Clarity Analysis:
For CY 2018, the staff determined that 100% of the Greater-than-Green findings issued in the CY contained adequate detail to enable the independent auditor to trace through the available documentation and conclude that the significance characterization was reasonably
justifiable from both programmatic and technical positions. This audit is documented in a memorandum dated February 3, 2019 (ADAMS Accession No. ML19009A129, non-publicly available).
R-3 Predictability of Agency Actions and Response Metric Criterion Met: Yes (Green)
Definition:
Deviations from the Action Matrix are expected to be infrequent to ensure reliable and predictable programmatic and technical positions.
Criteria:
Green Yellow Red 1 deviations 2 - 3 deviations
> 3 deviations
- Note: All deviations are individually evaluated for potential program improvements Basis:
IMC 0305 Program Area:
Assessment Lead/Data Source:
NRR/DIRS Related Principles:
Clarity Analysis:
For CY 2018, there were no ROP Action Matrix deviations.
R-4 Consideration of Operating Experience Insights Metric Criterion Met: Yes (Green)
Definition:
A summary of recent operating experience insights is provided and discussed for the mid-cycle and end-of-cycle assessments for each region to inform inspection planning.
Criteria:
Green Yellow Red Operating experience discussed during all regional assessment meetings N/A Operating experience not discussed during 1 or more regional assessment meetings Basis:
IMC 2523, NRC Application of the Reactor Operating Experience Program in NRC Oversight Processes Program Area:
Assessment, Inspection Lead/Data Source:
NRR/DIRS Related Principles:
Efficiency Analysis:
For CY 2018, 100% of all mid-cycle and end-of-cycle assessment meetings included discussions of operating experience insights to inform inspection planning.