ML18039A344
| ML18039A344 | |
| Person / Time | |
|---|---|
| Issue date: | 04/05/2018 |
| From: | Gregory Bowman NRC/NRR/DIRS/IRAB |
| To: | Chris Miller Division of Inspection and Regional Support |
| Anderson M, NRR/DIRS, 301-415-2939 | |
| Shared Package | |
| ML18039A288 | List: |
| References | |
| Download: ML18039A344 (19) | |
Text
Calendar Year 2017 Reactor Oversight Process Self-Assessment Metric Report (Enclosures 1 and 2)
Metric I-1 Completion of the Baseline Inspection Program I-2 Resident Inspector Objectivity Through Rotation Policy I-3 Resident Inspector Objectivity Through Diverse Experience I-4 Inspector Objectivity Reviews I-5 Fully Qualified Inspectors and Operator Licensing Examiners I-6 Analysis of Permanent Site Staffing Result Green Green Green Green Green Green Metric O-1 Issuance of Inspection Reports O-2 Issuance of Assessment Letters O-3 Conduct of Annual Assessment Meetings or Other Engagement Activities O-4 Reporting and Dissemination of PI Data O-5 Issuance of ROP Public Meeting Notices and Summaries O-6 Responsiveness to ROP Contact Us Forms Result Green Green Green Green Green Green Metric E-1 Completion of Supplemental Inspections E-2 Initiation of Reactive Inspections E-3 Completion of Temporary Instructions E-4 Completion of Performance Deficiency Determinations E-5 Completion of Final Significance Determinations E-6 Responsiveness to ROP Feedback Forms Result Green Green Green Red Yellow Red Metric C-1 Maintenance of ROP Web Pages C-2 Corrections to ROP Web Pages C-3 Traceability of Greater-than-Green Inspection Findings C-4 Maintenance of ROP Governance Documents Result Green Green Green Green Metric R-1 Performance of Lessons Learned Evaluations R-2 Predictability and Repeatability of Significance Determination Results R-3 Predictability of Agency Actions and
Response
R-4 Consideration of Operating Experience Insights Result Red Green Green Green Self-Assessment Metrics Overview/Dashboard Openness Metrics Efficiency Metrics Clarity Metrics Reliability Metrics Independence Metrics
INSPECTION MANUAL CHAPTER 0307, APPENDIX A REACTOR OVERSIGHT PROCESS SELF-ASSESSMENT METRICS 0307A-01 INDEPENDENCE PERFORMANCE METRICS (I)
I-1 Completion of Baseline Inspection Program Metric Criterion Met: Yes (Green)
Definition:
The baseline inspection program is completed annually in accordance with program requirements.
Criteria:
Green Yellow Red All regions and the Office of Nuclear Security and Incident Response (NSIR) met completion requirements N/A 1 or more regions or NSIR did not meet completion requirements
- Note: No less than 100% compliance for any region or NSIR.
Baseline inspection program completion is defined in Section 04.07 of Inspection Manual Chapter (IMC) 2515, Light-Water Reactor Inspection Program - Operations Phase. Any region (or NSIR) that does not complete the baseline inspection program per IMC 2515 is individually evaluated.
Basis:
IMC 2515 Program Area:
Inspection Lead/Data Source:
Regions, NSIR Related Principles:
Efficiency Analysis:
For CY 2017, all regions and the Office of Nuclear Security and Incident Response completed 100 percent of their baseline inspections within the allocated resources. Each region documented completion of the baseline inspection program in memoranda available at Agencywide Documents Access and Management System (ADAMS)
Accession Nos. ML18052A730 for Region I, ML18044A838 for Region II, ML18053A239 for Region III, and ML18057B084 for Region IV. NSIR documented completion of all its security baseline inspections in a memorandum at ADAMS Accession No. ML18011A830 (non-public).
I-2 Resident Inspector Objectivity Through Rotation Policy Metric Criterion Met: Yes (Green)
Definition:
Senior resident inspectors (SRIs) and resident inspectors (RIs) are stationed for a 7-year maximum tour length, unless specifically granted an extension per IMC 2515.
Criteria:
Green Yellow Red 0 inspectors exceeded 7 years without an extension N/A 1 or more inspectors exceeded 7 years without an extension
- Note: No less than 100% compliance for any region. Any region that has 1 or more inspectors exceed 7 years at a site without an extension is individually evaluated.
Basis:
IMC 0102, Oversight and Objectivity of Inspectors and Examiners at Reactor Facilities, and IMC 2515 Program Area:
Inspection Lead/Data Source:
Regions Related Principles:
Reliability Analysis:
For calendar year (CY) 2017, there was 100% compliance in all regions with no SRIs or RIs stationed at a plant beyond the maximum tour length without an extension.
I-3 Resident Inspector Objectivity Through Diverse Experience Metric Criterion Met: Yes (Green)
Definition:
Permanently-staffed SRIs and RIs spend a minimum of one week each year inspecting at another site.
Criteria:
Green Yellow Red 3 noncompliant
(~97.5%)
4-5 noncompliant 6 noncompliant
(~95%)
- Note: No more than two non-compliances in any region. Any region that has more than two non-compliances is individually evaluated.
Basis:
IMC 0102 Program Area:
Inspection Lead/Data Source:
Regions Related Principles:
Reliability
Analysis:
For CY 2017, 100% of all permanently-staffed resident inspectors completed their objectivity visits by spending a minimum of one week inspecting at another plant. No region had more than one noncompliance.
I-4 Inspector Objectivity Reviews Metric Criterion Met: Yes (Green)
Definition:
Line managers perform annual on-site employee performance and objectivity reviews of their assigned inspectors.
Criteria:
Green Yellow Red 3 noncompliant
(~97.5%)
4-5 noncompliant 6 noncompliant
(~95%)
- Note: No more than two non-compliances in any region. Any region that has more than two non-compliances is individually evaluated.
NSIR will also meet the 95% compliance expectation for their assigned employees.
Basis:
IMC 0102 Program Area:
Inspection Lead/Data Source:
Regions, NSIR Related Principles:
Reliability Analysis:
For CY 2017, there were two annual on-site employee performance and objectivity reviews not completed by a line manager. No single region or NSIR had more than two noncompliances.
I-5 Fully Qualified Inspectors and Operator Licensing Examiners Metric Criterion Met: Yes (Green)
Definition:
Inspectors and operator licensing examiners remain fully qualified in accordance with qualification requirements.
Criteria:
Green Yellow Red 5 noncompliant
(~97.5%)
6-9 noncompliant 10 noncompliant
(~95%)
- Note: No more than three non-compliances in any region. Any region that has more than three non-compliances is individually evaluated.
NSIR will also meet the 95% compliance expectation for their assigned
employees.
Basis:
IMC 1245, Qualification Program for Operating Reactor Programs, and Davis-Besse Reactor Vessel Head Degradation Lessons-Learned Task Force Report (ML022760172)
Program Area:
Inspection Lead/Data Source:
Regions, NSIR Related Principles:
Reliability Analysis:
For CY 2017, there was one individual who did not remain fully qualified as an inspector or operator licensing examiner. No single region or NSIR had more than one noncompliance.
I-6 Analysis of Permanent Site Staffing Metric Criterion Met: Yes (Green)
Definition:
Permanent inspector staffing levels at each of the reactor sites for both SRIs and RIs are maintained to provide continuity of regulatory oversight.
Criteria:
Green Yellow Red 95%
< 95% AND 90%
< 90%
- Note: No less than 90% compliance for any region. Any single site that falls below 90% is individually evaluated.
Inspectors assigned to the site permanently or through a rotation with a minimum duration of 6 weeks shall be counted. Inspectors on 6 week or longer rotational assignments will be identified as such. Inspectors assigned to the site for less than six weeks will not be counted, but should be indicated as such. Additionally, the regions shall indicate sites where permanently assigned resident or senior resident inspectors are away from the site for greater than 6 continuous weeks. Only inspectors who have attained at least a basic inspector certification status, as defined by Appendix A to Inspection Manual Chapter 1245, shall be counted.
Data will indicate number of days a qualified resident and senior resident inspector are permanently assigned to the site during the year divided by the number of days in the year. Number of days spent on training; meetings away from the site; participation in team inspections; leave; or other temporary duties (e.g. acting for branch chiefs in his/her absence) will not be counted against the metric unless the absence exceed 6 continuous weeks.
Basis:
Davis-Besse Reactor Vessel Head Degradation Lessons-Learned Task Force Report (ML022760172) and IMC 2515 Program Area:
Inspection Lead/Data Source:
Regions Related Principles:
Reliability Analysis:
For CY 2017, the average permanent inspector staffing was 98.5% for all regions with no single region falling below 97%. One site did fall below the 90% threshold. That site had a resident inspector position become vacant. The position was covered by qualified inspectors for several periods amounting to less than 6 weeks before the position could be permanently filled. No further evaluation or action is required for this site since the events were not reasonably within the regions ability to foresee and prevent.
0307A-02 OPENNESS PERFORMANCE METRICS (O)
O-1 Issuance of Inspection Reports Metric Criterion Met: Yes (Green)
Definition:
Reactor Oversight Process (ROP) inspection reports are issued within applicable timeliness goals.
Criteria:
Green Yellow Red 25 late
(~ 95% timely)
> 25 AND 50 late
> 50 late
(~ 90% timely)
- Note: No more than 15 late inspection reports in any region. Any region that has more than 15 late inspection reports is individually evaluated. NSIR will also meet the 90% timeliness expectation for reports generated by their office.
All inspection reports resulting from direct inspections of operating light water reactors and documented in accordance with IMC 0612, Power Reactor Inspection Reports, are counted for this metric (note that effective January 1, 2018, this guidance is now contained in IMC 0611).
For inspections not conducted by a resident inspector, inspection completion is normally defined as the day of the exit meeting. For integrated inspection reports, inspection completion is normally defined as the last day covered by the inspection report.
Basis:
IMC 0612 and IMC 2515 Program Area:
Inspection Lead/Data Source:
Regions, NSIR
Related Principles:
Reliability, Efficiency Analysis:
For CY 2017, there were six inspection reports not issued on time. No single region or NSIR had more than two late inspection reports.
O-2 Issuance of Assessment Letters Metric Criterion Met: Yes (Green)
Definition:
Annual, mid-cycle, and follow-up assessment letters are issued within the applicable timeliness goals.
Criteria:
Green Yellow Red 2 late 3 late 4 late
- Note: No more than one late letter in any region. Any region that has more than one late letter is individually evaluated.
Basis:
IMC 0305, Operating Reactor Assessment Program Program Area:
Assessment Lead/Data Source:
Regions Related Principles:
Reliability, Efficiency Analysis:
For CY 2017, 100% of the annual, mid-cycle, and follow-up assessment letters were issued on time.
O-3 Conduct of Annual Assessment Meetings or Other Engagement Activities Metric Criterion Met: Yes (Green)
Definition:
Public assessment meetings or other engagement activities that discuss the results of the U.S. Nuclear Regulatory Commissions (NRCs) annual assessment of the licensees performance, are conducted annually for all sites within the applicable timeliness goals.
Criteria:
Green Yellow Red 2 late 3 late 4 late
- Note: The level of public engagement and timeliness goals are determined by plant performance, as described in IMC 0305. No more than one late meeting/activity in any region. Any region that has more than one late meeting/activity is individually evaluated.
Basis:
IMC 0305 Program Area:
Assessment
Lead/Data Source:
Regions Related Principles:
Reliability, Efficiency Analysis:
For CY 2017, 100% of the public assessment meetings and other engagement activities met applicable timeliness goals.
O-4 Reporting and Dissemination of PI Data Metric Criterion Met: Yes (Green)
Definition:
Performance indicator (PI) data submittals by the licensees are posted to the NRCs external web site within the applicable timeliness requirements.
Criteria:
Green Yellow Red 0 late web posting 1-3 late web postings > 3 late web postings
- Note: Any licensee submittals that did not meet the timely reporting requirements will also be evaluated by NRC staff and discussed with industry to address corrective actions to prevent recurrence.
Basis:
IMC 0306, Information Technology Support for the Reactor Oversight Process, and Nuclear Energy Institute 99-02, Performance Indicator Data Collection Program Area:
Performance Indicators Lead/Data Source:
NRR/DIRS Related Principles:
Reliability, Efficiency Analysis:
For CY 2017, 100% of the licensee PI data submitted to the NRC was posted to the external ROP website on time. No licensee submitted data late to the NRC.
O-5 Issuance of ROP Public Meeting Notices and Summaries Metric Criterion Met: Yes (Green)
Definition:
ROP-related public meetings are noticed prior to the meeting and meeting summaries are posted after the meeting within the applicable timeliness requirements.
Criteria:
Green Yellow Red 95%
< 95% AND 90%
< 90%
- Note: ROP-related public meetings include ROP working group
meetings, annual assessment meetings, and other ROP-related meetings conducted by NRC staff. No more than two late notices or summaries for any region or office. Any region or office that has more than two late notices or summaries is individually evaluated.
Basis:
Management Directive (MD) 3.5, Attendance at NRC Staff-Sponsored Meetings, and COM-202, Meetings With Applicants, Licensees, Vendors or Other Members of the Public Program Area:
All Lead/Data Source:
NRR/DIRS, Regions, NSIR Related Principles:
Efficiency, Clarity Analysis:
For CY 2017, 100% of the public meeting notices and meeting summaries were completed within the timeliness goals.
O-6 Responsiveness to ROP Contact Us Forms Metric Criterion Met: Yes (Green)
Definition:
ROP Contact Us forms received through the public or internal website regarding the ROP are responded to within 45 days upon receipt.
Criteria:
Green Yellow Red 95%
< 95% AND 90%
< 90%
Basis:
IMC 0307 Program Area:
All Lead/Data Source:
NRR/DIRS Related Principles:
Efficiency, Clarity Analysis:
For CY 2017, 3 of 3 (100%) of all the completed NRCs internal ROP Contact Us feedback forms were responded to within the timeliness goals. The public website was updated to include a more prominent location for the ROP Contact Us link. In CY 2017, no comments were received from the public Contact Us form.
0307A-03 EFFICIENCY PERFORMANCE METRICS (E)
E-1 Completion of Supplemental Inspections Metric Criterion Met: Yes (Green)
Definition:
Exit meetings for supplemental inspections are completed within 180 days from licensee notification of readiness.
Criteria:
Green Yellow Red 1 late 2 late 3 late
- Note: No more than one late exit meeting in any region. Any region that has more than one late exit meeting is individually evaluated.
Basis:
Management Direction Program Area:
Inspection, Assessment Lead/Data Source:
Regions, NRR/DIRS Related Principles:
Reliability, Clarity Analysis:
For CY 2017, there was one late exit meeting. No single region had two or more late inspection reports.
E-2 Initiation of Reactive Inspections Metric Criterion Met: Yes (Green)
Definition:
Entrance meetings for reactive inspections are conducted within 30 days of a determination that an event or specific circumstances require a reactive inspection.
Criteria:
Green Yellow Red 2 late 3 late 4 late
- Note: No more than one late entrance meeting in any region. Any region that has more than one entrance meeting is individually evaluated.
Basis:
MD 8.3, NRC Incident Investigation Program Program Area:
Inspection Lead/Data Source:
Regions, NRR/DIRS Related Principles:
Reliability, Clarity Analysis:
For CY 2017, 100% of entrance meetings for reactive inspections were completed within the timeliness goal.
E-3 Completion of Temporary Instructions Metric Criterion Met: Yes (Green)
Definition:
Temporary Instruction (TI) inspections are completed within the required TI completion time.
Criteria:
Green Yellow Red 97.5% of documented completions are timely
< 97.5% AND 95%
< 95% of documented completions are timely
- Note: No less than 95% completion for any region. Any region that falls below 95% is individually evaluated. The term documented completions refers to cases where TI results are documented in an inspection report.
Basis:
Applicable TI Program Area:
Inspection Lead/Data Source:
Regions, NRR/DIRS Related Principles:
Reliability, Clarity Analysis:
In CY 2017, 100% of the TIs were completed within the timeliness goals.
E-4 Completion of Performance Deficiency Determinations Metric Criterion Met: No (Red)
Definition:
The time from the identification date (i.e., the date the issue of concern was brought to the licensees attention by the NRC, the date the performance deficiency was self-revealed, or the date the licensee documented the condition resulting from the performance deficiency in the corrective action program) to the start date used for consideration of inspection findings in the assessment process (as defined by IMC 0305) is within 120 days.
Criteria:
Green Yellow Red 90% timely
< 90% AND 75%
< 75% timely
- Note: No more than two untimely occurrences for any region. Any region that has more than two untimely occurrences is individually evaluated. NSIR will also meet the 75% timeliness expectation for security-related findings.
Basis:
Business Process Improvement, Process Improvement Review of the Significance Determination Process (ADAMS Accession No. ML14318A512), and IMC 0305
- Note: This is a pilot metric that will be evaluated through an effectiveness review in CY 2018, and only applies to those findings finalized as greater-than-Green.
Program Area:
Significance Determination Process Lead/Data Source:
Regions, NSIR, NRR/DIRS Related Principles:
Reliability, Clarity Analysis:
For CY 2017, 73% of the performance deficiency determinations were completed within the 120-day timeframe. One region had more than two untimely occurrences and is going to be individually evaluated. All other offices met the 75% timeliness expectation. Of the five performance deficiency determinations that were late, three were late by one week or less. This was the first year in which data was collected for this metric. The Inspection Finding Resolution Management (IFRM) process, which was the subject of a trial period in 2017, provides staff with timeliness insights on a number of milestones, including this metric, that are being collectively assessed as part of an IFRM effectiveness review that is currently underway.
E-5 Completion of Final Significance Determinations Metric Criterion Met: Yes (Yellow)
Definition:
Inspection items are finalized as greater-than-Green within 90 days since: (1) the date of initial licensee notification of the preliminary significance in an inspection report, or (2) the date the item was otherwise documented in an inspection report as an apparent violation or finding pending completion of a significance determination.
Criteria:
Green Yellow Red 95% timely
< 95% AND 90%
< 90% timely
- Note: No more than one late finalized significance determination for any region. Any region that has more than one late finalized significance determination is individually evaluated. NSIR will also meet the 90% timeliness expectation for security-related findings.
Basis:
IMC 0609, Significance Determination Process.
Program Area:
Significance Determination Process Lead/Data Source:
Regions, NSIR, NRR/DIRS Related Principles:
Clarity, Reliability Analysis:
For CY 2017, 92% of the greater-than-Green findings were completed within the timeliness goals. No single region or NSIR had more than one late finalized significance determination. The staff has increased monitoring of this metric to assure adequate oversight of this metric in the future.
E-6 Responsiveness to ROP Feedback Forms Metric Criterion Met: No (Red)
Definition:
ROP feedback forms are completed within applicable timeliness goals.
Criteria:
Green Yellow Red 90% timely
< 90% AND 80%
< 80% timely Basis:
IMC 0801, Reactor Oversight Process Feedback Program Program Area:
All Lead/Data Source:
NRR/DIRS Related Principles:
Clarity, Openness Analysis:
IMC 0801 was revised and issued on December 19, 2016, with an effective date of January 1, 2017. The more significant changes to the guidance include: (1) better defined roles, responsibilities, and process flow; (2) the addition of timeliness goals for closure of feedback forms; (3) a distinction between short-term and long-term feedback; and (4) creation of a Feedback Form Review Panel to establish the approach and timeliness goals for the resolution of the more complex long-term feedback forms.
For CY 2017, the staff did not meet the timeliness goals delineated in IMC 0801; 46 percent of CY 2017 feedback forms were dispositioned within 12 months. During CY 2017, 43 new feedback forms were initiated, and 37 feedback forms were dispositioned. As of February 2018, 229 feedback forms await disposition.
Interim timeliness goals are being developed to improve performance in this area. The focus of these interim goals will be to foster an environment where management and staff are more focused on feedback form inventory reduction.
0307A-04 CLARITY PERFORMANCE METRICS (C)
C-1 Maintenance of ROP Web Pages Metric Criterion Met: Yes (Green)
Definition:
ROP pages on the public website are reviewed at least quarterly to ensure that the content on the page is up-to-date with accurate information.
Criteria:
Green Yellow Red 90% Web pages reviewed
< 90% AND 80%
< 80% Web pages reviewed
- Note: All ROP-related Web pages will be reviewed for general content on a quarterly basis. This review will also include a sampling of hyperlinks for accuracy.
Basis:
Management Directive Program Area:
All Lead/Data Source:
NRR/DIRS Related Principles:
Openness Analysis:
For CY 2017, 100% of ROP Web pages on the public website were reviewed within timeliness goals.
C-2 Corrections to ROP Web Pages Metric Criterion Met: Yes (Green)
Definition:
Broken hyperlinks or out-of-date content on the ROP internal or external Website are corrected within 30 days upon discovery.
Criteria:
Green Yellow Red 95% corrected within 30 days
< 95% AND 90%
< 90% corrected within 30 days Basis:
Management Directive Program Area:
All Lead/Data Source:
NRR/DIRS Related Principles:
Efficiency Analysis:
For CY 2017, 100% of the reported broken hyperlinks were corrected within the timeless goals. An average of three to five broken hyperlinks per quarter were identified and corrected.
C-3 Traceability of Greater-than-Green Inspection Findings Metric Criterion Met: Yes (Green)
Definition:
Inspection findings are updated in the Reactor Program System (RPS) and posted to the ROP web page to ensure traceability of a greater-
than-Green inspection finding from discovery to final resolution. When a report or letter follows up on an existing item (i.e., final significance determination letters and supplemental inspection reports), the RPS entry is updated to reflect the new information.
Criteria:
Green Yellow Red 95% greater-than-Green findings traceable
< 95% AND 90%
< 90% greater-than-Green findings traceable
- Note: No more than one greater-than-Green inspection finding found to be untraceable for any region. Any region that has more than one untraceable issue is individually evaluated. NSIR will also meet the 90%
traceability expectation for security-related findings.
Basis:
IMC 0306 Program Area:
Significance Determination Process, Inspection Lead/Data Source:
Regions, NSIR, NRR/DIRS Related Principles:
Openness Analysis:
For CY 2017, 100% of the greater-than-Green inspection findings were found to be traceable from discovery to final resolution on both the internal and external websites. No region or NSIR had a greater-than-Green inspection finding that was deemed untraceable.
C-4 Maintenance of ROP Governance Documents Metric Criterion Met: Green Definition:
Baseline inspection procedures and other ROP-related inspection procedures and manual chapters are updated at least once every 4 years.
Criteria:
Green Yellow Red 95% updated within past 4 years
< 95% AND 90%
< 90% updated within past 4 years Basis:
IMC 0307 and Appendix B to IMC 0307 Program Area:
Inspection Lead/Data Source:
NRR/DIRS Related Principles:
Reliability Analysis:
This expectation was added to IMC 0307 when it was revised in November 2015. As noted in the CY 2015 metric report, the revised metric would be phased in beginning with the CY 2016 self-assessment, with the expectation that at least one-half of the ROP-related IMCs and
ROP-related IPs have been reviewed within the 4-year requirement.
The expectation is that at least three-quarters will have been reviewed within the 4-year requirement by CY 2018, and the metric will be fully evaluated for the CY 2019 self-assessment at which time all ROP-related governance documents are expected to be reviewed and verified for relevancy and up-to-date. 152 of the 246 (or 62%) of ROP-related IMCs and IPs were reviewed and revised within the past 4 years.
0307A-05 RELIABILITY PERFORMANCE METRICS (R)
R-1 Performance of Lessons Learned Evaluations Metric Criterion Met: No (Red)
Definition:
Lessons learned evaluations are performed, reports are issued, and recommendations are considered and entered into the tracking system for significant NRC activities to ensure their completion in accordance with program expectations.
Criteria:
Green Yellow Red All required evaluations completed and documented N/A 1 or more evaluations not completed
- Note: All supplemental inspections conducted in accordance with inspection procedure (IP) 95003, Supplemental Inspection For Repetitive Degraded Cornerstones, Multiple Degraded Cornerstones, Multiple Yellow Inputs Or One Red Input, implementations of IMC 0350, Oversight Of Reactor Facilities In A Shutdown Condition Due To Significant Performance and/or Operational Concerns, Incident Investigation Team (IIT) responses, and Augmented Inspection Team (AIT) responses are individually evaluated for potential program improvements.
Timeliness expectations will be determined by senior management on a case-by-case basis.
Basis:
IP 95003, IMC 0350, and MD 8.3 Program Area:
All Lead/Data Source:
Regions, NRR/DIRS Related Principles:
Efficiency Analysis:
For CY 2017, there was one IP 95003, and no IMC 0350, AIT, or IIT inspections completed. During CY17, the staff created a SharePoint-based lessons learned tracker. The purpose of this tracker is to enable more effective monitoring of lessons learned initiatives in a clear, transparent, and readily accessible manner. The tracker has been
tested and the majority of lessons learned initiatives have been entered; however, as of December 31, 2017, data entry had not been completed.
As a result, DIRS is marking this metric as Red. DIRS staff intends to complete data entry, rollout the tracker, and begin full implementation in CY18.
R-2 Predictability and Repeatability of Significance Determination Results Metric Criterion Met: Yes (Green)
Definition:
Greater-than-Green inspection findings and the associated degraded conditions contain adequate detail to enable an independent auditor to trace through the available documentation and conclude that the significance characterization is reasonably justifiable from both programmatic and technical positions.
Criteria:
Green Yellow Red 0 deemed unpredictable 1 deemed unpredictable 2 deemed unpredictable
- Note: Any significance determination documentation determined by the auditor to be inadequate will be evaluated and appropriate programmatic changes will be considered.
Basis:
IMC 0609, Significance Determination Process Program Area:
Significance Determination Process Lead/Data Source:
NRR/DIRS Related Principles:
Clarity Analysis:
For CY 2017, 100% of the greater-than-Green inspection findings significance characterizations were reasonably justifiable from both a programmatic and technical position as verified by an independent auditor using available documentation specified in a memorandum Completion of Audit of Greater-Than-Green Inspection Findings in support of Reactor Oversight Process Metric R-2 for the 2017 ROP Self-Assessment (ADAMS Accession No. ML18057B082 (nonpublic)).
R-3 Predictability of Agency Actions and Response Metric Criterion Met: Yes (Green)
Definition:
Deviations from the Action Matrix are expected to be infrequent to ensure reliable and predictable programmatic and technical positions.
Criteria:
Green Yellow Red 1 deviations 2 - 3 deviations
> 3 deviations
- Note: All deviations are individually evaluated for potential program improvements.
Basis:
IMC 0305 Program Area:
Assessment Lead/Data Source:
NRR/DIRS Related Principles:
Clarity Analysis:
For CY 2017, there were no Action Matrix deviations.
R-4 Consideration of Operating Experience Insights Metric Criterion Met: Yes (Green)
Definition:
A summary of recent operating experience insights is provided and discussed for the mid-cycle and end-of-cycle assessments for each region to inform inspection planning.
Criteria:
Green Yellow Red Operating experience discussed during all regional assessment meetings N/A Operating experience not discussed during 1 or more regional assessment meetings Basis:
IMC 2523, NRC Application of the Reactor Operating Experience Program in NRC Oversight Processes Program Area:
Assessment, Inspection Lead/Data Source:
NRR/DIRS Related Principles:
Efficiency Analysis:
For CY 2017, 100% of all mid-cycle and end-of-cycle assessment meetings included discussions of operating experience insights to inform inspection planning.