ML20247J599

Attachment 4 Basis and History

ML20247J599
Person / Time
Issue date:	11/12/2020
From:	Russell Gibbs NRC/NRR/DRO/IRSB
To:
Russell Gibbs, NRR/DRO/IRSB, 415-8578
Shared Package
ML20247J590	List: ML20247J599 ML20247J601 ML20247J602 ML20262G982 ML20274A133
References
Download: ML20247J599 (10)

Text

Problem Identification and Resolution Inspection Basis and History History, Staff Commitments and Direction from the Commission In SECY 99-007, Recommendations for Reactor Oversight Process Improvements, the staff committed to evaluating PI&R as a key element underlying licensee performance in each cornerstone area of the ROP1. Specifically, the staff stated that [a] fundamental goal of the NRCs reactor inspection and assessment process is to establish confidence that each licensee is detecting and correcting problems in a manner that limits the risk to members of the public.

This evaluation would consist of inspection by two means, within each cornerstone but also a standalone inspection procedure requiring a biannual review that does not duplicate the inspections within the cornerstones but does span across all of them. The procedure frequency was changed to annual when the pilot was implemented. The objectives of this initial inspection were to:

(1) conduct reviews of precursors to events which occur relatively infrequently but have significant consequences; (2) independently identify potentially generic concerns that a licensee may have missed, including specific problems involving safety equipment, procedure development, design control, etc.;

(3) assess the collective impact of all the items in the corrective action backlog which may not have individual risk significance. The cornerstone framework does not otherwise include a means to accomplish this assessment. A good understanding of plant-specific risk vulnerabilities would be needed while conducting this review; (4) have assurance that licensees adequately address potential common cause equipment failure concerns, identified either by internal events and issues or by receipt of operating experience feedback from other licensees, vendors, etc.;

(5) verify that licensees appropriately identify and capture issues that could affect the unavailability of equipment tracked by the SSPIs and the maintenance rule.

The Commission approved the replacement ROP pilot in SRM SECY 99-007A including implementing the PI&R inspection as described. They also directed the staff to consider ways to ensure that the assessment process is sufficiently robust to address programmatic breakdowns (e.g., breakdown of a corrective actions program or aspects of a particular quality assurance program) which are different from issues involving many minor findings.2 In SECY 00-0049, Results of the Revised Reactor Oversight Process Pilot Program summarizing the replacement ROP pilot results and recommending proceeding to full implementation, the staff informed the Commission that it (in part) performs IP 71152 in response to the Commissions direction in SRM SECY 99-007A. Based on stakeholder feedback, the staff implemented the ability to document observations found during PI&R samples and the ability to include a qualitative assessment of the PI&R inspections in the 1

SECY 99-007 Recommendations for Reactor Oversight Process Improvements 2

SRM SECY 99-007 Recommendations for Reactor Oversight Process Improvements and SECY 007A - Recommendations for Reactor Oversight Process Improvements (Follow-up to SECY-99-007)

annual assessment letter. The staff also reported a lack of consensus at the lessons learned workshop following the pilot program regarding indication of programmatic breakdowns:

One position was that, in keeping with the premise that a programmatic breakdown in problem identification and resolution performance would reveal itself through degraded performance indicators and risk-significant inspection findings, no focused inspection or assessment of a corrective action program should be conducted unless overall licensee performance was not within the licensee response band of the Action Matrix.

The second significant opinion expressed was that, without focused inspection of problem identification and resolution performance in the baseline inspection program, the NRC would not be able to provide identification of programmatic breakdowns that have in the past provided early indications of problems, which, if uncorrected, result in significant performance degradations. Therefore, inspection and assessment activities focused on problem identification and resolution in the baseline inspection and assessment programs should be retained and further refined.3 In SRM SECY 00-0049, the Commission directed the staff to emphasize the importance of the licensee corrective action programs (CAPs) and the integral role that each licensee must play, i.e., that the new process places greater responsibility on the utilities that operate nuclear power plants. The staff should more clearly articulate the role of licensee CAPs, with self-assessment, as the beginning and the end of this process. The Commission also approved the staffs response for addressing cross-cutting issues and programmatic breakdowns in the RROP.4 The Commission did not address the above two positions.

SECY 01-0114, Results of the Initial Implementation of the New Reactor Oversight Process shifted the procedure frequency to biennial where it has remained, added annual samples for select issues and an option to perform an additional team inspection for a licensee in Column

45. The SECY also described a continued divergence among stakeholders regarding the predictive abilities of the ROP:

Respondents indicated that the ROP provides sufficient attention to licensees whose performance is in the licensee response band (74%). However, they indicated less agreement regarding the ROPs ability to provide appropriate identification of declining safety performance before theres a significant reduction in safety margins (53%).

Following the Davis Besse Lessons Learned Task Force recommendations6, the staff incorporated the semiannual trends sample and the daily resident CAP screening along with some additional guidance for safety conscious work environment (SCWE) inspection. This work was detailed in SECYs 04-00537, 04-01118 and 05-01879. Those commitments are tracked within the procedure as [C1] and [C2]. Any alteration of these would have to address these 3

SECY 00-0049 Results of the Revised Reactor Oversight Process Pilot Program 4

SRM SECY 00-0049 Results of the Revised Reactor Oversight Process Pilot Program 5

SECY 01-0114, Results of the Initial Implementation of the New Reactor Oversight Process 6

Degradation of The Davis-Besse Nuclear Power Station Reactor Pressure Vessel Head Lessons Learned Report, ML022740211.

7 SECY 04-0053, Reactor Oversight Process Self-Assessment for Calendar Year 2003.

8 SECY 04-0111, Recommended Staff Actions Regarding Agency Guidance in the Areas of Safety Conscious Work Environment and Safety Culture 9

SECY 05-0187, Status of Safety Culture Initiatives and Schedule for Near-Term Deliverables.

2

commitments (e.g., moving daily screening to Plant Status would track the transfer of that commitment). In 2013, another commitment was made [C3] to continuously monitor for potential 10 CFR Part 21 issues, this was following the findings of an Office of Inspector General audit10. With those final additions, the basic structure and substance of the procedure that exists today were formed.

Regional IP 95003 Lessons Learned and Previous Working Group Efforts Feedback on the procedure has been provided by several means in recent years. One of these means is regional Inspection Procedure (IP) 95003 Lessons Learned documents. IP 95003, Supplemental Inspection for Repetitive Degraded Cornerstones, Multiple Degraded Cornerstones, Multiple Yellow Inputs or One Red Input11 directs inspection teams to perform a limited review of the NRCs assessment and inspection process. This review is supposed to compare the teams findings with current assessment data (both PIs and inspection findings) to determine if sufficient warning was provided. Reviewing the last five IP 95003 lessons learned, four concluded that the program provided sufficient warning of declining licensee performance prior to entry into Column 4, one (ANO) concluded it did not. In addition to these findings, three of the lessons learned made several specific observations and recommendations for the procedure. The following is a summary of IP 95003 lessons learned and previous reform efforts related to the procedure.

1. Browns Ferry (2013)12 Although the NRCs expectations of the licensees CAP and how we utilized the concept of a robust CAP in the development of our processes are well established, our process lacks guidance in several areas. Specifically:

• How to address licensees that have a long-standing poor performing CAP?

• What to do with the early warning signs of poor performance before a licensee crosses the threshold out of Column 1, the only effective guidance comes once the licensee reaches Column 4?

• What should be done with the review of NCVs, if the licensees CAP is poor?

The region made the following recommendations:

For human performance and or PI&R SCCIs, provide additional guidance for both annual sample and biennial inspections in IP 71152 to observe maintenance and other field activities to gather observations on human performance, and in-field performance of identifying and reporting issues into the CAP.

For PI&R SSCIs, recommend, that during biennial PI&R inspections, the corrective actions for all NCVs and findings issued since the last inspection be reviewed and 10 OIG-11-A-08, Audit of NRCs Implementation of 10 CFR Part 21, Reporting of Defects and Noncompliance, ML110820426.

11 IP 95003, Supplemental Inspection for Repetitive Degraded Cornerstones, Multiple Degraded Cornerstones, Multiple Yellow Inputs or One Red Input, ML15188A400.

12 Inspection Procedure 95003: Evaluation of NRC Assessment and Inspection Processes At Browns Ferry Unit 1, Effectiveness Review of The Recommendations from The Browns Ferry Inspection Procedure 95003 Lessons Learned Report, ML17005A186, ROP Feedback Form 71152-1968.

3

specifically documented in the inspection scope sections of the inspection report. The basis for this recommendation is that, as described in IMC 2515, Appendix A, confidence in a licensees CAP is one basis for closing NCVs when entered into the CAP, and if there is a SCCI associated with a licensees CAP there is some lack of confidence in their program.

Consider requiring the licensees to complete periodic third-party safety culture assessments with NRC review of the results and licensees actions to supplement the inspections insights of the PI&R inspections and the cross-cutting process.

Provide guidance in IMC 0612 associated with PI&R reports to explicitly document in the inspection scope section all the specific NCVs and minor violations reviewed for verification of the corrective actions.

2. Fort Calhoun (2014)13 Issue There is a need for additional inspection tools to evaluate the effectiveness of licensee corrective action programs to determine if additional NRC actions are warranted.

Description The lessons learned team noted the following critical assessment from the December 2011 biennial team inspection of problem identification and resolution activities at FCS:

The team noted deficiencies in all three areas of the problem identification and resolution process. Based on the inspection sample, the team concluded that the implementation of the corrective action program and overall performance related to identifying, evaluating, and resolving problems was frequently less than adequate.

The team expanded their historical review, based on the 2011 negative assessment of the FCS corrective action program, to determine if prior weaknesses had been identified through inspection or assessment processes. Indications of possible corrective action program weaknesses included inspection results which documented numerous Appendix B, Criterion XVI violations and a held open White finding due to root cause concerns. The team noted that the issues were dispositioned in accordance with existing regulatory guidance but questioned: (1) if additional processes for documenting, communicating, and responding to programmatic concerns would be beneficial and (2) if additional tools were necessary for the evaluation of licensee corrective action program adequacy. With respect to the first consideration, the concept is further developed in Lessons Learned No. 3 in this report. With respect to the second consideration, the team noted that additional inspection tools to aid inspectors in evaluating the effectiveness of FCSs corrective action program may have provided earlier opportunities to increase NRC engagement to determine if additional actions were warranted.

13 Lessons Learned from Implementing Inspection Manual Chapter 0350 at Fort Calhoun Station and Recommended Changes, ML14128A376. ROP Feedback Forms 71152-1984, 2025.

4

Recommendation Consistent with recommendations from the on-going ROP Enhancement Project effort (See Feedback Form No. 71152-2025), the team recommends that the NRC develop attributes of an effective corrective action program for use by inspectors and management. Additionally, regulatory actions such as additional team inspections, increased public meetings with senior NRC personnel, etc., should be considered and implemented consistent with the inspection results. Additional regulatory options are being documented as part of the ROP Enhancement Project.

3. Arkansas Nuclear One (2017)14 The experience at ANO reinforced previous evidence that the effectiveness of a licensees problem identification and resolution (PI&R) process is a primary indicator of safety performance. The IP 95003 diagnostic evaluation at ANO identified concerns in all six PI&R cross-cutting areas, as well as X.8, Benchmarking, which is an element related to self-assessment. However, the indications documented in the biennial PI&R inspections did not provide adequate warning of the scope or depth of the performance issues. The current biennial PI&R inspections does not provide objective measures to allow a determination when individual elements, or the process as a whole, is degraded to the point of regulatory concern, and no guidance exists to provide consistent regulatory action.

Recommendation 3: Improve the inspection and assessment programs to improve the ability to identify PI&R programs or individual program elements that may not be adequate to support the assumptions of the ROP.

a) Revise the objectives of IP 71152, Problem Identification and Resolution, to align with the six PI&R cross-cutting attributes (P.1 - Identification; P.2 -

Evaluation; P.3 - Resolution; P.4 - Trending; P.5 - Operating Experience; and P.6 - Self-Assessment), plus X.8 - Benchmarking.

b) Develop objective measures to be used to assess each of the seven attributes listed in Recommendation 3.a above for use during the biennial inspection conducted under IP 71152 to allow a determination whether any attributes exhibit marginal or ineffective performance.

c) Revise IP 71152 to add an objective to assess the overall effectiveness of the licensees PI&R programs.

d) Develop assessment program guidance to help identify when a licensees PI&R performance is no longer adequate to meet the ROP assumptions.

Thresholds or examples could be developed using information from plants known to have had significant PI&R performance problems, including recent Column 4 plants and plants viewed as having had cyclic performance based on repeatedly having been placed into Columns 2 and/or 3.

14 Inspection Procedure 95003: Evaluation of NRC Assessment and Inspection Processes, ML17160A290.

5

e) Develop actions to respond to the determination that one or more PI&R program elements are degraded or ineffective. It is suggested that this include a graded approach such that one or two degraded attributes might result in a letter requesting the licensee to evaluate and develop corrective action with subsequent NRC inspection of the effectiveness. For three or four degraded attributes, a more thorough focused inspection of the PI&R programs might be appropriate. More significant degradation could result in modifications to the inspection, assessment and enforcement programs to account for the fact that ROP assumptions concerning the effectiveness of the corrective action process is not effective.

f) Develop training for inspectors to improve the focus of biennial PI&R inspections on PI&R program performance assessments. Training should include: tools and methodology for inspection planning; techniques to develop sound assessments; and guidance to avoid diverting inspection time to performing detailed technical reviews which do not involve meeting the IP 71152 objectives.

4. ROP Enhancement Effort from 201415

• Recommended changes to the inspection process include clarification/expansion of inspection guidance and focusing the procedure on inspection of corrective action program implementation

• Recommended changes to the assessment process include improving procedural guidance on how to assess a licensees corrective action program and incorporating the outcome of the team inspection into the overall assessment of the plant...

• There appears to be some differences between the bases for the Problem Identification and Resolution inspection, as listed in IMC 0308, and the actual inspection procedure Following these recommendations, revisions were made to the procedure and IMC 0308 to attempt to align the two however this effort was minimal. As previously mentioned, one of the original objectives of the procedure, is to conduct reviews of precursors to events which occur relatively infrequently but have significant consequences. This objective was in the procedure until it was removed following the recommendation above. However, a deeper reading of the basis reveals the intent of that objective most likely is the concept that PI&R performance is seen as a precursor to declining licensee performance. This was discussed in SECY 00-0049.

A working group was established16 to address the significant comments regarding assessment and inspection. This group however was ultimately unsuccessful in addressing these recommendations due to a lack of interest by industry and senior NRC staff and competing priorities.

15 Reactor Oversight Process Enhancement Project - Baseline Inspection Program, ML14017A338 and Reactor Oversight Process Enhancement Project Baseline Inspection Program Enclosure 6, Inspection Area - Problem Identification and Resolution, ML14017A391.

16 Reactor Oversight Process Problem Identification and Resolution Working Group Charter - PHASE I, ML15290A004.

6

NRC Transformation Team Recommendations, ROP Enhancement and External Feedback The Executive Director for Operations established the Transformation Team in 2018 to identify potential transformational changes to the NRC's regulatory framework, culture, and infrastructure to further enhance effectiveness, efficiency, and agility in regulating novel technologies. The team provided the Commission the results of its review17 including 72 recommendations for improvements to the ROP, which were provided to NRR for consideration18. Subsequently, the Nuclear Energy Institute (NEI) submitted a letter to the Director of NRR to consolidate and prioritize NEl 's recommendations to enhance the ROP19.

All these recommendations were discussed in SECY 19-0067. Three substantial changes were recommended to the procedure, specifically making the team inspection performance based rather than a regularly performed (biennial) inspection.

ROP Transformation Recommendation #78:

Make frequency of Problem Identification & Resolution (PIR) Team Inspections performance based by using Big Data to better target inspection resources. This can be accomplished by moving PIR Team Inspections conducted under Inspection Procedure (IP) 71152 into the Reactor Oversight Process (ROP) from a biennial frequency to a as needed/performance based. Specifically, a PIR Team Inspection would be triggered when a 1) cross cutting them (first occurrence) is identified; or 2) a Greater-Than-Green Finding in which a PIR cross-cutting aspect is identified. Inspection historically not achieved the desired results as predicted by the ROP. Overall, results have not justified the level of effort. This has been expressed by industry and many within the NRC. Reduction in this area can be supported by the inspections done in reviewing licensees Corrective Action Program (CAP). Each IP in the ROP has a requirement to review CAP. IP 71152 also requires annual samples be performed (i.e. mini Team Inspection basically conducted by 1 person/sample), and semi-annual trend review which involves and in-depth review typically done by residents (but can be done regional inspectors if Team inspection frequency changes to allow for greater focus). This proposal would allow for better scheduling/planning so that the teams can get the right people for the job and not just who is available. It would also encourage the regions to work together to create diverse teams made up of inspectors from not just a single a region since inspection would be infrequent.

NEI Recommendation 1E:

Remove PI&R Inspection from Baseline Program: Change Inspection Procedure 71152, Problem Identification and Resolution,4 to an inspector follow-up or reactive procedure (described below) and remove it from the baseline inspection program. Resident Inspectors focus daily on the Corrective Action Program (CAP) by reviewing all new condition reports (CRs). Moreover, in most inspections, inspectors focus first on CAP entries. Thus, each inspection and inspector assesses the ability of the plant to find and fix its problems.

Focusing a separate inspection team solely on PI&R is redundant to the assessment of the PI&R function that occurs in almost every inspection. In addition, the cross-cutting issues element of the ROP captures trends in PI&R performance in every inspection.

17 SECY-18-0060, "Achieving Modern Risk-Informed Regulation," ML181QA186.

18 List of Recommendations for Reactor Oversight Process Enhancement Submitted to the Transformation Team, ML18292A594.

19 ROP Enhancement, ML18262A322.

7

NEI Recommendation 2B.4:

Revise IP 71152 to be Reactive: Change IP 71152, Problem Identification and Resolution, to an inspector follow-up or reactive procedure and remove it from the baseline inspection program. In just about every direct inspection, the focus is first on the corrective action program. Each inspector makes assessments of the ability of the plant to find and fix its problems; hence to focus a team on solely PI&R is redundant to almost every inspection.

The cross-cutting program monitors for trends in PI&R shortfalls.

SECY 19-0067 addressed these recommendations specifically by stating:

For recommendations 1E, 2B.4, and 78, the staff and stakeholders concluded that the fundamentals of the ROP were sound. As such, removal of IP 71152 from the baseline inspection program would constitute a significant change to the basis of the ROP that is not warranted at this time.

However, the staff explored other ways to increase efficiencies in implementation of the procedure, specifically shifting the team inspection from biennial to triennial and shifting the daily status review from the procedure to IMC 2515 Appendix D Plant Status. The SECY makes these recommendations but also states that a comprehensive review should be completed prior to significant changes in order to assure that the aggregate effect of changes is considered holistically.

Historical Inspection Data and OpE Studies The team reviewed historical inspection data utilizing the OpE Dashboard20 along with several operating experience studies of PI&R importance to gain more perspective of the historical performance of the procedure. Since 2002, when the team inspection was shifted to biennial, the procedure has been tracked under IP 71152 and IP 71152B (for the team inspection) in the Reactor Program System (RPS). IP 71152, which encompasses the annual and semiannual samples, has accounted for 1,968 inspection findings since 2001, placing it second to the IP 71153, Follow-up of Events and Notices of Enforcement Discretion21. IP 71152B, the team inspection, has the sixth most findings by inspection procedure, with 793. When combined, the procedure accounts for the most findings by any ROP baseline, supplemental or infrequently performed inspection. Of those, 26 findings were greater than Green according to RPS. While not necessarily a complete measure of effectiveness, this does communicate the importance of the procedure as an important oversight tool used to identify potentially significant issues of concern and as a means of collecting oversight data (inspection findings and observations). In 2011, the OpE staff internally published an OpE COMM22 along with an analysis23 of greater than Green findings that found that 90% of significant events from 2006 to 2012 included weaknesses in the corrective action program as a contributing factor. The discussion above justifies the SECY 19-0067 recommendation for a comprehensive review prior to any substantial changes to the procedure.

20 https://www.internal.nrc.gov/opE-Dashboards/FindingsDashboard.html 21 IP 71153, Follow-up of Events and Notices of Enforcement Discretion, ML19197A110.

22 OpE COMM: Contribution of Corrective Action Program Deficiencies to Greater-than-Green Inspection Findings, ML13154A209.

23 IFR 2012-010, STAFF EVALUATION REGARDING ISSUE FOR RESOLUTION (IFR) 2012-010:

Operating Experience Analysis of Significant Events 2006-2012 8

An informal OpE analysis of reports from 2011 to 2013 showed that teams utilized nine different undefined terms to describe licensees corrective action programs. These terms were only loosely tied to the number of findings or observations of the licensees PI&R performance.

Number of Sites with Given PI&R Assessment 35 30 25 20 15 10 5

0 PI&R Reports analyzed between 2011 and 2013 9

This inconsistency issue persisted until it was corrected by an unintended action. In 2016 the staff issued a revision to IMC 0611 Exhibit 4 Inspection Report Cover Letter Templates, including a new template for PI&R report cover letters with the statement the team determined that your staffs performance in each of these areas supported nuclear safety. As a result, of 38 reports reviewed from 2016 to 2019, 35 contained this or a similar statement in the cover letter.

Conclusion Based on the importance of PI&R inspections as an ROP foundational activity to (1) inform the staff on a licensees ability to identify, prioritize, evaluate and correct plant problems and (2) the extensive amount of internal and external feedback on the effectiveness of these inspections, an understanding of the basis and history of the PI&R inspection program is an essential element of a comprehensive review of the PI&R program.

END OF DOCUMENT 10