ML22321A104

Lessons Learned Recommendations Effectiveness Review

ML22321A104
Person / Time
Site:	Pilgrim
Issue date:	12/01/2022
From:	Philip Mckenna NRC/NRR/DRO/IRAB
To:	Chris Miller NRC/NRR/DRO
References
Download: ML22321A104 (8)

Text

C. Miller UNITED STATES NUCLEAR REGULATORY COMMISSION WASHINGTON, D.C. 20555-0001 November 29, 2022 MEMORANDUM TO: Christopher G. Miller, Director Division of Reactor Oversight Office of Nuclear Reactor Regulation Signed by McKenna, Philip FROM: Philip McKenna, Chief on 11/29/22 Reactor Assessment Branch Division of Reactor Oversight

SUBJECT:

RESULTS OF THE CALENDAR YEAR 2022 REACTOR OVERSIGHT PROCESS SELF-ASSESSMENT EFFECTIVENESS REVIEW OF THE PILGRIM 95003 LESSONS LEARNED REPORT RECOMMENDATIONS

SUMMARY

This memo presents results from the calendar year (CY) 2022 Reactor Oversight Process (ROP) self-assessment effectiveness review of the actions taken to address the recommendations from the Pilgrim Inspection Procedure (IP) 95003 Lessons Learned Report, dated June 7, 2018 (Agency Documents Access and Management System (ADAMS) Accession No. ML18158A104). The staff reviewed the actions associated with each recommendation, including any supporting data, to determine whether the ROP change was effective as implemented, and whether there were any unintended consequences from the change. As a result of the staffs analysis of the ROP program execution data, as well as other insights, the staff determined that the actions taken to address closure documentation for Licensee Event Reports (LERs) have been effective. There are some specific program recommendations to provide clarity that remain open.

BACKGROUND:

Annual effectiveness reviews were added to the ROP self-assessment program as part of the November 23, 2015, revision to IMC 0307, Reactor Oversight Process Self-Assessment Program. In the 2019 revision to IMC 0307, the effectiveness reviews were revised to focus on data-driven analysis and established standardized criteria for when an effectiveness review is needed. Effectiveness reviews are used to assess recently implemented significant ROP changes to evaluate their effectiveness to ensure that the intended results have been realized and to evaluate any unintended consequences.

CONTACT: Daniel J. Merzke, NRR/DRO/IRAB 678-602-7986

C. Miller DISCUSSION:

The staff reviewed the subject report to understand the bases for each of the recommendations.

The recommendations were:

1. Region I should consider an assessment/extent of condition review focused on Licensee Event Report (LER) closures for similar implementation deficiencies in documenting self-revealing performance issues.

2. Region I should consider training with regards to required disposition of performance deficiencies, to include a focus on performance deficiencies (self-revealing and licensee-identified) described in LER reports.

3. Region I should share the assessment and corrective actions with other regional and program offices, as applicable.

4. Region I should consider whether guidance changes in IMC 0305, Operating Reactor Assessment Program, or regional specific guidance (regarding quarterly and end-of-cycle reviews) is needed to ensure problem identification and resolution (PI&R) sample purpose/assessment for degraded plant performance is well documented.

5. NRR consider whether clarity is needed in IMC 0309, Reactive Inspection Decision Basis for Reactors, guidance to ensure a consistent application and understanding of the deterministic criteria with a particular focus on loss of a safety function.

6. NRR/DRO consider Column 4 plant experiences with cross-cutting issues program during its 2019 effectiveness review.

7. NRR/DRO consider improved guidance to better provide a consistent approach and conduct of these reviews. In particular, that guidance should consider defining key aspects like (a) scope of effort, (b) independence aspects of the team composition, and (c) resource expectations to achieve the objectives.

Recommendation 1 - Region I should consider an assessment/extent of condition review focused on LER closures for similar implementation deficiencies in documenting self-revealing performance issues.

Basis - The regional review identified that LERs in the 2011-2013 timeframe may not have been consistently evaluated and findings and violations may not have been documented consistent with ROP and enforcement guidance. Specifically, many of the issues described within the LERs appear to have been more appropriately evaluated as self-revealing issues with performance deficiencies, requiring a greater level of detail within the LER closeout inspection report descriptions.

Action - A regional Senior Reactor Analyst (SRA) completed an extent of condition review of 37 LERs from 2007 through 2014, at which time Pilgrim transitioned to Column 4 of the Action Matrix, to determine if any issues may have been greater than very low safety significance and not adequately evaluated consistent with ROP and enforcement guidance. This review was documented in a memo from F. Arner to J. Yerokun, dated June 6, 2018 (ML18158A130), and did not identify any LERs with clear performance deficiencies which would have exceeded the threshold of a very low safety significant issue.

Status - CLOSED Effectiveness - The action taken to address this recommendation was effective. The extent of condition review was thorough and conducted by a highly competent SRA. There was an

C. Miller element of uncertainty in the review as the analyst had to make assumptions of what some performance deficiencies may have existed based on the description provided in each LER without actually inspecting each issue to determine if a performance deficiency existed. Six detailed risk evaluations (DREs) were performed for issues that appeared to have potential performance deficiencies as described within the LER. In a few cases additional DREs were performed for LERs even if the analyst did not identify the potential for a performance deficiency within the LER. This was performed as a sensitivity study for what the safety significance of the issue may have been if a performance deficiency had been established.

Recommendation 2 - Region I should consider training with regards to required disposition of performance deficiencies, to include a focus on performance deficiencies (self-revealing and licensee-identified) described in LER reports.

Basis - Same basis as Recommendation 1.

Action - Training on documenting closure of LERs was provided to Region I staff in December 2018 in a presentation titled 2018 Year-End Review - Reactor Program Related Changes &

Initiatives. The training included reminders to use IMC 0611, Power Reactor Inspection Reports, dated January 7, 2020 (ML19317F647), to document findings and violations, even if of minor significance; document in the Scope section if there is or isnt a violation or performance deficiency with a basis; document when the review is for a revised LER; and to document if a violation or performance deficiency associated with the LER was previously documented. The Reactor Program System (RPS) was revised to include the appropriate language and considerations within the template for documenting LER samples reviewed.

Status - CLOSED Effectiveness - The actions to address this recommendation were effective. The staff reviewed 12 recent LER closures for Region I licensees and all 12 were documented in accordance with program guidance. Revising the RPS template for documenting LER reviews so that the appropriate closure text is automatically generated with the inspection report compels inspectors to document these samples in accordance with IMC 0611 guidance.

Recommendation 3 - Region I should share the assessment and corrective actions with other regional and program offices, as applicable.

Basis - This recommendation refers to the assessment of LER closures and appropriately documenting findings and violations resulting from LERs in accordance with program guidance.

The issues identified by the review team may be applicable to the other regions, so all regions should consider if inspection staff requires additional training or not.

Action - The assessment report and extent of condition review were shared with all Regional Administrators. This issue was a focus item during the Region III annual peer review conducted in September 2018 as part of the 2018 ROP self-assessment (ML19060A024). The findings and conclusions were shared with all regions.

Status - CLOSED Effectiveness - The actions to address this recommendation were effective in that all regions were made aware of the LER closure issues identified during the Pilgrim inspection procedure (IP) 95003 inspection, Supplemental Inspection Response to Action Matrix Column 4

C. Miller (Multiple/Repetitive Degraded Cornerstone) Inputs. The results of the Region III peer review indicated that the issue was not specific to Pilgrim or Region I. There were LER closures identified during the Region III peer review where closure without a finding or violation did not seem supported. The Region III review documented recommended changes to inspection report guidance similar to the recommendations from the Pilgrim 95003 review, e.g., enhance guidance in IMC 0611 to better clarify what needs to be documented when reviewing and closing out LERs. Specifically, require an explicit statement that no performance deficiency or finding/violation was identified when applicable. The Region IV response to the issues identified during the Region III peer review were documented in a memo, Region IV Evaluation in Response to Calendar Year 2018 Reactor Oversight Process Self-Assessment Regional Peer Review of Region III, dated March 28, 2019 (ML19088A135), indicating similar issues regarding LER closure documentation.

Recommendation 4 - Region I should consider whether guidance changes in IMC 0305 or regional specific guidance (regarding quarterly and end-of-cycle reviews) is needed to ensure problem identification and resolution (PI&R) sample purpose/assessment for degraded plant performance is well documented.

Basis - The documentation of PI&R sample reviews could have been improved to provide a more focused assessment of degraded performance in the corrective action program (CAP) area. The project branch and management team consistently described in-depth CAP insights in performance summary packages for mid-year and end-of-cycle assessments that identified weaknesses in CAP effectiveness; however, the supporting documentation from applicable PI&R samples in the inspection reports had not consistently matched the assessment.

Inspection insights and assessment of CAP performance during PI&R samples, most notably during semi-annual trend reviews, could have been better.

Action - Region I initiated feedback forms 0305-2342; 0611-2343; and 71152-2344 to provide recommendations to add additional guidance to IMC 0305, IMC 0611, Power Reactor Inspection Reports, and IP 71152, Problem Identification and Resolution. The revision to IMC 0305 is specific to Exhibit 3, Plant Performance Summary, recommending it be revised to state that the staff should summarize the conclusions from the last biennial PI&R inspection report, any PI&R inspection activities for following-up on cross-cutting issues, and insights from semi-annual trend review. This input may factor into the regions confidence in the licensees scope of efforts, or progress in addressing any cross-cutting themes, or if any adjustments are needed to future inspection plans. IMC 0305 Exhibit 3 was revised to include the recommended verbiage, and the feedback form closed.

Feedback form 71152-2344 was initiated in 2018 to add additional guidance to the PI&R semi-annual trend review, stating, This could include a review of findings and or events over a period of time which looks for any common root and contributing causes between the findings/events.

To address this issue and to gather more data, the topic of Problem Identification and Resolution sample inspection report write-ups (Semi-Annual Trend Review and Annual Follow-up of Selected Issues), was added as a focus area in the September 2022 Region I ROP Implementation Audit as part of the ROP self-assessment program. In the audit report, dated November 8, 2022 (ML22285A231), the audit team recommended, To address identified inconsistencies in documentation between Region I branches and between Region I and the other regions, revise and clarify IP 71152 guidance on documenting annual and semi-annual sample observations and assessments. To address this recommendation, the staff will add this guidance in the next revision of IP 71152, which is currently being drafted.

C. Miller Region I recommended adding additional guidance to IMC 0611 Section 0611-12 to include a statement, For PI&R - Annual Followup of Selected Samples, the basis for the selection and scope of review should be documented as part of the observations. The staff closed feedback form 0611-2343 to be tracked by feedback form 0611AppD-1616. In its closure, the IMC lead stated:

Documentation of the basis for sample selection is not something that needs to be documented nor justified in the report. The scope of review is already required to be documented for all samples. The inspectors will need to write more when the scope of inspection goes beyond the generalities list in the IP and the value of the information is important enough to warrant preservation. The specific guidance pertaining to the documentation of observations is not controlled by IMC 0611. Specific guidance related to observations is to be included in the IP and not in IMC 0611 proper. IMC 0611 states that observations can only be documented when specifically allowed by an inspection procedure or temporary instruction. IMC 0612 App D, Guidance for Problem Identification and Resolution Inspection Reports, (ML17130A997) should be revised regarding documentation guidance.

There is already an outstanding feedback form related to improving the semi-annual review - 0611AppD-1616. This feedback form can address the need for improved documentation guidance.

Feedback form 0611AppD-1616 was initiated in 2010 and remains open. There is relatively wide latitude for documenting PI&R samples. The necessary revision of IMC 0611 depends on implementing long-standing recommendations to revise IP 71152. There are differing views of how to revise IP 71152. Addressing these items is planned for action in the next revision of IP 71152 that is currently being drafted.

Status - OPEN Effectiveness - N/A Recommendation 5 - NRR consider whether clarity is needed in IMC 0309 guidance to ensure a consistent application and understanding of the deterministic criteria with a particular focus on loss of a safety function.

Basis - There is some judgment allowed by IMC 0309 to be applied by NRC decision-makers in response to these events, including whether a reactive inspection is warranted if the deterministic criteria and risk components are met. However, the evaluation team also determined there are differing perspectives in how staff and managers interpret the deterministic questions in IMC 0309 that may have a significant impact on the decision-making process. The evaluation team found that even amongst the most senior inspectors and risk analysts involved, there were differing perspectives about this criterion specific to whether the loss of offsite power constituted a loss of safety function as it relates to this process. The differing perspectives seem to be derived from the various understandings of what loss of safety function could mean in this process based upon review of different NRC guidance documents, such as NUREG-1022, Event Reporting Guidelines:10 CFR 50.72 and 50.73, dated January 2013 (ML13032A220) and applicable PRA guidance.

C. Miller Action - Feedback Form 0309-2316 was generated to track this recommendation. This feedback form remains open. The staff is developing a revision to IMC 0309 in conjunction with a revision to Management Directive 8.3, NRC Incident Investigation Program, dated June 24, 2014 (ML13175A294). Currently MD 8.3 and IMC 0309 are not aligned with respect to the launch criteria for reactive inspections, noting that MD 8.3 is the controlling document. The staff has developed clarifying criteria in consultation with the Senior Reactor Analyst (SRA) community and plans to issue those criteria into a future IMC 0309 revision that aligns with the needed changes to MD 8.3. NRR is currently waiting on NSIR for moving recommended MD 8.3 changes forward which will enable the IMC 0309 revision. While waiting for the MD 8.3 revision, NRR plans on issuing a minor revision to IMC 0309 to address items that can be revised without revising MD 8.3.

Status - OPEN Effectiveness - N/A Recommendation 6 - NRR/DRO consider Column 4 plant experiences with cross-cutting issues program during its 2019 effectiveness review.

Basis - The review team concluded that the cross-cutting issues program was appropriately implemented by Region I. The team attempted to evaluate the overall effectiveness of the cross-cutting issues program, considering similar observations documented in prior IP 95003 reviews conducted in other regions. However, because the review was limited to application of the program at Pilgrim, the team could not provide an objective assessment of the entire program.

Action - The staff considered the experiences of Column 4 plants during the cross-cutting issues (CCI) effectiveness review in 2019, as documented in a memo titled Results of the Reactor Oversight Process Self-Assessment Effectiveness Review of the Crosscutting Issues Program, dated September 21, 2020 (ML20239A835). The review team concluded that the CCI program continues to have value by providing a focus on patterns of safety culture behaviors, but recommended program changes to address issues of responsiveness resulting from the 2015 changes. The review team provided several options and recommendations to revise the CCI program. Two of the recommended changes were accepted for implementation, which were intended to enhance clarity identifying and closing cross-cutting issues. The program itself was substantially unchanged.

Status - CLOSED Effectiveness -The action taken was responsive to the recommendation. Since there have been no CCIs issued since the review was completed, the effectiveness of the enhancements to the CCI program is indeterminate.

Recommendation 7 - NRR/DRO consider improved guidance to better provide a consistent approach and conduct of these reviews. In particular, that guidance should consider defining key aspects like (a) scope of effort, (b) independence aspects of the team composition, and (c) resource expectations to achieve the objectives.

Basis - There is limited guidance available regarding the conduct of the IP 95003 review which has led to different approaches and scope of effort in completing them.

C. Miller Action - According to the Lessons Learned Tracker, IP 95003 was revised on 06/07/2022, and guidance directly associated with scope of effort, team composition, and resource expectations were enhanced. The published revision of IP 95003 stated, All team members should be involved in the assessment of their subject areas. Overall level of effort should be commensurate with the teams findings and the licensees underlying performance issues. The staff decided to leave the guidance generic because each lessons learned review is unique and has unique considerations. The decision to keep the guidance at a high level was made in consultation with the staff in Region I. This provides some flexibility to the staff, and there are several lessons learned reports that may be used as templates for future reviews.

Status - CLOSED Effectiveness - N/A; there have been no further 95003 inspections to conclude the effectiveness of this action.

CONCLUSION AND RECOMMENDATIONS:

As a result of the staffs analysis of the ROP program execution data, as well as other insights, the staff determined that the actions taken to address closure documentation for Licensee Event Reports (LERs) have been effective. The RPS pre-populated template text entered into the ISTAR (Inspection Scheduling, Tracking, and Reporting) program for generating inspection reports has been effective in ensuring LER closures are documented in accordance with program guidance. There were no unintended consequences identified during this review. This corrective action improved the efficiency, clarity, and reliability Principles of Good Regulation in that a basis is required to be documented when there are no findings or violations identified during LER closure, all LER closures are documented consistent with the program guidance, and the ghost text makes data entry quicker and more efficient. There are still some recommendations that remain open, and it is recommended that actions be taken as soon as possible to address those recommendations.

C. Miller

SUBJECT:

RESULTS OF THE CALENDAR YEAR 2022 REACTOR OVERSIGHT PROCESS SELF-ASSESSMENT EFFECTIVENESS REVIEWS ON NOVEMBER 29, 2022 DISTRIBUTION:

RidsNrrDro RidsNrrDroIrab RidsNrrDroIrib RidsNrrOd RidsRgnIVMailCenter RidsRgnIMailCenter RidsRgnIIMailCenter RidsRgnIIIMailCenter CMiller THipschman PMcKenna RFelts ADAMS ACCESSION NO: ML22321A104 *concurred via email OFFICE NRR/DRO/IRAB NRR/DRO/IRAB/BC NAME DMerzke PMcKenna DATE 11/17/22 11/29/22 OFFICIAL RECORD COPY