Text

Team Report for the Comprehensive Review of the Problem Identification and Resolution Inspection Program
	ML20247J602
Person / Time
Issue date:	11/12/2020
From:	Russell Gibbs; NRC/NRR/DRO/IRSB
To:	;
	Russell Gibbs, NRR/DRO/IRSB, 415-8578
Shared Package
ML20247J590	List: ML20247J599; ML20247J601; ML20247J602; ML20262G982; ML20274A133;
References
	Download: ML20247J602 (33)
	v • d • e

Team Report of the Reactor Oversight Comprehensive Review of the Problem Identification and Resolution Inspection Program1

Background

In August 2019 a multi-disciplinary team of staff from multiple agency offices was assigned to conduct a comprehensive review of the Problem Identification and Resolution (PI&R) inspection program. The review focused on two main areas: (1) procedure guidance and implementation; and (2) the Nuclear Regulatory Commissions (NRC) overall assessment of a licensees PI&R program. The team charter is available in Agencywide Documents Access and Management System (ADAMS) Accession No. ML19212A017. The team was comprised of the following individuals:

Shakur Walker SES Sponsor Jeffrey Bream Region I Thomas Hipschman NRR Sherlyn Haney Region I Zachary Hollcraft NRR Andy Rosebrook Region II Russell Gibbs NRR Ryan Taylor Region II Molly Keefe-Forsyth NRR Dan Tesar Region III Eric Thomas NRR Ami Agrawal Region IV Ross Telson NRR Jeremy Groom Region IV There was also a director-level Regional Advisory Panel comprised of the following individuals:

Jimi Yerokun Region I Steve Cochrum Region II Mohammed Shuaibi Region III Tony Vegel Region IV The history of the PI&R inspection program and the bases behind program changes over time is discussed in a separate document developed to accompany this report, available in ADAMS Accession No. ML20247J590. The PI&R inspection is used by NRC to evaluate a licensees processes and implementation for PI&R. Resident inspectors accomplish part of this inspection task through routine review of PI&R activities, semi-annual trend reviews, and an annual follow-up of selected issues. In addition, a regional inspection team evaluates each licensees PI&R program on a biennial basis.

Licensee procedures for implementing PI&R activities reflect the requirements of their Quality Assurance Programs, which are subject to NRC review. In addition, these programs require 1

Several differing perspectives are provided for the overall report to include: 1) Recommend removing references to baseline inspection hour reductions since it appears to message that the pupose of the review was to reduce the inspection program scope vice enhance the procedure itself; 2) Eliminate equating effectiveness of the procedure to the number of inspection findings since the number of findings does not address whether the procedure was effective because the PI&R inspection program tends to have many issues deferred to the team inspection; 3) Certain assessment recommendations, if adopted, may give a false impression that for example that an unsatisfactory performance area does not mean an overall ineffective licensee PI&R program; and 4) Time permitting, the paper would benefit by providing greater clarity and detail on what each of the options entail and basis/rationale for the teams recommendations.

1

adherence to Title 10 of the Code of Federal Regulations Part 50, Appendix B, and, Criterion XVI, Corrective Actions. The NRCs policy for giving licensees credit for use of the corrective action program (CAP) to resolve low-level inspection findings as non-cited violations is discussed in SECY 99-07A,2 as well as the NRC Enforcement Manual and Reactor Oversight Process (ROP) documents. This credit is contingent upon the licensees ability to effectively identify, evaluate, and correct plant problems.

NRC inspection reports for a biennial PI&R inspection normally include a statement regarding the inspectors overall assessment of the licensees CAP. During past ROP assessment cycles, these reports used language such as adequate, generally identified issues at an appropriately low threshold, and generally effective, which have been identified by internal and external stakeholders as ambiguous and undefined in ROP program documents.

Inspection procedure (IP) 71152, Problem Identification and Resolution (hereinafter referred to as the procedure), was introduced to establish confidence that each licensee is detecting and correcting problems in a manner that limits the risk to members of the public by inspection across all ROP cornerstones. Both the staff and the Commission have affirmed the importance of licensee PI&R performance to the ROP.

This comprehensive review determined that the procedure remains an effective oversight tool to assess the acceptability of licensee actions to identify, prioritize, evaluate and correct plant problems. However, aspects of the PI&R program could be strengthened to reduce subjectivity when assessing overall licensee performance in the areas of PI&R . Therefore, the staff identified several enhancements, as discussed in this report, that could improve the overall effectiveness of the PI&R inspection program.

Effectiveness Criteria The basis of the PI&R inspection program is stated in the procedure as:

A fundamental goal of the NRCs ROP is to establish confidence that each licensee is effectively detecting, correcting, and preventing problems which could impact cornerstone objectives. A key premise of the ROP is that weaknesses in licensees PI&R programs will manifest themselves as performance issues which will be identified during the baseline inspection program or performance indicators (PIs) crossing predetermined thresholds.

This goal is comprised of several key components that the team assessed to determine whether the objective of the program is being met by conducting:

1. routine review of licensees implementation of their corrective action program;

2. semi-annual reviews;

3. annual reviews; and

4. biennial reviews.

2 SECY-99-0007A, Recommendations for Reactor Oversight Process Improvements (Follow-up to SECY-99-007), dated March 22, 1999.

2

In addition, SECY-99-0073 discusses the importance of applying the NRCs Principles of Good Regulation to the ROP to instill confidence that nuclear power facilities are regulated in a manner that meets the NRCs mission. These principles can also be applied to assess the effectiveness of the PI&R inspection program.

Independence Openness Efficiency Clarity Reliability In assessing the effectiveness of the PI&R inspection program, the team considered the stated objective of the program, the team charter objectives, the basis and history of the program, internal and external feedback, and the NRC Principles of Good Regulation.

Information Reviewed Consistent with the team charter, the team reviewed the procedure to ensure there is:

(1) consistency in execution and documentation across the regions; (2) proper consideration of risk insights; (3) clear criteria for inspectors and regional management when assessing the overall effectiveness of a licensees PI&R program; and (4) a reliable or repeatable process.

In doing so, the team assessed the procedures structure, resource allocation, and inspection sample requirements. The team also reviewed baseline and supplemental inspection reports, previous PI&R assessments, operating experience studies, SECY papers, Staff Requirements Memorandums, ROP feedback forms, and IP 95003 lessons learned reports.

A number of internal recommendations were provided by staff as a result of lessons learned activities performed for plants that have entered Column 4 of the NRCs ROP Action Matrix as directed by IP 95003.4 These recommendations mostly focused on improving the procedure to enable a consistent and more effective approach to assess the overall effectiveness of the licensees PI&R program. Rather than holistically assessing the licensees PI&R program, the procedure includes discrete inspection samples and directs inspectors to draw conclusions on the licensees PI&R performance from these samples. Through recommendations made from IP 95003 lessons learned, Regional staff identified the need to improve the guidance regarding assessment of licensee PI&R performance and establish agency actions for declining licensee performance.

The team also reviewed feedback forms related to the procedure. Of the 28 currently open ROP feedback forms, 57 percent are directly associated with procedure guidance or licensee assessment. An additional 32 percent are associated with procedure structure and inspection requirements. The staffs feedback in these forms focused on enhancing sample selection, standardizing implementation across the regions, and providing clearer guidance for assessment of licensee PI&R performance. The proposed adjudication of open feedback forms is provided in ADAMS Accession No. ML20247J601.

The team explored the following documents to design a new approach for the assessment aspect of the PI&R program with defined criteria, performance tiers, and agency actions:

3 SECY-99-0007, Recommendations for Reactor Oversight Process Improvements, dated January 8, 1999.

4 IP 95003, Supplemental Inspection for Repetitive Degraded Cornerstones, Multiple Degraded Cornerstones, Multiple Yellow Inputs or One Red Input, dated December 18, 2015.

3

NUREG-2165, Safety Culture Common Language

Institute of Nuclear Power Operations12-012, Traits of a Healthy Nuclear Safety Culture

IMC 0310, Aspects Within Cross-Cutting Areas

MD 5.6, Integrated Materials Performance Evaluation Program Additionally, in 2018 the Nuclear Energy Institute (NEI) submitted several recommendations to enhance the procedure (ADAMS Accession No. ML18262A322). These recommendations were made as part of the NRCs transformation initiative. NEI provided feedback that the biennial team inspection is redundant to other NRC inspection and oversight activities (e.g., daily inspector review of licensee condition reports, semi-annual trend reviews, and the annual review of select PI&R issues). NEI recommended that the procedure be removed from the ROP baseline inspection program and be performed as an inspector follow-up or reactive inspection.

The NRC staff did not accept these recommendations. NRCs disposition of the NEI recommendations can be found in ADAMS Accession No. ML19101A334.

As discussed, the teams review of information included analysis of both the procedure and the assessment aspects of the program. Based on a holistic review of feedback received from both internal and external stakeholders, the team concluded that multiple enhancements could be made to the procedure.

Data Reviewed The relatively high number of inspection findings identified using the procedure compared to other ROP baseline IPs suggests that the procedure is effective and that any significant changes to the procedure should be considered carefully. As such, the options discussed in the Program Recommendations section of this report do not substantially alter the inspection requirements but attempt to clarify guidance and capitalize on the risk-informed nature of samples to improve their capability. Small reductions in inspection hours could be made to the estimated levels of effort through this process.

Total ROP Findings by Procedure since 2002 (OpE Dashboard data) 4

The teams comprehensive review of the inspection objectives and data did not produce data that supports or refutes shifting the team inspection from biennial to triennial. That is, reducing the periodicity of the team inspection increases the risk of missing a significant issue between inspections. Moreover, the team did not produce data that contradicts the conclusion made in SECY-19-0067, Recommendations for Enhancing the Reactor Oversight Process, that states, The staff has concluded that there are a sufficient number of touchpoints throughout the annual inspection cycle that assess the health of the licensee's CAP performance to justify changing to a triennial frequency now to reduce redundancies and improve efficiency for evaluating CAPs.

Program Recommendations The options identified by the team show a range of potential changes with varying flexibility of resource allocation and risk-informed sample selection. The team made the following general observations based on the information reviewed:

Although the PI&R procedure is an overall effective inspection tool, improvements to the assessment process are needed to improve objectivity, reliability, and repeatability.

Clearer inspection guidance is needed to improve the consistency of implementation of the PI&R inspection program.

Small inspection samples and inspection area adjustments should be considered to improve the risk-informed and performance-based aspects of the program.

The team examined several ideas when considering possible enhancements to the PI&R program to improve its overall effectiveness. Options considered by the team for the procedure ranged from making small changes to the procedure to completely revamping the procedure to 5

give it increased flexibility based on inspector feedback. For assessment, the options considered vary from providing a highly prescriptive and objective approach to one that incrementally improves the current assessment process.

The team recommends that the following changes be made irrespective of which option is selected. These changes collectively would improve guidance and structure, assign inspection hours to a more appropriate inspection procedures and provide more flexibility for Safety Conscious Work Environment (SCWE) inspections. The following recommendations would not require Commission approval or notification:

Update the procedure to conform to IMC 0040, Preparing, Revising, and Issuing Documents for the NRC Inspection Manual. Further, modify the structure of the procedure (based on the option approved) to either a single procedure with multiple sub-procedures or a base procedure with activity-specific attachments. This change would improve consistency across all the regional offices.

As discussed in SECY-19-0067, relocate the daily PI&R screening from the procedure to plant status activities defined in IMC 2515, Appendix D, Plant Status. This change would result in a more appropriate application of inspection activities to baseline IPs to inspect issues identified during PI&R screening. The change would shift approximately 129 to 175 of estimated annual inspection hours back to Appendix D. This change is not dependent on Commission decision on SECY-19-0067.

Update SCWE inspection guidance and inspector training to clarify expectations on how to assess SCWE in a graded approach using the existing list of interview questions (Attachment 4 of the procedure) and guidance in IP 93100, Safety Conscious Work Environment Issue Follow-up depending upon the level of NRC concern with the licensees SCWE.

In evaluating the proposed options presented below, the team considered the information reviewed, conclusions reached, NRC Principles of Good Regulation, and principles of the ROP.

The level of effort and complexity involved with implementation of these options varies along with the level of impact each option on its own would have. The options proposed in this report are conceptual and would require further development and stakeholder interaction for implementation. Specifically, Procedure Option 3 and Assessment Options 1 and 2 would require extensive internal and external stakeholder interaction before implemented. More detail about the dispositioning of all options considered is available in Appendices A and B to this report.

An abbreviated list of all options considered is shown in the following Tables 1 and 2.

6

Table 1: Procedure Enhancement Options Considered by the Team for PI&R Inspection Program Improvements Procedure Option 1 - Current Inspection Requirements with Small Enhancements

This Option addresses inspector feedback regarding the need for more clear guidance to enhance consistency.

This Option redistributes inspection resources in a more risk-informed and performance-based manner (as discussed in more detail in Appendix A to this report).

This Option is dependent on Commission direction on SECY-19-0067 for frequency of team inspection (triennial vs. biennial).

This Option would allow for a 15 - 17 percent reduction in team inspection hours (dependent on triennial vs.

biennial).

This Option would require minimal NRC staff effort overall to modify the program with routine industry interaction.

No Commission approval or notification is needed to implement this Option.

Procedure Option 2 - Susbstantive Changes to Inspection Sample Requirements

This Option addresses inspector feedback regarding the need for more clear guidance to enhance consistency.

This Option redistributes inspection resources in a more risk-informed and performance-based manner (as discussed in more detail in Appendix A to this report).

This Option is dependent on Commission direction on SECY-19-0067 for frequency of team inspection (triennial vs. biennial) separated into two smaller teams - one biennial; one triennial

This Option would allow for a 26 percent reduction in inspection hours.

This Option would require moderate NRC staff effort overall to modify the program (increase due to split of team inspection) with extensive industry interaction.

Commission approval would be needed to implement this Option.

Procedure Option 3 - Redistribution of Inspection Samples and Added Flexibility

This Option provides the following: improved PI&R inspection and assessment efficiency and effectiveness; increased reliance on documented and readily retrievable fact-based PI&R performance insights; enhanced billing resolution and data analytics.

This Option eliminates the current PI&R team inspection and provides increased flexibility by redistributing samples across various inspection modules (as discussed in more detail in Appendix A to this report).

This Option relocates licensee PI&R program assessment from the procedure to Inspection Manual Chapter (IMC) 0305, Operating Reactor Assessment Program elements of current team inspection performed for more risk-significant issues.

This Option would allow for 9 percent reduction in inspection hours.

This Option would require significant NRC staff effort overall to modify the program with extensive industry interaction.

Commission approval would be needed to implement this Option. Not recommended for short-term implementation - potential need for pilot.

The team recommends Procedure Option 1, Current Inspection Requirements with Small Enhancements, because it provides the most effective and efficient enhancement to the program.

7

Table 2: Assessment Enhancement Options Considered by the Team for PI&R Inspection Program Improvements5 Assessment Option 1 - New Assessment Tool Using Performance Areas and Criteria

This Option addresses inspector feedback regarding the need for guidance to assess the effectiveness of a licensees PI&R program.

This Option provides three assessment outcomes - satisfactory, satisfactory but needs improvement, unsatisfactory

- each outcome contains specific criteria for assessment which allows for improved objectivity.

This Option would require significant NRC staff resources to implement. The resources for licensees would remain neutral.

This Option would require significant NRC staff effort overall to modify the program with extensive industry interaction.

Commission approval is needed to implement this Option - potential need for pilot.

Assessment Option 2 - Binary Assessment with Simple Thresholds

This Option addresses inspector feedback regarding the need for guidance to assess the effectiveness of a licensees PI&R program - this includes binary (go/no go) assessment criteria and potential for inspection follow-up.

This Option must be coordinated with improvements to the CCI program as determined by the recent CCI effectiveness review (ML20239A806).

This Option provides objective criteria provided for assessment:

*more than two Greater-Than-Green inspection findings in a 24-month period

*PI&R CCI assigned

*SCWE CCI assigned

*supplemental inspection identifies and documents a significant weakness in PI&R program which the licensee did not identify

d) This Option would require moderate NRC staff resources to implement. The resources for licensees would remain neutral.

e) This Option would require moderate NRC staff effort overall to modify the program with extensive industry interaction.

f) Commission approval is needed to implement this Option.

Assessment Option 3 - Qualitative Evaluation of PI&R Areas

This Option uses traditional areas for PI&R evaluation (e.g., identification, evaluation and correction of problems).

This Option requires development of new guidance for qualitative evaluation of each area.

This Option relocates licensee PI&R program assessment from the procedure to Inspection Manual Chapter (IMC) 0305.

This Option would require moderate NRC staff resources to implement. The resources for licensees would remain neutral.

This Option would require moderate NRC staff effort overall to modify the program with moderate industry interaction.

Commission approval is not needed to implement this Option. However, Commission notification is needed.

5 A differing perspective is that the summary table does not appear to provide a summary of what the options entail. The reader should have a clear idea of the options if they wanted to only read the report and not appendices. In addition, the resource estimates should be removed from the table and provided in the appendices where more explanation is provided.

8

The team6 recommends Assessment Option 27, Binary Assessment with Simple Thresholds, because it provides the most objective and repeatable enhancement to the program without being overly burdensome. However, this recommendation hinges on the outcome of the recent CCI effectiveness review and whether or not Recommendation 2.d8 is accepted.9 There were some team member views that Assessment Option 1, New Assessment Tool Using Performance Areas and Criteria, would be the better option.

Furthermore, the team determined that Procedure Option 3 and Assessment Option 3 likely would not enhance the program in a manner that sufficiently balances the Principles of Good Regulation and the principles of the ROP.10 Pending a Commission decision on SECY-19-0067, both triennial and biennial frequencies were factored into the development of these options. The team recommends an effectiveness review of any substantive procedure changes be completed after a full cycle (whether triennial or biennial) in accordance with IMC 0307, Reactor Oversight Process Self-Assessment Program.

The team applied the Be riskSMART framework - a framework that was developed as part of the Office of the Executive Director for Operations (OEDO) transformation initiative for accepting risk in decision-making.11 This framework supports the NRCs vision of being a modern, risk-informed regulator by enabling staff to accept well-managed risks in NRCs decision-making without compromising the agencys mission. Throughout this review, the Be riskSMART framework was applied to help guide the staffs assessment of the challenges (what can go wrong/right? what are the consequences? how likely is it?) associated with the options and how some of those challenges could be managed. This is discussed more in Appendices A and B to this report.

6 The term team in this sentence is referring to a majority of the working group. There were several working group members that preferred Assessment Option 1.

7 Several reviewers in one organization generally endorsed Assessment Option 1 or 2. Other reviewers endorsed Assessment Option 3. The consensus was that Option 2 was a reasonable path forward.

Another organization believes that going with Assessment Option 3 would provide the greatest flexibility to be able to incorporate performance based on risk informed considerations, more so than the other options that might be to too prescriptive to some situations and result in unintended consequences.

8 CCI effectiveness review, Recommendation 2.d. states: Eliminate the concept of a cross-cutting theme and the requirement that a theme be exceeded for three consecutive periods before a CCI is opened; open a CCI with limited subjective consideration once a threshold is reached. See the CCI team report for more details (ADAMS Accession No. ML2039A806).

9 A differing perspective is that this assessment approach leads to more inspection without a clear path to how it plays out in assessment. It is not evident that this is an assessment tool but rather a tool requiring more hours of PI&R inspection. If the go/no go criteria says no-go, then what does it say about a licensees program? Following the inspection generated by a no-go assessment, it would be important to understand how a licensee would get assessed at the end of the year. This option also relies on the CCI program, with a fair amount of resources being required to have the changes implemented in that program. The CCI resources are not included in the review of this option. It could appear that the NRC is double hitting a licensee such that they get regulatory response in two program areas, CCI and PI&R.

10 A differing perspective is that Procedure Option 3 is the only option to explicitly seek to move assessment and its application from multiple small biennial teams into a larger better-structured annual regional licensee performance assessment process.

11 More information on the OEDO transformation initiative and the Be riskSMART framework can be found at https://usnrc.sharepoint.com/teams/FutureNRC/SitePages/ACCEPT-RISK.aspx.

9

The team also ensured that all options developed continued to support the Principles of Good Regulation, albeit in varying degrees. In determining a recommended path forward, the team discovered that some options better reflected certain principles over others. Provided below is a high-level summation of how the recommendations made address the principles.

Independence: All the options proposed support this principle with NRC independence remaining intact and essentially unchanged.

Clarity: The team determined that Procedure Option 1 would provide the greatest enhancement to clarity. Much of the internal feedback received on the procedure addressed the need for both structure and guidance improvements. The team determined that this option would provide a more consistent implementation of the procedure. For assessment, the current program lacks clear guidance. Therefore, all the proposed assessment options improve clarity and would therefore provide a more consistent assessment process.

Openness: The current PI&R inspection program and its results are open to public and the procedure options proposed do not degrade this principle in any capacity. All the proposed options for assessment would likely enhance openness by providing improved guidance for NRCs assessment of a licensees PI&R program.

Reliability: The procedure options provide enhanced reliability through the use of clear guidance and structure, along with adjustments to inspection samples and inspection areas.

The assessment options would also enhance reliability by providing specific guidance on the overall assessment of a licensees PI&R program. Assessment Options 1 and 2 provide the greatest enhancement for reliability.

Efficiency: Efficiency in the PI&R program would be enhanced through the application of risk insights for inspection sample and inspection area adjustments. In addition, there would be implementation efficiencies by improving the clarity of the procedure. Procedure Option 1 provides the greatest enhancement for efficiency. For assessment, Option 2 would be the most efficient by use of discrete criteria combined with a binary decision.

10

Table of Teams Review of the Options Consistency with the Principles of Good Regulation/ROP12 Principles of Good Procedure Procedure Procedure Assessment Assessment Assessment Regulation/ROP Option 1 Option 2 Option 3 Option 1 Option 2 Option 3 Independence + + + + + +

Clear/Understandable/Scrutable + + - + + -

Open/Transparent + + - + +* -

Reliable + + +/- + + -

Efficient + - +/- - +* +

Objective + + + + + -

Repeatable + + - + + -

Predictable + + - + +* -

Risk-informed + + + + + +

Performance-based + + + + + +

Reduces unnecessary regulatory N/A - +/- N/A N/A N/A burden Commission Approval or Neither Approval Approval Approval Approval Notification Notification NRC Staff Implementation Effort Minimum Moderate Significant Significant Moderate Moderate green shading = team recommended options

+ = consistent with Principles of Good Regulation/ROP

- = not consistent with Principles of Good Regulation/ROP

+/- = differing views on impacts to principles N/A = not applicable

= relies upon substantive changes to the CCI Program 12 A differing perspective for Assessment Option 2 is that moderate implementation effort is inconsistent with the effort it will take to get the CCI program to support and that it is not consistent with Commission approval effort. Additionally, this perspective believes it is not consistent with the level of effort to convince stakeholders (particularly industry) that more inspection in PI&R is needed before an assessment can be rendered and to agree with a method that would effectively imply that a licensees program is unsatisfactory because it failed to meet the go/no go criteria.

11

Procedure Review

1. Background

Consistent with the PI&R working group charter, the procedures subgroup reviewed the current structure of the baseline inspection procedure (IP 71152, or the procedure) to identify improvement opportunities to reduce unnecessary regulatory burden, allowing for more efficient and consistent use of agency resources while maintaining adequate levels of regulatory oversight. The primary goal of this review was to ensure that the structure of the procedure promoted consistent inspection that would ultimately support NRC assessment of licensee PI&R programs.

In performing this review, the procedure subgroup reviewed the basis documents for the procedure including Inspection Manual Chapter (IMC) 0308 and Attachment 2 to ensure the purpose of the inspection, including its relationship with the supplemental inspection procedures (IP 9500x series) and cross-cutting themes (IMC 0310) was well understood. As discussed in Section 2 of this Appendix, the team also evaluated options to better use risk insights and to integrate these insights/results into other inspections and the annual End of Cycle (EOC) assessment.

SECY 19-0067, Recommendations for Enhancing the Reactor Oversight Process, addressed NRC Transformation Initiative and industry recommendations regarding the procedure. The final recommendations the staff made for the procedure are as follows:

Transfer PI&R daily review and associated commitments, objectives, requirements, and resources to IMC 2515, Appendix D, Plant Status

Change frequency to triennial, reduction of 35-48 hours annually, and conduct a comprehensive review of the PI&R inspections The annual and semiannual samples from the procedure yield the second most findings for any procedure in the ROP. When combined with the team inspection (IP 71152B), the procedure is the highest contributor overall to ROP findings. While not the only measure of performance, the findings show that the procedure is risk-informed to identify more important licensee performance deficiencies. It is difficult to definitively conclude the reasons for this; however, some simple aspects of the procedure shed light on why:

The annual samples are a flexible means to inspect any issue that falls under any ROP cornerstone and requires entry into the licensees PI&R program.

The biennial inspection also yields a large number of findings, most likely due to the regional practice of holding certain potentially more significant issues, as appropriate for review by the team inspection combined with the in-depth review into these issues in the licensees PI&R programs.

The team developed three options for changes to the procedure structure. These procedure options vary in complexity and level of effort from a basic update to a more transformative change that seeks to optimize flexibilities and tracking of PI&R observations for regional assessment. Given the substantial backlog of change inputs that warrant incorporation and structural changes imposed by IMC 0040 since the procedures last revision in 2015, even a limited update would require a significant rewrite of the procedure.

APPENDIX A

2. Discussion and Procedure Options As discussed in the report, the team recommends that the following changes be made irrespective of which option is selected. These changes collectively would improve guidance and structure, assign inspection hours to a more appropriate inspection procedure and provide more flexibility for Safety Conscious Work Environment (SCWE) inspections. The following recommendations would not require Commission approval or notification:

A. As discussed in SECY 19-0067, the staff, pending Commission approval, would relocate the daily PI&R screening from IP 71152 to plant status activities defined in IMC 2515, Appendix D. This change would reduce a current bias toward overuse of IP 71152 in lieu of more appropriate baseline IPs to inspect issues identified during PI&R screening.

The change would shift approximately 129 to 175 of estimated annual resources back to plant status that had been previously transferred to IP 71152. The daily review was placed in the procedure in 2003 following a Davis Besse Lessons Learned recommendation; however, logically the daily review belongs in plant status as it is the only inspection activity under an IP that does not merit a documented sample in inspection reports. The staff would migrate the associated commitments, objectives, and requirements to not diminish the importance of the daily PI&R screening activity.

B. SCWE inspection guidance would be updated. The procedure contains the objective [t]o confirm licensees have established a safety conscious work environment and requirement 02.04.d supports this objective. In the original procedure, the inspection guidance included an attachment with a list of questions that can be used when discussing PI&R issues with licensee individuals to help assess whether there are impediments to the establishment of a safety conscious work environment. That attachment is still in the procedure. Later revisions included means by which regions can elect to utilize IMC 2515 Appendix C IP 93100, Safety Conscious Work Environment Issue Follow-up to perform more in-depth SCWE focus group interviews.

These revisions are commitments tied to SECY 04-0111. Some regions favor this more thorough approach to the individual interviews, whereas others favor conducting the individual interviews due to resource constraints. Other concerns are that the team members conducting the SCWE group interviews may not be formally trained Safety Culture Assessors in accordance with Manual Chapter 1245, Appendix C-12. This is due in part to the unclear guidance of IP 93100. The team recommends updating the procedure guidance to remove reference to IP 93100 to allow for focus groups under the scope of IP 71152. This would result in a data driven, risk-informed, graded-approach based on insights gained during the planning phase of the team inspection, or specific module for Option 3 as described below. There are several sets of data that can be reviewed throughout the inspection cycle, such as allegations, CAP inputs related to harassment or retaliation, employee concerns program data, and Nuclear Safety Culture Monitoring Panel data. This information would better inform the inspection team leader as to the general SCWE at the site. Guidance would be added to the planning section for the team inspection to ensure the team lead reviews this information and holds discussions with branch members to ascertain if a more in-depth inspection on SCWE should be made. Once a determination of the level of inspection is made, resources can be allotted as appropriate within the recommended resource constraints of the procedure. The level of effort can be tailored from individual and small group interviews utilizing IP 71152, Appendix A, to focus groups for certain licensee departments up to focus groups across all major departments. For formal focus groups APPENDIX A 2

the team recommends requiring qualified Safety Culture Assessors, but this would likely require increasing the number of qualified assessors across the regions. In the interim, teams may be supplemented with currently qualified assessors from other regions or headquarters offices. This approach accurately reflects current regional best practices and provides for a clearer and more flexible approach for assessing SCWE. For some licensees, this may result in an increase of inspection hours and for other licensees the hours may stay about the same.

C. The procedure would be updated to conform to revised IMC 0040 governance. Further, the staff would modify the structure of the procedure (based on the option approved) to either a single procedure with multiple sub procedures with separate CACs (IP 71111.04 as a model) or a base procedure with activity-specific attachments (IP 711124 as a model). This change would enhance fee billing and allow for better data analytics since each IP 71152 inspection would be assigned unique billing and inspection codes.

The staff developed resource estimates (see Section 3 of this appendix) and resultant efficiency gains for the three options presented in this appendix. These resources consider efficiencies gained through focusing of inspection efforts however they are estimates that the team considers reasonable and are not meant to be considered final.

Procedure Option 1 Discussion - Team Recommended Option:

Procedure Option 1 represents the smallest departure from the current PI&R inspection and was designed to enhance the existing inspection procedure based on operating experience, risk insights and feedback from internal and external stakeholders. Under this option, the staff would make small modifications to scope and frequency to improve efficiency and effectiveness.

The procedure maintains the same Semiannual Trend Review, Annual Follow-up of Selected Issues, and Team Inspection samples. The semiannual and annual samples would be formally broken into sub-procedures for resource tracking.

Option 1s frequency (biennial vs. triennial) of the PI&R team inspection is pending Commission decision relative to the options presented in SECY 19-0067. If Option 1 is implemented, the NRCs PI&R inspection program would include the following:

Inspection 71152S, Semiannual Trend Review. Inspectors would perform a review to identify trends that might indicate the existence of a more significant safety issue. This appendix would not be substantively different from the current inspection procedure requirement for semiannual trend reviews. Additionally, guidance would be added to trend SCWE issues if the triennial option is implemented.

Inspection 71152A, Annual Follow-up of Selected Issues. Inspectors would ensure that licensee corrective actions for select issues are planned or implemented commensurate with their significance.

Inspection 71152B (or T) Problem Identification and Resolution Team Inspection.

Inspectors would inspect licensee implementation of the PI&R program with a focus on identification, prioritization, evaluation and corrective action. The outcome of the procedure would be the identification and documentation of licensee PI&R program implementation trends and themes for use by NRC management during the End of Cycle assessment meetings. Elements of the current team inspection related to inspector review of licensee audits, self-assessments, and use of operating experience would be reduced based on a risk-informed assessment of the value of those requirements.

APPENDIX A 3

Development of Option 1 included a re-evaluation of the governance of the current team inspection. The purpose of the re-evaluation was to verify that the team inspection would adequately support the assessment of licensee PI&R (discussed in Appendix B of this report) but not unnecessarily duplicate inspection efforts. The team determined that in order to successfully support the programmatic assessment, the team inspection scope would be modified to focus on licensee implementation of PI&R in the areas of Identification or Problems, Evaluation of Problems, Timely and Effective Corrective Actions, and Safety Conscious Work Environment. Inspection scope and guidance would be modified to remove or transfer inspection elements that have historically not resulted in significant assessment input to other more appropriate samples. Elements of the current team inspection related to inspector review of licensee audits, self-assessments, and use of operating experience would be reduced in scope. There is no commitment to evaluate these programs separately as is current practice.

This option recommends reducing the review of these areas to only that required to support the evaluation of the main four areas. This change is supported by a review of findings data from Reactor Program System that revealed a large majority of findings are associated with performance issued that have cross-cutting aspects associated Identification or Problems (P.1),

Evaluation of Problems (P.2), Timely and Effective Corrective Actions (P.3) and less findings having cross-cutting aspects associated with Trending (P.4), Operating Experience (P.5) and Self-Assessment (P.6).

When developing Option 1, the team evaluated the current sample and resource estimates for the existing procedure and estimated revised sample and resource requirements to successfully implement the proposed guidance. These estimates were based on historical resource usage for the current IP 71152, risk insights, and professional judgement. One issue with data analysis is inspection hours for the daily review, semiannual trend review, and the annual samples are all binned under one CAC for 71152, the team inspection falls under 71152B. Due to the binning of the first three samples, it is difficult to analyze inspection hours for the procedure. All the procedure recommendations would solve this problem by separating the samples into individual CACs and moving the daily review to plant status (another reason this is recommended). That said, recent feedback on potential inefficiencies has focused on the team inspection rather than the other samples. The team determined that the following sample and resources13 are needed to implement Option 114:

For the semi-annual trend review, the team determined that completion of this inspection requirement would remain semiannual, with annual resource estimate of 20 +/- 4 hours4.62963e-5 days 0.00111 hours 6.613757e-6 weeks 1.522e-6 months .

This is largely unchanged from current sample and resource estimates included in IP 71152.

For the annual samples of select PI&R issues, the team determined that under Option 1 with a triennial frequency, completion of this requirement should increase to six to 10 samples per year, with an annual resource estimate of 96 +/- 24 hours2.777778e-4 days 0.00667 hours 3.968254e-5 weeks 9.132e-6 months . Under Option 1 with a biennial frequency, completion would remain at four to eight samples per year, with an annual resource estimate of 72 +/- 24 hours2.777778e-4 days 0.00667 hours 3.968254e-5 weeks 9.132e-6 months . This is a change from the current procedure guidance that breaks resource estimates out by number of units.

13 A detailed comparison of each option, to include sample requirements, sample frequencies and resource estimates is provided in Section 3 of this Appendix.

14 A differing perspective is that it is needed even if we do nothing on the procedure options. If the Commission picks a 3-year frequency, and we do nothing else, we should recommend the following bullets one and two.

APPENDIX A 4

For the modified PI&R team inspection (discussed above), under Option 1, this attachment would be performed either triennially with a reduced resource estimate of 210 +/- 30 hours3.472222e-4 days 0.00833 hours 4.960317e-5 weeks 1.1415e-5 months or biennially with a reduced resource estimate of 180 +/- 30 hours3.472222e-4 days 0.00833 hours 4.960317e-5 weeks 1.1415e-5 months .

In order to maintain the effectiveness of the revised team inspection at the longer frequency (triennial), a reasonable increase in team resources is merited to account for the additional items available to the team for sample selection due to the additional year between inspections.

Additionally, the team determined that in order to maintain reasonable confidence in the data available for assessment of licensee PI&R implementation for the triennial team inspection periodicity, additional samples under the Annual Follow-up of Select Issues attachment would be appropriate. Therefore, Option 1 with a triennial frequency includes an increase in nominal annual samples as described above.

While SCWE issues may not arise as frequently as violations related to the Corrective Action Program, the significance and potential consequences of licensees not addressing SCWE issues can be substantial. Therefore, the staff considered that if this portion of the inspection is changed to being performed on a triennial basis, there is other data that should be collected and inspected over the entire PI&R cycle.15 This data includes, but not limited to, anonymous CAP inputs especially those related to retaliation or harassment; Employee Concerns Program investigations; Nuclear Safety Culture Monitoring Panel inputs; any surveys or assessments that measure cultures; and other licensee programs that assess and address SCWE issues. Inspection and assessment of these inputs as well as the SCWE portion of the inspection would provide a holistic and programmatic review of the PI&R program and result in minimal increase in overall annual inspection hours over a 2 or 3-year periodicity (depending upon Commission direction).

Consideration of the four main focus areas:

Bases Consistency Risk Assessment and Actions This adheres to Improving the clarity of Level of effort and This recommendation does the current bases. the procedure would resource not address assessment.

improve inspection recommendations consistency. are risk informed.

Be riskSMART framework Spot Right (Opportunities) Wrong (Challenges) Consequences Manage

Minimal change to

Does not fully

Rapid

Resource procedure address resources implementation challenges may be

Clarify Guidance concerns with team

Improved addressed by future inspection consistency efforts (COVID-19

Continued lessons learned) resource/scheduling concerns Procedure Option 2 Discussion:

15 A differing perspective is that this other data should be collected and inspected over the entire PI&R cycle regardless of the frequency of the inspection cycle.

APPENDIX A 5

Similar to Option 1, Procedure Option 2 includes action for the staff to modify the structure of the current PI&R inspection procedure to a base procedure with activity specific attachments.

Under Option 2, the team evaluated significant changes in inspection scheduling to more evenly distribute the NRCs inspections. Specifically, Option 2 includes proposals to deconstruct the current PI&R team inspection into two separate, but smaller team inspections. One team inspection would be focused on licensee performance in identifying, evaluating and correcting problems, and the other focused on licensee performance for other PI&R program elements including use of audits, self-assessments, and operating experience. The smaller team inspections would sample PI&R elements currently reviewed under the existing program but at different frequencies and with different resource estimates. These new frequencies and resource estimates would be established on the teams assessment of several factors including:

(1) the relative risk-significance of the different elements in relation to PI&R programs; (2) historical licensee performance for the different elements; and (3) inspection trends and their overall contribution to the NRCs performance for the various program elements.

Under this option, IP 71152 would become a base procedure like IPs 71111, 71114, 71124 and 71126, and each inspection would be a separate sub-procedure (71152.01, 02, 03). The base procedure would contain generic guidance applicable to all attachments and additional guidance and clarification related to the routine review of licensee PI&R associated with all baseline inspection procedures. Each attachment would contain the applicable inspection objectives, requirements, and guidance associated with the current inspection procedure modules. The team determined this was an appropriate action applicable to any of the options in that it would reduce the complexity of each inspection procedure providing for greater clarity to inspectors and licensees. However, this would require a substantial formatting change to the IP with the issuance of several new inspection manual documents and would take several months of staff effort to modify the procedure and to engage with industry representatives on the new format.16 It is anticipated that this reformatting would not cause any appreciable impact on licensees.

This change option is only discussed under Options 2 or 3 but could be incorporated into Option 1 as well. If Option 2 is implemented, the NRCs PI&R inspection program would include the following:

IP 71152, Attachment 01, Semiannual Trend Review. Similar to Option 1, through this attachment, inspectors would perform a review to identify trends that might indicate the existence of a more significant safety issue. This attachment would not be substantively different from the current inspection procedure requirement for semiannual trend reviews. Guidance for SCWE trending would be included as well.

IP 71152, Attachment 02, Annual Follow-up of Selected Issues. Through this attachment, inspectors would ensure that licensee corrective actions for select issues are planned or implemented commensurate with their significance. Additional guidance related to sample selection would be added to provide inspectors with a hierarchy of required and optional samples from which to choose.

IP 71152, Attachment 03, Problem Identification and Resolution Team Inspection. This attachment includes samples to confirm the licensees performance in identifying, prioritizing, evaluating, and correcting problems.

16 A differing perspective is that substantial is not well defined and for clarity and ease of use, this effort of a couple staff working on it part time over a couple months is time well spent.

APPENDIX A 6

IP 71152, Attachment 04, Review of Audits, Self-Assessments, Operating Experience and SCWE. This attachment includes samples to confirm the licensees appropriate use of industry and NRC operating experience, and to evaluate the effectiveness of licensee audits and self-assessments.

When developing the scope for IP 71152, Attachment 03, Problem Identification and Resolution Team Inspection, the staff relocated the elements from the existing PI&R team inspection that typically involve the most risk-significant and greatest number of NRC findings. These include inspection samples of licensees ability to identify, prioritize, evaluate and correct problems.

Because the team viewed these samples as the most important elements (based on risk significance and historical findings trends), a biennial frequency is recommended, similar to the existing team inspection.17 The staff also more heavily weighted resource hours to review of these elements because of their importance in PI&R.

Similarly, when developing the scope for IP 71152, Attachment 04, Review of Audits, Self-Assessments, Operating Experience and SCWE, the staff relocated the remaining elements from the existing team inspection that, while important, did not typically result in a large number of NRC findings or findings with greater than green significance. The elements included NRC reviews of licensee audits and self-assessments, licensees appropriate use of industry and NRC operating experience, and confirmation of a safety conscious work environment. The staff also weighted resource hours such that Attachment 04 would be allotted less hours than 3.

Development of this option included a re-evaluation of the scope and elements of the current team inspection. The purpose of the re-evaluation was to verify that the team inspection would:

(1) adequately support the assessment of licensee PI&R; (2) not unnecessarily duplicate inspection performed in other procedures; and (3) provide for a more even distribution of NRC inspections of licensee PI&R programs. The concept associated with more even distribution of NRC inspections is similar to proposals made in SECY 18-0113, Recommendations for Modifying the Reactor Oversight Process Engineering Inspections. The team also established inspection intervals and resource estimates for the two team inspections based on factors including: (1) the relative risk-significance of the different elements in relation to PI&R programs; (2) historical licensee performance for the different elements; and (3) inspection trends and their overall contribution to the NRCs performance for the various program elements.

For the team inspection focused on licensee performance in identifying, evaluating and correcting problems, the team determined that the inspection interval should be biennial with a resource estimate of 128-160 hours every two years. This equates to approximately a 3-person inspection team with two on-site inspection weeks. The team established this inspection interval and resource estimate based on experience and judgment following review of inspection trends that revealed an overall large number of findings with PI&R cross-cutting issues relative to other area in the area of P.1, Identification, P.2, Evaluations, and P.3, Resolution. Additionally, the team noted that several of the lessons-learned for plants that entered Column 4 of the ROP Action Matrix or were under oversight under IMC 0350, Oversight of Reactor Facilities in a Shutdown Condition Due to Significant Performance and/or 17 A differing perspective is that this approach would cause scheduling problems with a biennial sub-procedure inside a triennial overall frequency. It could lead to two .03 inspections in a 3-year period, and effectively would overturn a 3-year frequency for the major team inspection of the PI&R inspections - a decision that is up before the Commission at this time. The differing perspective recommendation is that the 3-year frequency be kept if the Commission votes for a 3-year PI&R frequency.

APPENDIX A 7

Operational Concerns, documented common concerns related licensee weaknesses in the area of P.1, Identification, P.2, Evaluations, and P.3, Resolution, as common factors that resulted in the performance issues that dictated increased NRC oversight.

Under this option, the team that is focused on licensee performance in the use of audits, self-assessments, and operating experience, inspection interval should be triennial with a resource estimate of 72 hours8.333333e-4 days 0.02 hours 1.190476e-4 weeks 2.7396e-5 months every three years. This equates to approximately a 2-person inspection team with one on-site inspection week. The team established this inspection interval and resource estimate based on experience and judgment following review of inspection trends that revealed a lower number of findings with PI&R cross-cutting issues in the area of P.5, Operating Experience, and P.6, Self-Assessment, relative to other areas. The team also noted that historically, very few Greater-Than-Green findings have causal factors related to Operating Experience or Self-Assessment.

The benefit associated with Option 2 is a more even distribution of NRC inspection of PI&R programs when compared to the current program which is heavily weighted to the single biennial team inspection. Specifically, the use of two smaller team inspections would provide more frequent inspection touch points, similar to proposals made in SECY 18-0113, Recommendations for Modifying the Reactor Oversight Process Engineering Inspections.

This also spreads out the resource expenditure for licensees and regional offices. Further, the existing team inspection was evaluated against factors including: (1) the relative risk-significance of the different elements in relation to PI&R programs; (2) historical licensee performance for the different elements; and (3) inspection trends and their overall contribution to the NRCs performance for the various program elements. Therefore, Option 2 would result in a more targeted inspection, focusing greater resources on the more important elements of licensee PI&R programs.

Consideration of the four main focus areas:

Bases Consistency Risk Assessment and Actions This adheres to Improving the clarity of Level of effort and This recommendation does the current bases. the procedure would sample number not address assessment.

improve inspection recommendations consistency. are risk informed.

Be riskSMART framework Spot Manage Right (Opportunities) Wrong (Challenges) Consequences

Dispersed inspection

Resource impact of

Flexible resources

Stakeholder resources additional on region engagement

Clarify Guidance inspection

Licensee resource

Option 1 could be

Level of effort of impact could be implemented in interim changes could be higher to support significant multiple teams18

Better consistency 18 A differing perspective is that the licensees have told us that breaking large teams such as engineering teams into smaller chunks is a resource burden savings. It has been stated that it takes up less critical staff at one time and does not require as much team management. Additionally, this resource savings from splitting up inspections into smaller parts is the primary reason why the 3-week Component Design APPENDIX A 8

Difficulty/delays in implementation Additional Actions Required:

Internal and external stakeholder engagement

Commission notification or approval prior to implementation

Revisions to IP 71152 and IMC 0308 Procedure Option 3 Discussion:

Procedure Option 3 relocates all of IP 71152B biennial team inspection scope and effort to IP 71152 attachments associated with trend reviews and follow-up of selected issues. This option provides maximum flexibility in how the NRC inspects and assesses licensee PI&R programs and is provided as a transformative change to the procedure structure requiring significant resources to implement. It would require Commission approval based on the level of interest it would generate. Under Option 3, the IP 71152 PI&R inspection modules are structured into sections of a single procedure as follows:

Inspection Procedure 71152, Attachment 01, Trend Review. Formerly referred to as the semi-annual trend review, this attachment is like Option 1 and 2 where inspectors would perform a review to identify trends that might indicate the existence of a more significant safety issue. The trend review under Option 3 expands from the current scope and effort of 2 samples per year estimated at 20+/-4 hours to 3+/-1 sample at 50+/-20 hours because the team inspection is eliminated. Additionally, several of the inspection samples typically performed under IP71152B are shifted and characterized as required samples under IP71152, Attachment 1. Specifically, the following samples would be required samples under Attachment 1:

o Former 71152B inspection of licensee PI&R Audits and Self-Assessments, sampled no less than once every 2 or 3 years19; o Former 71152B inspection of licensee use of OpE, sampled no less than once every 2 or 3; o Former 71152B inspection of issues processed over the last five years, sampled no less than once every 2 or 3 years; o Former 71152B inspection of SCWE, sampled no less than once every 2 or 3 years.

Inspection Procedure 71152, Attachment 02, Follow-up of Selected Issues. Formerly referred to as annual follow-up of selected issues, this attachment is like Option 1 and 2 where inspectors ensure that licensee corrective actions for select issues are planned or implemented commensurate with their significance. Additional guidance related to Basis Assurance Inspection switched to a smaller Design Basis Assurance Inspection plus Focused Engineering Inspections.

19 Frequency would be based on pending Commission decision relative to the options presented in SECY 19-0067.

APPENDIX A 9

sample selection would be added to provide inspectors with a hierarchy of required and optional samples from which to choose. Because the team inspection is eliminated, the follow-up of select issues under Option 3 expands from the current annual scope and effort of 6+/-2 samples estimated at 94+/-10 hours to 9+/-3 samples at 125+/-35 hours for a dual-unit site upon receipt of the following former IP 71152 biennial team inspection scope and effort.

The requirement to perform a PI&R programmatic assessment is relocated from IP 71152 to IMC 0305. The goal of this relocation is to better focus IP 71152 on effective sample-based inspection, issue disposition, and documentation, while better integrating PI&R programmatic assessment into IMC 0305, Operating Reactor Assessment Program. The team determined that shifting assessment to IMC 0305 appropriately reflects the importance of PI&R assessment as a part of the holistic look at licensee performance that regions do under IMC 0305. This is another aspect, that while only provided in Option 3, the team determined could be incorporated under any option but would require additional effort and staff coordination. For Option 3, it is required due to the distributed nature of the samples.

Option 3 is intended to enhance flexibility by permitting former 71152B team inspection to be performed as: (1) multiple independent baseline samples, (2) a consolidated team inspection, or (3) a combination of both team and independent inspections. The team also noted that Option 3 improves the effectiveness of the inspection and assessment program by reallocating an approximately 250-hour single-sample biennial team combined inspection/assessment with approximately eight discrete non-redundant inspection-only samples combined with subsequent IMC 0305 annual assessment. This change also streamlines IP 71152, reducing the amount of training needed for inspectors and associated human capital required for upkeep.

Consideration of the four main focus areas:

Bases Consistency Risk Assessment and Actions This adheres to the Improving the clarity Level of effort and This recommendation current bases. of the procedure sample number addresses assessment.

would improve recommendations inspection are risk-informed.

consistency.

Be riskSMART framework Spot Manage Right (Opportunities) Wrong (Challenges) Consequences

Dispersed inspection

More effort to track

Flexible

Stakeholder resources inspection resources on engagement

Clarify Guidance completion region

Option 1 could be

Greatly increase

Significant changes

Better implemented in flexibility to procedure consistency interim

Regional

Requires IMC 0305

Difficulty/delays in

Inspector Training decisionmakers guidance implementation assess performance Additional Actions Required:

Significant Internal and external stakeholder engagement APPENDIX A 10

SECY for Commission approval for implementation

Potential Pilot

Revisions to IP 71152, IMCs 0305 and 0308 APPENDIX A 11

3. Resources The staff developed the following resource estimates and resultant efficiency gains for the three options presented in this Appendix. These resources consider efficiency gains through focusing of inspection efforts however they are estimates that the team considers reasonable and are not meant to be considered final. Fewer NRC inspection resources would result in fee-billable savings for licensees.

Option 1 - Option 2 - Option 3 -

Current Current Inspection Substantive Changes to Redistribution of Requirements with Small Inspection Sample Inspection Samples and Enhancements Requirements Added Flexibility Biennial: 182 +/- 43 hours4.976852e-4 days 0.0119 hours 7.109788e-5 weeks 1.63615e-5 months per year

[-37 hours nominal or 17%] 162 +/- 38 hours4.398148e-4 days 0.0106 hours 6.283069e-5 weeks 1.4459e-5 months per year 200 +/- 50 hours5.787037e-4 days 0.0139 hours 8.267196e-5 weeks 1.9025e-5 months per year 219 +/- 33 hours3.819444e-4 days 0.00917 hours 5.456349e-5 weeks 1.25565e-5 months per year Triennial: 186 +/- 38 hours4.398148e-4 days 0.0106 hours 6.283069e-5 weeks 1.4459e-5 months per [-57 hours nominal or 26%] [-19 hours nominal or 9%]

year

[-33 hours nominal or 15%]

IP 71152S - Semiannual Trend IP 71152.01 - Semiannual Section 02.02 - Semiannual IP 71152.01 - Trend Review Trend Trend (2 Samples) 2 Samples 2 - 4 Samples 2 Samples 20 +/- 4 hours4.62963e-5 days 0.00111 hours 6.613757e-6 weeks 1.522e-6 months per year 20 +/- 4 hours4.62963e-5 days 0.00111 hours 6.613757e-6 weeks 1.522e-6 months per year 60 +/- 15 hours1.736111e-4 days 0.00417 hours 2.480159e-5 weeks 5.7075e-6 months per year 20 +/- 4 hours4.62963e-5 days 0.00111 hours 6.613757e-6 weeks 1.522e-6 months per year Section 02.03 , Annual IP 71152A, Annual Follow-up Follow-up of Selected Issues of Selected Issues IP 71152.02 - Follow-up of IP 71152.02 - Annual Follow-4 - 8 Samples Biennial: 4 - 8 Samples up of Selected Issues Selected Issues with multiple sample requirements 71 +/- 10 hours1.157407e-4 days 0.00278 hours 1.653439e-5 weeks 3.805e-6 months (single unit) 72 +/- 24 hours2.777778e-4 days 0.00667 hours 3.968254e-5 weeks 9.132e-6 months per year 4 - 8 Samples 6 - 12 Samples 74 +/- 10 hours1.157407e-4 days 0.00278 hours 1.653439e-5 weeks 3.805e-6 months (dual unit) Triennial: 6 - 10 Samples 72 +/- 24 hours2.777778e-4 days 0.00667 hours 3.968254e-5 weeks 9.132e-6 months per year 140 +/- 35 hours4.050926e-4 days 0.00972 hours 5.787037e-5 weeks 1.33175e-5 months per year 77 +/- 10 hours1.157407e-4 days 0.00278 hours 1.653439e-5 weeks 3.805e-6 months (triple unit) 96 +/- 24 hours2.777778e-4 days 0.00667 hours 3.968254e-5 weeks 9.132e-6 months per year IP 71152.03 - PI&R Team IP 71152T(B)- PI&R Team Inspection Section 02.04 - Biennial Inspection Team Inspection Biennial: 128 -160 hours Guidance provided in base Biennial: 180 +/- 30 hours3.472222e-4 days 0.00833 hours 4.960317e-5 weeks 1.1415e-5 months IP 71152.04 - PI&R Team procedure for Routine 250 +/- 38 hours4.398148e-4 days 0.0106 hours 6.283069e-5 weeks 1.4459e-5 months biennially Review.

Triennial: 210 +/- 30 hours3.472222e-4 days 0.00833 hours 4.960317e-5 weeks 1.1415e-5 months Inspection Triennial: 72 hours8.333333e-4 days 0.02 hours 1.190476e-4 weeks 2.7396e-5 months APPENDIX A 12

Assessment Review

1. Background

As discussed in the basis and history of the Problem Identification & Resolution (PI&R) program document, [a] fundamental goal of the Nuclear Regulatory Commission's (NRCs) reactor inspection and assessment process is to establish confidence that each licensee is detecting and correcting problems in a manner that limits the risk to members of the public. This inspection basis was taken from SECY-99-02 and exists in Inspection Manual Chapter (IMC) 0308, Reactor Oversight Process Basis Document. In SRM-SECY-00-0049, the Commission affirmed this importance to the staff.

The team reviewed both the procedure and IMC 0305, Operating Reactor Assessment Program and determined there is limited guidance or standards for evaluating the effectiveness of a licensees PI&R. There is also limited guidance for any actions to be taken for a PI&R program with weaknesses or one deemed ineffective. The procedure for the biennial inspection provides guidance for inspecting discrete samples and includes the following under section 03.04.d:

Development of PI&R Program Performance Insights. By reviewing a sufficient number and breadth of samples, the inspection team should be able to develop insights into the licensees ability to identify, evaluate, and resolve problems using the corrective action program, operating experience, and results of self-assessments/audits. Inspectors should compare these results with the licensees performance reviews, including reviews of PI&R programs. Inspectors should determine whether licensee reviews are consistent with the NRC review of PI&R issues.

The intent of this IP (both the routine and biennial inspection effort) is to provide insights into licensee performance in the PI&R area based upon a performance-based review of corrective action issues, operating experience, and self-assessments/audits. More detailed programmatic reviews of licensee performance in the PI&R area will be conducted during supplemental inspections if established performance thresholds are crossed.

This guidance has existed with minimal changes since the procedure was initially issued in 2001. PI&R inspector comments have indicated that the guidance lacks the clarity to accomplish the procedure objective of being able to assess a licensees PI&R program.

A review of numerous inspection reports revealed concerns with licensee PI&R performance leading up to Column 4 of the Action Matrix shows trends in identified issues without drawing more holistic assessment of the effectiveness of a licensees PI&R program:

BROWNS FERRY. In the case of Browns Ferry, PI&R reports in 2007, 2008 and 2009 all described Regional staff concerns with the licensees ability to identify and correct issues, combined with three different PI&R Substantive Cross-cutting Issues (SCCIs). During this period the licensee went in and out of Column 3 of the Action Matrix prior to entering Column 4 in 2010 and receiving an IP 95003 supplemental inspection.

APPENDIX B

Browns Ferry Timeline:

deficiencies not previous issues not entered into entered into CAP corrective Numerous delays CAP corrective actions not fully effective increased backlogs actions were not significant deficiencies repeat problems effective identified through self-assessment of the CAP Column SCCIs FORT CALHOUN. Similarly, in the lead up to Fort Calhoun entering IMC 0350, Oversight of Reactor Facilities in a Shutdown Condition Due to Significant Performance and/or Operational Concerns, Regional staffs PI&R observations displayed a trend of concerns surrounding all aspects of the licensees PI&R performance. While the licensee had no PI&R SCCIs during this period, there were large swings through the Action Matrix.

Fort Calhoun Timeline:

did not consistently continuing challenges in several issues with the identify problems all three areas erosion quality of cause untimely corrective of confidence in the evaluations measures corrective action program Column SCCIs PILGRIM. Pilgrim entered Column 3 due to two white performance indicators in the Initiating Events cornerstone in 2013, a 95002 inspection determined the site had not proven that their corrective actions to be adequate and the PIs remained open. They entered Column 4 the subsequent year.

APPENDIX B 2

Pilgrim Timeline:

deficiencies regarding Entergys execution of corrective actions challenged Pilgrims not effective in evaluating ability to effectively and implementing classify and evaluate corrective actions. marginally effective.

issues.

Column CCIs

2. Discussion and Assessment Options Based on the initial tasking from the fourth charter focus area, the team focused the efforts of the assessment subgroup on addressing the above concerns and more specifically the multiple recommendations made by the IP 95003 Lessons Learned documents. The team evaluated each option versus all four of the charter focus areas and using the Be riskSMART framework to analyze what can go right or wrong and anticipate and mitigate potential consequences.

Any discussion of Cross-cutting Issues (CCIs) and increased agency action if/when a licensee develops a CCI considers the concurrent CCI effectiveness review. Efforts to implement recommendations from this effort and the CCI effort should be coordinated. This is not anticipated to be an issue.

Assessment Option 1 Discussion:

The team has introduced a concept and framework that provides an objective PI&R performance assessment tool that would provide (1) more clear criteria for assessment and (2) potential changes to agency response. One or both parts can be implemented using IP 71152 and/or IMC 0305 revisions (depending on procedure options discussed in Appendix A). A full description of Assessment Option 1, which is intended to be a strawman and a starting point or for further development if Option 1 is selected as the path forward, is provided in ADAMS Package Accession No. ML20247J590.

Under Option 1, assessment of the licensee PI&R program would be broken down into four discrete performance areas, including:

Identification of Problems APPENDIX B 3

Evaluation of Problems

Timely and Effective Corrective Actions, and

Safety Conscious Work Environment Each performance area is defined by the associated cross cutting aspects from IMC 0310, Aspects Within Cross-Cutting Areas.

The licensees performance in each of these areas would be compared to performance criteria that would provide examples of thresholds for assessment. The performance criteria would include, but not be limited to: all inspection findings with PI&R associated cross-cutting aspects; inspector identification of adverse trends; licensee PI&R program resolution of conditions adverse to quality; allegations of harassment, intimidation, retaliation, and discrimination; and results of SCWE interviews with licensee personnel. Each performance area would receive an individual assessment of Satisfactory, Satisfactory but Needs Improvement, or Unsatisfactory based on the performance criteria.

The following criteria would be used to determine the areas overall assessment.

Satisfactory - All the objectives of the performance area are met.

Satisfactory but Needs Improvement - All objectives of the performance area are met, but weaknesses were identified.

Unsatisfactory - The objectives of the performance area were not met through identification of repetitive or significant weaknesses.

The proposed criteria were developed utilizing operating experience and observations from past PI&R projects, Lessons Learned from Fort Calhoun and Davis Besse IMC 0350 periods, and past experience with long term SCCIs at Susquehanna, Indian Point, and Salem/Hope Creek and sites with longstanding PI&R weaknesses such as Pilgrim and Wolf Creek. The criteria are meant to be consistent with CCI thresholds such that new standards are not introduced.

The performance criteria would be applied during the PI&R team inspection and/or during IMC 0305 quarterly and annual assessment by using multiple sources of data including ROP and traditional enforcement findings, non-cited violations, notices of violations, allegation data, cross cutting aspects, as well as the samples selected during the biennial team inspection using the guidance in IP 71152B. The results of the assessment would inform NRC follow up inspection activities within the inspection program from annual samples to informed supplemental inspections (if appropriate, in accordance with the licensees position in the Action Matrix).

The team performed a case study of Column 4 plants and determined that if these criteria had been in place, the following sites would have been rated as follows:

Browns Ferry SNI SNI SNI U APPENDIX B 4

S: Satisfactory SNI: Satisfactory, but Needs Improvement U: Unsatisfactory Fort Calhoun SNI SNI S U Pilgrim SNI SNI U The team determined that the thresholds are such that most Column I licensees would be rated Satisfactory in all areas, with occasional sites needing additional attention under the rating of Satisfactory but Needs Improvement. Further study would be required to ensure the criteria allows for early identification of downward trends without focusing unnecessary resources on insignificant trends (false positives).

This option uses a graded approach to PI&R program weakness response actions by the NRC.

The first two steps address identified weaknesses by assigning a PI&R annual sample to review the issue with increasing scopes and level of effort or informing a supplemental inspection (if one is already required by the Action Matrix). This stays within the current ROP baseline or supplemental inspection programs. The next step allows for conducting an additional PI&R team inspection (this would be altered if procedure Option 3 was adopted). In the final step, NRC management would consider moving the licensee to Column 5 of the Action Matrix as discussed in IMCs 0305 and 0308. This would be in conjunction with concurrent Action Matrix inputs. The guidance sets out requirements, documentation, and checks and balances to ensure actions are applied consistently, objectively and based upon clearly demonstratable performance.

APPENDIX B 5

Consideration of the four charter focus areas:

Bases Consistency Risk Assessment and Actions This tool would seek A prescriptive tool The criteria is risk This action best addresses to address perceived with clear criteria and informed.20 the desire for clear criteria gaps in the current regional response and a reliable and bases but would would increase repeatable process for require additions for consistency. NRC response.

the criteria and actions.

Be riskSMART framework Spot Manage Right (Opportunities) Wrong (Challenges) Consequences

Clear criteria

Criteria could be

Assessment

Seek Commission

Pre-determined subjective communicated Approval regional actions

Substantial work publicly

Extensive required with

Prevention of stakeholder stakeholders prior to Column 4 engagement implementation (long performance

Pilot Program term project)

false positives

Other procedural

Negative improvements could stakeholder be implemented perception while this option is

Delayed being developed implementation Additional Actions Required:

Significant internal and external stakeholder engagement to discuss the details of the proposal, the need for a change, the benefits and downsides of such a proposal, and to fine-tune criteria and NRC response

SECY for Commission approval of a pilot

SECY for final Commission approval for implementation

Revisions to IP 71152, IMCs 0305 and 0308 Assessment Option 2 Discussion - Team Recommended Option:

The team has introduced a binary assessment tool to evaluate overall licensee PI&R performance with NRC follow-up inspection for specific issues. Under Assessment Option 2, the proposed criteria would be objective and substantially more simplified than Option 1. This assessment could be a part of a continuous risk informed process, much like the current Action 20 A differing perspective is that there is risk, which is not captured here, that the NRC would be perceived as coming up with a new criteria to declare a licensee unsatisfactory - to the point of moving them into Column 5 of the action matrix without demonstrating the need to do so. This perspective believes that the level of effort that would have to be applied to address this concern is a great risk to decision makers looking to choose an appropriate option. Also, there is a risk of the potential for an overlap of resources among the PI&R program, allegations program, and CCI program - such as using CCIs to determine how to rate a licensee or using allegations to change how we rate a program. Decision makers can already review these areas at End of Cycle meetings and at other times, without a formal grade being assigned.

APPENDIX B 6

Matrix and CCI program. Also, this proposal reflects similar options proposed by the CCI effectiveness review effort and would need to be coordinated with recommendations by the CCI effectiveness review team.

The team proposes the following criteria for assessment:

1. More than two Greater-Than-Green findings with significant causal factors related to PI&R in the previous 24-months.

2. PI&R CCI Assigned.21

3. SCWE CCI Assigned.

4. A supplemental inspection identifies and documents a significant weakness in PI&R program which the licensee did not identify.

Results of this assessment would dictate NRC follow-up, including licensee notification and additional inspection. Failure of any criteria would result in NRC activity similar in scope to a supplemental inspection. A new procedure to address criteria 1 and 2 would need to be developed, similar in scope to a 95001. IP 93100, Safety-Conscious Work Environment Issue of Concern Follow-up already can be utilized for Criterion 3. Criterion 4 already requires re-performance of the supplemental inspection (some additional guidance may be required in the supplemental procedures).

Results of this assessment would be that the licensees entire PI&R program is either Satisfactory or Needs Improvement.22 NRC follow-up inspections would inform NRC management assessment in accordance with IMC 0305.

For a plant evaluated as Needs Improvement and having completed all of the applicable follow-up inspections with documented unsatisfactory assessments and NRC management has documented ongoing concerns about the effectiveness of corrective actions for consecutive assessment periods, NRC management shall consider moving the licensee to Column 5 of the Action Matrix as discussed in IMC 0305 and its IMC 0308.

Although Assessment Option 2 would require further development the team recommends this option because it would potentially provide the most objective and repeatable enhancement to the program without being overly burdensome. However, as discussed in the report, this recommendation hinges on the outcome of the recent CCI effectiveness review and whether Recommendation 2.d is accepted. There were some technical staff views that Assessment Option 1, New Assessment Tool Using Performance Areas and Criteria, was the more effective method for enhancing the assessment process.

21 A differing perspective is that Criteria 2 and 3 rely on CCI which already has regulatory and licensee actions built into the program. These would essentially cause a licensee to get an additional 40 or so hour inspection, in addition to the regulatory actions that are being taken for SCWE or PI&R issues under the CCI program.

22 A differing perspective is whether two categories is sufficient. Would a satisfactory assessment include inputs for enhancing? Or, would that automatically put a licensee into the needs improvement bin. Would we have a sense of what portion of historical PI&Rs would fall into the needs improvement bin?

APPENDIX B 7

Consideration of the four charter focus areas:

Bases Consistency Risk Assessment and Actions This tool would seek Criteria is clear and While risk This action addresses the to address perceived objective, NRC informed, these desire for clear criteria and a gaps in the current actions are clear. criteria are more reliable and repeatable bases but would basic and the process for NRC response require additional threshold could be however actions are limited bases for the criteria too high or low. and response to licensee and actions. performance may not be timely.

Be riskSMART framework Spot Manage Right (Opportunities) Wrong (Challenges) Consequences

Objective criteria

Substantial work

Assessment

Seek Commission

Pre-determined required with communicated Approval actions stakeholders prior publicly

Extensive

Limited qualitative to implementation

Prevention of stakeholder assessment (long term project) Column 4 engagement

Limited tiers performance

Pilot Program

false positives

Other procedural

Negative improvements could stakeholder be implemented perception while this option is

Delayed being developed implementation Additional Actions Required:

Internal and external stakeholder engagement

SECY for Commission approval for implementation

Revisions to IP 71152, IMCs 0305 and 0308

Development of Criterion 2 procedures and revision on IP 9500x procedures.

Assessment Option 3 Discussion:

In Assessment Option 3, the Satisfactory assessment criteria in each of the assessment areas (i.e., identification, evaluation, correction of plant problems and SCWE) of the proposed methodology to support Assessment Option 1 could be modified and incorporated into the procedure as guidance to utilize for overall licensee PI&R performance assessment in accordance with the current inspection objectives and requirements. This guidance would provide for a more data driven qualitative observation of PI&R performance by the inspection lead, which is an improvement on the current process of developing insights based on discrete inspection samples. By breaking out the guidance by assessment area with more specific criteria, inspectors have a more guided means to document their observations.

Consistent with current ROP governance, there would be no additional actions beyond the baseline as a result of any team observations. However, the intent would be that regional decisionmakers have more data driven PI&R observations for trending and risk-informed sample selection or to consider if additional may be warranted through existing processes.

APPENDIX B 8

Consideration of the four main focus areas:

Bases Consistency Risk Assessment and Actions This adheres to the Additional guidance These criteria This action does not fully current bases and would improve would assist address the requests of would only require consistency, but not inspectors in risk some regional an addition to eliminate the informed decision stakeholders but does discuss the bases subjectivity of the making during their improve inspection of the elements. current PI&R inspection. guidance.

assessment process.

Be riskSMART framework Spot Manage Right (Opportunities) Wrong (Challenges) Consequences

Minimal change to

Subjective

Assessment

Inspector training procedure conclusions communicated

Improved guidance

No additional

No guidance to publicly on communications inspection beyond regions on actions

Predictive to licensees baseline for poor PI&R23 capabilities enhanced

No regional actions

Rapid implementation Additional Actions Required:

Stakeholder engagement prior to release of documents for comment

Revisions to IP 71152, IMCs 0305 and 0308

3. Other Considerations The team explored other options such as a PI&R performance indicator or incorporating licensee metrics into NRC assessment. This would require substantial stakeholder engagement and would constitute a transformational change to multiple ROP tenets, as such they were not pursued further.

Another path to address regional concerns with PI&R assessment is to update the bases to clarify expectations and limitations of the procedure. As discussed in the background, there has always been two staff views regarding response to licensee performance within the ROP. One staff view assumes that Action Matrix movement dictated by risk significant findings and performance indicators should be the only driver of agency response. The second staff view believes that programmatic breakdowns, especially in PI&R, may not yield risk significant 23 A differing perspective is that the NRC should consider how training or procedures could help with improving the lack of guidance on what to do with poor PI&R performance. It currently happens every cycle, where decision makers evaluate PI&R, SCWE issues, allegations, CCI info and other inspection input and determine how to best proceed with next years inspection plans. This perspective believes that documenting a proposed path would accomplish as much as a graded scale or go/no go approach which still do not determine how to best approach the inspections in the next assessment cycle other than through more inspections. PI&R is already one of the biggest sources of inspection we have of the 2800 plus hours of inspection at a two-unit site annually.

APPENDIX B 9

findings under the SDP but should be used to predict and prevent declining licensee performance.

In its current state, in addition to the previously discussed benefits, PI&R inspection allows for some qualitative assessment of PI&R performance to give the staff limited indications of licensee issues without going beyond the Action Matrix. Overall, the procedure has been effective in achieving this goal as demonstrated by the 95003 lessons learned effort (discussed above). With the exception of ANO, every lessons learned report in the last 10 years stated that the procedure, in conjunction with other inputs, provided sufficient warning of declining performance; the regional staff concern was the constraints placed on the ability to respond and prevent this declining performance. IMC 0308 could be updated to clarify the purposes and limitations of PI&R assessment to only that of indication and communication and reassert that the Action Matrix is the only method to dictate NRC response to declining licensee performance.

This would be in addition to the CCI program, which provides additional means of monitoring and communicating PI&R performance concerns.

APPENDIX B 10

ML20247J602

Contents

Text

Background

1. Background

1. Background

Navigation menu

ML20247J602

Text

Background

1. Background

1. Background

Navigation menu

Search