ML22340A452

Safety Culture Effectiveness Review Final Paper

ML22340A452
Person / Time
Issue date:	03/20/2023
From:	Philip Mckenna NRC/NRR/DRO/IRAB
To:	Russell Felts NRC/NRR/DRO
References
Download: ML22340A452 (32)

Text

March 20, 2023 MEMORANDUM TO: Russel Felts, Director Division of Reactor Oversight Office of Nuclear Reactor Regulation Signed by McKenna, Philip on 03/20/23 FROM: Philip J. McKenna, Chief Reactor Assessment Branch Division of Reactor Oversight Office of Nuclear Reactor Regulation

SUBJECT:

RESULTS OF THE SAFETY CULTURE PROGRAM EFFECTIVENESS REVIEW In January of 2022, a multi-disciplinary team of staff from multiple agency offices was assigned to conduct an effectiveness review of the Safety Culture program as part of the Reactor Oversight Process (ROP) self-assessment program. The team charter is available in Agencywide Documents Access and Management System (ADAMS) Accession No. ML22045A867. The purpose of this effectiveness review was to evaluate how safety culture is currently incorporated into the ROP and determine whether this was achieving its objectives laid out by the Commission in SRM-SECY-2004-011 and subsequent staff communications. In assessing the effectiveness of the Safety Culture program, the team applied the Principles of Good Regulation and the BeRiskSmart framework while considering: the stated objectives of the program; the history of the program; correlations between safety culture indicators and plant performance; and individual case studies. After completing the review and communicating potential recommendations with both internal and external stakeholders, the review team developed three recommendations for implementation.

Recommendation 1: Inspection Manual Chapter (IMC) 0305, Operating Reactor Assessment Program, and Inspection Procedure (IP) 95001, Supplemental Inspection Response to Action Matrix Column 2 (Regulatory Response) Inputs, should be updated to allow for an independent NRC evaluation of safety culture for Column 2 plants if the circumstances warrant. This would not necessarily constitute requiring the licensees to perform an independent third-party safety culture assessment. It would provide for independent NRC consideration of the need for sampling and assessing safety culture.

Enclosure:

Safety Culture Program Effectiveness Review Report CONTACT: Molly Keefe-Forsyth NRR/DRO 301415-5717

R. Felts 2 Recommendation 2: Improve the training available to inspectors to better meet Commission objectives stated in SRM-SECY-04-0111. (Note that steps are already in process that will support this recommendation, as NRR has a User Need open with RES to enhance training on safety culture based on review of ongoing inspector feedback and projected workload needs.)

Recommendation 3: IP 93100, Safety-Conscious Work Environment Issue of Concern Followup, should be revised to cover all aspects of safety culture, in addition to Safety Conscious Work Environment (SCWE). This will provide the NRC with a tool that can be used for following up on safety culture concerns to encourage meaningful action before licensees are assessed in Column 3.

The team developed three options for Recommendation Three:

a. IP 71152, Problem Identification and Resolution (PI&R), review of specific safety culture attributes should be expanded with a reference to the revised IP 93100 to provide guidance for a more comprehensive review of safety culture as necessary. Considerations already factored into resource allocations for PI&R inspections, including review of data trends and inspector observations, should add consideration for including a qualified safety culture assessor on the team if warranted. The team estimates that this option would be essentially resource neutral.

b. Adjustments should be made to the existing inspection framework to allow for safety culture assessment when indicated using the revised IP 93100 procedure. This would leave IP 93100 as an IMC 2515, Appendix C, Special and Infrequently Performed Inspections, procedure, available for use at Regional Administrator discretion using criteria to be developed separately if this option is chosen (The effectiveness of previous NRC management discussions with licensees based on quarterly trend reviews discussed in IMC 0305 may be one criterion for consideration). IP 93800, Augmented Inspection Team, and IP 93812, Special Inspection, should be updated to reference the revised IP 93100 and consideration of including a qualified safety culture assessor on the inspection team, as well as to emphasize the importance of reviewing safety culture attributes as within scope of the fact-finding purpose of the inspection, to better align the use of these procedures with the intent of changes initiated in response to the original Commission direction.

c. A regular safety culture assessment, based on the revised IP 93100, should be included as part of the baseline inspection program on a tri-/biennial basis. This could be performed through a graded assessment allotting more or less resources depending on safety culture/SCWE related data amassed throughout the inspection cycle. This may require additional inspection resources beyond what is currently allocated, approximately 4-5 (which could include team members who are training to be qualified) team members on site for a week gives an estimate of 60-80 inspection hours. Although SRM-SECY-04-0111 specifically decided against a baseline inspection option for safety culture in lieu of the current approach, this effectiveness review finds that the cross-cutting issues program that was developed instead has not been able to fully meet the stated objectives of the program.

During discussions with NRC Regional Administrators and Division Directors, an alternative recommendation was proposed which leverages the existing IMC 0305 quarterly ROP trend reviews of licensee performance. The recommendation is to revise existing assessment guidance to prompt NRC Management discussions with licensees on any safety culture aspects discussed during quarterly reviews. The recommendation would encourage licensee action to address safety culture aspects before significant performance degradation occurs. This

R. Felts 3 recommendation would not require additional inspection or significant additional resources, consistent with the graded approach in SRM-04-0111 for licensees in Column 1 or Column 2 of the ROP Action Matrix.

The staff discusses the effectiveness criteria and assessment of effectiveness that was used to arrive at the above recommendations in the enclosed report.

ML22340A452 NRR-106 OFFICE NRR/DRO/IRAB NRR/DRO/IRAB NAME MKeefe-Forsyth PMcKenna DATE 3/20/2023 3/20/2023 Safety Culture Program Effectiveness Review In January of 2022, a multi-disciplinary team of staff from multiple agency offices was assigned to conduct an effectiveness review of the Safety Culture program as part of the Reactor Oversight Process (ROP) self-assessment program. The team charter is available in ADAMS at Accession No. ML22045A867. The team was comprised of the following individuals:

Molly Keefe-Forsyth, Safety Culture Program Manager NRR/DRO/IRAB (Team Lead)

Rebecca Sigmon, Reactor Systems Engineer, NRR/DRO/IOEB April Nguyen, Senior Resident Inspector, Region III Elise Eve, Senior Reactor Inspector, Region I Dori Willis, OE/Headquarters Allegation Team Lead Chris Hunt, Senior Resident Inspector, Region III Alex Garmoe, Senior Reactor Analyst NRR/DRO/IRAB (Team consultant)

Effectiveness reviews were added to the ROP self-assessment program as part of the November 23, 2015, revision to Inspection Manual Chapter (IMC) 0307, Reactor Oversight Process Self-Assessment Program. These effectiveness reviews are used to assess recently implemented ROP changes to evaluate their effectiveness to ensure that the intended results have been realized and to evaluate any unintended consequences. The agency has not performed a holistic review of the effectiveness of safety culture oversight since Commission-directed changes were implemented in 2006.

The purpose of this effectiveness review is to evaluate how safety culture is currently incorporated into the ROP and determine whether this is achieving the objectives laid out by the Commission in SRM-SECY-2004-011 and subsequent staff communications. In assessing the effectiveness of the Safety Culture program, the team applied the Principles of Good Regulation1 and the Be riskSMART2 framework while considering: the stated objectives of the program; the history of the program; correlations between safety culture indicators and plant performance; and individual case studies.

Effectiveness Criteria While the cross-cutting areas of Human Performance, Problem Identification and Resolution (PI&R), and Safety Conscious Work Environment (SCWE) have been a part of the ROP since its inception in 1999, the current framework of the safety culture program evolved out of Commission direction3 following the discovery of the reactor vessel head degradation at Davis-Besse in 2002. While rejecting staff options to develop a regulation or baseline inspection procedure directly addressing safety culture, the Commission approved the staff option to enhance the ROP treatment of cross-cutting areas.

1 As discussed in relation to the ROP in SECY-99-007 2 NUREG/KM-0016, ADAMS Accession Number ML21071A238 3 SRM-SECY 2004-0111, ADAMS Accession Number ML042340661 Enclosure

The relevant Commission direction laid out in SRM-2004-0111 included:

Staff should not use surveys of licensee personnel, but rather rely on inspector observations and other indicators already available to the NRC. Staff should develop tools that allow inspectors to rely on more objective findings.

Staff should consider including enhanced problem identification and resolution initiatives as part of this effort.

Staff should ensure that the inspectors are properly trained in the area of Safety Culture.

Staff should consider developing an enhanced training program for its inspectors and resident inspectors on Safety Culture that uses both insights from INPOs work in this area and insights from the international community.

Staff should include as part of its enhanced inspection activities for plants in the Degraded Cornerstone Column a determination of the need for a specific evaluation of the licensees Safety Culture.

The staffs methodology for using the treatment of cross-cutting issues to more fully address Safety Culture should require a specific determination for plants in the Degraded Cornerstone Column.

As discussed in SECY-06-0122, based on the Commission direction, the staff developed an approach with the participation of stakeholders to better align the three cross-cutting areas to those aspects of performance that are important to safety culture. The staff adjusted selected baseline, event response, and supplemental inspection procedures and inspection manual chapters within the pre-existing framework of the ROP and consistent with the principles that guided the development of the ROP. The staff also developed inspector training on safety culture and the changes to the ROP.

The approach provided for a graded regulatory response to plant performance issues so that the regulatory response increases as licensees move to the right in the ROP Action Matrix. Key features for increasing the focus on safety culture for all plants included:

The assessment of performance deficiencies for cross-cutting aspects during the development of inspection findings Revision of IP 71152, Identification and Resolution of Problems, to modify the existing guidance for inspectors to assess the effectiveness of the corrective action program, the use of operating experience information, the results of independent and self-assessments, and the licensees safety conscious work environment.

Update of response procedures IP 71153, Event Follow-up, IP 93812, Special Inspection, and IP 93800, Augmented Inspection Team, to direct inspection teams to also consider contributing causes related to the safety culture components as part of efforts to fully understand the circumstances surrounding an event and its probable causes.

2

The staffs implementation of the program was communicated in RIS 2006-013, Information on Changes Made to the Reactor Oversight Process to More Fully Address Safety Culture, 4 and included the following summary of ROP changes discussed in SECY 06-0122 to implement the Commissions original direction, which the team used to guide the effectiveness review:

Provide better opportunities for the NRC staff to consider safety culture weaknesses and to encourage licensees to take appropriate actions before significant performance degradation occurs. (Column 1 & 2)

Provide the NRC staff with a process to determine the need to specifically evaluate a licensees safety culture after performance problems have resulted in the placement of a licensee in the degraded cornerstone column of the Action Matrix. (Column 3)

Provide the NRC staff with a structured process to evaluate the licensees safety culture assessment and to independently conduct a safety culture assessment for a licensee in the multiple/repetitive degraded cornerstone column of the Action Matrix. (Column 4)

The group determined that to be considered effective, the program should be able to demonstrate that it is meeting each of these objectives. The group concluded that while the third objective is fully met under the ROP framework, there is opportunity for improvement in the ROPs approach to meeting the first two objectives.

In practice, ROP review and assessment of safety culture for plants in the Multiple/Repetitive Degraded Cornerstone Column of the Action Matrix (Column 4) have been highly effective at identifying factors that contributed to the performance degradations that resulted in Column 4 assessment.

When an NRC safety culture assessment has been leveraged as allowed under IP 95002, Supplemental Inspection Response to Action Matrix Column 3 (Degraded Performance)

Inputs, it has been effective at addressing issues for plants in the Degraded Cornerstone Column (Column 3) of the Action Matrix. However, prior to this effectiveness review, changes implemented in 2015 that increased the number of White inputs required before entry into column 3 of the Action Matrix did not appear, based on SECY-15-0108, to fully consider the effect on safety culture oversight. Safety culture concerns since that change have arisen at plants in the Regulatory Response Column (Column 2) that, under the original implementation of the safety culture framework, might have been more effectively addressed with the type of safety culture review provided for Column 3 plants.

As for plants in the Licensee Response and Regulatory Response Columns (Columns 1 & 2) of the Action Matrix, inspection activities, and in particular the assignment of cross-cutting aspects to inspection findings, have allowed NRC staff to identify licensees with potential safety culture weaknesses, but the program has been challenged in some cases to encourage appropriate actions, before Action Matrix movement occurs. The conclusions of the team are summarized in Figure 1:

4 RIS 2006-013, Information on the Changes Made to the Reactor Oversight Process to More Fully Address Safety Culture, July 31, 2006, ADAMS Accession Number ML061880341.

3

Figure 1: This effectiveness review found that ROP safety culture oversight became progressively more effective at meeting the objectives summarized in RIS 2006-013 as plants moved across the Action Matrix.

Potential Gaps Identified in Current Implementation of Safety Culture in the ROP Based on this effectiveness review, the team has identified the following opportunities for improvement:

SECY 06-0122 states, IP 71152 was updated to include explicit requirements for reviewing certain attributes of a healthy safety culture, including the effectiveness of the corrective action program, operating experience, independent and self-assessments, and safety conscious work environment.

A thorough review of effectiveness of problem identification and resolution can identify trends in other safety culture traits. Additional guidance in the inspection procedure for following through on these trends, including the addition of trained safety culture assessors to the team when appropriate, would facilitate enhanced continuing communication of inspector observations about developing trends and encourage licensee action prior to significant performance declines.

In SRM-04-0111, the Commission directed: Staff should include as part of its enhanced inspection activities for plants in the Degraded Cornerstone Column a determination of the need for a specific evaluation of the licensees Safety Culture.

Evaluation of the impact of changing the threshold for Column 3 in 2015 focused on the additive risk impact of multiple White Action Matrix inputs, but there is no indication that this evaluation considered the impact of the change for safety culture oversight. Prior to 2015, NRC would independently assess two White findings for safety culture contributors under IP95002, however when the post 2015 changes occurred, the IP95001 procedure does not include an independent assessment, only confirmation from licensees that they considered safety culture weaknesses. There is no indication in SECY-15-0108 that the changes considered impacts to oversight of safety culture. This has created a gap where the safety culture attributes contributing to multiple White inputs are not receiving the level of objective NRC consideration that were explicitly of interest to the Commission during the formulation of the program. However, a differing perspective expressed by some staff and managers is that the risk-informed decision in 2015 to change the threshold of licensee performance to enter Column 3 appropriately changed the threshold for all significant oversight actions, including the determination of the need for a specific evaluation of the licensees Safety Culture, to occur at 3 white findings instead of 2 consistent with Commission direction in SRM-04-0111.

4

SECY 2006-0122 states:

The event response procedures IP 71153, Event Follow-up, IP 93812, Special Inspection, and IP 93800, Augmented Inspection Team, were revised to direct inspection teams to also consider contributing causes related to the safety culture components as part of efforts to fully understand the circumstances surrounding an event and its probable causes.

While identifying safety culture contributors as part of the fact-finding is mentioned in event response and reactive inspection procedures, in practice this is limited to assignment of cross-cutting aspects to any identified performance deficiencies rather than their broader contribution to inspector understanding of the event.

SRM-04-0111 states, Staff should consider developing an enhanced training program for its inspectors and resident inspectors on Safety Culture.

Training for inspection staff is currently limited to an Individual study activity in Appendix B of Inspection Manual Chapter (IMC) 1245, Qualification Program for New and Operating Reactor Programs, a computer-based training course on the fundamentals, and some discussions of safety culture concepts imbedded within various other training courses such as Root Cause Evaluation and Conducting inspections.

When non-SCWE safety culture concerns are identified for plants in Column 1 or Column 2, there is no inspection procedure for reviewing licensee safety culture or the effectiveness of any actions taken beyond what is provided for under the ROP cross-cutting issues program.

Assessment of Effectiveness To perform the effectiveness review, the team reviewed the information listed in the teams Charter (ADAMS at Accession No. ML22045A867). The teams review of information and data focused on 15 years worth of data since the current framework has been in place, 2007-2021.

The team reviewed the history of the program (including extensive interactions with industry, as detailed in the background section included as attachment 1); allegation data (as well as allegations specifically associated with a chilled work environment); observations and lessons-learned reports from Column 4/Column 5 plants; inspection reports and selected observations from Column 1, Column 2, and Column 3 plants; feedback from inspectors about the effectiveness of the current program and possible improvements; and NRC management feedback from earlier effectiveness reviews related to safety culture. The team also reviewed inspection guidance and sample requirements related to safety culture in the following inspection governance documents:

IP 71152, Problem Identification and Resolution; IP 93100, Safety Conscious Work Environment Issue of Concern Follow-up; IP 95003, Supplemental Inspection Response to Action Matrix Column 4 (Multiple/Repetitive Degraded Cornerstone) Inputs; IP 40100, Independent Safety Culture Assessment Follow-Up; 5

IP 95003.02, Guidance for Conducting an Independent NRC Safety Culture Assessment; IP 95002, Supplemental Inspection Response to Action Matrix Column 3 (Degraded Performance) Inputs; IP 95001, Supplemental Inspection Response to Action Matrix Column 2 (Regulatory Response) Inputs; IP 93800, Augmented Inspection Team; IP 93812, Special Inspection; Inspection Manual Chapter (IMC) 0305, Operating Reactor Assessment Program; and IMC 0310, Aspects within the Cross-Cutting Areas.

In performing the assessment, the team broke out the three objectives for the ROP safety culture program by Action Matrix column as delineated in the RIS and evaluated the effectiveness of each. Below we summarize the results of the assessment, starting with the most involved safety culture actions, for plants in the multiple/repetitive degraded cornerstone column of the Action Matrix, then moving through the degraded cornerstone and regulatory response columns, and finally the effectiveness of actions for plants in the licensee response column.

Assessment of Effectiveness: Column 4 - Multiple/Repetitive Degraded Cornerstone Objective: Provide the NRC staff with a structured process to evaluate the licensees safety culture assessment and to independently conduct a safety culture assessment for a licensee in the multiple/repetitive degraded cornerstone column of the Action Matrix.

Through a review of inspection report input and lessons learned feedback from the most recent plants inspected under IP 95003, the team determined that section .02 of IP 95003 allows NRC inspectors to develop a comprehensive understanding of licensee weaknesses in the safety culture traits that contributed to the performance issues that led to plant assessment in Column

4. Within the framework of Column 4 response requirements, NRC staff were able to communicate observations and concerns with licensees and encourage the actions necessary to address underlying issues.

For example:

Licensee 1: In response to the NRC teams concerns, [the licensee] performed a common cause analysis of all of the safety culture attributes that were identified in the

[root cause evaluations] in order to assess the collective significance and causes From this cause evaluation, [the licensee] identified the following apparent cause: [The licensee] did not have an adequate explicit management focus on safety culture and the associated infrastructure to support a healthy nuclear safety culture. This apparent cause allowed the specific nuclear safety culture weaknesses to exist at [the licensee]

and to impact the ability of the leadership team to recognize and address the overall decline in nuclear safety culture. (ML16161B279)

Licensee 2: The NRC team concluded that [the licensees] nuclear safety culture evaluations were comprehensive. The evaluation report documented multiple conditions that contributed to the failure at the site and corporate level to identify and arrest declining performance. The NRC team also concluded that the identified corrective 6

actions, if properly implemented, could be effective in addressing nuclear safety cultures declines at [the site]. (ML17129A217)

No plant that has undergone an independent NRC safety culture assessment under IP 95003 since it was revised following the issuance of RIS 2006-013 has moved back to Column 3 or Column 4. While only 5 plants have had a safety culture assessment under IP 95003 since 2006, this indicates that the comprehensive IP 95003 process, including the safety culture assessment, allowed licensees to identify and comprehensively address the issues underpinning the events that led to the Column 4 assessment, contributing to sustainable levels of performance.

Conclusion The team determined that the ROP is highly effective at meeting safety culture objectives for plants in Column 4.

Recommendations None.

Assessment of Effectiveness: Column 3 - Degraded Cornerstone Objective: Provide the NRC staff with a process to determine the need to specifically evaluate a licensees safety culture after performance problems have resulted in the placement of a licensee in the degraded cornerstone column of the Action Matrix.

IP 95002 states, Independently determine whether the licensee root cause, extent-of-condition, and extent-of-cause evaluations appropriately considered if any safety culture component caused or significantly contributed to a significant performance issue If the licensee did not consider whether a possible weakness in a particular safety culture component could have been a root cause or a significant contributing cause of the deficiency, and if the inspectors determined that a weakness in the same component could reasonably have been a root cause or a significant contributing cause of the deficiency, then an independent NRC evaluation should be performed.

For plants in Column 3, this allows the NRC to perform a more in-depth review of safety culture if there are concerns about the licensees ability to appropriately identify and correct safety culture issues that contributed to performance declines. Pointedly, it does not require a significant investment of inspection resources for reviewing safety culture if there are no apparent concerns in this area, but it allows for the option. The potential for an expanded safety culture review, particularly if licensee causal evaluations fail to critically examine and address potential safety culture shortfalls that contributed to the performance decline, can serve as additional encouragement for licensees to give an appropriate level of attention to safety culture issues, including the performance of an independent safety culture assessment to identify underlying issues.

The following examples from IP 95002 inspection reports illustrate the impact that additional NRC scrutiny on safety culture during the inspection can have:

7

Licensee 3: As part of the cause analysis, the licensee identified root and contributing causes against the safety culture components that could have contributed to the issues [The] licensee determined that weaknesses in management oversight of supplemental workers, decision making, work practices, and the corrective action program were the most prevalent safety culture attributes. The licensee also included the results of 2010 and 2011 safety culture surveys and a 2012 independent assessment of the [affected] department. (ML13224A382)

Licensee 4: The inspectors independently determined whether the licensees root cause, extent-of-condition, and extent-of-cause evaluations appropriately considered if safety culture components caused or significantly contributed to the performance issues

[involved]. The inspectors also reviewed the third-party safety culture assessment that was performed prior to the inspection activities onsite The team identified a few additional attributes that the licensee should have considered. One such attribute, was in the area of operating experience (OE) The inspectors determined that the licensee was not effectively implementing the OE program to ensure that valuable information was reviewed and leveraged to preclude avoidable events from occurring. The team also identified benchmarking and training as gaps [in the licensees cause evaluations].

(ML21306A311)

In both these cases, the NRC staff documented and took appropriate regulatory action to address long standing concerns with licensee safety culture and the ability of the licensee to effectively address those issues.5 IP 95002 provided a process for NRC staff to perform an in-depth review of licensee actions, interview plant staff, and provide observations to the licensee about weaknesses in the cause evaluations and proposed corrective actions. Neither licensee has been assessed in Column 3 or 4 since these reviews.

Two separate licensees had been assessed with substantive cross-cutting issues in 2008 and entered Column 3 of the action matrix. Per the IP 95002 inspection procedure, inspectors performed an independent review of licensee root cause evaluations, determined that the licensee had appropriately identified the safety culture components that contributed to the performance deficiencies and decided not to exercise the option to conduct an independent NRC evaluation of safety culture. These sites were subsequently assessed in Column 3 again in 2010, and then Column 4 and IMC 0350 respectively by 2011. In these cases, the cross-cutting issues (CCI) program was identifying that there were potential safety culture weaknesses.

However, as stated in IMC 0310, the CCI program itself does not fully assess safety culture, just certain components of safety culture. Program reviews in 2014 and 2020 have found that while it is effective at identifying safety culture concerns, it is not always fully effective at communicating the concerns it does identify to encourage timely action to address safety culture weaknesses before performance declines occur.

Not every licensee in Column 3 necessarily needs a full NRC safety culture assessment. The team considered two licensees that were assessed in Column 3 in 2010 and 2012 respectively.

Neither station had prior indications of sustained safety culture weaknesses as shown by either 5 Licensee 3 had open substantive cross-cutting issues from 2007 through 2011 when the plant was assessed in Column 3 with a Yellow input to the Action Matrix. Safety culture concerns were documented in inspection reports for Licensee 4 as early as 2016, including ML16315A372 and ML17074A265.

Licensee 4 was assessed in Column 3 with a Yellow input to the Action Matrix in 2020.

8

accumulated cross-cutting aspects or observations from IP 71152 inspections. NRC staff inspection under 95002 found that both sites adequately identified safety culture aspects contributing to the performance decline. Since returning to Column 1, neither site has been assessed beyond Column 2 in the ten years since.

Change to Column 3 Assessment Criteria - Impact on Plants That Now Fall under Column 2 (Regulatory Response)

In SECY-15-0108, Recommendation to Revise the Definition of Degraded Cornerstone as Used in the Reactor Oversight Process, (ML15076A066), NRC staff recommended Commission approval to change the threshold for the definition of Degraded Cornerstone for Column 3 from two White inputs in one cornerstone to three White inputs in one cornerstone (or strategic performance area)6. The Commission approved this change,7 and it took effect with the end-of-cycle assessment for 2015. Shortly before the change, Licensee 2 (discussed in the Column 4 section above) had transitioned to Column 4 based on a failure to adequately address two White performance indicators in the Initiating Events cornerstone in a timely manner. Under the criteria at the time, the licensee was assessed in Column 3 for two White inputs in a cornerstone, and moved to Column 4 when they failed to effectively address the performance issues within four quarters (considered a repetitive degraded cornerstone). The IP 95003 inspection found multiple safety culture weaknesses that the licensee had failed to identify and address prior to the Column 4 assessment.8 Since the Degraded Cornerstone definition change, two sites that would have met the original criteria for assessment in Column 3 with two White inputs in a cornerstone were both identified by NRC staff as outliers in the accumulation of cross-cutting aspects in the Human Performance cross-cutting area. Changes to the cross-cutting issues program in 2015 mean that neither plant triggered a cross-cutting issue, though both tripped the threshold for a cross-cutting theme. Both plants experienced multiple White inputs, multiple reactive inspections, and were assessed in Column 2, Regulatory Response Column, multiple times from 2016-2021. IP 95001, inspection follow-up for plants in Column 2, does not provide the option for any safety culture assessment beyond verifying that the licensee has identified safety culture attributes that contributed to the performance issues.

After the 2015 changes to the column 3 criteria were approved, IP 95001 was updated to review common cause evaluations of multiple White inputs, but did not consider the need for more in-depth NRC evaluation of the safety culture considerations of those common causes. This effectiveness review has identified a gap between the original 2004 Commission direction and the current oversight provided in 95001. Multiple safety-significant issues indicate a higher potential for a deeper underlying safety culture issue that may be causing the performance deficiencies, yet the language of IP95001 does not provide any stronger tools for addressing or evaluating safety culture attributes as a potential common cause than it did when evaluating a single White input. A safety culture assessment would have been allowed if the plants had shifted to Column 3 as they would have under the previous definition, which might have prevented additional significant performance issues that did occur, including Licensee 4, referenced earlier in this section, which tripped the threshold for a Yellow input in the Initiating 6 The definition of a degraded cornerstone with respect to one Yellow input remained unchanged 7 SRM-SECY-15-0108 (ML15335A559) 8 Other plants that were assessed in Column 3 under the original criteria but which would now be assessed in Column 2, including Duane Arnold and Point Beach in 2013, did not exhibit any significant safety culture weaknesses and have not been assessed beyond Column 2 since.

9

Events Cornerstone, after 3 previous White inputs in the same cornerstone in 4 years.

However, a differing perspective expressed by some staff and managers is that the risk-informed decision in 2015 to change the threshold of licensee performance to enter Column 3 appropriately changed the threshold for all significant oversight actions, including the determination of the need for a specific evaluation of the licensees Safety Culture, to occur at 3 white findings instead of 2 consistent with commission direction in SRM-04-0111. In addition, the update made to IP 95001 to review common cause evaluations of multiple white inputs, directs inspectors to look for shared causes (e.g., Cross-Cutting Aspects as discussed in IMC 0310; shared systems, structures, and components; shared procedures, processes, or personnel) for programmatic weaknesses in performance. This change also inherently enables inspectors to evaluate if the common-cause evaluation for multiple white findings appropriately considers the safety culture traits in NUREG-2165, Safety Culture Common Language, referenced in IMC 0310-06 and is consistent with a graded approach to oversight for licensees which are in Column 2, but may have multiple white findings.

Conclusion The ROP has effectively implemented a process that can address safety culture weaknesses for plants in Column 3. However, the risk-informed changes made to the criteria for reaching Column 3 (change from two to three white findings) and the associated NRC supplemental inspection activity have similarly impacted the ROP safety culture program. The effectiveness is potentially reduced by delaying the point at which a determination of the need for a specific evaluation of the licensees Safety Culture would occur. Safety culture concerns since that change have arisen at plants in the Regulatory Response Column (Column 2) with multiple White inputs that, under the original implementation of the safety culture framework, might have been more effectively addressed with the type of assessment provided for Column 3 plants.

Recommendation One:

IMC 0305 and IP 95001 should be updated to allow for an independent NRC evaluation of safety culture for Column 2 plants if the circumstances warrant. This would not require the licensees to perform an independent third-party safety culture assessment. It would provide for independent NRC consideration of the need for sampling and assessing safety culture.

Assessment of Effectiveness: Column 1 & 2 - Licensee Response / Regulatory Response Objective: Provide better opportunities for the NRC staff to consider safety culture weaknesses and to encourage licensees to take appropriate actions before significant performance degradation occurs.

Following the direction of SRM-SECY-04-0111, oversight of safety culture was enhanced in the ROP within the framework of the pre-existing ROP program, which included the cross-cutting areas of Human Performance, PI&R, and SCWE. This was accomplished by directing inspectors to assign inspection findings - issues of concern that were determined to be more-than-minor performance deficiencies as defined in IMC 0612 App. B - with the cross-cutting aspect that is the primary cause or most significant contributor to the performance deficiency.

Cross-cutting aspects are defined in IMC 0310.

10

The Cross-Cutting Issue program provides a process to inform licensees that the NRC has a concern with the licensees performance with respect to safety culture attributes under the cross-cutting areas and encourage the licensee to take appropriate actions before more significant issues emerge. From 2006-2015, a substantive cross-cutting issue was identified if a licensee met both of the following criteria:

four or more findings with a given cross-cutting aspect in a four-quarter timeframe during the mid-cycle or end-of-cycle review.

the Region had concerns about the licensees ability to address those issues.

A 2014 review of the program found that the threshold of four findings was too low to allow licensees to take meaningful action prior to reaching the threshold.9 Furthermore, while it is impossible to determine how many licensees took action to prevent more significant performance declines because the existing threshold prompted timely action, the review at the time did not identify any correlation between licensees assessed with a substantive cross-cutting issue and licensee performance as assessed in the action matrix. In addition, there were licensees with multiple cross-cutting issues identified over several assessment cycles that did not appear to take any meaningful action, raising questions about whether the program was meeting its objective to encourage appropriate actions.

A working group was established to develop recommendations for increasing the effectiveness of the program.10 As a result of this review, IMC 0305 was updated to change the definition of a cross-cutting issue11 to three consecutive cycles with a common cross-cutting aspect assigned to at least six inspection findings, or a backstop number of inspection findings - 20 findings in the Human Performance cross-cutting area or 12 findings in the PI&R cross-cutting area.12 The thresholds established following the 2014 review were based on an analysis of cross-cutting aspect assignments and inspection findings from 2009-2013, identifying outliers in both the number of cross-cutting aspects, plants with repetitive/long-standing cross-cutting issues, and plants with noted performance issues. After remaining relatively steady from 2006-2014, the number of Green ROP inspection findings declined by almost 75% from 2015-2020 before rising again beginning in 2021, a decline driven largely by a change to NRC implementation of the inspection finding screening process.13 As discussed in detail in the 2020 Cross-Cutting Issues 9 Effectiveness Review of Substantive Cross-Cutting Issues, April 23, 2014, ADAMS Accession Number ML14099A171 10 Working Group Recommendations to Revise the SCCI Process, December 3, 2014, ADAMS Accession Number ML14328A544 11 The designation was changed from substantive cross-cutting issue to cross-cutting issue as a result of a separate recommendation in the working group paper.

12 The definition for a cross-cutting issue in the SCWE cross-cutting area - 1 finding tagged with a SCWE cross-cutting aspect or a chilling effects letter or an escalated enforcement action/confirmatory order involving discrimination - remained unchanged, except that it must apply over four assessment cycles (2 years) and not be considered an isolated issue before a cross-cutting issue is opened.

13 As discussed in SECY 22-0029, Reactor Oversight Process Self-Assessment for Calendar Year 2021, ADAMS Accession Number ML22033A287, staff analysis of the drop in inspection findings determined that the trend is primarily driven by an increased emphasis on risk in the inspection finding screening process. While some staff consider that this trend, along with declines in other indicators, demonstrate an improvement in licensee performance over the past twenty years of the ROP to the point where improvements in the oversight of safety culture are not indicated, staff analysis of this trend has found 11

Effectiveness Review,14 while still providing some encouragement to monitor cross-cutting aspects, the changes made may be less responsive to crosscutting behavior indicators impacting the ability of the cross-cutting issue program to encourage actions.

Below, this objective is broken out in its two pieces to assess the programs effectiveness at considering safety culture weaknesses as well as its effectiveness at encouraging action before significant performance degradation occurs.

Consider Safety Culture Weaknesses There have not been any plants assessed as having cross-cutting issues in Human Performance or PI&R since the change to the threshold in 2015. Nonetheless, indicators of weaknesses in plant safety culture have been identified through the inspection process. The assignment of cross-cutting aspects identifies areas where inspectors can raise observations with licensees about developing weaknesses; IP 71152 samples allow inspectors to review trends in specific safety culture attributes that licensees are tracking; and allegations of a chilled work environment result in targeted inspection follow-up by a trained safety culture assessor using IP 93100.

It is notable that across 15 years of implementing the cross-cutting issues program, the fraction of findings assigned a cross-cutting aspect has remained remarkably stable, as shown in Figure 2.15 This indicates that inspectors in general have a good understanding of the program and its intent; that they are able to identify safety culture issues with respect to their impact on performance deficiencies; and it validates the premise of the cross-cutting issue program as one means of identifying potential safety culture weaknesses, even when cross-cutting issues are not actually assessed.

To meet Commission direction in SRM-SECY-04-0111 to ensure inspectors are trained on safety culture, the basic inspector qualification was updated to include an activity specifically focused on safety culture. Additionally, required inspection courses discuss the assignment of cross-cutting aspects to inspection findings and the application of safety culture attributes in root cause evaluations. Nonetheless, feedback from inspectors with additional training in safety culture, particularly as part of the IMC 1245 App. C-12 Safety Culture Assessor qualification, noted that with additional training they were able to recognize developing safety culture issues to engage the licensee in discussion before more significant concerns emerged.

that it is not readily attributable to licensee performance overall or improvements in safety culture specifically.

14 Results of the ROP Self-Assessment Effectiveness Review of the Cross-Cutting Issues Program, September 21, 2020, ADAMS Accession Number ML20239A806 15 The noticeable dip in 2013 occurred with the implementation of the common language. This shifted the labels used for individual aspects, and changed the language, particularly for many of the Human Performance cross-cutting aspects.

12

Percent of Green Findings with a Cross-Cutting Aspect All CCAs Human Perf PI&R 100%

80%

60%

40%

20%

0%

2007 2008 2009 2010 2011 2012 2013 2014 2015 2016 2017 2018 2019 2020 2021 Figure 2: Approximately 80% of Green findings have been tagged with a cross-cutting aspect each year since the current cross-cutting issues framework was established.

Conclusion The assignment of cross-cutting aspects to inspection findings under the ROP provides an effective mechanism for identifying potential safety culture weaknesses. While inspectors are aware of the process for applying cross-cutting aspects to findings and employ it consistently, there is room for development of a better understanding of the impacts of safety culture on a wider scope of inspection activities.

Recommendation Two Improve the training available to inspectors to better meet Commission objectives stated in SRM-SECY-04-0111. (Note that steps are already in process to support this recommendation, as NRR has a User Need open with RES to enhance training on safety culture based on review of ongoing inspector feedback and projected workload needs.)

Encourage Action Before Significant Performance Degradation Occurs When determining whether this program objective has been met, it is necessary to establish what is meant by the term significant performance degradation. While the term significant degradation in safety performance is used in Figure 1 of IMC 0305 to describe Column 4 of the Action Matrix, IMC 0305 also defines Safety Significant as Having greater than very low (i.e.

Green) safety significance. From a qualitative perspective, 99% of assessment inputs are Green, so anything greater-than-Green is by definition more significant. The Accident Sequence Precursor (ASP) Program defines a Precursor Event as a White finding in the reactor safety cornerstones or an event with a CCDP (or change in core damage probability, CDP) 1E-6.

IMC 0309, Reactive Inspection Basis for Reactors, establishes a CCDP or CDP 1E-6 as a lower threshold for considering a reactive inspection. For the purposes of data analysis during this evaluation, the team therefore established greater-than-Green as the threshold for 13

significant when establishing a degradation in performance. This also allowed the team to compare the risk-informed ASP data and reactive inspection information from events that are not necessarily related to degraded performance with data from the performance-driven Action Matrix.

As part of a holistic look at safety culture data, the team considered how developing safety culture issues might be flagged not just by cross-cutting aspect assignments, but also other indicators including an increase in allegations or concerns. Additionally, when considering potential impacts on safety from declining safety culture, the team looked at Action Matrix movement, but also precursor events as defined by the ASP program, and reactive inspections.

Figure 3: The team considered a variety of inputs that could be considered to fall under the designation of safety culture, and also considered multiple risk-informed outcomes IMC 0310 states, Although the presence of CCAs or the assignment of a cross-cutting issue may be indicative of a potentially degraded safety culture, the NRC draws conclusions about safety culture based on the results of licensee and NRC safety culture assessments conducted by qualified staff, not based on the presence of CCAs or cross-cutting issues.

The goal of this analysis was not to identify a specific threshold for what would constitute indications of safety culture weakness. Rather, the team was looking to determine whether the data supports the assumption that there is an association between safety culture weaknesses and declines in performance, and whether there is any indication that it might be a predictive association (that is, are safety culture weaknesses evident prior to safety/risk significant events indicative of degraded performance).

The team approached the analysis from two sides:

Quantitative analysis evaluating the accumulated inputs for plants in Column 1 as compared to the indicators of performance the following year Qualitative analysis from review of documentation from IP 95003 inspection reports and Lessons Learned reviews to determine if the safety culture assessment found issues that were apparent prior to Action Matrix movement 14

Quantitative Data Analysis Using risk ratio calculations to determine the increased risk of a plant exhibiting degraded performance given a range of the safety culture inputs described above, the team found the following statistically significant predictive associations:

Figure 4: This figure shows which safety culture inputs from Figure 3 showed a statistically significant association with each risk-informed indicator the following year.

The strength of the association varied, but for all of these, the result showed that plants in Column 1 of the Action Matrix with higher numbers of these parameters (at least 1.5 standard deviations above the mean) were twice as likely to have the associated risk-informed outcome the following year.

Looking at plants in the bottom 10th percentile for accumulating cross-cutting aspects each year (14 or more findings with a Human Performance cross-cutting aspect in a year, or at least six findings with a PI&R cross-cutting aspects in a year), this is illustrated below:

15

Human Performance Cross-Cutting Aspects

% Plants in Col 1 with 14 42.4%

% Plants in Col 1 with <14 17.5%

0.0% 10.0% 20.0% 30.0% 40.0% 50.0% 60.0% 70.0% 80.0% 90.0% 100.0%

Increased Oversight the Following Year Remained in Col 1 the Following Year Figure 5a: Looking at all plants in Column 1, 42% of plants with 14 or more Human Performance cross-cutting aspects in a year moved right in the Action Matrix the following year, compared with 18% of plants that had fewer than 14 Human Performance cross-cutting aspects.

PI&R Cross-Cutting Aspects

% Plants in Col 1 with 6 43.2%

% Plants in Col 1 with <6 17.0%

0.0% 10.0% 20.0% 30.0% 40.0% 50.0% 60.0% 70.0% 80.0% 90.0% 100.0%

Increased Oversight the Following Year Remained in Col 1 the Following Year Figure 5b: 43% of plants in Column 1 with six or more PI&R cross-cutting aspects moved right in the Action Matrix the following year compared with 17% of plants with fewer than six PI&R cross-cutting aspects The strongest associations were seen between cross-cutting aspects (both Human Performance and PI&R) and Action Matrix movement. This suggests that the assignment of high numbers of cross-cutting aspects reflecting safety culture attributes is more predictive of safety-significant events driven by performance versus risk-significant events overall (including those not associated with a performance deficiency). In other words, the accumulation of increasing numbers of cross-cutting aspects is a statistically meaningful indicator of the potential for performance decline, as measured by the occurrence of safety-significant events.

The strongest association for reactive inspections is not actually with the more extreme outliers

- the plants with the highest numbers of allegations or cross-cutting aspect assignments - but with plants in the 4th quintile (20th-40th percentile). The deterministic criteria of IMC 0309 used for the reactive inspection decision-making process include:

16

Involved repetitive failures Involved questions or concerns pertaining to licensee operational performance.

These two criteria are loosely associated with the cross-cutting areas of PI&R and Human Performance respectively. While a reactive inspection is not performed unless there is an event or discovery of degraded condition with risk significance to trigger it, it may be that awareness of developing safety culture weaknesses is a factor in determining when a reactive inspection is needed, particularly when the risk evaluation lies in the overlap region where there is discretion about whether to charter a reactive inspection or follow the event under the baseline inspection program. Nonetheless, while reactive inspection procedures mention consideration of safety culture components as described in SECY-06-122, the focus of the inspection charter is to establish immediate technical fact-finding specifically related to the event. In practice, safety culture is generally considered only in the assigned cross-cutting aspects if there is an identified performance deficiency. The team did not identify any documented reactive inspection activities and subsequent observations that examined the overarching safety culture impact on the event occurance, even when the deterministic criteria driving the inspection had strong safety culture ties. However, if an event results in significant inspection findings, inspectors will have the opportunity to evaluate the contribution of safety culture aspects to the cause of any underlying performance deficiencies to the extent prescribed by the applicable supplemental inspection procedure. .16 Qualitative Data Analysis Plants that reach Column 4 of the Action Matrix, though rare (five instances since 200617) present a significant degradation of safety performance. Oversight and inspection for Column 4 plants consumes significant resources for both NRC and the licensee. As discussed above, IP 95003 provides for an independent NRC safety culture assessment of the site.

Each of these assessments has found that weaknesses in multiple safety culture traits contributed to the performance degradation experienced by the plants. In some cases the plant had multiple, open cross-cutting issues; other sites had experienced a history of repeated baseline PI&R assessments noting weaknesses in the corrective action program. For others, the performance degradation appeared at first glance to be a sudden change, but in hindsight the indicators of declining performance, including safety culture weaknesses, were apparent.

Examples below show the gaps identified during the follow-on lessons learned reviews conducted after each IP 95003 inspection.

Licensee 2: During the evaluated period, there were two cross-cutting themes identified.

The evaluation team determined that consistent with the existing program documents, Region I appropriately documented and supported its rationale that no substantive cross-cutting issues (SCCls) were identified as a result of the themes. (ML18157A040)

Licensee 5: Based on the review of inspection findings and associated cross-cutting aspect assignments, along with the substantive cross-cutting issues as communicated in the middle-of-cycle and end-of-cycle letters, the NRC used the process in accordance with the program. Moreover, the NRCs actions taken in response to longstanding 17 Palo Verde, Browns Ferry, Fort Calhoun (IMC 0350), ANO, and Pilgrim 17

substantive cross-cutting issues (SCCIs) used the full extent of the options available by the program. Nonetheless, these efforts and actions were ineffective in creating substantial and sustained improvement in the licensees performance. (ML13270A473)

Although NRC Safety Culture Assessments are just one of the many tools the NRC uses to address poor licensee performance, in every case reviewed, the inability of the ROP safety culture framework to encourage early meaningful action to address safety culture weaknesses was a contributing factor to the Column 4 assessment. The framework of the ROP is mainly reactive and performance-based, however, the safety culture framework is deliberately proactive, meaning that safety culture inspection tools were intended to identify developing or underlying issues in a formal, structured way, before they result in Action Matrix movement.

Changes to the cross-cutting issue program in 2015, which primarily addressed the threshold at which potential weaknesses might be communicated, did not fully consider the impact the changes would have on NRC oversight and assessment of safety culture or to encourage timely action to address safety culture weaknesses before performance declines occur.

SCWE as One Part of Safety Culture Safety Conscious Work Environment (SCWE) is defined as, an environment in which employees are encouraged to raise safety concerns without fear of retaliation, SCWE occupies a unique place in the ROP oversight of safety culture. As one of the three cross-cutting areas, it was incorporated into the ROP from Safety the beginning. It is included in the common language SCWE as the safety culture trait Environment for Raising Culture Concerns, and is available to tag findings in the baseline inspection program, though this has only been done three times.18 It also falls under traditional enforcement, through allegation follow-up or issuance of a Chilling Effects Letter, and is the only part of safety Figure 6: SCWE is one of the safety culture traits considered under the ROP, but is also culture that is tied to regulatory requirements in 10 a factor in traditional enforcement CFR 50.7 prohibiting discrimination against an employee for raising safety concerns.

When the NRC receives an allegation raising SCWE concerns, trained safety culture assessors will perform an inspection using IP 93100. While the procedure is narrowly focused on the SCWE trait, inspector interviews and review may identify additional issues in other safety culture traits that contributed to the development of SCWE concern.

Licensee 6: The NRC requested the licensee investigate a potential SCWE issue in the licensees Radiation Protection department. The licensee determined that the SCWE in the department was chilled. The licensee swiftly enacted corrective actions and was able to successfully effect change in the environment before performance in the department declined. The IP 93100 follow-up inspection showed that the environment was no longer chilled.

Licensee 7: Inspectors assessed the sites safety conscious work environment and determined that personnel were willing to raise nuclear safety concerns and felt they 18 Peach Bottom 2007, Watts Bar 2016, Wolf Creek 2017 18

were empowered to stop work when they identified issues. However, the inspectors identified two weaknesses in the sites Safety Culture associated with the Leadership, Safety, Values & Actions and the Respectful Work Environment safety culture traits.

IP 93100 is available under IMC 2515 App. C, meaning that it is not part of the baseline inspection program, but may be used with Regional Administrator approval. As currently written, it looks only at SCWE. As the examples above demonstrate though, the very act of performing the inspection opens the door to gain insights about other safety culture traits. The use of this procedure by appropriately trained inspectors demonstrates the potential for looking at safety culture within the inspection program to more effectively encourage action before significant performance issues develop.

Conclusion The existence of safety culture oversight within the ROP has likely prompted licensees to identify safety culture issues early and take action that they might not otherwise have taken, thereby avoiding further degradation of safety performance, though it is impossible to measure this quantitatively. However, even as it does appear to provide useful insights for flagging developing safety culture weaknesses, the program appears to be challenged in its objective to encourage action before significant performance degradation occurs.

Recommendation Three IP 93100 should be revised to cover all aspects of safety culture, in addition to Safety Conscious Work Environment. This will provide the NRC with a tool that can be used for following up on safety culture concerns (either in focused areas or as whole) to encourage meaningful action before licensees are assessed in Column 3.

The team has developed three options for Recommendation Three:

a. IP 71152 review of specific safety culture attributes should be expanded with a reference to the revised IP 93100 to provide guidance for a more comprehensive review of safety culture as necessary. Considerations already factored into resource allocations for PI&R inspections, including review of data trends and inspector observations, should add consideration for including a qualified safety culture assessor on the team if warranted.

The team estimates that this option would be essentially resource neutral.

b. Adjustments should be made to the existing inspection framework to allow for safety culture assessment when warranted using the revised IP 93100 procedure. This would leave IP 93100 as an App. C procedure, available for use at Regional Administrator discretion using criteria to be developed separately if this option is chosen. (The effectiveness of previous NRC management discussions with licensees based on quarterly trend reviews discussed in IMC 0305 may be one criterion for consideration).

IP 93800 and IP 93812 should be updated to reference the revised IP 93100 and consideration of including a qualified safety culture assessor on the inspection team, as well as to emphasize the importance of reviewing safety culture attributes as within scope of the fact-finding purpose of the inspection, to better align the use of these procedures with the intent of changes initiated in response to the original Commission direction.

19

c. A regular safety culture assessment, based on the revised IP 93100, should be included as part of the baseline inspection program on a tri-/biennial basis. This could be performed through a graded assessment allotting more or less resources depending on safety culture/SCWE related data amassed throughout the inspection cycle. This may require additional inspection resources beyond what is currently allocated, approximately 4-5 (which could include team members who are training to be qualified) team members on site for a week gives an estimate of 60-80 inspection hours. Although SRM-SECY 0111 specifically decided against a baseline inspection option for safety culture in lieu of the current approach, this effectiveness review finds that the cross-cutting issues program in combination with a periodic safety culture assessment would more effectively meet the stated objectives of the program.

During discussions with NRC Regional Administrators and Division Directors, an alternative recommendation was proposed which leverages the existing IMC 0305 quarterly ROP trend reviews of licensee performance. The recommendation is to revise existing assessment guidance to prompt NRC Management discussions with licensees on any safety culture aspects discussed during quarterly reviews. The recommendation would encourage licensee action to address safety culture aspects before significant performance degradation occurs. This recommendation would not require additional inspection or significant additional resources, consistent with the graded approach in SRM-04-0111 for licensees in Column 1 or Column 2 of the ROP Action Matrix.

Conclusion This effectiveness review concludes that oversight of safety culture in the ROP is:

Highly effective for plants in Column 4 Mostly effective for plants in Column 3 Partially effective for plants in Columns 1 and 2 o Effective as a source of information for identifying developing safety culture weaknesses o Partially effective at encouraging action before significant performance degradation occurs In the original Commission direction, the staff was directed to improve safety-culture oversight in the ROP. The intent of incorporating safety-culture oversight into the ROP was by design proactive, not reactive, as stated in GAO- 04-0415 and recommendation 3B in SECY-04-0111.

It is noteworthy that the reactive aspects of safety-culture oversight are more effective than the proactive aspects, though as stated previously, the existence of safety culture oversight within the ROP has likely prompted licensees to identify safety culture issues early and take action that they might not otherwise have taken, thereby avoiding further degradation of safety performance, though it is impossible to measure this quantitatively.

As described above, the team provides the following recommendations to increase the ability of the ROP to provide effective oversight of safety culture to meet program objectives:

1) IMC 0305 and IP 95001 should be updated to allow for an independent NRC evaluation of safety culture for Column 2 plants if the circumstances warrant.

20

2) Improve the training available to inspectors to better meet Commission objectives stated in SRM-SECY-04-0111. (Note that steps are already in process to support this recommendation, as NRR has a User Need open with RES to enhance training on safety culture based on review of ongoing inspector feedback and projected workload needs.)

3) IP 93100 should be revised to cover all aspects of safety culture, in addition to Safety Conscious Work Environment. This will provide the NRC with a tool that can be used for following up on safety culture concerns (either in focused areas or as a whole) to encourage meaningful action before licensees are assessed in Column 3, using one of the following three options:

a) IP 71152 review of specific safety culture attributes should be expanded with a reference to the revised IP 93100 to provide guidance for a more comprehensive review of safety culture as necessary. Considerations already factored into resource allocations for PI&R inspections, including review of data trends and inspector observations, should add consideration for including a qualified safety culture assessor on the team if warranted.

b) Adjustments should be made to the existing inspection framework to allow for safety culture assessment when warranted using the revised IP 93100 procedure. This would leave IP 93100 as an App. C procedure, available for use at Regional Administrator discretion using criteria to be developed separately if this option is chosen. (The effectiveness of previous NRC management discussions with licensees based on quarterly trend reviews discussed in IMC 0305 may be one criterion for consideration).

IP 93800 and IP 93812 should be updated to reference the revised IP 93100 and consideration of including a qualified safety culture assessor on the inspection team, as well as to emphasize the importance of reviewing safety culture attributes as within scope of the fact-finding purpose of the inspection to better align the use of these procedures with the intent of the changes initiated in response to the original Commission direction.

c) A regular safety culture assessment, based on the revised IP 93100, should be included as part of the baseline inspection program on a tri-/biennial basis. Although SRM-SECY-04-0111 specifically decided against a baseline inspection option for safety culture in lieu of the current approach, this effectiveness review finds that the cross-cutting issues program in combination with a periodic safety culture assessment would more effectively meet the stated objectives of the program. Note: this recommendation was explicitly rejected by the Commission in the past and would require Commission approval to go forward.

21

=

Background===

Incorporation of safety culture into the agencys oversight of operating power reactors first occurred in 1989 with the publication of the NRCs Policy Statement on the Conduct of Nuclear Power Plant Operations, drawing a nexus between the precursors of todays safety culture traits and safe nuclear power plant operations. The 1996 policy statement, Freedom of Employees in the Nuclear Industry to Raise Safety Concerns without Fear of Retaliation, underscored the necessity of a safety conscious work environment (SCWE), a key component of nuclear safety culture, to ensuring continued safe operations.

In 2002, investigations into the discovery of degradation of the reactor pressure vessel head at Davis-Besse Nuclear Power Station revealed that safety culture weaknesses were the root cause of the event. The NRC made significant changes to the Reactor Oversight Process (ROP) to strengthen the agencys ability to detect potential safety culture weaknesses during baseline inspections and performance assessments, even if performance remained in the licensee response column of the Action Matrix. The Davis-Besse Lessons Learned Task Force Report (LLTFR), issued on September 30, 2002, (ML022740211) discussed numerous causes of the Davis-Besse vessel head degradation including contributions related to SCWE and the lack of a questioning attitude. Section 3.3.4 of the LLTFR stated that the implementation of NRCs guidance relative to the inspection and assessment of employee concerns programs and SCWE did not result in the identification of existing problems at Davis-Besse within these areas, as discussed in Section 3.2.5. Recommendation 3.3.4(5) was included in the report and stated:

The NRC should review the range of NRC baseline inspections and plant assessment processes, as well as other NRC programs, to determine whether sufficient programs and processes are in place to identify and appropriately disposition the types of problems experienced at DBNPS [Davis-Besse Nuclear Power Station].

On August 30, 2004, in the Staff Requirements Memorandum (SRM) to SECY-04-011 (ML051160076), the Commission directed the staff to enhance the ROP to more fully address safety culture. Regulatory Issue Summary 2006-13, Information on the Changes Made to the Reactor Oversight Process to More Fully Address Safety Culture, dated July 31, 2006, (ML061880341) described to licensees the revised ROP. Most notably, the NRC revised the existing cross-cutting areas of human performance, problem identification and resolution, and SCWE to incorporate aspects that are important to safety culture. The intent of the revisions to the ROP was threefold:

1. Provide better opportunities for the NRC staff to consider safety culture weaknesses and to encourage licensees to take appropriate actions before significant performance degradation occurs.

2. Provide the NRC staff with a process to determine the need to specifically evaluate a licensees safety culture after performance problems have placed the licensee in the Degraded Performance Cornerstone column of the Action Matrix.

3. Provide the NRC staff with a structured process to evaluate the licensees safety culture assessment and to independently conduct a safety culture assessment for a licensee in the Multiple/Repetitive Degraded Cornerstone column of the Action Matrix.

22

The staff presented numerous options for Commission consideration in SECY-04-0111:

Option 1A: Publish in the Federal Register for comment a draft guidance document (attached to the SECY) that is intended to augment the 1996 Policy Statement Option 1B: Issue the draft guidance document as a Regulatory Issue Summary without further public comment Option 1C: Notice in the Federal Register a decision not to pursue further development of such guidance Option 2A: Revise the 1989 Policy Statement on the Conduct of Nuclear Power Operations to broaden its focus beyond the control room and licensed operators and update the definition and attributes of Safety Culture to improve alignment with international standards Option 2B: Emphasize the 1989 Policy Statement through development of a guidance document on Safety Culture Option 2C: Monitor industry developments to assess Safety Culture without additional guidance from the NRC Option 3A: Develop an inspection process for systematic assessment of Safety Culture with additional agency action if issues are identified Option 3B: Enhance the ROP treatment of cross-cutting issues to more fully address Safety Culture and allow for more agency action when a cross-cutting issue is identified Option 3C: Proactively work with international communities to develop objective performance indicators of possible safety culture problems Option 3D: Engage industry to develop an industry process for assessment of Safety Culture Option 3E: Develop criteria for and intervention strategies for when downward trends in SCWE or Safety Culture already exist, and the licensee has failed to take action The staff recommended Commission approval of options 1B and 2B and requested Commission authorization to explore the feasibility of options 3B, 3C, 3D, and 3E.

In SRM-SECY-04-0111, the Commission approved option 1A and disapproved options 1B and 1C, approved option 2C and disapproved options 2A and 2B, approved option 3B, supported option 3C with limitations to monitoring foreign developments within existing programmed levels, and disapproved options 3A, 3D, and 3E.

Thus, this effectiveness review focused on an assessment of whether options 1A, 2C, and 3B were effectively implemented and maintained consistent with the Commission direction in SRM-SECY-04-0111 and considering any subsequent Commission direction in the area of safety culture.

In November 2004, also in response to events at Davis-Besse, the Institute of Nuclear Power Operations (INPO) published Principles for a Strong Nuclear Safety Culture, which describes principles and attributes of a healthy nuclear safety culture as developed by an industry advisory group.

The incorporation of safety culture into the baseline inspection program of the ROP in 2006 was, by direction, to assess safety culture only in terms of the cross-cutting areas of SCWE, Human Performance, and Problem Identification and Resolution (PI&R), by assigning a cross-23

cutting aspect to each inspection finding. A broader, more holistic assessment of safety culture only happens following a decline in performance that leads to plant assessment in Column 3 or Column 4.

The lessons learned review following the Palo Verde IP 95003 Supplemental Inspection for Repetitive Degraded Cornerstones, Multiple Degraded Cornerstones, Multiple Yellow Inputs, or One Red Input inspection in 2009 identified improvements that could be made to the oversight of safety culture in Column 4. The changes were incorporated in the January 2009 revision to IMC 0305 Operating Reactor Assessment Program. The staff also revised IP 95003 and IP 71152, Identification and Resolution of Problems. Some of the more significant changes related to safety culture to IMC 0305 included: (1) lengthening the time period for considering safety conscious work environment (SCWE) items in the substantive cross-cutting issue (SCCI) process; (2) using a graded approach to NRC independent safety culture assessments to align with the corresponding changes made to the IP 95003 revision that was issued on January 15, 2009; (3) clarifying that for plants in Column 4 and in the Unacceptable Performance column (Column 5), the licensee is expected to perform a third-party safety culture assessment; and (4) clarifying that the generic SCCI closure criteria applies when the closure criteria are not specified in the assessment letter.

In 2009, in partnership with NEI and INPO, the nuclear power industry began an initiative to further enhance safety culture. The industrys process for monitoring and improving safety culture used INPOs principles and attributes of a healthy nuclear safety culture as a framework, as described in NEI 09-07. Through NEI, in partnership with INPO, the nuclear power industry pilot tested a broad initiative to monitor and improve its nuclear safety culture.

Four nuclear power plants volunteered to participate in the industrys pilot application of the Site Nuclear Safety Culture Process, documented in NEI 09-07.

During the Nuclear Strategic Issues Advisory Committee meeting on December 16, 2010, the industry adopted two components from NEI 09-07 for use at each power plant:

The first component is a nuclear safety culture monitoring panel (NSCMP), which is used at most sites. The NSCMP meets quarterly and is made up of management representatives from most departments and includes the employee concerns manager. The purpose of this panel is to self-assess and monitor trends and faint signals of safety culture before they manifest into larger, safety-significant issues.

The second component is a safety culture leadership team which, at most utilities, meets twice a year. The purpose of this meeting is to glean insights from the NSCMP about the overall health of the station and make decisions related to necessary changes to maintain a positive safety culture.

The NRCs final Safety Culture Policy Statement (SCPS) was published on June 14, 2011. The SCPS provides the NRCs expectation that individuals and organizations performing regulated activities establish and maintain a healthy safety culture that recognizes the safety and security significance of their activities and the nature and complexity of their organizations and functions.

Because safety and security are the primary pillars of the NRCs regulatory mission, consideration of both safety and security issues, commensurate with their significance, is an underlying principle of the SCPS.

24

Changes to the oversight of safety culture in the ROP were also made as a result of lessons learned from The ROP Independent Assessment report (ML14035A571), dated February 14, 2014. The 2014 ROP Assessment Report included a recommendation to perform a comprehensive analysis to determine whether the use of CCIs and safety culture provided regulatory value in terms of licensee safety performance for the resources expended. The report also recommended that the staff clarify and document the goals, purposes, uses, and desired outcomes associated with the inclusion of CCIs and safety culture in the ROP. This aligned with the staffs planned efforts to review the CCI program as part of an ongoing look in the 2013 timeframe at possible enhancements to the ROP. The effectiveness review report was transmitted on April 23, 2014 (ADAMS ML14099A171). An NRC staff working group subsequently recommended the following changes to the CCI program (ADAMS ML14309A612):

Increase the threshold for a cross-cutting theme to six for all cross-cutting aspects, except for SCWE.

The subjective questions should be revised to set more objective criteria for opening an SCCI.

For Column 4 plants, all SCCIs would be closed out in the Confirmatory Action Letter (CAL) closing out the 95003 inspection.

Develop a backstop at the cross-cutting area level.

Develop standard SCCI closure criteria.

Consider additional actions for licensees after the second consecutive assessment cycle with the same SCCI.

In March 2014, the staff published NUREG-2165, Safety Culture Common Language, which documents the outcomes of public workshops to develop a common language to describe safety culture in the nuclear industry. The purpose of this initiative was to align terminology used by both licensees and the NRC when describing safety culture at nuclear power facilities. These workshops, held in December 2011, April 2012, November 2012, and January 2013, included subject matter experts from the NRC, the nuclear power industry, and the public. The Common Language was finalized and agreed upon at the January 2013 workshop. The NRC staff uses the agreed-upon common language to implement elements of its programs that provide oversight of regulated activities. Parts of the common language were incorporated into the ROP for operating nuclear reactors. All changes to oversight programs, including the ROP, have been documented in their associated Inspection Manual Chapters and Inspection Procedures.

Program changes based on the 2014 effectiveness review were implemented in the April 9, 2015, revision to IMC 0305. Cross-cutting areas and aspects were not modified, though the threshold for a cross-cutting theme was elevated from four to six findings with a common aspect. Backstops were added such that a cross-cutting theme would also be defined as 20 total findings in the human performance cross-cutting area and 12 total findings in the PI&R cross-cutting area. Along with adjusting the definition of a cross-cutting theme, the subjective criteria for considering whether to open an SCCI were removed. The revised program also resulted in removal of the term substantive such that a cross-cutting issue would be opened once a cross-cutting theme was present for three consecutive assessment periods.

At a November 2017 ROP public meeting, NEI and other utility representatives gave a presentation about the current monitoring of safety culture industrywide. Most sites still use NEI 09-07, Revision 1, issued March 2014, which provides different options for implementing the monitoring program. In the first quarter of 2018, the staff conducted an effectiveness 25

review on industrys implementation of the NEI 09-07 program and has determined that the current process for monitoring and assessing safety culture at nuclear power plants, including the industrys implementation of NEI 09-07, is effective.

In 2018, the Reactor Assessment and Human Factors Branch in the Division of Inspection and Regional Support (DIRS) (now DRO) sent a survey (refer to Enclosure 1 for additional details) to a sample of regional inspectors from all regions, to gain insight on the effectiveness of these programs. As discussed above, feedback on the common language initiative was also specifically solicited from industry representatives, including NEI, and other external stakeholders at monthly ROP public meetings. The survey results indicated that the safety culture common language is viewed favorably, and the changes made to the previous language were deemed as improvements. DIRS did not identify any additional areas for improvement or actions associated with this initiative.

26

References SECY-04-0111 Recommended Staff Actions Regarding Agency Guidance in the Areas of Safety Conscious Work Environment and Safety Culture Dated July 1, 2004 (ML041750238) and Corresponding SRM Dated August 30, 2004 (ML042430661)

GAO Report to Congressional Requesters (GAO-04-415): NRC Needs to More Aggressively and Comprehensively Resolve Issues Related to the Davis-Besse Nuclear Power Plants Shutdown Dated May 17, 2004 SECY-05-0187 Status of Safety Culture Initiatives and Schedule for Near-Term Deliverables Dated October 19, 2005 (ML052590283)

SECY-06-0122 Safety Culture Initiative Activities to Enhance the Reactor Oversight Process and Outcomes of the Initiatives Dated May, 24, 2006 (ML060660390)

NRC Regulatory Issues Summary (RIS) 2006-13 Information on The Changes Made to The Reactor Oversight Process to More Fully Address Safety Culture dated July 31, 2006 (ML061880341)

SECY-15-0108 Recommendation to Revise the Definition of Degraded Cornerstone as Used in the Reactor Oversight Process Dated December 2, 2015 (ML15076A066) and corresponding SRM-SECY-15-0108 (ML15335A559)

COMSECY-16-0022: Proposed Criteria for Reactor Oversight Process Changes Requiring Commission Approval and Notification Dated April 11, 2017 (ML17132A338)

Pilgrim Nuclear Power Station - Supplemental Inspection Report (Inspection Procedure 95003 Phase C) 05000293/2016011 And Preliminary Greater-Than-Green Finding Dated May 10, 2017 (ML17129A217)

Arkansas Nuclear One - NRC Supplemental Inspection Report 05000313/2016007 And 05000368/2016007 Dated June 09, 2016 (ML16161B279)

Watts Bar SCWE inspection Reports and Confirmatory Orders (ML16083A479, ML18229A153, ML18073A202, ML18242A458, ML17069A133, ML17208A647, ML19357A240, ML20218A483, ML20310A353, and ML21048A200)

Grand Gulf Nuclear Station - NRC Supplemental Inspection Report 05000416/2021040 And Assessment Follow-Up Letter dated November 18, 2021 (ML21306A311)

Results of the ROP Self-Assessment Effectiveness Review of the Cross-Cutting Issues Program (ML20239A806)

Dispositioning Of Cross-Cutting Issues Program Effectiveness Review Recommendations dated September 17, 2021 (ML21209A993)

Dispositioning Of Degraded Performance Column Effectiveness Review Recommendations, Dated September 17, 2021 (ML21209B001) 27

Working Group Recommendations to Revise the Substantive Cross-Cutting Issue Process (ML14328A544)

PI&R Comprehensive Review Final Report and Associated Documents Dated November 12, 2020 (ML20247J590)

Results of the Calendar Year 2018 Reactor Oversight Process Self-Assessment Effectiveness Review On The Safety Culture Program Dated January 9, 2019 (ML18219A687)

Inspection Procedure 95003: Evaluation of NRC Assessment and Inspection Processes at Pilgrim Nuclear Power Station: 95003 Lessons Learned Report Dated June 06, 2018 (ML18157A040)

Inspection Procedure 95003: Evaluation of NRC Assessment Process and Recommendations: Palo Verde 95003 Lessons Learned Report Dated February 01, 2008 (ML080320136)

Inspection Procedure 95003: Evaluation of NRC Assessment and Inspection Processes:

Arkansas Nuclear One (ANO) 95003 Lessons Learned Report Dated June 08, 2017 (ML17160A290)

Inspection Procedure 95003: Evaluation of NRC Assessment and Inspection Processes at Browns Ferry Unit 1: Browns Ferry 95003 Lessons Learned Report Dated September 27, 2013 (ML13270A473)

Lessons Learned from Implementing Inspection Manual Chapter 0350 at Fort Calhoun Station and Recommended Changes Inspection Procedure 95003: Evaluation of NRC Assessment Process and Recommendations: Fort Calhoun IMC 0350 Lessons Learned Report Dated May 08, 2014 (ML14128A376)

NRC Safety Culture Policy Statement Website (https://www.nrc.gov/about-nrc/safety-culture/sc-policy-statement.html) 28