ML23207A041
| ML23207A041 | |
| Person / Time | |
|---|---|
| Site: | National Bureau of Standards Reactor |
| Issue date: | 06/02/2023 |
| From: | US Dept of Commerce, National Institute of Standards & Technology (NIST) |
| To: | Office of Nuclear Reactor Regulation |
| Shared Package | |
| ML23207A039 | List: |
| References | |
| Download: ML23207A041 (1) | |
Text
FINAL REPORT INDEPENDENT THIRD-PARTY NUCLEAR SAFETY CULTURE ASSESSMENT OF THE NIST CENTER FOR NEUTRON RESEARCH JANUARY 31 THROUGH JUNE 2, 2023
INDEPENDENT THIRD-PARTY NUCLEAR SAFETY CULTURE ASSESSMENT Executive Summary This report is an independent and critical assessment of NCNR-ROEs performance compared to the INPO 12-012 / NUREG 2165 Nuclear Safety Culture Traits. This assessment establishes the state of NCNR-ROEs Nuclear Safety Culture and describes it by NSC traits and attributes. Further it is an initial benchmark of the facilitys performance to the INPO 12-012 / NUREG-2165 Traits. NCNR-ROE has been working to instill Nuclear Safety Culture behaviors into its organization and is making progress.
Much work remains. This report aims to identify gaps and make recommendations for improvement in several organizational areas. Organizations that have an established and sustaining healthy Nuclear Safety Culture are self-critical and continually challenge themselves to improve performance.
1.1 BACKGROUND
The National Institute of Standards and Technology (NIST) Center for Neutron Research (NCNR) owns and operates a heavy water-moderated nuclear test reactor and associated neutron beam research facility. The NCNRs reactor is licensed by the Nuclear Regulatory Commission (NRC) (USNRC License TR-5) under the name National Bureau of Standards Reactor (NBSR).
On February 3, 2021, the reactor exceeded its nuclear fuel design temperature limit that resulted in melted fuel. The Chief of NCNR Reactor Operations and Engineering notified the NRC of an alert concerning elevated radiation levels at the NBSR. Pursuant to the event notification received from NCNR staff on February 3, 2021, the NRC initiated a special inspection at the NBSR. The NRC staff issued an interim special inspection report on April 14, 2021, to provide an initial assessment of their understanding of the event sequence, consequences, and the NCNRs response to the event.
On October 1, 2021, NCNR submitted a restart request to the NRC. This restart request documented the NCNR-identified root causes and corrective actions to preclude recurrence of the February 3, 2021, event.
On March 16, 2022, the NRC released a final report of its initial conclusion from its Special Inspection Report (SIT). The NRCs final report confirmed and expanded on many aspects of NISTs analysis of the incident, pointing to deficiencies in policies, procedures, training, and Nuclear Safety Culture as contributing to the incident.
The SIT report included: (a) two apparent violations related to exceeding the fuel cladding temperature safety limit; (b) three apparent violations related to inadequate fuel handling, startup, and emergency response procedures; and (c) two apparent violations related to inadequate fuel handling within the vessel and inadequate modifications that invalidated operators ability to meet a TS requirement. In addition to the apparent violations, the SIT also identified weaknesses in the licensees root cause analysis as well as the proposed corrective actions.
FINAL REPORT JUNE 2, 2023 i
INDEPENDENT THIRD-PARTY NUCLEAR SAFETY CULTURE ASSESSMENT Following the issuance of the SIT report, the NRC staff and NIST engaged in an Alternative Dispute Resolution (ADR) process and agreed to a series of corrective actions that were documented in an NRC Confirmatory Order (CO). The goal of the ADR was to allow the NRC and NIST to reach agreement on actions necessary to restore the facility to its design basis conditions. The NRC staff and NIST held three ADR sessions between May 10 and June 2, 2022.
On August 1, 2022, a Confirmatory Order Modifying License No. TR-5 was issued by the NRC. The Confirmatory Order documents action completed and planned by NCNR as well as the commitments made by NIST to enable the safe operation of the NBSR.
NCNR engaged Advanced Technologies and Laboratories International, Inc. (ATL, Inc.)
to support recovery from this event and to enhance facility safety. This task-based contract covers a wide array of activities in six task areas. Task 1 is the conduct of an independent Third-Party Nuclear Safety Culture Performance Assessment (Assessment) of NCNR Groups that directly support Reactor Operations. These are Reactor Operations and Engineering, Health Physics, and Safety.
The Independent Nuclear Safety Culture Assessment Team (Task 1) was commissioned on January 26, 2023.
Three Assessment Team members made short scoping visits to the NCNR facility between mid-February and late March 2023. During these scoping visits team members received training, badging, were provided tours of the facility, and observed management meetings, Plan of the Day meetings, and Pre-Job Briefs. The Assessment Team members also interviewed operators, individual contributors, and management team members. The scoping visits provided significant context to the framing of the April 3-14, 2023 Team Assessment period. During an early scoping visit an NCNR manager referred to the February 3, 2021 fuel melt event as a Performance Deficiency Event.
During the second visit, two team members attended a public meeting during which the US NRC presented its restart review at NIST Headquarters on March 2, 2023. On March 10, the US NRC authorized NCNR to restart the research reactor. During the third and last scoping visit team members piloted the interview question set with several individuals and one Operations Focus Group consisting of three operators.
The Assessment Team developed and administered a Nuclear Safety Survey to the ROE group during the time frame February 24 - March 10. Upon closing, the survey had a 96% participation rate.
The Assessment Team analyzed the data from the 2023 survey, compared results to previous assessments, and developed question sets for individual interviews, Focus Group interviews, and Program Assessment reviews.
The Team conducted 23 individual interviews, 14 Focus Groups interviews, nine (9)
Program Assessment Reviews, and several Observations, including one reactor startup and power ascension to 100kw.
FINAL REPORT JUNE 2, 2023 ii
INDEPENDENT THIRD-PARTY NUCLEAR SAFETY CULTURE ASSESSMENT 1.2 NUCLEAR SAFETY CULTURE (NSC)
US NRC expectations to establish and maintain a positive Nuclear Safety Culture apply to all licensees (among other entities) that are subject to NRC authority; this includes NCNR.
Nuclear Safety Culture: The set of core values and behaviors resulting from a collective commitment by leaders and individuals to emphasize safety over competing goals to ensure protection of people and the environment.1 SCWE ConOpslndustrial Safety Human Pl&R Quality Culture Performance Culture The Team observed that a number of NCNR-ROE members equated Industrial Safety Culture with Nuclear Safety Culture. While Industrial Safety Culture is, indeed, part of NSC, so are many other components and programs beyond Industrial Safety. This Assessment discusses each of these areas, as well as the NSC Traits and Attributes, as they have shaped NCNRs current NSC.
Nuclear safety is a collective responsibility. The concept of Nuclear Safety Culture applies to every employee in the nuclear organization, from the highest level to the 1 NUREG-2165, Safety Culture Common Language, pg. 5; INPO-12-012 Rev 1, Traits of a Healthy Nuclear Safety Culture, pg. 6.
FINAL REPORT JUNE 2, 2023 iii
INDEPENDENT THIRD-PARTY NUCLEAR SAFETY CULTURE ASSESSMENT individual contributor. No one in the organization is exempt from the obligation to commit to safety first. While Nuclear Safety Culture is defined as the set of core values and beliefs it is measured by observing and assessing organizational behaviors related to the protection of people and the environment.
As a minimum, the groups that interface with Operations need to understand and embrace Nuclear Safety Culture. This independent NSC assessment was limited in scope to the NCNR-ROE, and the Health Physics and Safety organizations.
Other NCNR groups that interact with Reactor Operations were not assessed. The Neutron Condensed Matter Science Group, the Research Facility Operations Group, and other NIST groups that support NCNR need to understand and embrace Nuclear Safety Culture standards. These other groups support and interface with NCNR-ROE and should be a part of the overall effort to enhance the Nuclear Safety Culture at NIST.
The Independent Third Party Nuclear Safety Culture Assessment Team (Team) mission was to evaluate the Nuclear Safety Culture of NCNR-ROE against the Nuclear Safety Culture traits defined by the Common Language Initiative as documented in NUREG 2165/INPO 12-012 and alluded to in NRC Manual Chapter 0350 (Oversight of Reactor Facilities in a Shutdown Condition due to Significant Performance and/or Operational Concerns) and NRC Inspection Procedure 95003.02, (Guidance for Conducting an Independent NRC Safety Culture Assessment) as revised to reflect NUREG-2165.
Based upon this evaluation, the Team identified Areas For Improvement (AFIs) in the NCNR-ROE Nuclear Safety Culture.
1.3 ASSESSMENT PROCESS This Assessment focused on assessing traits and programs, and identifying Limiting Weaknesses.
Limiting Weaknesses: Those problems that limit current performance and, if addressed in the near-term, will lead to a step-change in observable and measurable performance.
Note: Failures to comply with regulations or regulatory commitments are automatically assumed to be limiting weaknesses.
1.3.1 Information Gathering The Nuclear Safety Culture survey was administered over a two-week period. The participants were provided anonymity. The questions were developed to measure the degree that the personnel in the survey group believe the traits have been implemented.
The survey results were generally higher (more positive) than results from other sources, such as from interviews. Leadership traits received the lowest scores. A small number of personnel responded with either all 5s (absolutely positive) or with mostly 1s FINAL REPORT JUNE 2, 2023 iv
INDEPENDENT THIRD-PARTY NUCLEAR SAFETY CULTURE ASSESSMENT (absolutely negative). If the results from these individuals were removed, the data correlated more closely with the results from interviews and field observations.
Following the survey, the Nuclear Safety Culture Assessment Team conducted individual interviews, focus group interviews, program review sessions with program owners, and field observations. The goal was to interview all individuals who had participated in the survey, and to observe activities in progress. While the survey indicated what personnel believe, the interviews and field observations indicated what is currently being implemented. As part of the interviews, personnel were asked to give traits a numerical grade near the end of the interview. These subjective ratings were significantly lower than the initial computerized survey results. Also, when asked to prioritize facility needs, the most common response was to increase operations staffing level. During the interviews, the interviewees were open, interactive, and provided key information on issues and needs.
Field observations were conducted on a limited basis. Based on the current plant conditions very few evolutions were conducted. A reactor startup was observed which was assigned a positive observation designation based on the conduct of the evolution, communications, and control of trainee. Some minor issues were noted but they were quickly addressed by the CRO. A number of Plan of the Day meetings were observed which indicated more structure is needed to improve its usefulness as a management tool.
The bulk of summary discussion in this report is distributed between NSC Traits (approximately 45 pages) and NSC-related Programs (approximately 23 pages). While Nuclear Safety Culture is defined as the set of core values and beliefs it is measured by observing and assessing organizational behaviors related to the protection of people and the environment. One of the primary means by which organizations establish, manage, and modify organizational behavior is through processes and their implementing procedures. Thus, a thorough evaluation of Nuclear Safety Culture must, necessarily, consider the key processes and what they indicate about the collective commitment of the organization to protection of people and the environment.
Program Reviews are required by NRC IP 95003 Attachment 2 (the requirements of which are applicable to this Assessment as stated in the Task Statement of Work from NCNR) and were clearly identified in the scope of the NSC Assessment Plan, which was reviewed and approved by NCNR prior to the on-site assessment phase.
1.3.2 Data Analysis Process
- 1. The Team analyzed the raw data captured from each evaluated data source (Functional analysis, Review of archived data; Structured interviews; Focus Groups; Observations; Safety Culture Survey; Program Reviews) to identify behavioral characteristics.
- 2. The Team analyzed the information captured by each individual data source to determine cultural themes.
FINAL REPORT JUNE 2, 2023 v
INDEPENDENT THIRD-PARTY NUCLEAR SAFETY CULTURE ASSESSMENT
- 3. Compared the behavioral characteristics and cultural themes derived from the seven data sources, then drew conclusions with regard to Limiting Weaknesses.
- 4. Compared conclusions with the NUREG-2165 / INPO 12-012 Safety Culture Framework Traits
- 5. Determined Assessment Categories:
Area of Strength (AOS)
Positive Observation (PO)
Area in Need of Attention (ANA)
Area for Improvement (AFI)
- 6. Developed Recommendations 1.4 ASSESSMENT CONCLUSIONS The Team drew the following conclusions:
The Team evaluated implementation of the NCNR Nuclear Safety Culture as a whole to be Marginally Effective. In the aggregate, programs that support Nuclear Safety Culture were in the early stages of development and implementation and were determined to be ineffective at this stage. However, many of the programs were trending toward the positive.
The analysis of the 2023 raw survey data did not reveal consistent cultural or behavioral themes. There were, however, several data patterns (discussed in the body of the report), that at first were seen as curiosities, then later were determined to be detracting from the credibility of the collective survey data rollup.
Interviews and Focus Groups provided the richest source of data, by far, that enabled the Team to understand the state of the Nuclear Safety Culture at NCNR-ROE.
Extensive interactions with personnel provided the Team with access to personal insights, real-time information exchanges, a chance to observe group interaction dynamics, and personal engagement.
Regarding basic understanding of Nuclear Safety Culture:
In the survey, over 90% of respondents stated they had a basic understanding of the term Nuclear Safety Culture.
Interviews and focus groups showed that only two individuals (out of 50+
participants) recognized that Nuclear Safety Culture was related to behaviors.
FINAL REPORT JUNE 2, 2023 vi
INDEPENDENT THIRD-PARTY NUCLEAR SAFETY CULTURE ASSESSMENT Nuclear Safety Culture Survey:
The Team observed that both the 2021 and 2023 raw survey data set each showed a high aggregate average; much higher than the Team has observed in culture surveys conducted at nuclear power facilities performing at average levels.
The survey data reported much higher response values than the comparative interview and focus group responses. In nearly every category, the Interview and Focus Group responses were lower than the corresponding 2023 raw survey data.
2023 data variation was very low for many statement responses (put another way, many respondents used just one number or predominantly one number when responding to 79 survey statements, including several managers).
Interviews, Focus Groups, and Program Reviews:
Many interview questions derived from highly rated survey statements elicited interview and focus group responses that in aggregate showed little corroboration. Thats one of many reasons this nuclear safety culture Team assessment was comprised of seven (7) assessment areas, and not solely on an organizational survey.
The Team conducted nine (9) Program Reviews.
The Team noted that the majority of the programs are either under development or in very early stages of implementation. Programs that have not experienced sufficient run-time to produce consistent, reliable results cannot be considered to be effective. The Team determined that three programs were marginally effective, and six were not effective. That said, the Team observed that NCNR-ROE was expending a great deal of thought and effort on initial program development and implementation.
The Team noted that one of the most currently limiting weaknesses at NCNR-ROE is insufficient funding to obtain and prioritize external resources who can take on:
Operations Department administrative workload and free up operators (see LA.1, Section 7.4.1.2.2, last table entry); and, Program development and implementation (see 7.3, Assessment
Conclusions:
Programmatic Support to NSC for discussion of nine key programs).
1.5 ASSESSMENT RECOMMENDATIONS Develop a corrective action plan to implement the 12 core recommendations presented below as soon as practical. NCNR-ROE should take the lead in setting the standards that all NIST groups that interface with the Reactor in any way, are educated in and commit to the ten NSC Traits and how these apply to their organizations.
FINAL REPORT JUNE 2, 2023 vii
INDEPENDENT THIRD-PARTY NUCLEAR SAFETY CULTURE ASSESSMENT
- 1. Designate Operations and the I & C portion of the Aging Reactor Management section as priority groups and develop communications and team building strategies to better clarify and address the quality of the safety culture environment and implement corrective actions.
During individual and focus group interviews, multiple individuals informed the assessors that a toxic work environment between Operations and I & C may exist.
2.Develop and deliver a competence-based understanding of Nuclear Safety Culture.
Develop and implement a formal NCNR-ROE Nuclear Safety Culture Program based upon INPO 12-012 Traits of a Healthy Nuclear Safety Culture / NUREG 2165 to understand Nuclear Safety Culture (including SCWE), Industrial Safety Culture, and the difference.
Consider implementing a Nuclear Safety Culture Monitoring Panel per NEI 09-7 Fostering a Healthy Nuclear Safety Culture.
Develop and implement the vision, values, principles, and objectives that define the expected behaviors and attitudes needed to achieve and sustain a high-reliability Nuclear Safety Culture.
Develop enabling objectives that focus on all NCNR-ROE leadership and staff to own Nuclear Safety Culture attributes and role-modeling the behaviors including, for example to (for instance):
o Building trust, o Breaking down silos, and o Improving daily communication behaviors.
Develop and implement NSC Training, and socialize expected NSC behaviors, teamwork and interactions amongst departments and between people.
Engage external resources to assist in developing, delivering, implementing, and coaching sustainable actions.
Interim: Re-implement daily use of the Gray Book INPO 12-012, Traits of a Healthy Nuclear Safety Culture.
- 3. Mitigate the staffing and resource shortage in the Operations Department:
Obtain and prioritize external resources (including funding) who can take on the Operations Department administrative workload (see LA.1, Section 7.4.1.2.2, last table entry).
Prioritize the engagement of external resources to coach Operations Crew Chiefs, SROs, and non-licensed operators regarding shift Operational Focus.
Complete the planned selection and appointment of a Deputy Chief Reactor Operations.
FINAL REPORT JUNE 2, 2023 viii
INDEPENDENT THIRD-PARTY NUCLEAR SAFETY CULTURE ASSESSMENT
- 4. Develop, with Human Resources:
An effective and timely policy to attract and retain Operations staff.
Build Operator ranks with full support to stand up a fifth shift with a prescribed minimum reserve bench metric. Aggressively pursue increasing the number of licensed individuals.
Mitigate the resource shortage in the Aging Reactor Management (ARM) Group with external resources to:
o Take on the administrative tasks.
o Develop and implement organization-wide programs (e.g., CAP, Observation Program, et.al.).
o Coach and mentor ARM staff with regards to programmatic responsibilities.
- 5. Corrective Action Program: Complete the PI&R Assessment as soon as practicable. Continue with plans to implement a right-sized PI&R process. In the interim, prioritize the following immediate actions:
Establish and enforce an expectation for a reporting threshold across NCNR-ROE.
Select external resources to assist with program implementation, administration, and coaching.
Develop and implement an easy and personal input method for CAP.
Establish a Corrective Action Review Board (CARB) / Management Operations Review Group utilizing external resources to assist with development, charter, startup, administration, and member coaching.
- 6. Conduct behavioral assessments of the Operations Crew Chiefs to assess attitudes, alignment, leadership capabilities, and provide targeted development feedback.
Behavioral assessments, feedback and coaching have been shown to be critical to aligning leadership behavior to obtain consistent performance improvements.
Consider extending these assessments to other key leaders as priorities allow.
- 7. Develop a behavior-based mentoring/coaching program focused on safety culture attributes, leadership, and process improvements.
Provide the top leadership team with a team coach at the Director/Chiefs level to promote organizational alignment and teamwork.
Identify key leaders in ROE and establish individual mentors to develop manager behaviors and their interface to the organizational recovery processes.
Conduct behavioral assessments of current organizational leadership starting at Crew Chiefs level and going to top level management (Director/Chiefs).
FINAL REPORT JUNE 2, 2023 ix
INDEPENDENT THIRD-PARTY NUCLEAR SAFETY CULTURE ASSESSMENT
- 8. Training and Procedures Programs: Complete the Training and Procedures Assessments as soon as practical.
Implement the Systematic Approach to Training (SAT).
Implement the PPA / INPO 11-003 requirements for procedures.
- 9. Establish and maintain housekeeping/combustibles and gas cylinder loading that reflect OSHA standards in:
Guide Hall, Confinement Building, Laboratories, and Other spaces.
- 10. Complete the planned development and implementation of an Employee Concerns Program (ECP).
Strongly recommend for the foreseeable future, establish and maintain a regular and consistent onsite presence.
Include within the scope of follow-on Nuclear Safety Culture Assessments.
- 11. Develop and implement a problem-solving approach for emergent issues to include:
Define the Problem Statement.
Conduct significance assessment.
Identify known consequences.
Perform extent of condition evaluation.
Pre-establish a Command-and-Control NCNR-ROE response structure.
Establish response teams with issue-specific roles & responsibilities.
Develop a written response plan.
Implement and revise response plan as necessary.
- 12. As a function of development of the comprehensive improvement plan that results from planned assessments and the collective evaluation process, consider the following elements to address Cross-Cutting, Distributed Function Programs.
Every line manager owns a piece of distributed functions; examples of distributed functions include, but are not limited to:
Nuclear Safety Culture and Roles, Responsibilities, Authorities, and Accountabilities Emergent Issue Management Process FINAL REPORT JUNE 2, 2023 x
INDEPENDENT THIRD-PARTY NUCLEAR SAFETY CULTURE ASSESSMENT Conduct of Operations Operational Focus PI&R/Corrective Action Program Employee Concerns Program Training Management Observation Program Work Planning, Management and Control Management of Change QA and Oversight Function Risk Management Obtain and prioritize external resources (including funding) to support development and implementation of the programs listed above.
In the interim, establish a daily framework: Morning meetings, daily schedule reviews, CAP Screening, Management Reviews, Plan of the Day, Break-out groups as needed.
FINAL REPORT JUNE 2, 2023 xi
INDEPENDENT THIRD-PARTY NUCLEAR SAFETY CULTURE ASSESSMENT Table of Contents Executive Summary ..................................................................................................................... i 1.1 Background.................................................................................................................... i 1.2 Nuclear Safety Culture (NSC) ...................................................................................... iii 1.3 Assessment Process ................................................................................................... iv 1.3.1 Information Gathering ........................................................................................... iv 1.3.2 Data Analysis Process ........................................................................................... v 1.4 Assessment Conclusions ............................................................................................. vi 1.5 Assessment Recommendations .................................................................................. vii 2.0 Assessment Objective ........................................................................................................ 1 3.0 Nuclear Safety Culture ........................................................................................................ 1 4.0 Assessment Scope ............................................................................................................. 2 5.0 Team Members................................................................................................................... 3 6.0 Assessment Process and Criteria ....................................................................................... 7 6.1 Assessment Process ................................................................................................... 7 6.1.1 Methodology ......................................................................................................... 7 6.1.2 Data Analysis and Assessment ............................................................................. 9 6.2 Assessment Criteria ....................................................................................................10 7.0 Assessment Results ..........................................................................................................11 7.1 Assessment
Conclusions:
Summary ...........................................................................11 7.2 Insights by Assessment Activity ..................................................................................12 7.2.1 Nuclear Safety Culture Survey .............................................................................12 7.2.2 Interviews and Focus Groups ..............................................................................16 7.2.3 Workplace and Meeting Observations .................................................................17 7.3 Assessment
Conclusions:
Programmatic Support to NSC ..........................................18 7.3.1 Management Observation Program .....................................................................22 7.3.2 Corrective Action Program ...................................................................................23 7.3.3 Audit/Surveillance/QA Program ...........................................................................26 7.3.4 Document Control Program .................................................................................28 7.3.5 Conduct of Operations Program...........................................................................30 7.3.6 Training Program .................................................................................................33 7.3.7 Procedures ..........................................................................................................35 7.3.8 Causal Analysis Program .....................................................................................38 7.3.9 Employee Concerns Program ..............................................................................40 FINAL REPORT JUNE 2, 2023 xii
INDEPENDENT THIRD-PARTY NUCLEAR SAFETY CULTURE ASSESSMENT 7.4 Assessment
Conclusions:
Nuclear Safety Culture Traits .............................................43 7.4.1 Leadership Safety Values and Actions (LA) .........................................................47 7.4.2 Problem Identification and Resolution (PI) ...........................................................55 7.4.3 Personal Accountability (PA)................................................................................61 7.4.4 Work Processes (WP)..........................................................................................66 7.4.5 Continuous Learning (CL) ....................................................................................70 7.4.6 Environment for Raising Concerns (RC) ..............................................................73 7.4.7 Effective Safety Communication (CO) ..................................................................76 7.4.8 Respectful Work Environment (WE).....................................................................80 7.4.9 Questioning Attitude (QA) ....................................................................................82 7.4.10 Decision making (DM)..........................................................................................86 7.5 Assessment Recommendations ..................................................................................90 8.0 Attachments:......................................................................................................................95 A. List of Acronyms ................................................................................................................96 B. People Contacted ..............................................................................................................98 C. Documents Reviewed ......................................................................................................100 D. Meetings and Workplace Evolutions Observed ................................................................113 E. Nuclear Safety Culture Assessment Plan.........................................................................122 F. Nuclear Safety Culture Traits Evaluation Summary .........................................................129 FINAL REPORT JUNE 2, 2023 xiii
INDEPENDENT THIRD-PARTY NUCLEAR SAFETY CULTURE ASSESSMENT 2.0 Assessment Objective Provide an independent and comprehensive assessment of the existing Organizational Nuclear Safety Culture, including the Safety Conscious Work Environment (SCWE) at the NIST Center for Neutron Research. The assessment was performed in accordance with the requirements of the August 1, 2022, Confirmatory Order Modifying License No.
TR-5. The assessment identified areas for improvement and corrective actions, and other improvement opportunities. The assessment also evaluated the rigor, self-criticality, and overall quality of NCNRs internal self-assessment activities in this performance area.
This Assessment focused on assessing traits and programs, and identifying Limiting Weaknesses.
Limiting Weaknesses: Those problems that limit current performance and, if addressed in the near-term, will lead to a step-change in observable and measurable performance.
Note: Failure to comply with regulations or regulatory commitments is always a limiting weakness.
3.0 Nuclear Safety Culture Organizational culture is the shared basic assumptions that are developed in an organization as it learns and copes with emerging issues and problems. The basic assumptions that have worked well enough to be considered valid are taught to new members of the organization as the correct way to perceive, think, act, and feel. Culture is the sum total of a groups learning. Culture is for the group what character and personality are for the individual.2 In addition to a healthy organizational culture, each nuclear facility, because of the special characteristics and unique hazards of the technologyradioactive byproducts, concentration of energy in the reactor core, and decay heatneeds a healthy safety culture. NRC expectations to establish and maintain a positive Nuclear Safety Culture apply to all licensees (among other entities) that are subject to NRC authority; this includes NCNR-ROE.
Nuclear Safety Culture: The set of core values and behaviors resulting from a collective commitment by leaders and individuals to emphasize safety over competing goals to ensure protection of people and the environment.3 Nuclear safety is a collective responsibility. The concept of Nuclear Safety Culture applies to every employee in the nuclear organization, from the highest level to the 2 Edgar Schein, Organizational Culture and Leadership.
3 NUREG-2165, Safety Culture Common Language, pg. 5; INPO-12-012 Rev 1, Traits of a Healthy Nuclear Safety Culture, pg. 6.
FINAL REPORT JUNE 2, 2023 1
INDEPENDENT THIRD-PARTY NUCLEAR SAFETY CULTURE ASSESSMENT individual contributor. No one in the organization is exempt from the obligation to commit to nuclear safety as an overriding priority.
As a minimum, the groups that interface with Operations need to also understand and embrace Nuclear Safety Culture. The NSC assessment was limited in scope to the NCNR-ROE, Health Physics, and Safety organizations. Neutron Condensed Matter Science and Research Facility Ops should also understand and embrace Nuclear Safety Culture standards. All NCNR groups support and interface with NCNR-ROE and should be a part of the overall plan to enhance the Nuclear Safety Culture at NIST.
NCNR-ROE should take the lead in setting the standards that all NIST groups that interface with the Reactor in any way, are educated in and committed to the ten NSC Traits and how they apply to their organizations.
4.0 Assessment Scope The Assessment was limited to examining the existing Nuclear Safety Culture of the NCNR ROE and Safety and Health Physics organizations, using the Ten NSC Traits (quoted from NUREG-2165, below) described in NUREG-2165 and INPO 12-012 as a framework:
I. Leadership Safety Values and Actions (LA) - Leaders demonstrate a commitment to safety in their decisions and behaviors.
II. Problem Identification and Resolution (PI) - Issues potentially impacting safety are promptly identified, fully evaluated, and promptly addressed and corrected commensurate with their significance.
III. Personal Accountability (PA) - All individuals take personal responsibility for safety.
IV. Work Processes (WP) - The process of planning and controlling work activities is implemented so that safety is maintained.
V. Continuous Learning (CL) - Opportunities to learn about ways to ensure safety are sought out and implemented.
VI. Environment for Raising Concerns (RC) - A safety-conscious work environment (SCWE) is maintained where personnel feel free to raise safety concerns without fear of retaliation, intimidation, harassment, or discrimination.
VII. Effective Safety Communication (CO) - Communications maintain a focus on safety.
VIII. Respectful Work Environment (WE) - Trust and respect permeate the organization.
IX. Questioning Attitude (QA) - Individuals avoid complacency and continuously challenge existing conditions and activities in order to identify discrepancies that might result in error or inappropriate action.
X. Decision making (DM) - Decisions that support or affect nuclear safety are systematic, rigorous, and thorough.
FINAL REPORT JUNE 2, 2023 2
INDEPENDENT THIRD-PARTY NUCLEAR SAFETY CULTURE ASSESSMENT 5.0 Team Members The independent Assessment Team membership is as follows:
Dr. Michael D. Quinn, Task Leader, and Analyst Michael T. Coyle, RADM, USN (ret.), Analyst Steven K. Crowe, CAPT, USNR (ret.), Analyst Michael J. Fecht, SRO, Analyst Richard N. Swanson, P.E. (ret.), Analyst, and Assessment Report Leader This team of highly experienced nuclear professionals has a combined experience in the nuclear industry of over 240 years. Each member has participated in multiple performance recoveries of nuclear organizations, in both DOE and NRC-regulated facilities. Additionally, each member has operated nuclear reactors and managed and supervised nuclear personnel.
Dr. Mary Jo Rogers, Behavioral Scientist, SME, advised the team throughout the Assessment.
Michael D. Quinn, Sc.D., SRO, Team Leader.
Dr. Quinn has over 45 years of nuclear industry experience. He is a Principal in a consulting group that focuses on high reliability organizational performance and assessments, nuclear organizational recoveries in the USA and Canada, event analysis and root cause training, Conduct of Operations, Corrective Action Programs, Safety Culture, and Human Performance remediation.
Prior to his present role, Michael collected 25 years of commercial nuclear power plant operations leadership in various capacities, including Director of Nuclear Station Services; Director of Nuclear Station Emergency Operations and Station Duty Officer (US NRC Senior Reactor Operator Licensed); Radio-Chemist and Manager of Chemistry/Radio-Chemistry; Station Operations Review Committee member and Chair; Corrective Action Review Board member and Chair, and a member of an Offsite Nuclear Review Board subcommittee.
In his assignments Michael has worked closely with commercial nuclear power stations and nuclear suppliers in the US and Canada, as well as with U.S. Federal agencies, including the US Department of Energy (e.g., LANL, Hanford, WIPP) and the US Nuclear Regulatory Commission. Experience includes all stages of nuclear facilities:
design, new builds, rebuilds, power operations, and decommissioning.
Michael has taught the US Nuclear Regulatory Commission (US NRC), the Canadian Nuclear Safety Commission (CNSC), and the Japan Nuclear Regulation Authority (JNRA) how to evaluate Root Cause Analysis Reports (engaging IPs 95001, 95002, and 95003) conducted by their respective licensees. He has co-taught this RCA Evaluation course over 40 times to more than 600 US NRC inspectors since 2006, as well as to two US NRC 95003 Teams, and is contracted into 2023.
FINAL REPORT JUNE 2, 2023 3
INDEPENDENT THIRD-PARTY NUCLEAR SAFETY CULTURE ASSESSMENT Michael is certified in the Phoenix Causal Analysis Method and the FPI/PII Causal Analysis methodology. He has co-taught the 40-hour Phoenix Causal Analysis Course for commercial nuclear organizations in the US and Canada, as well as to the US DOE.
Michael holds a Doctorate of Science in Organizational Management Systems, a Masters in Business Administration, and a Bachelor of Science Degree in Chemistry.
Michael T. Coyle, RADM, USN (Ret), Analyst.
Rear Admiral Coyle has over 58 years of experience in the maintenance and operation of Naval and commercial nuclear power, including significant positions of leadership.
After completing the Navy Nuclear Propulsion and Submarine training programs he served in nuclear submarines before transitioning to the Navys Engineering Duty Officer program where his duties included extensive experience in naval shipyards that specialized in nuclear submarine and surface ship maintenance and repair. He eventually commanded Pearl Harbor, Hawaii and Mare Island, California Naval Shipyards. After selection for Rear Admiral his Flag Officer assignments were Deputy Commander for Submarines at the Naval Sea Systems Command, Maintenance Officer for the Pacific Fleet, and Deputy Commander for Engineering at the Naval Sea Systems Command.
Following retirement from the Navy in 1998 he was the Site Vice President at the Clinton, Illinois Nuclear Power Station and later, the Site Vice President at the Cooper Nuclear Plant in Brownville, Nebraska. From 2004 to 2006 he was on loan to the Nuclear Energy Institute (NEI) in Washington, DC. From 2006 to 2009 he was Vice President, Special Projects for Exelon, the largest operator of commercial nuclear plants in the United States. From 2009 he has been an employee and consultant to several contractors performing work for the Department of Energy. RADM Coyle received a Bachelors degree from the U.S. Naval Academy where he graduated with distinction in 1965, and a Master of Science degree in Mechanical Engineering from the U.S. Naval Postgraduate School in Monterey, California where he received the Naval Sea Systems Command award for Naval Engineering.
Steven K. Crowe, Analyst.
Mr. Crowe has 46 years of experience in the operation and licensing of nuclear facilities, safe management of nuclear materials, decommissioning and demolition of nuclear facilities, and disposal of hazardous chemicals. He is a recognized subject matter expert (SME) in operational readiness and Conduct of Operations (CONOPS) and has mentored the recovery of troubled facilities prior to Operational Readiness Review (ORR) resumption.
His relevant experience includes leading multiple root cause investigations and serving as a qualified team leader/advisor for numerous U.S. Department of Energy (DOE)
ORRs and other assessments, including the Waste Isolation Pilot Plant (WIPP) restart.
He is trained in MORT, Kepner-Tregoe, PII, and INPO Human Performance Evaluation/Root Cause Methodology. He provided specialty nuclear engineering, operations support, and consulting for the Depleted Uranium Hexafluoride (DUF6)
FINAL REPORT JUNE 2, 2023 4
INDEPENDENT THIRD-PARTY NUCLEAR SAFETY CULTURE ASSESSMENT facilities startup at the Portsmouth Gaseous Diffusion Plant (PORTS) and the Paducah Gaseous Diffusion Plant (PGDP). He served as a member of the Nuclear Safety Basis Review Board that reviewed and approved for submittal over 20 Documented Safety Analysis (DSA) documents for Bechtel Jacobs Company LLC (BJC). At the Rocky Flats Environmental Technology Site (RFETS), he managed all safety, health, and quality aspects for nuclear material-handling and rule and order compliance for the quality assurance (QA), nuclear safety, fire protection, waste management, and construction programs. He also managed the Westinghouse Government Services Quality Assurance Manual (QAM) Review and Gap Analysis. Steve received a B.S. in Operations Analyses, with merit, from the U.S. Naval Academy.
Michael J. Fecht, SRO Analyst.
Mr. Fecht has 49 years of nuclear experience. He has held positions as Commercial Senior Reactor Operator, Reactor Operator, Lessons Learned Manager, Nuclear Assurance and Licensing Manager, Corporate Quality Assurance Director and was the corrective action program troubleshooting lead in which capacity he led corporate teams at troubled nuclear sites to assist in recovery. He has established Management Assessment, self-assessment, and Observation programs. He has taught personnel the basics of how to perform assessments. In DOE, he was a Senior Advisor to the Yucca Mountain DOE Quality Assurance Director where he established a Nuclear Safety Culture (NSC) training program and established NSC guidelines and observation metrics for QA when performing audits and assessments. Within the DOE, he has conducted multiple Operational Readiness Reviews, Readiness Assessments, Nuclear Safety Culture assessments, and performed root cause analyzes at both commercial nuclear and DOE sites.
He has mentored troubled facilities at TVA and DOE facilities and performed the root cause analysis on the 2018 Idaho National Drum event where four re-packed Transuranic Waste drums ejected their lids and radioactive contents within a clean area in the ARP-V repackaging tent. He is knowledgeable with both commercial and DOE Operations Technical Specifications (TS) and has taught TS to Operations personnel.
He was appointed the operational team lead for the WIPP readiness restart team. He was the team lead for a Management Self-Assessment on the underground ventilation system. In February 2020 he assumed the lead as the TFE acting project manager for the WIPP Records Management program to recover a troubled Records Management Program. He is trained in MORT, Kepner-Tregoe, PII, and INPO Human Performance Evaluation/Root Cause Methodology. He established the initial Human Performance Enhancement System root cause training course for TVA and has conducted Corrective Action management training for TVA and DOE sites.
Richard N. Swanson, P.E. (Ret.), Analyst/Report Coordinator.
Mr. Swanson has 51 years of nuclear experience in the performance, oversight, training, and mentoring of assessment and management personnel in causal analysis, event investigations, investigation of allegations, and organizational performance FINAL REPORT JUNE 2, 2023 5
INDEPENDENT THIRD-PARTY NUCLEAR SAFETY CULTURE ASSESSMENT evaluations. He has been a Director/General Manager of Engineering and Projects at three nuclear utilities, as well as Quality Assurance Director, Backfit Construction Manager, and Regulatory Affairs Manager. He was the Emergency Director for the joint FEMA/NRC-evaluated nuclear emergency preparedness exercise that was pre-requisite to the restart of Pilgrim Station following an extended shutdown and troubled plant status.
He is a co-developer for several Root Cause Analysis courses and currently trains NRC staff in evaluating root cause analysis reports since 2006 (including relevant content of IPs 95001, 95002, and 95003). As an independent consultant for the past 27 years, he has supported companies engaged in power generation, uranium enrichment, fuel fabrication, electric transmission and distribution, heavy manufacturing, and DOE, NRC, and state agencies.
He was an assessment team member for four major Hanford Waste Treatment Plant project-wide performance assessments. He was a Senior Advisor to the Yucca Mountain DOE QA Director, and has conducted investigations at Yucca Mountain, Paducah Gaseous Diffusion Plant, Portsmouth Gaseous Diffusion Plant, Idaho National Labs, the URENCO Enrichment Plant, and over 35 nuclear power plants. Credentials include B.S. with merit (United States Naval Academy), M.S. with merit (Northeastern University), M.B.A with distinction (Babson College), Naval Nuclear Power Training Program, certification as Naval Nuclear Engineer, and Licensed Professional Engineer (retired). He is formally trained in several causal analysis methodologies, including The Phoenix Method, SOURCE, Kepner-Tregoe, and FPI/PIl, and is conversant with several others.
Mary Jo Rogers, Ph.D., Behavioral Scientist Dr. Rogers is the founder and president of Rogers Leadership Group. Her work is dedicated to improving leadership and culture in safety significant and high hazard industries. In 2013, she published the book, Nuclear Energy Leadership: Lessons Learned from U.S. Operators (PennWell) and has published articles in HazardEx and Power Engineering magazines.
She has 25 years of experience consulting to commercial nuclear power, electric and gas utility companies, the U.S. Department of Energy, DOE contractors, and the National Institute of Standards and Technology (NCNR), among others. She has conducted board-requested effectiveness assessments of risk reduction programs and multiple safety culture assessments.
Previously, she was managing partner at a global management consultancy firm and a leader in management development at Exelon Corporation. Her extensive background and education enable her to apply psychological and behavioral science principles, combined with real-world utility and business experience, to help individuals and organizations uncover the values, beliefs, and behaviors that may be limiting their performance.
FINAL REPORT JUNE 2, 2023 6
INDEPENDENT THIRD-PARTY NUCLEAR SAFETY CULTURE ASSESSMENT She received her Ph.D. in clinical psychology at Loyola University Chicago, her M.A. in developmental psychology at Northern Illinois University, and B.A. in psychology from the University of Wisconsin.
6.0 Assessment Process and Criteria 6.1 ASSESSMENT PROCESS The Team performed its work in several phases:
Phase One: Data request and review Phase Two: Development and administration of the 2023 NCNR Nuclear Safety Culture Survey. Results were compared to output from the 2021 NCNR-ROE Nuclear Safety Culture Survey. The survey provided insight into what NCNR-ROE personnel believe.
Phase Three: Observation and evaluation of workplace behaviors and interactions, as well as further exploration of survey results.
Phase Four: Conducted Interviews, Focus Groups, and Program Assessments Phase Five: Data Analysis and Processing 6.1.1 Methodology The Team and the Behavioral Scientist collectively determined the data collection methods that were needed for this assessment, which included seven domains to evaluate the state of the NCNR-ROE Nuclear Safety Culture.
The Team assessed these Data Sources:
Functional analysis Review of archived data, procedures, programs, policies Structured interviews Focus Groups Observations Safety Culture Survey Program Reviews Actions included:
Administered and evaluated a confidential Nuclear Safety Culture Survey developed by the Team, which performed follow-up reviews and interviews to capture more context on safety culture issues identified by the survey results.
Followed up on comments provided during the survey. This included specific focus on organizational elements that the survey identified as having a relatively weaker or stronger Nuclear Safety Culture.
FINAL REPORT JUNE 2, 2023 7
INDEPENDENT THIRD-PARTY NUCLEAR SAFETY CULTURE ASSESSMENT Reviewed NCNR and NIST corporate policies, programs, procedures, and training as they related to Nuclear Safety Culture.
Sampled communications reviews related to NCNR-ROE activities with emphasis on Nuclear Safety Culture events and issues.
Reviewed information regarding NCNR performance and issues, including, but not limited to, results of previous assessments, NRC inspections and findings, employee concerns, allegations, Human Resources information, Corrective Action Program (CAP) data, performance metrics, site assessment programs, organizational performance goals, resolution of significant issues, safety culture analyses in NCNR event causal evaluations, and site performance indicators.
The confidentiality of sensitive security- and personnel-related information was respected and protected.
Many of the aforementioned areas were limited in depth in this report given that subsequent Task activities will be comprehensively evaluating these areas.
Observed selected ROE activities and meetings to determine the extent to which the conduct and results of those activities and meetings were indicative of a healthy Nuclear Safety Culture.
Received information provided by NCNR-ROE management on selected programmatic issues and emergent operational issues.
Interviewed selected ROE and supporting personnel external to NCNR-ROE.
Conducted Focus Group interviews to gather information on the Nuclear Safety Culture within the various ROE groups.
Individual interviews were conducted to gather information on particular topics and issues.
The Team compared the collected information with the normative NSC Traits, Attributes, and examples cited in NUREG-2165 / INPO 12-012 to evaluate the NCNR Nuclear Safety Culture.
FINAL REPORT JUNE 2, 2023 8
INDEPENDENT THIRD-PARTY NUCLEAR SAFETY CULTURE ASSESSMENT 6.1.2 Data Analysis and Assessment FIGURE 2: Cultural Theme Data Sources and Assessment Flow Cultural Themes I
NUREG-2165I INPO-12-012 Safety Culture Framework: Tra its I
Assessment and Recommendations
- 1. The Team analyzed the raw data captured from each evaluated data source (Functional analysis, Review of archived data; Structured interviews; Focus Groups; Observations; Safety Culture Survey; Program Reviews) to identify behavioral characteristics.
- 2. The Team analyzed the information captured by each individual data source to determine cultural themes.
- 3. Compared the behavioral characteristics and cultural themes derived from the different data sources, then drew conclusions.
- 4. Compared conclusions with the NUREG-2165 / INPO 12-012 Safety Culture Traits
- 5. Determined Assessment Categories:
Area of Strength (AOS)
Positive Observation (PO)
Area in Need of Attention (ANA)
Area for Improvement (AFI)
- 6. Developed Recommendations FINAL REPORT JUNE 2, 2023 9
INDEPENDENT THIRD-PARTY NUCLEAR SAFETY CULTURE ASSESSMENT 6.2 ASSESSMENT CRITERIA Assessment Observation Definitions:
a) Area of Strength - Performance that is exceptionally effective in achieving its desired results; a program, process, or activity of such high quality that it could serve as an example for other similar elements.
b) Positive Observation (PO) - A performance, program, or element that is sufficient to meet its basic intent and does not require additional management attention to achieve full effectiveness and consistency.
c) Area in Need of Attention (ANA) - A performance, program, or process element that is sufficient to meet its basic intent, but that requires management attention to achieve full effectiveness and consistency.
d) Area for Improvement (AFI) - A performance, program, or process element that requires significant improvement to obtain the desired results in a consistent and effective manner.
e) Highly Effective - Notable areas of strength and/or positive observations; no or few ANAs, no AFIs. Performance, programs, and processes are more than sufficient to obtain the desired results with consistency and effectiveness.
f) Effective - One or several ANAs and no or a few AFIs. Performance, programs, and processes are sufficient to obtain the desired results with consistency and effectiveness.
g) Marginally Effective - Several or more ANAs and more than several AFIs are clearly evident. The basic intent of the program is achieved; however, the performance, program, or process is challenged to obtain the desired results with consistency and effectiveness. Prompt management action and scheduled periodic trending is required.
h) Not Effective - Significant shortcomings such that the basic intent of the program or process is not being achieved. AFIs identified as Not Effective require immediate and continuous management attention.
FINAL REPORT JUNE 2, 2023 10
INDEPENDENT THIRD-PARTY NUCLEAR SAFETY CULTURE ASSESSMENT 7.0 Assessment Results 7.1 ASSESSMENT CONCLUSIONS:
SUMMARY
Limiting Weakness(es):
NCNR personnel are inconsistent in their understanding that values and behaviors are driven by the desired principles and functions. Many have confused Nuclear Safety Culture with Industrial Safety Culture. Changes in values can only be brought about by a concerted and consistent effort directed toward changing behaviors.
NCNR personnel are inconsistent in their understanding of what nuclear safety culture is.
The Leadership Team does not demonstrate what behaviors constitute a healthy Nuclear Safety Culture, and that the Leadership Team has a prominent role in establishing and fostering a healthy Nuclear Safety Culture.
Leadership has not embraced succession planning and taken effective action to establish an understanding by NIST HR of the unique nature of NCNR-ROE hiring needs, especially of operators.
NCNR did not take effective action to address nuclear safety culture-related deficiencies identified in a 2012 IAEA report. Similar Nuclear Safety Culture issues were identified in this assessment report.
NCNR-ROE has insufficient funding to obtain and prioritize external resources who can take on:
o Operations Department administrative workload and free up operators (see LA.1, Section 7.4.1.2.2, last table entry); and, o Program development and implementation (see 7.3, Assessment
Conclusions:
Programmatic Support to NSC for discussion of nine key programs).
Discussion When asked, What does the term Nuclear Safety Culture mean? few interviewees identified the appropriate Nuclear Safety Culture behaviors, despite the fact that the INPO 12-012 Handbook had been distributed to NCNR employees for at least the past year. The NUREG 2165 / INPO 12-012 definition of nuclear safety culture states: The set of core values and behaviors resulting from a collective commitment by leaders and individuals to emphasize safety over competing goals to ensure protection of people and the environment.
NCNR-ROE requested an independent assessment of Operations and Maintenance in 2012 that was conducted by the International Atomic Energy Agency (IAEA) in 2012 and identified a number of deficiencies at NCNR. Most deficiencies remained unaddressed as of this assessment conducted in 2023. Examples include:
Existing procedures, in general, required improvement. One of the root causes of the 02/03/2021 event identified the same issue. NCNR did not implement effective actions in 2012 that may have prevented the 2021 event.
FINAL REPORT JUNE 2, 2023 11
INDEPENDENT THIRD-PARTY NUCLEAR SAFETY CULTURE ASSESSMENT Lack of a Training Program at NCNR.
Numerous programmatic issues with the following programs identified in 2012 that were not corrected prior to the 2021 fuel melt event:
o 2012 Benchmarking Plan development o Change Management Program o Corrective Action Program o Human Performance Improvement o Configuration Management o Training addressing the ten Nuclear Safety Culture Traits o Facility Cleanliness/housekeeping o No key performance indicators o Hazard Identification and Control 7.2 INSIGHTS BY ASSESSMENT ACTIVITY 7.2.1 Nuclear Safety Culture Survey The Safety Culture Team developed the 2023 NCNR-ROE Safety Culture Survey during February 2023 following document reviews and a site visit in mid-February. The survey was administered during the period February 24 through March 10, 2023.
The Team used the 2023 Nuclear Safety Culture survey as one of seven (7) inputs to developing its overall assessment of NCNR-ROEs Nuclear Safety Culture:
Functional Analysis Structured Interviews Archived Data Review Observations Focus Groups Program Reviews Nuclear Safety Culture Survey Results The 2021 NBSR safety culture assessment survey effort did not engage the other six supplementary inputs to fully characterize the state of NSCR-ROEs Nuclear Safety Culture status at that time. Given the very narrow, incomplete scope of the late-2021 safety culture assessment, engaging just one of seven key attributes, the 2021 survey data results could not provide a reasonable indicator of NCNR-ROEs Nuclear Safety Culture.
FINAL REPORT JUNE 2, 2023 12
INDEPENDENT THIRD-PARTY NUCLEAR SAFETY CULTURE ASSESSMENT High-level metrics from the 2021 and 2023 Surveys:
- Times NCNR-ROE Nuclear
- Surveys #
Survey Staff % Returned Safety Returned Comments*
Number Culture Mentioned 2021 54 53 98 88 0 2023 57 55 96 15 8 Comparing the 2021 survey results to 2023 survey results, the Team observed that each survey data set had conveyed a high aggregate average:
2021 2023 Metric Survey Survey Aggregate average of all inputs on a scale of 1-5 4.13 4.10 Lowest average rating for any of the Survey Statements: 2.15 3.25 Highest individual assessment rating 4.85 4.67 With these data points in mind, the Team developed lines of inquiry for the onsite assessment phase to understand what the drivers were for these survey results. If taken as a sole indicator of the nuclear safety health of the organization, these survey data results would be (or could be) very misleading. Looking back, the self-administered 2021 Safety Culture Survey results were not supplemented with any other assessment method.
Following a review of the 2023 raw survey data that entailed 23 different queries, the Team determined that the most beneficial product of this survey effort was the collective set of interview lines of inquiry. The Team developed these raw data themes into focused lines of inquiry to learn more about NCNR-ROEs behaviors engaging the structured interviews, focus groups, and Program Reviews that were done during the two-week April 3-14, 2023 onsite assessment.
The 2023 survey data also provided insight into the Teams functional analysis review, archived data review, and local observations of scheduled work evolutions and meetings.
There were no major validating cultural or behavioral themes emerging from the 2023 raw data assessment. There were however several data patterns (in addition to those just noted), that at first were seen as curiosities, then later were determined to be detracting from the credibility of the collective survey data rollup. Examples include:
Many individuals responded with the same value for each of the 79 statements (all 5s); variance was actually zero. Some managers inputs were amongst this group of zero-variance responders. (Note: Zero variance in a set of data from an individual or a given statement raises questions regarding the credibility of the data).
FINAL REPORT JUNE 2, 2023 13
INDEPENDENT THIRD-PARTY NUCLEAR SAFETY CULTURE ASSESSMENT Many individuals responded with the same value for nearly all of the 79 statements (all 5s, save for one or two 4s); variance was again very low. Some managers inputs again were amongst this group of very low variance responders.
Contrasting the first two observations, a few respondents provided predominately low values (1 or 2) to most of the 79 statements.
Low safety culture data variation in survey data is generally not a healthy characteristic in an organization that is not a long-running high-reliability organization.
Diversity of thought focused on the mission generally underpins organizational evolution, resilience, readiness for change, and sustainability. Absence of evolutionary movement toward a measurable positive Nuclear Safety Culture can lead to lagging diversity of thought, resistance to needed change, ability to recover from what has been comfortable and a diminished probability to change, increasing the likelihood of permanent shutdown.
Additionally:
Survey Q13: I understand what the term Nuclear Safety Culture means received an aggregate average rating of 4.18 out of 5.0 with a very small variance.
Survey Q13 gets to the underpinning of this independent third-party assessment. The survey data results suggest that survey respondents overwhelmingly believe they understand the term Nuclear Safety Culture. Of the those who responded to this statement:
Responses 3.50% 0.50%
7%
23%
66%
- Selected a 5
- Selected a 4
- Selected a 3
- Selected a 2
- Selected a 1 FINAL REPORT JUNE 2, 2023 14
INDEPENDENT THIRD-PARTY NUCLEAR SAFETY CULTURE ASSESSMENT 37 of 56 respondents selected a 5 (66%)
13 of 56 respondents selected a 4 (23%)
4 of 56 respondents selected a 3 (7%)
2 of 56 respondents selected a 2 (3.5%)
0 of 56 respondents selected a 1.
Data from the survey suggests that 50/56 (89%) of individuals believe they know what the term means (using those data that were rated 4 and 5).
When the Team asked interviewees and focus group members, What does the term Nuclear Safety Culture mean? the Team received just two (2) responses of 57 individuals that demonstrated an actual understanding of what Nuclear Safety Culture is. This is just 3.5% of the population, a long way from the 89% who claimed in the 2023 NSC Survey that they understood what Nuclear Safety Culture meant.
This was a very telling metric that added to the Teams view that the 2023 Nuclear Safety Culture Survey results by themselves were an unreliable indicator of NCNR-ROEs safety culture understanding. This also implies that the earlier 2021 Safety Culture Survey (which was not supported by additional assessment activities) was similarly unreliable.
Continuing, the Team asked interviewees and Focus Group members to rate order the following statement, regarding the four sub-categories:
On a scale of 1 to 10, with 10 being always/outstanding and 1 being never/poor, how frequently do you see:
- a. Leaders
- i. Walk the talk? [LA]
ii. Communicate effectively? [CO]
iii. Respect and respond effectively to differing opinions? [WE]
iv. Respond to employees and the concerns they raise without defensiveness to questions, problems, issues and concerns? [RC]
- b. Workers
- i. Stop when unexpected or uncertain conditions exist? [DM]
ii. Challenge assumptions and/or offer opposing views? [QA]
iii. Are actively engaged in changes to processes and procedures? [WP]
- c. Organization
- i. You receive feedback? [CL]
ii. You give feedback? [LA]
iii. Effective organizational communications? [CO]
iv. Individuals taking ownership of issues, problems, and processes? [PA]
- v. Clearly and unambiguously defined roles and responsibilities? [LA]
- d. Processes
- i. Effective use of the problem reporting and resolution process? [PI]
ii. Effective resolution of employee concerns? [RC]
FINAL REPORT JUNE 2, 2023 15
INDEPENDENT THIRD-PARTY NUCLEAR SAFETY CULTURE ASSESSMENT Survey Data Conclusion In nearly every category, the Interview and Focus Group responses were lower than the corresponding 2023 survey results. The results of the survey refined the interview questions.
7.2.2 Interviews and Focus Groups The Team conducted 23 interviews and 14 focus groups during the safety culture assessment period at NCNR-ROE. Interviews and Focus Groups provided by far the richest set of data to enable the Team to understand the state of the ROE Nuclear Safety Culture: personal insights, real-time exchange, group interaction dynamics and engagement - all attributes that a survey cannot provide.
Developing lines of inquiry from the raw survey data, previously discussed as being overly optimistic, the Team developed question sets that would enable the Team to understand the bases behind the many statements that received a very high percentage of 4s and 5s. The Team sought to understand those survey statements that received very low ratings, though there were but a few of those compared to the abundance of highly graded survey statements.
Many other insights were gained from the interviews and focus groups that provided the Team more lines of inquiry. The Team only included comments below when they were corroborated by other interviewees and inputs. They include:
Example Employee Responses:
Comment There is a strong push to get the reactor up and running, and that means a lot of pressure. Regarding getting things done that have complicating factors, we hear things like, If this were 10 years ago, wed give it to the mid-shift to fix at 2 AM when no one is around. That mindset is still there.
Its easy to stovepipe here, thats been an issue forever. As an example, we had helium build up this week, and Operations didnt talk to Engineering.
We get our candidates ourselves, mostly through Linked-In. HR doesnt understand that what we need are operators with at least one years experience.
We are essentially writing an encyclopedia for each procedure. There are bound to be errors in themthe question is how will we handle those errors and how will we move forward from there?
A past manager was the antithesis of Nuclear Safety Culture. There were a lot of people who retired because of that individual.
There was a strong feeling about procedures - they shouldnt be too detailed, because I dont want to write myself into a compliance trap.
FINAL REPORT JUNE 2, 2023 16
INDEPENDENT THIRD-PARTY NUCLEAR SAFETY CULTURE ASSESSMENT Comment PRA is not done here, just neutrons; Nuclear grade components here are not determined the way theyre determined in a commercial reactor.
We cannot do PRA here due to no documentation. We do not do safety margin here either.
We generally know what they [contractors] are doing. We had a recent issue with a propane tank under a fork truck.
We now have in some cases 20 procedures to do a task that formerly took one procedure. Five years from now well probably go back to one.
Some procedures need to be vague - and these have been made more prescriptive.
We often have a tough time finding procedures: cant find them in SharePoint even though they may be in there. SharePoint is not easy to move through. The R Drive has some Maintenance Procedures.
A few years ago, everyone knew eight (8) individuals were retiring; management did not hire anyone to replace those leaving. We dont have enough people to refuel.
7.2.3 Workplace and Meeting Observations The Team conducted 23 interviews and 14 focus groups during the contract mandated 2-week Nuclear Safety Culture assessment period at NCNR-ROE. At the same time, the Team also recorded 11 Performance Observations. NCNR-ROE had limited reactor operations during the assessment period due to emerging plant conditions. Startup beyond several hundred kW was hampered by two predominant issues: tramp fuel particle fissioning that resulted in higher than estimated fission gases, and lack of effective routing of fission gas due to deficient ventilation system integrity. These conditions dominated the management team's attention and activities.
The Team observed multiple Plan of the Day meetings, management team discussions and brainstorming sessions, one all-hands meeting and one reactor startup.
Additionally, all team members toured the facility with plant personnel.
Each observation was documented and identified conditions were evaluated for behaviors that could impact Nuclear Safety Culture traits.
The results of the performance observation activities and their implications regarding Nuclear Safety Culture traits implementation are shown in Attachment D.
FINAL REPORT JUNE 2, 2023 17
INDEPENDENT THIRD-PARTY NUCLEAR SAFETY CULTURE ASSESSMENT 7.3 ASSESSMENT CONCLUSIONS: PROGRAMMATIC SUPPORT TO NSC The Team reviewed nine Programs important to Nuclear Safety Culture:
Most programs were in the very early stages of development and implementation.
Most programs were determined to be marginally effective to not effective at this stage, with many trending positive.
One of the primary means by which organizations establish, manage, and modify organizational behavior is through processes and their implementing procedures. Thus, a thorough evaluation of Nuclear Safety Culture must, necessarily consider the key processes. When reviewing programs, the NSC Assessment restricted itself to considering program effectiveness and did not drill deeply into details. While the Team notes that many of the programs reviewed are in the early stages of development and/or implementation, it would also point out that these programs have been required to be in place for some time prior to the 2021 Fuel Melt event. The fact that NCNR is currently working hard to establish them in 2023 does not change the fact that, as of this Assessment, most are ineffective.
Program Reviews are required by NRC IP 95003 Attachment 2 (the requirements of which are applicable to this Assessment as stated in the Task Statement of Work from NCNR) and were clearly identified in the scope of the NSC Assessment Plan, which was reviewed and approved by NCNR prior to the on-site assessment phase.
Many of the processes considered in this NSC Assessment will be revisited in more depth in future assessment activities to provide additional details of specific strengths and weaknesses, and targeted recommendations to address identified issues.
Progress in terms of NSC program implementation and behavioral change appear to have stalled. The personnel in charge of creating and implementing a number of new programs (i.e., CAP, ECP, etc.) are understaffed in terms of FTE (full-time equivalent) and experience in these areas. Although many of these individuals are intelligent, educated, and have valuable prior experiences, the lift currently is too much for this group.
For instance, CAP understandably has had a great deal of attention. While senior leaders and others report that it is off to a good start and just needs fine-tuning and various adjustments, there is little evidence of effective usage. Although CAP has rolled out, adoption and effective functioning of CAP is limited currently.
The Team reviewed nine key NCNR-ROE programs related to Nuclear Safety Culture behaviors:
- 1. Management Observation Program
- 2. Corrective Action Program
- 3. Audit/Surveillance/QA Program
- 4. Document Control Program
- 5. Conduct of Operations Program
- 6. Training Program
- 7. Procedures
- 8. Causal Analysis Program
- 9. Employee Concerns Program FINAL REPORT JUNE 2, 2023 18
INDEPENDENT THIRD-PARTY NUCLEAR SAFETY CULTURE ASSESSMENT The Team rated each of the nine programs for effectiveness, and determined the following aggregate result:
Marginally Highly Effective Effective Not Effective RATING: Effective NUMBER OF 0 0 3 6 PROGRAMS:
Team conclusions and individual recommendations for each program are discussed in Section 7.3 Assessment
Conclusions:
Programmatic Support to NSC.
The basis for each rating is summarized in the table below.
The Team performed an in-depth assessment on NCNR-ROEs performance in each of the nine (9) evaluated programs and their supporting attributes.
Data input was drawn from each evaluated data source (Functional analysis; Archived Data; Structured Interviews; Focus Groups; Observations; Safety Culture Survey and individual program reviews) to feed into a collective assessment. Trends are assigned on the basis of extensive Team member experience in the nuclear industry, discussions with NCNR personnel regarding the progress they achieved since 2021, and professional judgment.
The Teams collective rollup is presented in the table below:
PROGRAM RATINGS AND BASIS REPORT BASIS PROGRAM RATING TREND SECTION [LIMITING WEAKNESSES & OTHER]
Existing program is limited to procedural observations.
Many observations, including some from senior Not management, not documented Management 7.3.1 Observation Program Effective Flat by NCNR observer; some meet AFI minimum requirements for 2 per quarter.
CAPs not consistently written in a timely manner for problems identified.
Less than adequate staffing to develop CAP.
Less than adequate training re:
Not CAP.
Corrective Action Program exists, recently 7.3.2 Program Effective Flat revised.
AFI Threshold not well understood by staff.
Lack of management engagement.
FINAL REPORT JUNE 2, 2023 19
INDEPENDENT THIRD-PARTY NUCLEAR SAFETY CULTURE ASSESSMENT PROGRAM RATINGS AND BASIS REPORT BASIS PROGRAM RATING TREND SECTION [LIMITING WEAKNESSES & OTHER]
QA Program applies to only Engineering.
Program does not identify important programs needed for safe and effective nuclear plant Not facility operation.
Audit/Surveillance/QA Of those programs that are 7.3.3 Program Effective Flat identified, most have short run-AFI times and are in very early stages.
Program definition is fragmented among several documents, causing confusion and lack of use/compliance.
No Records/Document Control Manager No plan to meet Federal Document Control Not requirements (36 CFR 1200 Document Control 7.3.4 Program Effective Flat Att. B).
AFI Lack of a Records Management Program that meets federal government requirements (36 CFR 1200 Att B)
Critical elements of CONOPs are insufficiently addressed in NBSR Conduct of Operations guidance. (See Section 7.3.5)
Marginally Operations staffing shortages, Conduct of 7.3.5 Effective Positive limited currently to two shifts.
Operations Program AFI Ops Training weaknesses (see Training program)
Lack of Ops staff understanding of NSC and how it applies to CONOPs FINAL REPORT JUNE 2, 2023 20
INDEPENDENT THIRD-PARTY NUCLEAR SAFETY CULTURE ASSESSMENT PROGRAM RATINGS AND BASIS REPORT BASIS PROGRAM RATING TREND SECTION [LIMITING WEAKNESSES & OTHER]
Current training program does not support a continuing Operator Training program.
Training that has been Marginally developed was not done using 7.3.6 Training Program Effective Positive the Systematic Approach to AFI Training.
Operator training is limited in scope and does not address full spectrum of Operations Department needs.
Procedures are inadequate for rule-based operations required by operators with less experience.
Some revised procedures are Marginally overly complex and 7.3.7 Procedures Effective Positive cumbersome.
Pen and ink changes to AFI important procedures (i.e.,
reactor startup) did not receive 10CFR50.59 review.
Inexperienced (rather than experienced) operators are writing procedures.
Program has not been implemented & is under Not development with no written Causal Analysis 7.3.8 Effective Flat plans.
Program AFI Few trained causal analysts.
The few trained causal analysts have limited experience.
Program has not been Not implemented.
Employee Concerns 7.3.9 Effective Flat ECP Coordinator has not yet Program AFI been trained or started in the position.
The Team noted that the majority of the programs are either under development or in very early stages of implementation. Programs that have not experienced sufficient run-time to produce consistent, reliable results cannot be considered to be effective. The Team determined that three programs were marginally effective, and six were not effective. That said, the Team observed that NCNR-ROE was expending a great deal of thought and effort on initial program development and implementation.
FINAL REPORT JUNE 2, 2023 21
INDEPENDENT THIRD-PARTY NUCLEAR SAFETY CULTURE ASSESSMENT Team observations and discussions of individual program reviews follow.
7.3.1 Management Observation Program AR 5.4, Observation Program and Checklist was written after the February 2021 fuel melt event. The existing procedure is narrowly based on improvement of procedures, increased understanding of procedures, and management oversight of procedural work performed.
The first milestone achieved in the CAP development was the deployment of an Observation Program ensuring management oversight of procedures and procedure adherence. The current Observation Program is tied to CAP improvements.
7.3.1.1 Conclusions Management Observation Program Rating: Marginally Effective Trend: Flat The current observation procedure AR 5.4 inappropriately limits observations to management oversight of procedural work performed. Interviews noted a number of individuals believed that no CAPs are written from Management Observations.
The current NCNR Observation Program was an initial attempt for the site to provide guidance on how to conduct and document observations. The original program limits observations to management observations of procedural procedure implementation which states, Limiting Weakness(es):
The existing management observation program is limited to procedural observations.
Many observations, including some from senior management, are not consistently documented in a timely manner by NCNR observers; some managers meet minimum requirements for 2 per quarter.
CAPs are not consistently being written by observers when observations document problems.
The Team evaluated the Management Observation Program as Marginally Effective, based upon (1) the limiting weaknesses noted and (2) the fact that the program is in place and in the early stages of implementation.
FINAL REPORT JUNE 2, 2023 22
INDEPENDENT THIRD-PARTY NUCLEAR SAFETY CULTURE ASSESSMENT 7.3.1.2 Basis, Discussion, and Observations Employee Survey Attribute Question Score WP.1 Q49. Plant activities are governed by comprehensive high- 3.40 quality programs, PI.2 Q70. We have processes to identify and resolve existing 3.44 organizational weaknesses.
CL.4 Q71. Leaders foster an environment in which individuals 3.71 value and seek continuous learning opportunities PI.5 Q77. A broad set of performance indicators is utilized with a 3.48 focus on early detection of problems The Team only included comments below when they were corroborated by other interviewees and inputs. They include:
Example Employee Responses:
Comment Observation program is still in development. The purpose is to provide feedback from people not directly involved in the evolution.
The NRC needs to observe us do some things. Observation of our revised approach to conform to procedures. [NCNR should not rely on the NRC to identify NCNRs problems.]
A number of programs need to be integrated, starting with CAP and the Observation Program, and it needs to have teeth.
Q: What management observations, self-assessments, QA assessments, or independent assessments have evaluated your Program?
A: None right now.
OBSERVATION program: in 2022 Q4 we were at 40% of target (participation), and at the end of Q1 2023 we were 50-60% target for observations.
7.3.2 Corrective Action Program The existing CAP program AR 7.2, R2 Corrective Action Program was developed after the February 2021 fuel event. Currently this procedure does not meet the requirements of ANSI/ANS 15.8 Quality Assurance Program requirements.
ANSI/ANS-15.8-1995 Quality Assurance Program requirements for research reactors, section 2.18 Corrective Actions, states that conditions adverse to quality shall be identified and promptly corrected as soon as practical. In the case of significant conditions adverse to quality the cause of the condition shall be investigated, and corrective actions taken to preclude recurrence. AR 7.2 does not identify that conditions including significant conditions adverse to quality shall be documented and corrected.
FINAL REPORT JUNE 2, 2023 23
INDEPENDENT THIRD-PARTY NUCLEAR SAFETY CULTURE ASSESSMENT NCNR also implements the Incident Reporting and Investigation (IRIS) process. The purpose establishes the requirements and roles and responsibilities for the reporting and investigation of work-related safety incidents and near misses to determine why these events occurred and what actions must be taken to prevent their recurrence.
7.3.2.1 Conclusions Corrective Action Program Rating: Not Effective Trend: Flat NCNR has not developed or implemented a PI&R process that would identify, evaluate, correct, and trend issues. Additionally, the NCNR CAP does not meet ANSI 15.8 Quality Assurance Program Requirements for Research Reactors identified in section 2.15 control of Nonconforming Items and Services.
Prior to the February 2021 event NCNR had a CAP (AR-20) but had not implemented it and has just implemented the current CAP in 2023. The existing NIST IRIS program was previously implemented at NCNR to capture near misses and industrial safety issues.
Limiting Weakness(es):
Less than adequate staffing to develop CAP.
Less than adequate training re: CAP.
Program exists, recently revised.
Threshold not well understood by staff.
Lack of management engagement (as indicated by lack of effective CAP).
The Team evaluated the Corrective Action Program as Not Effective, based upon (1) the limiting weakness noted and (2) the fact that the program is in place and in the early stages of implementation.
7.3.2.2 Basis, Discussion, and Observations Prior to the February 2021 event NCNR had a CAP (AR-20) but had not implemented it and has just implemented the current CAP in 2023. The current NCNR CAP program was approved on 03/31/2023. (Also see PI&R Trait, PI.2, for additional supporting information on the lack of implementing an effective CAP).
CAP Challenges and Resistance Although most interviewees have had the CAP training, the problems with CAP revolve around its actual functionality (reportedly), organizational resources, and general confusion. Some organizations, such as Operations, are in a crisis mode with the ongoing technical and current licensed staffing issues, and thus have not prioritized CAP. Ops personnel are focused on restart, getting operators licensed without a reactor running at 100%, as well as conduct of ops and control room behavior.
FINAL REPORT JUNE 2, 2023 24
INDEPENDENT THIRD-PARTY NUCLEAR SAFETY CULTURE ASSESSMENT Other employees that have had the training, report technical difficulties entering a CAP item, confusion about entering Trouble Tickets vs CAP items, perceive that taking the time to figure out how to use the new CAP system is not worthwhile as it is expected to change, and/or have insufficient time to learn and adopt its usage. Employees in some organizations report not having received the training.
Employee Survey Responses:
ATTRIBUTE QUESTION SCORE Q83. I understand and use human error reduction WP.4 4.45 techniques, such as self-check, STAR, and pre-job briefs.
PI.4 Q77. A broad set of performance indicators is utilized with a 3.48 focus on early detection of problems.
Q71. Leaders foster an environment in which individuals CL.4 3.71 value and seek continuous learning opportunities Q70. We have processes to identify and resolve existing PI.2 3.44 organizational weaknesses Q67. Leaders evaluate serious events and implement 4.16 CL.1 actions to learn from the experience.
Q60. I promptly challenge unanticipated test results or 4.44 QA.2 unexpected system response The Team only included comments below when they were corroborated by other interviewees and inputs. They include:
Employee and Group Interview Responses:
Comment Ops needs additional staffing to allow implementation of corrective actions Previous similarities on latching issues not captured and shared [1980s and 1990s]
Corrective action program - we had a program 3 years ago; not really used, not in our IT system; we are just now starting to use the CAP Engineers are expected to use the CAP process. But CAP is new, they have not tried it Number of programs need to be integrated, starting with CAP and the Observation Program, and it needs to have teeth CAP was used as a punishment. If you raised a CAP issue, you were assigned to fix it. This was in 2018. The SRTs felt like just more work to do. If we raise things, we get more work to do.
We have a Corrective Action Program, but it is very high level and is largely a paper process. The NRC didnt push on it FINAL REPORT JUNE 2, 2023 25
INDEPENDENT THIRD-PARTY NUCLEAR SAFETY CULTURE ASSESSMENT Comment Just rolled [the CAP] process out. Went to training last week. Believes program can achieve success in long term. Data tracking is poor so far. No IT backup now.
Concerned that small issues may not be identified 7.3.3 Audit/Surveillance/QA Program 7.3.3.1 Conclusions Audit/Surveillance/QA Program Rating: Not Effective Trend: Flat NCNR does not embrace the concept of having a fully functional QA Program.
Interviews identified this concern and the assessment team review of the QA Program identified that procedures that would implement the program have not been developed or are developed but have not been implemented.
Limiting Weakness(es):
QA Program applied to only Engineering.
Program does not identify important programs needed for safe and effective nuclear plant facility operation.
Of those programs that are identified, most have short run-times and are in very early stages.
Program definition is fragmented among several documents, causing confusion and lack of use/compliance.
The Team evaluated the Audit/Surveillance/QA Program as Not Effective, based upon the limiting weaknesses noted.
7.3.3.2 Basis, Discussion, and Observations Employee Survey Responses:
ATTRIBUTE QUESTION SCORE Q54. Work is effectively planned and executed by WP.1 3.63 incorporating risk-informed insights.
Q55. Plant activities are governed by comprehensive high-WP.3 3.47 quality programs, processes, and procedures.
Q28. I treat other employees with dignity and respect.
WE.1 4.65 (Note: responses were either 4 or 5)
Q31. Differing opinions are welcomed and respected.
WE.2 3.96 (Note: most responses were 4 or 5; six responded with 2)
FINAL REPORT JUNE 2, 2023 26
INDEPENDENT THIRD-PARTY NUCLEAR SAFETY CULTURE ASSESSMENT The Team only included comments below when they were corroborated by other interviewees and inputs. They include:
Employee and Group Interview Responses:
Comment The site has not embraced the QA Program Previous similarities on latching issues not captured and shared Number of programs need to be integrated, starting with CAP and the Observation Program, and it needs to have teeth CAP was used as a punishment. If you raised a CAP issue, you were assigned to fix it. This was in 2018. The SRTs felt like just more work to do. If we raise things, we get more work to do We have a Corrective Action Program, but it is very high level and is largely a paper process. The NRC didnt push on it Just rolled process out. Went to training last week. Believes program can achieve success in long term. Data tracking is poor so far. No IT backup now. Concerned that small issues may not be identified A QA Program will identify the programs and processes that are required to safely operate a nuclear facility in a quality manner. The Program will also identify roles, responsibilities, accountabilities, and authorities to successfully develop, implement, and provide oversight of the Program. An effective QA Program will prevent events from occurring.
The current NCNR Program is fragmented, inconsistently being implemented, and provides for no effective independent oversight of its implementation. The NCNR Safety Evaluation Committee (SEC) is established to provide an independent review of NCNR reactor operations to ensure the facility is operated and maintained in such a manner that the general public, facility personnel, and property shall not be exposed to undue risk. This function will be the subject of a separate Task assessment. The NSC Assessment Team noted that it was not effective at identifying the numerous issues that were identified in the NCNR root cause evaluation and did not identify a lack of a healthy Nuclear Safety Culture which would have prevented the fuel failure event.
The NCNR Safety Assessment Committee (SAC) was established to provide an independent review or audit of NCNR reactor operations. This audit is to ensure that safety reviews and reactor operations are being performed in accordance with regulatory requirements and public safety is being maintained. SAC will also be part of another Task assessment; however, this independent review or audit was also not effective in identifying the underlying issues that led up to the fuel failure event.
ANSI15.8 also discusses management assessments that should provide oversight of the facility. NCNR does not have a management self-assessment process.
FINAL REPORT JUNE 2, 2023 27
INDEPENDENT THIRD-PARTY NUCLEAR SAFETY CULTURE ASSESSMENT 7.3.4 Document Control Program NBSR-0001-DOC-023 Document Control Plan was last revised in May of 2009. This document provides general guidance for a document control program. The program plan also discusses preparation of records and having a retention schedule.
Upper tier Federal requirements for a Document Control and Records Management Program include the following:
36 CFR 1220 Federal Records M-23-07 Transition to Federal Records 7.3.4.1 Conclusions Document Control Program Rating: Not Effective Trend: Flat NCNR does not have a documented document control and records program that meets the Code of Federal Regulations and current government requirements. A manager other than the QA Manager should be responsible for document control and records.
The NCNR document control and records management program plan, NBSR-0001-DOC-023, was in place in May 2009 and was not being implemented prior to the February 2021 fuel event. Currently the QA Program Manager is working on a Document Control and Records Management Program, however there is not a procedure that is effectively being implemented.
As stated in an interview with the QA Program Manager, NCNR has not implemented a Document Control and Records Management Program. The Team notes that records can become nuclear safety issues when they impact nuclear safety components, Tech Spec requirements, and/or not meeting ANSI-15.8.
Limiting Weakness(es):
NCNR does not have a Records and Document Control Manager NCNR does not have a plan to meet Federal Document Control requirements.
7.3.4.2 Basis, Discussion, and Observations The requirements of a Document Control and Records Management Program are not being effectively implemented.
Technical Specifications state the following.
Records to be Retained for a Period of at Least Five Years or for the Life of the Component Involved if Less than Five Years.
Records to be Retained for at Least One Operator Licensing Cycle.
Records to be Retained for the Life of the Reactor Facility.
FINAL REPORT JUNE 2, 2023 28
INDEPENDENT THIRD-PARTY NUCLEAR SAFETY CULTURE ASSESSMENT ANSI15.8 states the following:
Quality Records - A records system or systems shall be established at the earliest practicable time consistent with the time of completion of the work activity.
Document Control - The preparation, issue, and change of documents which specify requirements that affect quality or prescribe activities affecting quality shall be controlled to assure that correct documents are used.
NCNRs Document Control Plan does not meet 36 CFR 1220 through 1236.
Management roles and responsibilities for supporting a document control and records management program are not defined.
Employee Survey Responses:
ATTRIBUTE QUESTION, SCORE Q70. We have processes to identify and resolve PI.2 3.44 existing organizational weaknesses.
Q55. Plant activities are governed by WP.3 comprehensive high-quality programs, processes, 3.47 and procedures.
Q49. Plant activities are governed by WP.1 3.40 comprehensive high-quality programs, Q33. Leaders implement change in a way that builds LA.5 3.56 organizational trust.
Q17 Leaders from all levels in the organization are LA.2 3.83 involved in oversight of work The Team only included comments below when they were corroborated by other interviewees and inputs. They include:
Employee and Group Interview Responses:
Comment All of the programs need a lot of work. Program stand up is done as (level of effort).
ANSI 15.8 (Quality Assurance Program Requirements for Research Reactors) has not been embraced.
NCNR-ROE had about 550 active legacy procedures and documents prior to the accident, all of which need to get reviewed, checked against CONOPS, and probably revised. The commitment date is January 2025, and there is no plan to get it done, and no schedule. (Trait-Documentation: The organization creates and maintains complete, accurate and up-to-date documentation).
FINAL REPORT JUNE 2, 2023 29
INDEPENDENT THIRD-PARTY NUCLEAR SAFETY CULTURE ASSESSMENT 7.3.5 Conduct of Operations Program The Assessment Team undertook a limited scope review of the NCNR-ROE Conduct of Operations (CONOPs) program and its implementation. The basis document Administrative Rule 1.0 NBSR Conduct of Operations (Rev 02 dated 5/31/2022) was reviewed and approved by SEC and the NCNR-ROE Chiefs. Following review of this gateway procedure and prior to meeting with the program owner, the Team developed initial lines of inquiry related to a nuclear CONOPs program based upon standard practice in the US DOE and the commercial nuclear sector.
7.3.5.1 Conclusions ConOps Program Rating: Marginally Effective Trend: Positive The NCNR CONOPS program does not fully describe numerous nuclear industry CONOPS program elements.
The Team determined that AR1.0 addresses some attributes from nuclear Industry related CONOPS programs, however, does not describe some critical programs such as Operations Focus, Configuration Management and Control, Reactivity Control, and procedure control.
Provides for 50.59 review and determination, although not in a procedural context.
Specifies support programs, policies, and procedures that need to be in place to support a nuclear operations department.
Provides clearly defined roles, responsibilities, authority, and accountability requirements referenced for each Operations position.
Prescribes minimum staffing requirements for qualified/licensed operators.
References technical support personnel (e.g., Engineering, Health Physics, Maintenance) are in place to support operations.
Prescribes staff development, attraction, retention, and a succession matrix in a documented plan.
Details expectations for professional behavior and decorum.
Discusses emerging issue management.
Prescribes how the corrective action plan is a part of day-to-day activities taken.
Provides for investigating abnormal instances/events, determining consequences and understanding significance.
FINAL REPORT JUNE 2, 2023 30
INDEPENDENT THIRD-PARTY NUCLEAR SAFETY CULTURE ASSESSMENT The NCNR CONOPs program guidance did not sufficiently address the following aspects of CONOPs:
Operations Focus Configuration Management and Control Intent Procedure Changes Independent Verification Component Labeling Temporary Modifications Issue/event Causal Analysis Operational Limits Behaviors that underly CONOPS elements In a nuclear facility, the operators should set the example for how Nuclear Safety Culture will be implemented. NCNR operators and Operations supervisors/managers do not demonstrate a working knowledge of Nuclear Safety Culture behaviors.
Limiting Weakness(es):
Critical elements of a CONOPs program are Insufficiently referenced in NBSR Conduct of Operations:
Operations staffing shortages hamper operator performance, training, and recruiting.
Operations Training weaknesses (see Section 7.3.6 Training Program) discourage operator learning and learning the plant vs. pass a test.
Lack of Operations staff understanding of Nuclear Safety Culture and how it applies to CONOPs.
7.3.5.2 Basis, Discussion, and Observations Nuclear Safety Culture Survey Excerpts Attribute Question Score Q51. The organization conducts activities that could QA.1 affect reactivity with caution, in accordance with 4.33 procedures.
WP.4 Q52. Design and operating margins are carefully 4.16 guarded and changed as defined by procedures.
WP.2 Q53. Safety-related equipment is operated and 4.26 maintained within design requirements.
WP.1 Q54. Work is effectively planned and executed by 3.63 incorporating risk-informed insights.
Q55. Plant activities are governed by WP.3 comprehensive high-quality programs, processes, 3.47 and procedures.
FINAL REPORT JUNE 2, 2023 31
INDEPENDENT THIRD-PARTY NUCLEAR SAFETY CULTURE ASSESSMENT Attribute Question Score Q56. ROE employees master reactor operations to CL.4 establish a solid foundation for decisions and 3.74 behaviors.
WE Q28. I treat other employees with dignity and 4.65 respect. (Note: responses were either 4 or 5)
Q31. Differing opinions are welcomed and WE respected. (Note: most responses were 4 or 5; six 3.96 responded with 2)
The Team only included comments below when they were corroborated by other interviewees and inputs. They include:
Example Employee Responses:
Comment Q: Are you aware of any examples where technical specification requirements are not met on initial attempts, but are eventually met after several attempts?
A: Yes. One example is the Building Leak rate Test, where we have run the testing until we get a value that passes We didnt have much training before [the 02/03/2021 incident]. We did have hearing loss training, and electrical shock hazard training before, and maybe a bit more. But not much. We now have daily safety meetings to get people thinking about safety.
TTQ5: Who is accountable for the performance of your Program? Besides yourself, who else needs to support your Program for it to succeed?
A: Everybody.
There is a lack of experienced SROs, and procedures were written for the Good Old Boy club.
ANSI 15.8 (Quality Assurance Program Requirements for Research Reactors) has not been embraced.
Training was non-existent. Requal training was focused on passing the test. Licensing training emphasized reactor theory and 10 CFR. There was no emphasis on our systems including our reactor safety systems.
After the event we went overboard with the procedures (making them cumbersome). We should have spent much more time on Training. Some procedures are overly prescriptive.
We need more time and commitment to the training program. We need more people, and we need the right people, because right now its not working.
FINAL REPORT JUNE 2, 2023 32
INDEPENDENT THIRD-PARTY NUCLEAR SAFETY CULTURE ASSESSMENT Comment Training is a top issue, from day one. The SRO continuing training amounted to a one-or two-week biannual cram session for re-licensing. It also took two years [Note: in reality, a little over one year] from the time the position went vacant for the Training Manager position to get posted.
Q: How is the relationship communications with Operations?
A: Communication has not been good for some time Trouble Ticket (TT) and Preventive Maintenance and Material History need to be integrated AND they need a system that can handle nuclear rated components.
Operations is pushing hard for restarting as soon as possible, and at times Im uncomfortable with how hard they push.
Q9: What currently limits the performance of your Program? What are you doing about that?
A: CONOPs is hindered by lack of licenses. Operations has 5 potential licenses hanging in the balance, waiting for plant conditions to complete. We have 7 current licenses, giving us 12. The plan is to have 20 licenses total in the next two years.
Q: What do you expect the next two levels of management above you to do to support ConOps?
A: To respect the operator schedulelike not do things that were done in the past, like take operators on the mid-shift, make them attend training the following day, and then have them come back on mids to stand watch again. I need resources. Weve got to be able to maintain and improve the infrastructure, equipment, and people.
We still have stove-piping between Engineering and Operations.
The relationship between Ops-Engineering is not optimal.
7.3.6 Training Program Administrative rule 2.0 provides general qualification and training guidelines for NCNR ROE operations staff in order to ensure that personnel are capable of independent performance of facility activities. The NCNR training program implements the training attribute of the Nuclear Safety Culture trait for continuous learning (CL). The latest revision, Rev 1, to AR 2.0 was approved on December 21, 2021 (See section 7.4.5 of this report).
Other components of the NCNR training program are contained in the following documents:
AR 2.2.0 (August 2, 2022), ROE Personnel Onboarding AR 2.2.1 (August 3, 2022), Equipment Operator Qualification AR 2.2.2 (August 14, 2022, RO/SRO Qualification AR 2.2.3 (August 14, 2022), Reactor Supervisor Qualification AR 4.0 (March 31, 2022), Fuel Manipulation Proficiency Requirements FINAL REPORT JUNE 2, 2023 33
INDEPENDENT THIRD-PARTY NUCLEAR SAFETY CULTURE ASSESSMENT 7.3.6.1 Conclusions Training Program Rating: Marginally Effective Trend: Positive Prior to the February 2021 event the NCNR training program was conducted through self-study and signoffs by qualified operators. A new training manager has recently been selected. He has undertaken the task of making the program more formal in order to better address the fact that many of the new-hire operators lacked the industry experience of previous operators. In addition, he is developing lessons plans. The team was shown the recently developed lesson plan for Facility Rounds - B2 Basement Equipment Operator. Following the Nuclear Safety Culture assessment NCNR-ROE intends to perform an independent third-party assessment of the training program.
Limiting Weakness(es):
The current training program does not support a rule-based operator workforce.
Training that has been developed was not done using the Systematic Approach to Training.
Operator training is limited in scope and does not address the full spectrum of Operations Department needs.
The Team evaluated the Training Program as Marginally Effective, based upon (1) the limiting weaknesses noted and (2) the fact that the program is in place and has a recently named Training Manager.
7.3.6.2 Basis, Discussion, and Observations Employee Survey Training Question Score Attribute Q41. Leaders provide training and knowledge transfer to CL.4 3.71 establish and maintain technical competence.
Survey comment:
CL.4 The training program is under development, and improving, N/A but that aspect has historically been lacking.
Survey comment:
CL.4 We still have only one person as the technical trainer for all N/A of ROE.
FINAL REPORT JUNE 2, 2023 34
INDEPENDENT THIRD-PARTY NUCLEAR SAFETY CULTURE ASSESSMENT The Team only included comments below when they were corroborated by other interviewees and inputs. They include:
Employee and Group Interview Responses:
Comment We didnt have much training before [the 02/03/2021 incident]. We did have hearing loss training, and electrical shock hazard training before, and maybe a bit more. But not much. We now have daily safety meetings to get people thinking about safety.
We need a leadership needs assessment and leadership development. Leading people is a skill that has to be learned. We have to assess where we need to improve (behaviorally) to be a leader, understand where they need to develop, then coaching and training. Especially soft skills.
Training was non-existent. Requal training was focused on passing the test. Licensing training emphasized reactor theory and 10 CFR. There was no emphasis on our systems including our reactor safety systems.
After the event we went overboard with the procedures (we made them cumbersome).
We should have spent much more time on Training. Some procedures are overly prescriptive.
We need more time and commitment to the training program. We need more people, and we need the right people, because right now its not working.
The perception that requalification training focused on passing the test rather than understanding the systems was voiced by several individuals.
7.3.7 Procedures (NBSR) Procedure Administrative Rule 5.0 Rev 1 of August 12, 2021, provides for NCNRs Procedure Use and Adherence administration. The NCNR procedures program implements the WP.3 and WP.4 attributes for planning and controlling work activities.
(See section 7.4.4 of this report).
The purpose of Administrative Rule 5.0 is to:
Establish administrative controls governing adherence to written instructions in support of excellent human performance.
Identify actions to be taken when adherence to written instructions is not possible or is unsafe.
Describe requirements for verifying activities affecting alignment or status of the NIST test reactor (NBSR) systems, portions of systems, or components.
This document provides supplemental guidance to TR-5 section 6.4 to include instruction for procedure use and correction for all ROE and HP procedures.
AR 5.0 applies to the procedures, instructions, rules and manuals used by the Reactor Operations and Engineering group, to include Reactor Operations, Reactor FINAL REPORT JUNE 2, 2023 35
INDEPENDENT THIRD-PARTY NUCLEAR SAFETY CULTURE ASSESSMENT Engineering, and NCNR Health Physics, for the safe and reliable operation of the NBSR.
7.3.7.1 Conclusions Procedures Program Rating: Marginally Effective Trend: Positive The root cause investigation conducted after the February 2021 event identified the following inadequacies in the NCNR procedures:
Procedures as written did not capture necessary steps in assuring elements are latched.
Procedural compliance was not enforced.
For most of its operating life NCNR relied upon industry experienced operators. The need for more detail in procedures was not necessary. As the number of these experienced operators retired or left, less experienced operators were hired. These new employees needed more structure and detail in the operating procedures. Corrective actions included:
Rewrite fueling procedures to capture aspects of movements to align with training.
Update procedures to require training for all personnel on procedure use, place keeping, and adherence.
Revise procedures to be consistent with INPO 11-003 (Guideline for Excellence in Procedure and Work Instruction Use and Adherence.)
While the intention of the root cause investigation improvement initiatives may have been beneficial, they require an evaluation of its overall effectiveness.
Limiting Weakness(es):
Inadequate procedures to support implementation of a rule-based method of operation required by less industry experienced operators.
Some of the revised procedures are overly complex and cumbersome.
Pen and ink changes to important procedures (i.e., reactor startup) did not receive 10CFR50.59 review.
Inexperienced (rather than experienced) operators are writing procedures.
The Team evaluated the Procedures Program as Marginally Effective, based upon (1) the limiting weaknesses noted and (2) the fact that procedure upgrades are being implemented.
FINAL REPORT JUNE 2, 2023 36
INDEPENDENT THIRD-PARTY NUCLEAR SAFETY CULTURE ASSESSMENT 7.3.7.2 Basis, Discussion, and Observations Employee Survey Procedures Program Question Score Attribute WP.3 Q49. Plant activities are governed by comprehensive 3.40 high-quality programs, processes, and procedures.
WP.3 Q52. Design and operating margins are carefully 4.16 guarded and changed as defined by procedures.
WP.4 Q43. I apply a rigorous approach to problem solving 4.24 in accordance with procedures.
Q51. The organization conducts activities that could WP.4 affect reactivity with caution, in accordance with 4.33 procedures.
WP.4 Q38. I follow processes, procedures, and work 4.67 instructions.
Some policies may actually be overly restrictive WP.4 (some aspects of the procedural updates Survey compensating for weaknesses in the training comment program, for example)
The incredibly heavy focus on procedures that "anyone can perform" tells me again that WP.3 Survey management is not concerned about maintaining comment experienced people, only minimally trained individuals The quantity of procedures and administrative rules has multiplied 3-4x. Information which was contained WP.3 in just a few pages is now fragmented across several Survey documents. This makes finding the needed info and comment following the procedure per AR 5 rules excessively time consuming, cumbersome, and often impossible.
Others question the validity of the procedures. Survey WP.3 comment While most graded questions were scored high, Q38 was the highest scored question in the Survey. The comments paint a different picture.
FINAL REPORT JUNE 2, 2023 37
INDEPENDENT THIRD-PARTY NUCLEAR SAFETY CULTURE ASSESSMENT The Team only included comments below when they were corroborated by other interviewees and inputs. They include:
Employee and Group Interview Responses:
Comment We revised Administrative Rule (AR) 5.0 on Procedure Use and Adherence We now have much better references in our procedures, programs, and policies and in our conduct of operations.
NCNR-ROE had about 550 active legacy procedures and documents prior to the accident, all of which need to get reviewed, checked against CONOPS, and probably revised. The commitment date is January 2025, and they have no plan to get it done, and no schedule.
I think people want to get to where we need to be, and they see the opportunity to make the shiftbut we dont have the processes that we need, and the changes wont stay without that.
Operations has about 169 procedures of approximately 600 NCNR procedures. Per the Confirmatory Order, all procedures need to be upgraded by 2025.
There is no consistency in the quality of the procedure reviews. Fidelity is off - we identify issues after procedure is approved.
We are governed by procedures, but not high-quality procedures. The procedures are not cross tied.
There is no flow down of requirements Procedure review process is not formal. Pen and ink changes approved by SRO and Im not sure if they use the 10CFR 50.59 process.
Some drawings are not as shown, and configuration management is a known issue.
Some procedures are overly prescriptive.
Procedure use prior to the event was mostly informal. It has increased by an order of magnitude since but changing procedures is cumbersome 7.3.8 Causal Analysis Program 7.3.8.1 Conclusions Causal Analysis Program Rating: Not Effective Trend: Flat Causal Analysis/Root Cause Analysis (RCA) is a method of establishing logically complete, evidence-based, tightly coupled chains of factors from the least acceptable FINAL REPORT JUNE 2, 2023 38
INDEPENDENT THIRD-PARTY NUCLEAR SAFETY CULTURE ASSESSMENT consequences to the deepest significant underlying causes. Any systematic approach that accomplishes this is acceptable.
Limiting Weakness(es):
The NCNR Causal Analysis Program does not formally exist; it is under development without written development or implementation plans.
NCNR has few trained causal analysts.
Causal analysts that are trained have limited experience.
The Team evaluated the Causal Analysis Program as Not Effective, based upon the limiting weaknesses noted.
7.3.8.2 Basis, Discussion, and Observations Employee Survey Attribute Question Score Q69. Root cause analysis is rigorously applied to identify 3.75 and correct the fundamental causes of significant events.
Until lack of experience is considered a possible root cause or a contributing factor that needs correcting, I believe root cause analysis is not a useful tool. In general, Comment I believe everything so far has been deemed a management, procedural, or previously unknown engineering/equipment error.
The Team only included comments below when they were corroborated by other interviewees and inputs. They include:
Example Employee Responses:
Comment Q: brief overview of the Causal Analysis Program at NCNR.
A: It doesnt exist; its still in development. NIST itself has issues with Root Cause Analysis. We got the best information about Causal Analysis they hadit amounted to Build a timeline and Use the 5-Whys.
TapRooT was used for the [02/03/2021] event, but TapRooT wont get to Safety Culture, and the NRC had a big problem with that.
Q: Who is responsible for selecting an appropriate RCA method?
A: [redacted to protect personal identifying information]
Q: What requirements are NRC inspectors who look at RCAs expected to meet, particularly when theyre looking at a key RCA related to a Confirmatory Order?
A: [redacted to protect personal identifying information]
FINAL REPORT JUNE 2, 2023 39
INDEPENDENT THIRD-PARTY NUCLEAR SAFETY CULTURE ASSESSMENT Comment Q: Is there a plan to define, establish, and implement the RCA Program?
A: Yes.
Q: Is it written down?
A: No.
Q: What documents govern implementation of RCA?
A: They are under development.
Q: How important is the RCA Program to NCNR success?
A: It should be a pillar and it isnt. This organization has a history of figuring out how to solve the problem before figuring out what the problem is. Its going to be an uphill battle.
An important part to me is that we will be doing RCAs and coming up with corrective actions in the future. Right now, the program is in its infancywe arent there yet.
Q: How good do you think the RCA in response to the 2/3/2021 incident was?
A: Pretty good, considering that only a handful of us ever did RCAs before. There were a lot of people trying to find singular root causes instead of looking at the organization as a whole. The fuel melt was just one symptom of many things that were wrong. We have some real things that we have to improve.
7.3.9 Employee Concerns Program Employee Concerns Programs (ECPs) exist to provide employees with an alternate means of raising concerns and having them addressed, independent of the management structure. ECP Coordinators typically report to the Chief Nuclear Officer.
Reasons for establishing ECPs include:
An employee may not feel comfortable raising a specific concern through the standard management structure.
Managers or supervisors may not be adequately skilled or trained to deal with specific concerns.
Other systems or processes for resolving concerns/problems may be perceived as slow or ineffective.
The purpose of an ECP is to:
Provide employee anonymity and confidentiality.
Bring significant, valid concerns to managements attention.
Provides employees with a constructive alternative to utilizing outside organizations.
FINAL REPORT JUNE 2, 2023 40
INDEPENDENT THIRD-PARTY NUCLEAR SAFETY CULTURE ASSESSMENT The most common categories of concerns raised through ECP programs are allegations from individuals engaged in regulatorily protected activities and involve:
Harassment Intimidation Retaliation Discrimination Chilling Effect 7.3.9.1 Conclusions Employee Concerns Program Rating: Not Effective Trend: Flat The survey result for raising concerns was healthy but there was no Employee Concerns Program; the program is in an embryonic stage.
Limiting Weakness(es):
The NCNR Employee Concerns Program has not been implemented.
The ECP Coordinator has not yet been trained or started in the position.
The Team evaluated the Employee Concerns Program as Not Effective, based upon the limiting weakness noted.
Employee Survey Employee Concerns Question Score Program Attribute Q84. Leaders take ownership when receiving and RC.1 responding to concerns, while respecting confidentiality as 4.29 appropriate.
Q85. Leaders create an environment in which individuals RC.1 4.37 feel free to raise concerns.
RC.1 Q27. My supervisor responds to questions and concerns 4.54 in an open and honest manner.
Example Employee Responses:
Comment Q: Please give me a brief overview of the Employee Concerns Program.?
A: Its a new program for us. It was initiated as a result of the Confirmatory Order.
FINAL REPORT JUNE 2, 2023 41
INDEPENDENT THIRD-PARTY NUCLEAR SAFETY CULTURE ASSESSMENT Comment We finalized the Employee Concerns program and submitted it to the NRC for review and comment. Weve hired a Program Manager specifically to lead ECP here at NCNR.
7.3.9.2 Basis, Discussion, and Observations NCNR has taken initial steps to establish an Employee Concerns Program in response to the Confirmatory Order by submitting the draft Program document to the NRC for review and comment. NCNR is considering an internal hire to work part-time on the ECP and part-time on other projects for the Director, with an expectation that in six months the position will be working remotely.
The relationship of trust established between the ECP Coordinator and site personnel, and the availability of face-to-face communications are keys to an effective ECP. The Assessment Team has not seen an effective ECP that lacked a frequent (often continuous) on-site presence.
FINAL REPORT JUNE 2, 2023 42
INDEPENDENT THIRD-PARTY NUCLEAR SAFETY CULTURE ASSESSMENT 7.4 ASSESSMENT CONCLUSIONS: NUCLEAR SAFETY CULTURE TRAITS Limiting Weakness(es):
Lack of cross-functional alignment at the top (leaders and chiefs) and throughout the organization.
Lack of development/retention of key, talented mid-level managers.
Lack of visible leadership.
Insufficient funding to obtain and prioritize external resources who can take on:
o Operations Department administrative workload and free up operators (see LA.1, Section 7.4.1.2.2, last table entry); and, o Program development and implementation (see 7.3, Assessment
Conclusions:
Programmatic Support to NSC for discussion of nine key programs).
Discussion:
Lack of cross-functional alignment at the top (leaders and chiefs) and throughout the organization.
The chiefs are not in agreement on deployment of resources or priorities, and communication is dismal. Moreover, they are not brought into alignment by their leadership. Results in poor cross-functional teamwork at all levels.
Lack of development/retention of key, talented mid-level managers.
These critical, high potential individuals are key to recovery, but they are being overwhelmed yet unsupported in the current environment. They are not receiving the guidance, resources, and direction they need to be successful and are a flight risk.
Lack of visible leadership.
In the current simultaneous plant recovery and NSC program and behavior change, leadership direction and visibility are crucial to:
o Driving senior leadership (chiefs) to cascade an aligned direction, expectations, motivation, down through their organizations. They must set up the proper routines and communications to do so.
o Advocating for the organization for the immediate resources and appropriate schedule expectations, particularly as it relates to culture change.
The Team performed an in-depth assessment of NCNR-ROEs performance in each of the 10 Traits and their supporting attributes.
Data input was drawn from each evaluated data source (Functional analysis; Archived Data; Structured Interviews; Focus Groups; Observations; and Safety Culture Survey) to feed into a collective assessment. The Teams collective rollup is presented in the FINAL REPORT JUNE 2, 2023 43
INDEPENDENT THIRD-PARTY NUCLEAR SAFETY CULTURE ASSESSMENT table below with more details presented in an Attribute-by-Attribute evaluation summary in Attachment F:
NSC EFFECTIVENESS BY TRAIT TRAIT RATING TREND BASIS - LIMITING WEAKNESSES There is a lack of understanding Nuclear Safety Culture core values/behaviors and the need for improvements across ROE.
Leaders have not fully embraced newly developed programs that are fundamental
- 1. Leadership Marginally to an effective Nuclear Safety Culture Values & Effective Positive The Operator training program is under-Actions (LA) resourced with respect to staffing and AFI/ANA program development and staff has too few licensed reactor operators.
Improvement plans lack structure, milestones, and teamwork.
Management has not implemented a Management of Change process.
NCNR-ROE has not embraced the PI&R concepts of problem identification, reporting, documenting, evaluating, resolving, and trending issues occurring on site.
& Resolution Effective Flat Corrective Action Program does not meet (PI) the requirements of ANSI/ANS 15.8 AFI Quality Assurance Program requirements.
A broad set of performance indicators with a focus on early detection of problems does not exist.
A consistent CAP reporting threshold has yet to be established.
Organizational silos exist that weaken teamwork.
Single-point accountability for Nuclear
- 3. Personal Marginally Safety Culture decisions has not been Accountability Effective Flat clearly established.
(PA) Collaborative ownership of cross-AFI cutting/distributive programs such as training, corrective action, and conduct of operations is not occurring at the department head level FINAL REPORT JUNE 2, 2023 44
INDEPENDENT THIRD-PARTY NUCLEAR SAFETY CULTURE ASSESSMENT NSC EFFECTIVENESS BY TRAIT TRAIT RATING TREND BASIS - LIMITING WEAKNESSES Procedures coming out of the current upgrade program are often overly detailed, complex, cumbersome, and have not been effectively verified and validated.
- 4. Work Marginally Work evolutions are not consistently Processes Effective Positive planned to consider contingencies and (WP) engaging supporting work groups.
PO/AFI Procedure development is currently being done by operators as a collateral duty without benefit of technical writers.
An Emerging Issue Management process has not been established.
The Operator Continuing Training program is under development.
The programs for managing Operating Marginally Experience, Benchmarking, and Self-
- 5. Continuous Effective Positive Assessments have not been developed.
Learning (CL)
ANA/AFI Knowledge transfer is largely based upon tribal knowledge.
The organization does not conduct critical self-assessments.
The Alternate Process for Raising Concerns (RC.2), the Employee Concerns program has been submitted to the NRC for review.
- 6. Environment Marginally Some employees expressed reluctance for Raising Effective Positive to bring up concerns related to safety Concerns (RC) because they dont think that AFI management would do anything about them.
A formal Differing Professional Opinion process does not exist.
FINAL REPORT JUNE 2, 2023 45
INDEPENDENT THIRD-PARTY NUCLEAR SAFETY CULTURE ASSESSMENT NSC EFFECTIVENESS BY TRAIT TRAIT RATING TREND BASIS - LIMITING WEAKNESSES Leadership communication about the importance of nuclear safety sometimes gets confused with the importance of schedule.
- 7. Effective Communication of operational and Marginally organizational decisions is inconsistent.
Safety Effective Positive Ineffective communications between Communication (CO) ANA different departments impacts Nuclear Safety Culture.
Important organizational issues are not consistently bubbled up to the director level.
The NCNR-ROE organization is siloed.
A potential toxic work environment exists between Operations and I&C.
Leaders at times implement change in a
- 8. Respectful way that does not build organizational Marginally Work trust.
Effective Positive Environment There are elements of the organization (WE) ANA/AFI that have no trust in other parts of the organization.
Differing opinions are inconsistently respected and considered.
Risks are not always evaluated and managed before proceeding on infrequently performed tasks and emergent issues.
9, Questioning Effective Some employees stated that they are not Positive always listened to when challenging Attitude (QA) ANA/AFI some assumptions.
There is an apparent resistance to change or slow acceptance for the need to change regarding key Nuclear Safety Culture traits.
FINAL REPORT JUNE 2, 2023 46
INDEPENDENT THIRD-PARTY NUCLEAR SAFETY CULTURE ASSESSMENT NSC EFFECTIVENESS BY TRAIT TRAIT RATING TREND BASIS - LIMITING WEAKNESSES Operations staff acceptance of some surveillances that require multiple executions until an acceptable reading is obtained - for years - and not addressing the underlying issues (physical and procedural).
Marginally Operational decisions on emerging
- 10. Decision-Effective Positive issues are occasionally made without the Making (DM)
ANA input of all expected functional groups.
Lack of single-point accountability for nuclear safety decisions. (This was the lowest scoring question in the survey).
Conservative assumptions are inconsistently engaged when addressing emerging issues or work.
See Attachment F, Nuclear Safety Culture Traits Evaluation Summary, for additional details.
The Team evaluated each NSC Trait by considering what the collective information revealed with regard to each of the attributes defined by INPO 12-012 / NUREG-2165.
Team conclusions are summarized below, trait by trait.
7.4.1 Leadership Safety Values and Actions (LA)
Leaders demonstrate a commitment to safety in their decisions and behaviors.
Safety Culture Common Language (NUREG 2165) and INPO 12-012 identify the following Nuclear Safety Culture attributes of the Trait- Leadership Safety Values and Actions:
LA.1 Resources: Leaders ensure that personnel, equipment, procedures, and other resources are available and adequate to support nuclear safety.
LA.2 Field Presence: Leaders are commonly seen in working areas of the plant observing, coaching, and reinforcing standards and expectations. Deviations from standards and expectations are corrected promptly.
LA.3 Incentives, Sanctions and Rewards: Leaders ensure incentives, sanctions, and rewards are aligned with nuclear safety policies and reinforce behaviors and outcomes that reflect safety as the overriding priority.
LA.4 Strategic Commitment to Safety: Leaders ensure plant priorities are aligned to reflect nuclear safety as the overriding priority.
FINAL REPORT JUNE 2, 2023 47
INDEPENDENT THIRD-PARTY NUCLEAR SAFETY CULTURE ASSESSMENT LA.5 Change Management: Leaders use a systematic process for evaluating and implementing change so that nuclear safety remains the overriding priority.
LA.6 Roles, Responsibilities, and Authorities: Leaders clearly define roles, responsibilities, and authorities to ensure nuclear safety.
LA.7 Constant Examination: Leaders ensure that nuclear safety is constantly scrutinized through a variety of monitoring techniques, including assessments of Nuclear Safety Culture.
LA.8 Leader Behaviors: Leaders exhibit behaviors that set the standard for safety.
7.4.1.1 Conclusions Leadership Values/Actions (LA): Marginally Effective (AFI) Trend: Positive Although Leadership Safety Values and Actions is a stand-alone trait per NUREG 2165 and INPO 12-012, it is foundational to the other nine traits. After the February 2021 event NCNR leadership implemented several measures to develop a healthy Nuclear Safety Culture. These measures, in large part, were not integrated into an overall strategy for achieving it. In particular, the effort did not consider the challenge in obtaining the necessary resources, particularly manpower.
Capabilities and Resources The February 2021 reactor event, shutdown, and recovery efforts have visibly stretched the capabilities and resources of the organization as a whole. The loss of talent, knowledge, and experience are ongoing through attrition and retirement; recovery expectations and effortstechnical and programmaticare ongoing in the form of procedure changes, creation and implementation of significant programs, and behavioral change, among others.
The result has been an apparent increase in conflicts over resources, resistance to changes, and functional siloing. Although leaders will mention that there has been approval for various staff positions, relief has been slow, and many employees were pessimistic. This environment may also have spurred increased directive leadership for some, and at the other end of the continuum, passive leadership.
Talent Retention The assessment team was impressed by the number of talented and motivated employees and leaders who had realistic perspectives on the NSC challenges at NCNR. However, the level of frustration and the number of individuals actively looking to leave NIST were alarming. This talent pool is vital to the recovery and future leadership of NCNR.
It should be noted that although Operations has short-term and long-term plans to increase staffing and licensed operators, there is a risk now of a loss of licensees and near-term licenses that could impact license requirements in the near future.
FINAL REPORT JUNE 2, 2023 48
INDEPENDENT THIRD-PARTY NUCLEAR SAFETY CULTURE ASSESSMENT Limiting Weakness(es):
There is a lack of understanding Nuclear Safety Culture core values/behaviors and the need for improvements across ROE.
Leaders have not fully embraced newly developed programs that are fundamental to an effective Nuclear Safety Culture The Operator training program is under-resourced with respect to staffing and program development and staff has too few licensed reactor operators.
Improvement plans lack structure, milestones, and teamwork.
Management has not implemented a Management of Change process.
Insufficient funding to obtain and prioritize external resources who can take on the Operations Department administrative workload and free up operators (see LA.1, Section 7.4.1.2.2, last table entry).
7.4.1.2 Basis, Discussion, and Observations 7.4.1.2.1 Nuclear Safety Culture Survey Of the ten lowest scored questions on the employee survey, five dealt with leadership.
Survey average score was 4.1. Twenty-one (21) questions dealing with leadership fell below the survey average score.
LEADERSHIP QUESTION SCORE ATTRIBUTE Q44 Leaders assign single-point accountability for nuclear 3.25 LA.6 safety decisions.
Q23 Team Leaders are selected based upon fostering a strong 3.38 LA.4 nuclear safety environment that promotes accountability.
Q12 Leaders provide incentives and rewards that are aligned 3.44 LA.3 with nuclear safety policies.
Q33. Leaders implement change in a way that builds 3.56 LA.5 organizational trust.
Q21. Leaders at all levels ensure that the basis for operational 3.58 LA.8 and organizational decisions is communicated to staff in a timely manner.
Q59. Leaders develop contingencies to deal with the LA.7 3.70 possibility of emergent problems.
Q41. Leaders provide training and knowledge transfer to LA.1 3.71 establish and maintain technical competence.
Q71. Leaders foster an environment in which individuals LA.1 3.71 value and seek continuous learning opportunities.
Q17. Leaders from all levels in the organization are LA.2 3.83 involved in oversight of work activities.
Q24. Leaders 'walk the talk modeling behaviors when LA.8 3.84 resolving conflicts between nuclear safety and production.
FINAL REPORT JUNE 2, 2023 49
INDEPENDENT THIRD-PARTY NUCLEAR SAFETY CULTURE ASSESSMENT LEADERSHIP QUESTION SCORE ATTRIBUTE Q8. Leaders reinforce the focus on nuclear safety through LA.2 3.90 site visits.
Q20. Leaders provide goals for production of neutrons that LA.4 are aligned to reflect nuclear safety as the overriding 3.92 priority.
Q16. Team Leaders are visible in the plant reinforcing LA.2 3.92 Nuclear Safety Culture behaviors.
Q63. Leaders apply conservative decision making to LA.7 3.94 mitigate unpredicted failures.
Q6. Leaders clearly define roles, responsibilities, LA.6 4.00 authorities, and accountabilities to ensure nuclear safety.
LA.2 Q26. Our Team Leaders are frequently present in the field. 4.00 Q14. Leaders exhibit behaviors that set the standard for LA.8 4.02 nuclear safety.
Q45. Leaders ask for input when evaluating nuclear safety LA.8 4.02 issues.
Q36. Leaders enhance trust and nuclear safety through LA.4 4.04 communications.
Q72. Leadership enforces management standards and LA.4 4.08 expectations that reflect nuclear safety values.
Q47. When previous operational decisions are called into LA.8 question by emerging facts, leaders re-evaluate and adjust 4.08 as needed.
Employee surveys, when taken seriously, can be helpful to identify potential strengths and weaknesses in Nuclear Safety Culture to be investigated in subsequent interviews and observations. The concerns with accountability, selection of leaders, organizational change, and communication of organizational decisions were, to some extent, confirmed in subsequent interviews and focus groups. A number of those interviewed selected a 5, the highest number, for each question.
7.4.1.2.2 Employee and Group Interview Responses A representative sample of interview responses is provided below, arranged by Attributes of the LA Trait, with Attributes arranged in descending order of significance to the NCNR Nuclear Safety Culture.
LA.1: Resources: Leaders ensure that personnel, equipment, procedures, and other resources are available and adequate to support nuclear safety.
One of the most serious existential threats to reactor operations for NCNR leadership is the difficulty of establishing and maintaining sufficient numbers of qualified operators, due to difficulties in hiring, training, and qualifying them. Many interviewees expressed frustration with the hiring process coupled with the need for improved procedures and training for new hires that lack the experience of veterans.
FINAL REPORT JUNE 2, 2023 50
INDEPENDENT THIRD-PARTY NUCLEAR SAFETY CULTURE ASSESSMENT The Team only included comments below when they were corroborated by other interviewees and inputs. They include:
Example Employee Responses:
Comment We need licenses, and it is difficult to get NRC License examiners here to NCNR.
We need to get the hiring process revised and made consistent; cut back on the competitive process; it takes months to hire someone.
We need more people, and we need the right people, because right now its not working. It is very hard to hire people.
Ops needs additional staffing to allow implementation of corrective actions. Ops needs more licensed operators to function.
Q: Do you have anything documenting your plans? Even personal hand-written notes?
A: No.
Q: What are the top three issues at NCNR that impact the Nuclear Safety Culture?
A:
Schedule pressure.
Resource allocation Training and experience Communications between different departments Resistance to change The experienced operators are willing to follow the rules, but theyll complain about it.
Operations carries a substantial administrative load, including:
Procedure updates (90% operations)
Procedure reviews (on every procedure)
Procedure required reading for all updated procedures SRTs (operations organizes the meetings and fills out the paperwork)
Hazard Reviews ECN reviews Maintaining regulatory required logs Trouble Ticket system maintenance CAP entries and corrective actions Security posture and access monitoring Surveillance maintenance and documentation Preventive maintenance and documentation Material history Inventory of consumables Training Records per regulations and requalification Industrial Safety Training OJT for 13 NLOs and future NLOs FINAL REPORT JUNE 2, 2023 51
INDEPENDENT THIRD-PARTY NUCLEAR SAFETY CULTURE ASSESSMENT LA.4: Strategic Commitment to Safety: Leaders ensure plant priorities are aligned to reflect nuclear safety as the overriding priority.
Management must ensure that no schedule pressure, even self-induced, distracts people from the focus on safe operations. Efforts to resume normal operations must focus on the improved safety of the facility.
The Team only included comments below when they were corroborated by other interviewees and inputs. They include:
Example Employee Responses:
Comment People are under self-induced pressure here, and we must make it easier for them to do things right Theres always schedule pressure.
Management is pushing to get to the next level of power/schedule pressure. Some management is visibly upset because of not meeting schedule. Because of the visible reaction, management sets the tone for schedule pressure.
Operations is pushing hard for restarting as soon as possible, and at times Im uncomfortable with how hard they push.
Q. To what extent do you agree that NCNR is ready to restart and enter power operations?
A: No, we should not be at power yet. We are dealing with the fission gas issue, and we dont have enough operators to support operating. The problem is that we dont know what will happen during start up.
Q: What behaviors do you believe are NOT aligned well with where NCNRs culture needs to go?
A. There is a strong push to get the reactor up and running, and that means a lot of pressure. Regarding getting things done that have complicating factors, we hear things like, If this were 10 years ago, wed give it to the mid-shift to fix at 2 AM when no one is around. That mindset is still there.
LA.8: Leader Behaviors: Leaders exhibit behaviors that set the standard for safety.
Continuous reinforcement of Nuclear Safety Culture behaviors will be necessary.
Management should take every opportunity to stress nuclear safety as an overriding priority.
The Team only included comments below when they were corroborated by other interviewees and inputs. They include:
FINAL REPORT JUNE 2, 2023 52
INDEPENDENT THIRD-PARTY NUCLEAR SAFETY CULTURE ASSESSMENT Example Employee Responses:
Comment Q: What examples can you provide of managers/supervisors encouraging identification and reporting of nuclear safety issues or concerns?
A: Ive heard upper management say that in meetings. Upper management is very concerned about safety.
Management Talks the Talk but does not Walk the Walk.
LA.6: Roles, Responsibilities, and Authorities: Leaders clearly define roles, responsibilities, and authorities to ensure nuclear safety.
NCNR leadership must continue to improve organizational alignment and eliminate stovepipes. Formal documentation of roles and responsibilities for each employee is essential for everyone to know their role in safe operations.
The Team only included comments below when they were corroborated by other interviewees and inputs. They include:
Example Employee Responses:
Comment After the event, we have daily meetings between Operations and Engineering. Safety culture has improved markedly.
Its easy to stovepipe here, thats been an issue forever. As an example, we had helium build up this week, and Operations didnt talk to Engineering.
Need improvements in organizational communication. everyone & no-one is responsible Q. If you have a procedure that affects Engineering or HP, how do you train them?
A: That is a gap we have to work on. Other groups that are affected are on the list to review the procedure before its approved, but that is all. We dont have any training for other organizations set up.
Q: Has your HR support improved in the last few years?
A: No. We get our candidates ourselves, mostly through Linked-In. HR doesnt understand that what we need are operators with at least one years experience.
Q: Youve told us about external audits and surveillance. Do you do any internally performed audits and surveillances?
A: That would be QAs responsibility.
Q. Who is responsible for selecting an appropriate RCA method?
A: [Pause] I do not know.
FINAL REPORT JUNE 2, 2023 53
INDEPENDENT THIRD-PARTY NUCLEAR SAFETY CULTURE ASSESSMENT LA.2: Field Presence: Leaders are commonly seen in working areas of the plant observing, coaching, and reinforcing standards and expectations. Deviations from standards and expectations are corrected promptly.
Nuclear Safety Culture leadership requires management to be frequently visible and engaged in meaningful communications with employees.in the field. Some progress has been achieved but it must become a regular part of each day.
The Team only included comments below when they were corroborated by other interviewees and inputs. They include:
Example Employee Responses:
Comment Management has no problem with us bringing up issuestalking about the CRO, CRE, and Director. We see Jim Adams regularly touring the facility.
Observation Program: in 2022 Q4 we were at 40% of target (participation), and at the end of Q1 2023 we were 50-60% target for observations.
In the old days they were always in the Control Room. Not now - the managers do not do that.
LA.5: Change Management: Leaders use a systematic process for evaluating and implementing change so that nuclear safety remains the overriding priority.
Ensuring that employees understand that necessary organizational changes are properly communicated and understood builds trust. Implementing a Nuclear Safety Culture will involve frequent changes.
The Team only included comments below when they were corroborated by other interviewees and inputs. They include:
Example Employee Responses:
Comment Employees here do not like change.
We are headed in the right direction, but we cant rush, were on the right path.
Culture change isnt a check the box and youre done.
ANSI 15.8 (Quality Assurance Program Requirements for Research Reactors) has not been embraced.
LA.7: Constant Examination: Leaders ensure that nuclear safety is constantly scrutinized through a variety of monitoring techniques, including assessments of Nuclear Safety Culture.
Regular follow-up Nuclear Safety Cultures surveys will be necessary as NCNR leadership continues to implement a healthy Nuclear Safety Culture. In order to ensure the veracity of the survey it is vital that employees recognize their responsibility to provide honest answers to the survey questions.
FINAL REPORT JUNE 2, 2023 54
INDEPENDENT THIRD-PARTY NUCLEAR SAFETY CULTURE ASSESSMENT The Team only included comments below when they were corroborated by other interviewees and inputs. They include:
Example Employee Responses:
Comment Q: Why do you think a number of people answered all 5s on the safety culture survey?
A: They did not take the survey seriously.
Q: Why do you think a number of people answered predominantly 5s or predominantly 1s on the safety culture survey?
A: 1s decided that they were not listened to thus a poor rating. 5s individuals are doing well so thats what they observe.
In the past, we have had a culture of, If you cant measure it, they cant nail you for it. Thats still here, to some extent.
LA.3 Incentives, Sanctions and Rewards: Leaders ensure incentives, sanctions, and rewards are aligned with nuclear safety policies and reinforce behaviors and outcomes that reflect safety as the overriding priority.
The Team only included comments below when they were corroborated by other interviewees and inputs. They include:
Example Employee Responses:
Comment Nuclear Safety Culture has improved. Prior to the 2021 event we had a level of Nuclear Safety Culture, however there was complacency. Now we have regular meetings, a Plan of the Day meeting, rewards and incentives.
Management established rewards and other recognition to encourage people to do the right thing, like raising issues.
7.4.2 Problem Identification and Resolution (PI)
Issues potentially impacting safety are promptly identified, fully evaluated, and promptly addressed and corrected commensurate with their significance.
Safety Culture Common Language (NUREG 2165) and INPO 12-012 identify the following Nuclear Safety Culture attributes of the Trait-Problem Identification and Resolution PI.1 Identification: The organization implements a corrective action program with a low threshold for identifying issues. Individuals identify issues completely, accurately, and in a timely manner in accordance with the program.
FINAL REPORT JUNE 2, 2023 55
INDEPENDENT THIRD-PARTY NUCLEAR SAFETY CULTURE ASSESSMENT PI.2 Evaluation: The organization thoroughly evaluates problems to ensure that resolutions address causes and extent of conditions, commensurate with their safety significance.
PI.3 Resolution: The organization takes effective corrective actions to address issues in a timely manner, commensurate with their safety significance.
PI.4 Trending: The organization periodically analyzes information from the corrective action program and other assessments in the aggregate to identify programmatic and common cause issues.
7.4.2.1 Conclusions Problem Identification & Resolution (PI) Rating: Not Effective (AFI) Trend: Flat Problem Identification and Resolution is a key Trait of INPO 12-012 and NUREG-2165.
The Traits and Attributes lay out the expectations for NCNR to identify, evaluate, promptly address issues commensurate with their significance. Prior to the February 2021 event, Leadership had not effectively implemented a PI&R process. Since the 2021 event, Leadership has developed a process implemented in March of 2023, trained some personnel, but has not integrated its implementation throughout the organization.
Limiting Weakness(es):
NCNR has not embraced the PI&R concepts of problem identification, reporting, documenting, evaluating, resolving, and trending issues occurring on site.
The existing CAP program AR 7.2, Corrective Action Program does not meet the requirements of ANSI/ANS 15.8 Quality Assurance Program requirements.
A broad set of performance indicators with a focus on early detection of problems does not exist.
A consistent CAP reporting threshold has yet to be established.
7.4.2.2 Basis, Discussion, and Observations 7.4.2.2.1 Nuclear Safety Culture Survey Of the ten lowest scored questions on the employee survey, five dealt with PI.
Survey average score was 4.2.
Leadership Question Score Attribute Q77. A broad set of performance indicators is utilized with a 3.48 PI.4 focus on early detection of problems.
FINAL REPORT JUNE 2, 2023 56
INDEPENDENT THIRD-PARTY NUCLEAR SAFETY CULTURE ASSESSMENT Leadership Question Score Attribute Q70. We have processes to identify and resolve existing 3.44 PI.3 organizational weaknesses.
Q69. Root cause analysis is rigorously applied to identify 3.75 PI.2 and correct the fundamental causes of significant events.
Q49. Plant activities are governed by comprehensive high- 3.40 PI.1 quality programs, processes, and procedures.
Employee surveys, if taken seriously, can be helpful to identify potential strengths and weaknesses in Nuclear Safety Culture to be investigated in subsequent interviews and observations - when provided data is thought to be reliable. The concerns with accountability, selection of leaders, organizational change, and communication of organizational decisions were, to some extent, confirmed in subsequent interviews and focus groups. A number of those interviewed selected a 5, the highest number, for each survey statement.
The Assessment Team reviewed the PI&R process AR 7.2, survey results, interviews, and CAP data provided by NCNR ROE. The Team reviewed the CAP and basic requirements to clearly state expectations for identifying low threshold issues.
The Team also reviewed a list of NCNR CAPS through 03/31/2023 for issues being properly categorized. One event that was classified as a level 0 (AR 7.2 definition states, No Reactor Safety or Health and Safety consequences) was a dropped fuel event. The descriptions stated that The tool was on the roll pins rather than the on the "ears". The fuel element was inspected, and damage was found inside the nozzle. The Team determined that the significance level of 0 was inappropriately assigned to the CAP. The Teams conclusion that identifying this CAP as a level 0 was inappropriate, is based on the description of the CAP, potential procedure weaknesses, verification that the fuel element was on the ears rather that the roll pins, human performance issues, training, and management oversight weaknesses contributed to this event. Because of the level 0 determination, no previous event review was conducted (looking for previous dropped fuel events and corrective action effectiveness) no cause analysis, no corrective actions for contributing causes.
The 2021 root cause self-performed by NCNR-ROE for the failed fuel element is being evaluated in Task 2. However, the Team learned that the NRC staffs review of the NIST root cause investigation into the February 3, 2021, event identified that an inadequate safety culture (not identified in the NCNR-ROE root cause report) was a significant contributor to the actions leading to the event.
The NRC also required that NCNR take various actions to improve leadership accountability including safety culture training for all senior leaders and plans to address staffing challenges. The Assessment Team identified that the 2023 NCNR survey results identified there was a misconception of what Nuclear Safety Culture is. This was validated during interviews and observation that only 2 of the 55 people interviewed had an understanding of Nuclear Safety Culture and most personnel discussed an industrial safety culture not the behaviors of a healthy nuclear culture.
FINAL REPORT JUNE 2, 2023 57
INDEPENDENT THIRD-PARTY NUCLEAR SAFETY CULTURE ASSESSMENT When NSC training was discussed, virtually everyone stated that they had not had NSC training. So, meeting the area of improving leadership accountability, the site falls short in the area of training for NSC.
Both of these examples indicate a lack of a healthy Nuclear Safety Culture and not having a PI&R process which should have identified a lack of a healthy Nuclear Safety Culture as a cause.
7.4.2.2.2 Employee and Group Interview Response The Assessment Team conducted interviews using three methods: individual interviews, group interviews of 3-4 persons, and program leader interviews.
The interviewers documented interviews in formal summaries that were reviewed by the entire Assessment Team. A representative sample of interview responses is provided below, arranged by Attributes of the LA Trait, with Attributes arranged in descending order of significance to the NCNR Nuclear Safety Culture:
A representative sample of interview responses is provided below, arranged by Attributes of the PI Trait.:
PI.1: Identification: The organization implements a corrective action program with a low threshold for identifying issues. Individuals identify issues completely, accurately, and in a timely manner in accordance with the program.
Leaderships role for an effective CAP is to support the issuance of a documented procedure that provides guidance in meeting the four attributes of a healthy CAP.
Leadership should discuss issues and ask the most important question by asking who has written a CAP on this issue?
The NCNR PI&R process has not been effectively developed and implemented for years. Employees provided numerous comments on the current status and how one should be implemented. Expectations should be provided and enforced from the Leadership down to every employee on the importance of effectively implementing PI&R.
The Team only included comments below when they were corroborated by other interviewees and inputs. They include:
Example Employee Responses:
Comment We have a Corrective Action Program, but it is very high level and is largely a paper process Our staff was reluctant to bring up concerns related to safety because they dont think that we would do anything about them.
Corrective action program - we had a program 3 years ago; not really used, not in our IT system; we are just now starting to use the CAP Ive seen recently that when there are incidents, people bring it up now, saying This probably needs reporting.
FINAL REPORT JUNE 2, 2023 58
INDEPENDENT THIRD-PARTY NUCLEAR SAFETY CULTURE ASSESSMENT Comment Engineers are expected to use the CAP process. But CAP is new, they have not tried it.
We need to pay attention to minor, non-consequential events too CAP was just rolled out. Data tracking is poor so far. No IT backup now. Concerned that small issues may not be identified.
A Number of programs need to be integrated, starting with CAP and the Observation Program, and it needs to have teeth.
We have a Corrective Action Program and within that we have a Corrective Action Report that we can fill out. Its brand new, a work in progress. It has nothing in it now.
CAP is essentially the observation program with teeth PI.2: Evaluation: The organization thoroughly evaluates problems to ensure that resolutions address causes and extent of conditions, commensurate with their safety significance.
Once identified, each issue should be screened and labeled by a significance level. The significance level determination will provide guidance on actions that must be taken such as extent of condition, root cause, and corrective actions to prevent recurrence or the issue is corrected, closed, and trended.
Issues that are identified can affect NCNR Technical Specification requirements and must be reported and documented promptly. Early screening of the issue for significance will guide the analyst to the level of properly evaluating the issue including performing an extent of condition and root cause analysis. This would also include identifying a nonsignificant issue that was documented, actions taken and closed for trending.
The Team only included comments below when they were corroborated by other interviewees and inputs. They include:
Example Employee Responses:
Comment Building leak rate test did not consistently meet requirements Q. Please give me a brief overview of the Causal Analysis Program at NCNR.?
A: It doesnt exist; its still in development.
No HPI Program Q: Please give me a brief overview of the Causal Analysis Program at NCNR.?
A: It doesnt exist; its still in development. The CAP program is developed, but NIST itself has issues with Root Cause Analysis.
FINAL REPORT JUNE 2, 2023 59
INDEPENDENT THIRD-PARTY NUCLEAR SAFETY CULTURE ASSESSMENT Comment When the NRC reviewed the RCA, they said, Your RCA is flawedyou didnt get to Nuclear Safety Culture. I agree with them that the NSC is a problem.
Q: Where is the pushback with regard to CAP closeout coming from?
A: Primarily from Operations. Some people dont see the value. Randy [Strader, CRO]
sees the value. Operations sees action closeout as an administrative load.
Q: How do you plan to get the NCNR organization involved and actively participate and supportive of Program implementation?
A: That is a very good question. I dont have much in my repertoire to push forward.
PI.3: Resolution: The organization takes effective corrective actions to address issues in a timely manner, commensurate with their safety significance.
Timely initiation and resolution of issues are important aspects of a healthy PI&R program. Employees see that after timely initiation of an issue, the program is being effectively implemented by taking prompt action to resolve the issue.
The Team only included comments below when they were corroborated by other interviewees and inputs. They include:
Example Employee Responses:
Comment CAP was used as a punishment. If you raised a CAP issue, you were assigned to fix it. This was in 2018. The SRTs felt like just more work to do. If we raise things, we get more work to do.
One employee stated that they had recently initiated CAPs but based on the lack of not being processed within the process in a timely manner, the employee said they gave up for initiating CAPs.
PI.4: Trending: The organization periodically analyzes information from the corrective action program and other assessments in the aggregate to identify programmatic and common cause issues.
The importance of effectively trending issues will focus management attention and also identify lower threshold issues to be evaluated before they recur and become more significant issues.
FINAL REPORT JUNE 2, 2023 60
INDEPENDENT THIRD-PARTY NUCLEAR SAFETY CULTURE ASSESSMENT The Team only included comments below when they were corroborated by other interviewees and inputs. They include:
Example Employee Responses:
Comment Q: What performance metrics measure Program performance? Please provide copies.
A: I havent thought about measuring ConOps. [Pause] I would have to look at different things.
I have done performance metrics elsewhere in manufacturing jobs, but we dont do that here.
Q: How do you keep issues from happening again.
A: We should get the first major trend report late Q2 or early Q3 7.4.3 Personal Accountability (PA)
All individuals take personal responsibility for safety.
Safety Culture Common Language (NUREG 2165) and INPO 12-012 identify the following Nuclear Safety Culture attributes of the Trait-Personal Accountability:
PA.1 Standards: Individuals understand the importance of adherence to nuclear standards. All levels of the organization exercise accountability for shortfalls in meeting standards.
PA.2 Job Ownership: Individuals understand and demonstrate personal responsibility for the behaviors and work practices that support nuclear safety.
PA.3 Teamwork: Individuals and workgroups communicate and coordinate their activities within and across organizational boundaries to ensure nuclear safety is maintained.
7.4.3.1 Conclusions Personal Accountability (PA) Rating: Marginally Effective Trend: Flat Accountability is a cross cutting Trait and affects the other nine. The Attributes lay out the expectations for NCNR to adhere to nuclear standards, behaviors and work practices that support nuclear safety, and teamwork by communicating and coordinating their activities within and across organizational boundaries to ensure nuclear safety.
The nuclear standards that NCNR personnel are expected to follow are not always being rolled down into programs and processes (i.e., programs that are required to be effectively documented and implemented have not been developed such as Nuclear Safety Culture, Conduct of Operations, Quality Assurance, Document Control).
FINAL REPORT JUNE 2, 2023 61
INDEPENDENT THIRD-PARTY NUCLEAR SAFETY CULTURE ASSESSMENT Limiting Weakness(es):
Organizational silos exist that weaken teamwork.
Single-point accountability for Nuclear Safety Culture decisions has not been clearly established.
Collaborative ownership of cross-cutting/distributive programs such as training, corrective action, and conduct of operations is not occurring at the department head level.
7.4.3.2 Basis, Discussion, and Observations 7.4.3.2.1 Nuclear Safety Culture Survey Of the ten lowest scored questions on the employee survey, five dealt with accountability. Survey average score was 4.2.
Leadership Question Score Attribute Q44 Leaders assign single-point accountability for nuclear 3.25 PA.1 safety decisions.
Q49. Plant activities are governed by comprehensive 3.40 PA.2 high-quality programs, processes, and procedures Q70. We have processes to identify and resolve existing 3.44 PA.3 organizational weaknesses.
Q55. Plant activities are governed by comprehensive 3.47 PA.2 high-quality programs, Q54. Work is effectively planned and executed by 3.63 PA.3 incorporating risk-informed insights.
The assessment team reviewed the PI&R process AR 7.2, survey results, interviews, and CAP data provided by NCNR. Personnel at all organizational levels must be held accountable for standards and expectations. Accountability must be demonstrated both by recognizing excellent performance as well as identifying less-than-adequate performance. Accountability considers behaviors that may contribute to undesirable outcomes.
The Team reviewed an accountability document from the NCNR Director titled- Safety Culture and NCNR leadership Accountability/Accountability of leadership for safety culture. This document was sent to the Acting NIST Director and then submitted to the NRC on October 15, 2021.
FINAL REPORT JUNE 2, 2023 62
INDEPENDENT THIRD-PARTY NUCLEAR SAFETY CULTURE ASSESSMENT This document described the following: Accountability for these root causes and the responsibility for implementing measures to address them resides with the licensee -
the NCNR Director - and with the leadership within the Reactor Operations &
Engineering (ROE) Group, including the Chief of Reactor Operations & Engineering (CROE), Chief of Reactor Operations (CRO) and Chief of Reactor Engineering (CRE).
This accountability follows from the licensees responsibility for the safety of the public and environment. Within the NIST organization, the accountability follows the NIST management chain beginning with the NIST Director.
The NIST Director holds the NCNR Director accountable for effective leadership of the NCNR, including the safe operations of the reactor.2 The NCNR Director holds the CROE accountable for effective leadership of the ROE group and the safe operations of the reactor. Accountability flows from the CROE to the CRO and CRE and then to the reactor crew chiefs and supervisors.
On December 7, 2022, the NCNR Director issued an update titled, 2022 Update on Status of Safety Improvements in Reactor Operations and Engineering. This report discussed accountability and the Nuclear Safety Culture Improvement Program.
The Assessment Team also identified a Power Point that provided actions, however, the Team could not identify that the actions were taken or being tracked.
7.4.3.2.2 Employee and Group Interview Responses A representative sample of interview responses is provided below, arranged by Attributes of the PA Trait:
PA.1 Standards: Individuals understand the importance of adherence to nuclear standards. All levels of the organization exercise accountability for shortfalls in meeting standards.
NCNR has operated for many years without having documented programs and procedures to operate a nuclear facility. Personnel understand they are required to follow programs and processes; however, when they are not documented, NCNR does not have to follow them.
The Team only included comments below when they were corroborated by other interviewees and inputs. They include:
Example Employee Responses:
Comment Q: Do you have anything documenting your plans? Even personal hand-written notes? [Personal Accountability includes documenting plans.]
A: No Q: What performance metrics measure Program performance? Please provide copies.
A: I would have to look at different things FINAL REPORT JUNE 2, 2023 63
INDEPENDENT THIRD-PARTY NUCLEAR SAFETY CULTURE ASSESSMENT Comment Q: How do you plan to get the NCNR organization involved and actively participating and supportive of Program implementation?
A: Use the same methods using the Observation Program, and the rest of the organization falls under AR-5.0 as well Q: Performance metrics in the Commercial Nuclear Generation world are almost a religion, with top-level quarterly reviews in front of corporate executives. Do you have anything like that here?
A: No, not even remotely. We have done performance metrics elsewhere in manufacturing jobs, but we dont do that here.
Q: What management observations, self-assessments, QA assessments, or independent assessments have evaluated your Program? Please provide copies of each.
A: Other than the Observation Program, we have no assessments. We rely on you guys and other external organizations. We dont have people who can do assessments. There is an SEC audit that occurs, but there are no assessments PA.2 Job Ownership: Individuals understand and demonstrate personal responsibility for the behaviors and work practices that support nuclear safety.
NCNR does not understand Nuclear Safety Culture behaviors. Without having required programs in place to operate the facility, personnel operate the facility how they were instructed by others on how to perform.
Without having a detailed Training program, personnel were self-taught. There is no question NCNR has job ownership as indicated in the Survey and interviews, but these behaviors and work practices do not support Nuclear Safety.
The Team only included comments below when they were corroborated by other interviewees and inputs. They include:
Example Employee Responses:
Comment Ive had situations where I was watching someone doing a valve lineup and start to turn a valve the wrong way. I said, Stop! Dont do that.
Q. Who owns the Corrective Action Program?
A. CHARM QA oversees CAP Q What interface does your oversight have with other programs?
A: Were trying to do that through CAP FINAL REPORT JUNE 2, 2023 64
INDEPENDENT THIRD-PARTY NUCLEAR SAFETY CULTURE ASSESSMENT Comment Q Youve told us about external audits and surveillance. Do you do any internally performed audits and surveillances?
A: That would be QA Q: What management observations, self-assessments, QA assessments, or independent assessments have evaluated your Program? Please provide copies of each.
A: Not many - we do need a comprehensive 3rd party assessment Q: Where in the decision process for pen & ink changes does the 50.59 decision get made?
A: If there is a change in scope that affects a Safety System, we go to 50.59.
Q: Who makes that call?
A: The SRO can make it, but usually they will talk to me, and I make the call. I have not delegated that authority.
Q: Do you have a Confined Space program?
A: Yes, but it needs to be upgraded.
PA.3 Teamwork: Individuals and workgroups communicate and coordinate their activities within and across organizational boundaries to ensure nuclear safety is maintained.
Having long standing silos (as identified in interviews) is a leadership issue. Teamwork needs to come from clear expectations from the Leadership Team. Individuals and workgroups understand that they must work together, however clear teamwork expectations have not been communicated and enforced to perform as a team. Just talking about teamwork does not, by itself, address the issue.
The Team only included comments below when they were corroborated by other interviewees and inputs. They include:
Example Employee Responses:
Comment Historical behaviors run the place, and the place is siloed.
Siloing has to be addressed We still have stove-piping between Engineering and Operations.
The relationship between Ops-Engineering is not optimal We need Leadership Training amongst ourselves to get onto common ground and stop these pissing contests FINAL REPORT JUNE 2, 2023 65
INDEPENDENT THIRD-PARTY NUCLEAR SAFETY CULTURE ASSESSMENT Comment We have interdisciplinary teams now that work together to solve issues, and they have a lot of discussion. Before, wed write something down and pass it around. Often now, we talk about it before it becomes a major issue.
It was very hard to stand up SRT because Ops dug in and said they dont have time/resources.
Its easy to stovepipe here, thats been an issue forever. As an example, we had helium build up this week, and Operations didnt talk to Engineering.
Q: If you have a procedure that affects Engineering or HP, how do you train them?
A: That is a gap we have to work on. Other groups that are affected are on the list to review the procedure before its approved, but that is all. We dont have any training for other organizations set up.
7.4.4 Work Processes (WP)
The process of planning and controlling work activities is implemented so that safety is maintained.
Safety Culture Common Language (NUREG 2165) and INPO 12-012 identify the following Nuclear Safety Culture attributes of the Trait-Work Processes.
WP.1 Work Management: The organization implements a process of planning, controlling, and executing work activities such that nuclear safety is the overriding priority. The work process includes the identification and management of risk commensurate to the work.
WP.2 Design Margins: The organization operates and maintains equipment within design margins. Margins are carefully guarded and changed only through a systematic and rigorous process. Special attention is placed on maintaining fission product barriers, defense-in-depth, and safety-related equipment.
WP.3 Documentation: The organization creates and maintains complete, accurate and up-to-date documentation.
WP.4 Procedure Adherence: Individuals follow processes, procedures, and work instructions.
7.4.4.1 Conclusions Work Processes (WP) Rating: Marginally Effective Trend: Positive Prior to the February 2021 event NCNR-ROE was operated by experienced operators who were able to operate the reactor without detailed procedures. As the experienced operators were replaced by less experienced ones, there was a need for procedures that were more prescriptive. Following the event, NCNR leadership recognized this need and embarked on an aggressive remediation plan. However, the plan lacked structure and many new procedures, developed by operators, were overly complex and cumbersome.
FINAL REPORT JUNE 2, 2023 66
INDEPENDENT THIRD-PARTY NUCLEAR SAFETY CULTURE ASSESSMENT Limiting Weakness(es):
Procedures coming out of the current upgrade program are often overly detailed, complex, cumbersome, and have not been effectively verified and validated.
Work evolutions are not consistently planned to consider contingencies and engaging supporting work groups.
Procedure development is currently being done by operators as a collateral duty without benefit of technical writers.
An Emerging Issue Management process has not been established.
7.4.4.2 Basis, Discussion, and Observations 7.4.4.2.1 Nuclear Safety Culture Survey Work Processes Question Score Attribute 3.40 WP.3 Q49. Plant activities are governed by comprehensive high-quality programs, processes, and procedures.
4.16 WP.2 Q52. Design and operating margins are carefully guarded and changed as defined by procedures.
4.24 WP.4 Q43. I apply a rigorous approach to problem solving in accordance with procedures.
4.33 WP.2 Q51. The organization conducts activities that could affect reactivity with caution, in accordance with procedures.
WP.4 Q38. I follow processes, procedures, and work instructions. 4.67 Some policies may actually be overly restrictive (some WP.4 Survey aspects of the procedural updates compensating for comment weaknesses in the training program, for example)
The incredibly heavy focus on procedures that "anyone can WP.3 perform" tells me again that management is not concerned Survey about maintaining experienced people, only minimally trained comment individuals The quantity of procedures and administrative rules has multiplied 3-4x. Information which was contained in just a few WP.3 pages is now fragmented across several documents. This Survey makes finding the needed info and following the procedure per comment AR 5 rules excessively time consuming, cumbersome, and often impossible.
WP.3 Others question the validity of the procedures. Survey comment Employee surveys are helpful to identify potential strengths and weaknesses in Nuclear Safety Culture to be investigated in subsequent interviews and observations. The survey results in the area of Work Processes were generally positive, however, FINAL REPORT JUNE 2, 2023 67
INDEPENDENT THIRD-PARTY NUCLEAR SAFETY CULTURE ASSESSMENT Question 49 was the lowest scored question in the survey. The other four questions scored above the survey average of 4.10. A number of those interviewed selected a 5, the highest number.
7.4.4.2.2 Employee and Group Interview Responses A representative sample of interview responses is provided below, arranged by Attributes of the WP Trait, with Attributes arranged in descending order of significance to the NCNR Nuclear Safety Culture.
WP.3 Documentation: The organization creates and maintains complete, accurate and up-to-date documentation.
The Team only included comments below when they were corroborated by other interviewees and inputs. They include:
Example Employee Responses:
Comment We revised Administrative Rule (AR) 5.0 on Procedure Use and Adherence We now have much better references in our procedures, programs, and policies and in our conduct of operations.
NCNR-ROE had about 550 active legacy procedures and documents prior to the accident, all of which need to get reviewed, checked against CONOPS, and probably revised. The commitment date is January 2025, and they have no plan to get it done, and no schedule.
I think people want to get to where we need to be, and they see the opportunity to make the shiftbut we dont have the processes that we need, and the changes wont stay without that.
Operations has about 169 procedures of approximately 600 NCNR procedures. Per the Confirmatory Order, all procedures need to be upgraded by 2025.
There is no consistency in the quality of the procedure reviews. Fidelity is off - we identify issues after procedure is approved.
We are governed by procedures, but not high-quality procedures. The procedures are not cross tied.
There is no flow down of requirements Procedure review process is not formal. Pen and ink changes approved by SRO and Im not sure if they use the 10CFR 50.59 process Some drawings are not as shown, and configuration management is a known issue.
After the event we went overboard with the procedures (making them cumbersome).
We should have spent much more time on Training. Some procedures are overly prescriptive.
FINAL REPORT JUNE 2, 2023 68
INDEPENDENT THIRD-PARTY NUCLEAR SAFETY CULTURE ASSESSMENT WP.2 Design Margins: The organization operates and maintains equipment within design margins. Margins are carefully guarded and changed only through a systematic and rigorous process. Special attention is placed on maintaining fission product barriers, defense-in-depth, and safety-related equipment.
The Team only included comments below when they were corroborated by other interviewees and inputs. They include:
Example Employee Responses:
Comment We delayed a lot of maintenance while we were shut down Q-Are you aware of any examples where technical specification requirements are not met on initial attempts, but are eventually met after several attempts?
A- Yes. One example is the Building Leak rate Test, where we have run the testing until we get a value that passes.
WP.4 Procedure Adherence: Individuals follow processes, procedures, and work instructions.
The Team only included comments below when they were corroborated by other interviewees and inputs. They include:
Example Employee Responses:
Comment Procedure use prior to the event was mostly informal. It has increased by an order of magnitude since but changing procedures is cumbersome Latching and unlatching shim arms is performed outside the processes. Need to manually increase power to latch and then reduce power after latching. A work around.
NRC did not look at Human Performance.
WP.1 Work Management: The organization implements a process of planning, controlling, and executing work activities such that nuclear safety is the overriding priority. The work process includes the identification and management of risk commensurate to the work.
Interview and survey responses on this Attribute were all positive.
The Team only included comments below when they were corroborated by other interviewees and inputs. They include:
FINAL REPORT JUNE 2, 2023 69
INDEPENDENT THIRD-PARTY NUCLEAR SAFETY CULTURE ASSESSMENT Example Employee Responses:
Comment 20 years ago in a different era, what we would do is wave our hands and take more risks. That changed. The problem is that some people will say that were having more problems because we are operating more conservatively, which is not true.
But we have to deal with [that perception].
Where in the decision process for pen & ink changes does the 50.59 decision get made? A: If there is a change in scope that affects a Safety System, we go to 50.59.
7.4.5 Continuous Learning (CL)
Opportunities to learn about ways to ensure safety are sought out and implemented.
Safety Culture Common Language (NUREG 2165) and INPO 12-012 identify the following Nuclear Safety Culture attributes of the Trait-Continuous Learning (CL)
CL.1 Operating Experience: The organization systematically and effectively collects, evaluates, and implements relevant internal and external operating experience in a timely manner.
CL.2 Self-Assessment: The organization routinely conducts self-critical and objective assessments of its programs and practices.
CL.3 Benchmarking: The organization learns from other organizations to continuously improve knowledge, skills, and safety performance.
CL.4 Training: The organization provides training and ensures knowledge transfer to maintain a knowledgeable, technically competent workforce and instill nuclear safety values.
7.4.5.1 Conclusions Continuous Learning (CL) Rating: Marginally Effective Trend: Positive Continuous improvement in Nuclear Safety Culture requires a vibrant program of learning from a variety of sources. The first three attributes of this trait are key elements of a corrective action program, part of the Problem Identification and Resolution Trait (PI).
Prior to the February 2021 event most operators were previously experienced in the nuclear industry and were able to progress with much of their training self-taught. As these workers left due to retirement and other reasons, newer workers, especially operators did not have comparable experience and required formal training.
This need has been recognized by NCNR leadership, but progress has been slow and actions to develop the necessary changes have not made a significant improvement. At FINAL REPORT JUNE 2, 2023 70
INDEPENDENT THIRD-PARTY NUCLEAR SAFETY CULTURE ASSESSMENT the time of the on-site assessment the training manager was acting and could only spend about half of his time in Training.
Limiting Weakness(es):
The Operator Continuing Training program is under development.
The programs for managing Operating Experience, Benchmarking, and Self-Assessments have not been developed.
Knowledge transfer is largely based upon tribal knowledge.
The organization does not conduct critical self-assessments.
7.4.5.2 Basis, Discussion, and Observations 7.4.5.2.1 Nuclear Safety Culture Survey Continuous Learning Question Score Attribute 3.71 CL.4 Q41. Leaders provide training and knowledge transfer to establish and maintain technical competence.
3.71 CL.4 Q71. Leaders foster an environment in which individuals value and seek continuous learning opportunities.
4.16 CL.1 Q67. Leaders evaluate serious events and implement actions to learn from the experience.
Q75. The organization uses both self-assessments and 4.26 CL.2 independent oversight.
Survey comment:
CL.4 The training program is under development, and improving, but that aspect has historically been lacking.
Survey comment:
CL.4 We still have only one person as the technical trainer for all of ROE.
Employee surveys can be helpful to identify potential strengths and weaknesses in Nuclear Safety Culture to be investigated in subsequent interviews and observations.
The Teams observations in the areas of training and continuous learning were well below the survey average of 4.10. A number of those interviewed selected a 5, the highest number, for each question.
7.4.5.2.2 Employee and Group Interview Responses A representative sample of interview responses is provided below, arranged by Attributes of the CL Trait, with Attributes arranged in descending order of significance to the NCNR Nuclear Safety Culture:
CL.4 Training: The organization provides training and ensures knowledge transfer to maintain a knowledgeable, technically competent workforce and instill nuclear safety values.
FINAL REPORT JUNE 2, 2023 71
INDEPENDENT THIRD-PARTY NUCLEAR SAFETY CULTURE ASSESSMENT The Team only included comments below when they were corroborated by other interviewees and inputs. They include:
Example Employee Responses:
Comment We didnt have much training before [the 02/03/2021 incident]. We did have hearing loss training, and electrical shock hazard training before, and maybe a bit more. But not much. We now have daily safety meetings to get people thinking about safety Q- What are your plans for training? Are you using the SAT process to develop training?
A: Plan to use a systematic approach.
Training was non-existent. Requal training was focused on passing the test. Licensing training emphasized reactor theory and 10 CFR. There was no emphasis on our systems including our reactor safety systems After the event we went overboard with the procedures (making them cumbersome).
We should have spent much more time on Training. Some procedures are overly prescriptive.
We need more time and commitment to the training program. We need more people, and we need the right people, because right now its not working.
We need a leadership needs assessment and leadership development. Leading people is a skill that has to be learned. We have to assess where we need to improve (behaviorally) to be a leader, understand where they need to develop, then coaching and training. Especially soft skills.
Training is a top issue, from day one. The SRO continuing training amounted to a one-or two-week biannual cram session for re-licensing. It also took two years from the time the position went vacant for the Training Manager position to get posted.
CL.1 Operating Experience: The organization systematically and effectively collects, evaluates, and implements relevant internal and external operating experience in a timely manner.
The Team only included comments below when they were corroborated by other interviewees and inputs. They include:
Example Employee Responses:
Comment Q-How has the Nuclear Safety Culture changed since the February 3, 2021, event?
A-Previously knowledge was passed down through OJT. Need to learn from the past.
Previous similarities on latching issues not captured and shared.
Feb 4 [2021] we almost killed someone with CO2, and no one wants to talk about it, and other potential hazards. Its a systemic problem, NIST-wide, not attending to near misses or precursors, or extent of conditions.
FINAL REPORT JUNE 2, 2023 72
INDEPENDENT THIRD-PARTY NUCLEAR SAFETY CULTURE ASSESSMENT CL.2 Self-Assessment: The organization routinely conducts self-critical and objective assessments of its programs and practices.
The Team only included comments below when they were corroborated by other interviewees and inputs. They include:
Example Employee Responses:
Comment Q-Why do you think a number of people answered predominantly 5s or predominantly 1s on the safety culture survey?
A: They are surveyed out. You are the 4th group that has come in to survey and interview us. Its like a broken record.
Q: Why do you think a number of people answered all 5s on the safety culture survey?
A: Prima facie evidence they are extremely passive-aggressive; it is shameful.
CL.3 Benchmarking: The organization learns from other organizations to continuously improve knowledge, skills, and safety performance.
The Team only included comments below when they were corroborated by other interviewees and inputs. They include:
Example Employee Responses:
Comment Tom Newton supported benchmarking [another research reactors] training program.
Need to benchmark procedures and initiate a formal procedure writing process for consistency Benchmarking is perceived as a political hot potato.
7.4.6 Environment for Raising Concerns (RC)
A safety-conscious work environment (SCWE) is maintained where personnel feel free to raise safety concerns without fear of retaliation, intimidation, harassment, or discrimination.
Safety Culture Common Language (NUREG 2165) and INPO 12-012 identify the following Nuclear Safety Culture attributes of the Trait-Environment for Raising Concerns (RC)
RC.1 Safety-Conscious Work Environment Policy: The organization effectively implements a policy that supports individuals rights and responsibilities to FINAL REPORT JUNE 2, 2023 73
INDEPENDENT THIRD-PARTY NUCLEAR SAFETY CULTURE ASSESSMENT raise safety concerns, and does not tolerate harassment, intimidation, retaliation, or discrimination for doing so.
RC.2 Alternate Process for Raising Concerns: The organization effectively implements a process for raising and resolving concerns that is independent of line-management influence. Safety issues may be raised in confidence and are resolved in a timely and effective manner.
7.4.6.1 Conclusions Environment for Raising Concerns (RC) Rating: Marginally Effective Trend: Positive NCNR-ROE does not have a functioning employee concerns program (ECP). Actions are underway to implement and staff the program. Most of the safety conscious work environment (SCWE) survey and interview responses were positive. While the survey and interview responses were positive, the lack of an ECP leads the Team to evaluate this Trait as Marginally Effective.
Limiting Weakness(es):
The Alternate Process for Raising Concerns (RC.2), the Employee Concerns program has been submitted to the NRC for review but has not been implemented.
Some employees expressed reluctance to bring up concerns related to safety because they dont think that management would do anything about them.
A formal Differing Professional Opinion process does not exist.
7.4.6.2 Basis, Discussion, and Observations 7.4.6.2.1 Nuclear Safety Culture Survey Environment for Raising Question Score Concerns Attribute RC.1 Q31. Differing opinions are welcomed and respected. 3.96 RC.2 Q86. Claims of harassment, intimidation, retaliation, and 4.05 discrimination are investigated.
RC.1 Q18. Employees have input into resolving issues. 4.21 RC.1 Q32. Supervisors respond to questions and concerns. 4.27 Q84. Leaders take ownership when receiving and RC.1 responding to concerns, while respecting confidentiality as 4.29 appropriate.
FINAL REPORT JUNE 2, 2023 74
INDEPENDENT THIRD-PARTY NUCLEAR SAFETY CULTURE ASSESSMENT Environment for Raising Question Score Concerns Attribute Q85. Leaders create an environment in which individuals RC.1 4.37 feel free to raise concerns.
Q29. I freely raise nuclear safety concerns without fear of RC.1 4.62 retribution.
Q61. I stop when I identify unexpected or uncertain RC.1 4.63 conditions.
Employee surveys are helpful to identify potential strengths and weaknesses in Nuclear Safety Culture to be investigated in subsequent interviews and observations. Most of the survey results in the area of Environment for Raising Concerns scored above the survey average of 4.10. A number of those interviewed selected a 5, the highest number, for each question.
7.4.6.2.2 Employee and Group Interview Responses A representative sample of interview responses is provided below, arranged by Attributes of the RC Trait, with Attributes arranged in descending order of significance to the NCNR Nuclear Safety Culture:
RC.2 Alternate Process for Raising Concerns: The organization effectively implements a process for raising and resolving concerns that is independent of line-management influence. Safety issues may be raised in confidence and are resolved in a timely and effective manner.
The Team only included comments below when they were corroborated by other interviewees and inputs. They include:
Example Employee Responses:
Comment We finalized the program and submitted it to the NRC for review and comment. Weve hired a Program Manager specifically to lead ECP here at NCNR.
I wonder why something like ECP hasnt been institutionalized at a higher level than NCNR?
Q: Please give me a brief overview of the Employee Concerns Program.?
A: Its a new program for us. The NCNR Director initiated it as a result of the Confirmatory Order. Im not really sure why its in the Confirmatory Order.
FINAL REPORT JUNE 2, 2023 75
INDEPENDENT THIRD-PARTY NUCLEAR SAFETY CULTURE ASSESSMENT RC.1 Safety-Conscious Work Environment Policy: The organization effectively implements a policy that supports individuals rights and responsibilities to raise safety concerns, and does not tolerate harassment, intimidation, retaliation, or discrimination for doing so.
The Team only included comments below when they were corroborated by other interviewees and inputs. They include:
Example Employee Responses:
Comment We have an admin rule that says everyone has Stop Work authority if they feel something is unsafe or incorrect Q-Do managers respond without defensiveness to questions, problems, issues and concerns?
A-They are not defensive now.
Q7 What examples can you provide of managers/supervisors encouraging identification and reporting of nuclear safety issues or concerns?
A: Happens every day in Engineering.
Management has no problem with us bringing up issues Q: What are some examples of you stopping work when you identified unexpected or uncertain conditions?
A: I know I stopped work several times last year, but I cant remember any specific examples. [Pause] I stopped work when they found a hot particle on the floor of the process room.
Our staff was reluctant to bring up concerns related to safety because they dont think that we would do anything about them.
7.4.7 Effective Safety Communication (CO)
Communications maintain a focus on safety.
Safety Culture Common Language (NUREG 2165) and INPO 12-012 identify the following Nuclear Safety Culture attributes of the Trait-Effective Safety Communication.
CO.1 Work Process Communications: Individuals incorporate safety communications in work activities.
CO.2 Basis for Decisions: Leaders ensure that the basis for operational and organizational decisions is communicated in a timely manner.
CO.3 Free Flow of Information: Individuals communicate openly and candidly, both up, down, and across the organization, and with oversight, audit, and regulatory organizations.
FINAL REPORT JUNE 2, 2023 76
INDEPENDENT THIRD-PARTY NUCLEAR SAFETY CULTURE ASSESSMENT CO.4 Expectations: Leaders frequently communicate and reinforce the expectation that nuclear safety is the organizations overriding priority.
7.4.7.1 Conclusions Effective Safety Communications (CO) Rating: Marginally Effective Trend: Positive Leadership communications since the event have properly focused on the importance of NCNR-ROE having a healthy safety culture. The survey results and interviews indicate that progress is being made but more work remains. It will be very important to not only communicate with words but with actions.
Limiting Weakness(es):
Leadership communication about the importance of nuclear safety sometimes gets confused with the importance of schedule.
Communication of operational and organizational decisions is inconsistent.
Ineffective communications between different departments impacts Nuclear Safety Culture.
Important organizational issues are not consistently bubbled up to the director level (e.g., potential toxic work environment not communicated to NCNR Director).
The NCNR-ROE organization is siloed.
7.4.7.2 Basis, Discussion, and Observations 7.4.7.2.1 Nuclear Safety Culture Survey Safety Communications Question Score Attribute Q21. Leaders at all levels ensure that the basis CO.2 for operational and organizational decisions is 3.58 communicated to staff in a timely manner.
Q36. Leaders enhance trust and nuclear safety CO.3 4.04 through communications.
Q72. Leadership enforces management CO.4 standards and expectations that reflect nuclear 4.08 safety values.
Q66. I share information from a wide variety of CO.3 4.00 sources with other site groups.
Q78. We value insights provided by operational CO.3 4.04 support or oversight groups.
FINAL REPORT JUNE 2, 2023 77
INDEPENDENT THIRD-PARTY NUCLEAR SAFETY CULTURE ASSESSMENT Survey comment CO.3 Communication between Engineering, Ops., and HP has room for improvement 7.4.7.2.2 Employee and Group Interview Responses A representative sample of interview responses is provided below, arranged by Attributes of the CO Trait, with Attributes arranged in descending order of significance to the NCNR Nuclear Safety Culture:
CO.4 Expectations: Leaders frequently communicate and reinforce the expectation that nuclear safety is the organizations overriding priority.
The Team only included comments below when they were corroborated by other interviewees and inputs. They include:
Example Employee Responses:
Comment Im a little burned out on Safety culture stuff; we have been beat over the head over and over, theres a potential of getting desensitized.
Leadership is betting that this event is just Nuclear Safety Culture, and the causes are fixed. They continue to push to learn why things are wrong, but there is some pressure to just get it done.
How has the Nuclear Safety Culture changed at NCNR overall since the February 3, 2021, event?
A: We have more lectures and meetings on safety which seems a little overboard.
CO.2 Basis for Decisions: Leaders ensure that the basis for operational and organizational decisions is communicated in a timely manner.
The Team only included comments below when they were corroborated by other interviewees and inputs. They include:
Example Employee Responses:
Comment Need improvements in organizational communication.
The root cause of coordination issues is communications issues.
FINAL REPORT JUNE 2, 2023 78
INDEPENDENT THIRD-PARTY NUCLEAR SAFETY CULTURE ASSESSMENT CO.3 Free Flow of Information: Individuals communicate openly and candidly, both up, down, and across the organization, and with oversight, audit, and regulatory organizations.
The Team only included comments below when they were corroborated by other interviewees and inputs. They include:
Example Employee Responses:
Comment Since the event, Engineering and Operations are talking more.
NRC-site communications are fantastic now. Not an ounce of secrecy, nothing is held back.
Q: How have communications changed?
A: Like night and day. Prior to the event, the ECN process pushed things to Operations, and the communications with Operations were vague. Now it is much more collaborative.
Cross-organizational communications have increased, and people seem willing to say why what theyre suggesting is better.
Communications between different departments impacts Nuclear Safety Culture.
One of the top issues impacting Nuclear Safety Culture is communications between different departments Q: How is the relationship/ communications with Operations?
A: Communication has not been good for some time.
CO.1 Work Process Communications: Individuals incorporate safety communications in work activities.
The Team only included comments below when they were corroborated by other interviewees and inputs. They include:
Example Employee Responses:
Comment After the event, we have daily meetings between Operations and Engineering. Safety culture has improved markedly.
Q: What examples can you provide of managers/supervisors encouraging identification and reporting of nuclear safety issues or concerns?
A: Happens every day in Engineering.
Management Talks the Talk but does not Walk the Walk.
FINAL REPORT JUNE 2, 2023 79
INDEPENDENT THIRD-PARTY NUCLEAR SAFETY CULTURE ASSESSMENT 7.4.8 Respectful Work Environment (WE)
Trust and respect permeate the organization.
Safety Culture Common Language (NUREG 2165) and INPO 12-012 identifies the following Nuclear Safety Culture attributes of the Trait-Respectful Work Environment.
WE.1 Respect is Evident: Everyone is treated with dignity and respect.
WE.2 Opinions are Valued: Individuals are encouraged to voice concerns, provide suggestions, and offer questions. Differing opinions are respected.
WE.3 High Level of Trust: Trust is fostered among individuals and workgroups throughout the organization.
WE.4 Conflict Resolution: Fair and objective methods are used to resolve conflict.
7.4.8.1 Conclusions Respectful Work Environment (WE) Rating: Marginally Effective Trend: Positive A healthy Nuclear Safety Culture requires teamwork based on mutual respect. Allowing a potentially toxic work environment must be addressed on a priority basis.
Limiting Weakness(es):
A potential toxic work environment between Operations and I&C.
Leaders at times implement change in a way that does not build organizational trust.
There are elements of the organization that have no trust in other parts of the organization.
Differing opinions are inconsistently respected and considered.
7.4.8.2 Basis, Discussion, and Observations 7.4.8.2.1 Nuclear Safety Culture Survey PROCEDURES QUESTION SCORE ATTRIBUTE Q33. Leaders implement change in a way that builds 3.56 WE.3 organizational trust.
Q24. Leaders 'walk the talk modeling behaviors when 3.84 WE.4 resolving conflicts between nuclear safety and production.
WE.2 Q31. Differing opinions are welcomed and respected. 3.96 Q36. Leaders enhance trust and nuclear safety through 4.04 WE.3 communications.
WE.1 Q28. I treat other employees with dignity and respect. 4.65 Employee surveys are helpful to identify potential strengths and weaknesses in Nuclear Safety Culture to be investigated in subsequent interviews and observations. 4 of the 5 FINAL REPORT JUNE 2, 2023 80
INDEPENDENT THIRD-PARTY NUCLEAR SAFETY CULTURE ASSESSMENT survey questions for Trust and Respect scored below the survey average score of 4.10.
A number of those interviewed selected a 5, the highest number, for each question.
7.4.8.2.2 Employee and Group Interview Responses A representative sample of interview responses is provided below, arranged by Attributes of the WE Trait, with Attributes arranged in descending order of significance to the NCNR Nuclear Safety Culture.
WE.1 Respect is Evident: Everyone is treated with dignity and respect.
The Team only included comments below when they were corroborated by other interviewees and inputs. They include:
Example Employee Responses:
Comment I&C staff are picking shifts on which to do work to avoid the toxic work environment.
There have been screaming matches.
During individual and focus group interviews, multiple individuals informed the assessors that a toxic work environment between Operations and I & C may exist.
WE.3 High Level of Trust: Trust is fostered among individuals and workgroups throughout the organization.
The Team only included comments below when they were corroborated by other interviewees and inputs. They include:
Example Employee Responses:
Comment The weakness in trust very much concerns me. Ive heard that there are elements of the organizations that have no trust in other parts of the organization.
There is a higher opinion of self vs. groups they work with; this points to a trust issue.
I trust myself, but not so much others.
WE.4 Conflict Resolution: Fair and objective methods are used to resolve conflict.
The Team only included comments below when they were corroborated by other interviewees and inputs. They include:
Example Employee Responses:
Comment We need better processes and plans of how to resolve problems.
FINAL REPORT JUNE 2, 2023 81
INDEPENDENT THIRD-PARTY NUCLEAR SAFETY CULTURE ASSESSMENT WE.2 Opinions are Valued: Individuals are encouraged to voice concerns, provide suggestions, and offer questions. Differing opinions are respected.
The Team only included comments below when they were corroborated by other interviewees and inputs. They include:
Example Employee Responses:
Comment Our discussions are more thought out, deliberate, and systematic. Now using a group approach to safety issues. Modeled after INPO Nuclear Safety Culture 7.4.9 Questioning Attitude (QA)
Individuals avoid complacency and continuously challenge existing conditions and activities in order to identify discrepancies that might result in error or inappropriate action.
Safety Culture Common Language (NUREG 2165 / /INPO 12-012) identifies the following Nuclear Safety Culture attributes of the Trait Questioning Attitude (QA)
QA.1 Nuclear Is Recognized as Special and Unique: Individuals understand that complex technologies can fail in unpredictable ways.
QA.2 Challenge the Unknown: Individuals stop when faced with uncertain conditions. Risks are evaluated and managed before proceeding.
QA.3 Challenge Assumptions: Individuals challenge assumptions and offer opposing views when they think something is not correct.
QA.4 Avoid Complacency: Individuals recognize and plan for the possibility of mistakes, latent problems, or inherent risk, even while expecting successful outcomes.
7.4.9.1 Conclusions Questioning Attitude (QA) Rating: Effective Trend: Positive NCNR personnel have not exhibited behaviors that challenge existing conditions and when they do, they are sometimes criticized by leadership. Discrepancies that might result in errors and inappropriate actions have been observed.
Limiting Weakness(es):
Risks are not always evaluated and managed before proceeding on infrequently performed tasks and emergent issues.
Some employees stated that they are not always listened to when challenging some assumptions.
There is an apparent resistance to change or slow acceptance for the need to change regarding key Nuclear Safety Culture traits.
FINAL REPORT JUNE 2, 2023 82
INDEPENDENT THIRD-PARTY NUCLEAR SAFETY CULTURE ASSESSMENT 7.4.9.2 Basis, Discussion, and Observations 7.4.9.2.1 Nuclear Safety Culture Survey Of the ten lowest scored questions on the employee survey, five dealt with questioning attitude. Survey average score was 4.2.
QUESTIONING ATTITUDE QUESTION SCORE ATTRIBUTE Q44 Leaders assign single-point accountability for nuclear 3.25 QA.1 safety decisions.
Q49. Plant activities are governed by comprehensive high- 3.40 QA.4 quality programs, processes, and procedures Q54. Work is effectively planned and executed by incorporating 3.63 QA.2 risk-informed insights Q55. Plant activities are governed by comprehensive high- 3.56 QA.3 quality programs, processes, and procedures.
Q21. Leaders at all levels ensure that the basis for operational 3.58 QA.1 and organizational decisions is communicated to staff in a timely manner.
Questioning attitude is an excellent Trait for preventing issues before they can occur.
When personnel question the team and those questions are listened to and evaluated, the decision-making process is made stronger. When employees are listened to, they feel part of the team. Questioning attitude should be encouraged and implemented as a prevention tool for site activities.
Interviews identified that teamwork is a challenge at NCNR. Long standing silos have been established and breaking through those silos will not be accomplished in a short period of time.
Avoiding complacency to develop a continually improving culture is a very difficult task.
A complacent culture appears to have been institutionalized for many years at NCNR.
Numerous issues identified in an International Atomic Energy Agency (IAEA) report existed prior to the February 2021 Fuel event and continue to exist today.
In 2012, NCNR did not have a formal root cause program and this issue continues to exist in 2023.
In 2012, a robust self-assessment program was not in place and the lack of a program continues to not be in place in 2023.
In 2012, procedures were being implemented with handwritten notes with a recommendation to develop a plan to prioritize, track, procedure upgrades. One of the FINAL REPORT JUNE 2, 2023 83
INDEPENDENT THIRD-PARTY NUCLEAR SAFETY CULTURE ASSESSMENT root causes identified in the February 2021 fuel event was the lack of detailed procedures.
In 2012, the team recommended Nuclear Safety Culture 10 traits training. In 2023 Nuclear Safety Culture continues to be an issue while effective training has not been conducted in order for employees to understand the 10 traits.
In 2012, the team identified cleanliness and housekeeping as an issue. In 2023 cleanliness and housekeeping continues to be an issue.
In 2012 the team identified that a documented classroom and field training program was not in place. In 2023, the training issues continue to exist.
In 2012 there was not a PI&R program. In 2023 there continues to be a lack of a PI&R program.
When complacency becomes the normal, when others identify issues for continual improvement, and no actions are taken, issues continue to occur without effective corrective action and a significant event will occur. The NCNR identified in February 2021 is an example of complacent organization that was in place for years.
If an effective root cause was conducted for the IAEA report from 2012, an extent of condition performed, and corrective actions developed to correct the issues from the report, the February 2021 event may not have occurred.
7.4.9.2.2 Employee and Group Interview Responses A representative sample of interview responses is provided below, arranged by Attributes of the QA Trait:
QA.1 Nuclear Is Recognized as Special and Unique: Individuals understand that complex technologies can fail in unpredictable ways.
Nuclear is not being recognized as Special and Unique. Nuclear Safety Culture is seen and discussed as Industrial Safety Culture. Nuclear type programs are seen as an administrative burden and not as nuclear programs required to operate a nuclear facility.
The Team only included comments below when they were corroborated by other interviewees and inputs. They include:
Example Employee Responses:
Comment NIST HR is a significant challenge as they dont seem to appreciate operating a reactor is special and unique.
FINAL REPORT JUNE 2, 2023 84
INDEPENDENT THIRD-PARTY NUCLEAR SAFETY CULTURE ASSESSMENT Comment Q: What management observations, self-assessments, QA assessments, or independent assessments have evaluated your Program? Please provide copies of each.
A: None right now We went back to the RCA and are folding in that information into the emergent training program, however not all done before startup. All staff did go through ADM 4.0 Fuel Manipulation Proficiency Requirements.
Question posed to a Crew Chief: If an EO takes a reading thats not presently on a rounds log sheet should we have a place on the log sheet for such readings?
A: No, itll make the log sheet too long. [Takeaway: We dont want to do that.]
QA.2 Challenge the Unknown: Individuals stop when faced with uncertain conditions. Risks are evaluated and managed before proceeding.
Having a questioning attitude to question the unknown will prevent issues when effective actions are taken. Personnel stated in interviews that they would stop with uncertain conditions, risk are not always evaluated and managed before proceeding.
The Team only included comments below when they were corroborated by other interviewees and inputs. They include:
Example Employee Responses:
Comment Engineering looked into why operations did not take a manual scram and allowed the automatic in the time they started observing power oscillations.
QA.3 Challenge Assumptions: Individuals challenge assumptions and offer opposing views when they think something is not correct.
Some employees stated that they are not always listened to when challenging some assumptions. One individual stated that he challenged management for not signing on to an RWP and was laughed at by the managers. The managers ended up being contaminated for the area they had entered. Leadership must walk the talk when telling employees can challenge assumptions.
The Team only included comments below when they were corroborated by other interviewees and inputs. They include:
Example Employee Responses:
Comment Safety culture has gotten a lot better in the past 2 years since the incident: as seen in people being comfortable speaking up, questioning.
FINAL REPORT JUNE 2, 2023 85
INDEPENDENT THIRD-PARTY NUCLEAR SAFETY CULTURE ASSESSMENT Comment One HP technician stated that when they called the control room to inform them that external gates had been left open. The operator told the HP technician that they were no longer in operations to forget about it.
Q: What are the challenges you have in your responsibilities to stand up multiple programs that are part of the NRC Confirmatory Action letter and recovery plans?
A: OPS. The success path will require direct confrontation with Ops. For 2/3 of this their view is we dont have time for this.
NIST HR is a significant challenge as they dont seem to appreciate operating a reactor is special and unique.
Safety culture has gotten a lot better in the past 2 years since the incident: as seen in people being comfortable speaking up, questioning.
QA.4 Avoid Complacency: Individuals recognize and plan for the possibility of mistakes, latent problems, or inherent risk, even while expecting successful outcomes.
The Team only included comments below when they were corroborated by other interviewees and inputs. They include:
Example Employee Responses:
Comment Nuclear Safety Culture has improved. Prior to the 2021 event we had a level of Nuclear Safety Culture, however there was complacency.
I think we just took our eyes off the switch. We became complacent.
7.4.10 Decision making (DM)
Decisions that support or affect nuclear safety are systematic, rigorous, and thorough.
Safety Culture Common Language (NUREG 2165) and INPO 12-012 identify the following Nuclear Safety Culture attributes of the Trait Decision making (DM)
DM.1 Consistent Process: Individuals use a consistent, systematic approach to make decisions. Risk insights are incorporated as appropriate.
DM.2 Conservative Bias: Individuals use decision making practices that emphasize prudent choices over those that are simply allowable. A proposed action is determined to be safe to proceed, rather than unsafe in order to stop.
FINAL REPORT JUNE 2, 2023 86
INDEPENDENT THIRD-PARTY NUCLEAR SAFETY CULTURE ASSESSMENT DM.3 Accountability for Decisions: Single-point accountability is maintained for nuclear safety decisions.
7.4.10.1 Conclusions Decision Making (DM) Rating: Marginally Effective Trend: Positive There seems to be a disconnect between survey data and interview results. The survey data identifies concerns, e.g., Q44 was the lowest scored question in the survey whereas many interview responses were positive. During the onsite assessment period the Team observed decision-making on an emergent issue without all expected functional groups weighing in. The approach did not go as planned, further delaying startup.
Limiting Weakness(es):
Operations staff acceptance of some surveillances that require multiple executions until an acceptable reading is obtained - for years - and not addressing the underlying issues (physical and procedural).
Operational decisions on emerging issues are occasionally made without the input of all expected functional groups.
Lack of single-point accountability for nuclear safety decisions. (This was the lowest scoring question in the survey).
Conservative assumptions are inconsistently engaged when addressing emerging issues or work.
7.4.10.2 Basis, Discussion, and Observations 7.4.10.2.1 Nuclear Safety Culture Survey Decision Making Question Score Attribute Q44. Leaders assign single-point accountability for DM.3 3.25 nuclear safety decisions.
Q21. Leaders at all levels ensure that the basis for DM.1 operational and organizational decisions is 3.58 communicated to staff in a timely manner.
Q56. ROE employees master reactor operations to DM.1 establish a solid foundation for decisions and 3.74 behaviors.
Q63. Leaders apply conservative decision making to DM.2 3.94 mitigate unpredicted failures.
FINAL REPORT JUNE 2, 2023 87
INDEPENDENT THIRD-PARTY NUCLEAR SAFETY CULTURE ASSESSMENT Q47. When previous operational decisions are called DM.1 into question by emerging facts, leaders re-evaluate 4.08 and adjust as needed.
Q46. I use a conservative approach to nuclear safety DM.2 4.49 when making decisions.
Could the 50.59 decision tree benefit from more risk Survey DM.1 informed questions for more complicated tasks? Comment Employee surveys are helpful to identify potential strengths and weaknesses in Nuclear Safety Culture to be investigated in subsequent interviews and observations. 5 of the 6 survey questions for Decision-Making scored below the survey average score of 4.10.
Q44 was the single lowest scored question in the survey. A number of those interviewed selected a 5, the highest number, for each question.
7.4.10.2.2 Employee and Group Interview Responses A representative sample of interview responses is provided below, arranged by Attributes of the DM Trait, with Attributes arranged in descending order of significance to the NCNR Nuclear Safety Culture:
DM.2 Conservative Bias: Individuals use decision making practices that emphasize prudent choices over those that are simply allowable. A proposed action is determined to be safe to proceed, rather than unsafe in order to stop.
The Team only included comments below when they were corroborated by other interviewees and inputs. They include:
Example Employee Responses:
Comment Im working hard to drive decision-making down below my level.
So, I work with them, so they learn how to make the right decision, and I correct them when they dont.
Our focus is on safety. We learn about decisions after the fact.
Engineering looked into why operations did not take a manual scram and allowed the automatic in the time they started observing power oscillations.
Q: To what extent do your peers question assumptions, decisions, and justifications that do not appear to sufficiently consider impacts to nuclear safety?
A: Yes, okay. Some things get overlooked.
FINAL REPORT JUNE 2, 2023 88
INDEPENDENT THIRD-PARTY NUCLEAR SAFETY CULTURE ASSESSMENT DM.3 Accountability for Decisions: Single-point accountability is maintained for nuclear safety decisions.
The Team only included comments below when they were corroborated by other interviewees and inputs. They include:
Example Employee Responses:
Comment Q: To what extent do your peers question assumptions, decisions, and justifications that do not appear to sufficiently consider impacts to nuclear safety?
A: If I made a mistake, I expect to be called out.
We dont hear enough about management decisions.
DM.1 Consistent Process: Individuals use a consistent, systematic approach to make decisions. Risk insights are incorporated as appropriate.
The Team only included comments below when they were corroborated by other interviewees and inputs. They include:
Example Employee Responses:
Comment Q: To what extent do your peers question assumptions, decisions, and justifications that do not appear to sufficiently consider impacts to nuclear safety?
A: We question things in Engineering.
Our alarms are set much lower than before, and we will shut down much sooner if we have another incident.
FINAL REPORT JUNE 2, 2023 89
INDEPENDENT THIRD-PARTY NUCLEAR SAFETY CULTURE ASSESSMENT 7.5 ASSESSMENT RECOMMENDATIONS The Independent Nuclear Culture Assessment Team was commissioned on January 26, 2023 to assess the current status of NCNR-ROEs nuclear safety culture and to make recommendations to address identified NSC weaknesses. The NSC Assessment is one of six assessment tasks that will identify performance issues and provide recommendations.
The NSC Team focused on identifying the top-level actions that:
Would be possible for NCNR-ROE to initiate in the short term; Address the weaknesses currently limiting NCNR-ROE performance; and, Would not, in the aggregate, overwhelm NCNR-ROEs ability to implement.
Regarding the latter point, it is no challenge for an independent group to come into an organization with performance weaknesses and to identify more issues than the organization could possibly deal with over the next several years. To avoid this pitfall and add the most value, the Team identified those relatively few things that, if addressed, will produce measurable performance improvements.
As the Team reviewed nine selected programs related to NSC performance and analyzed NCNR-ROE performance within the ten NSC Traits, it identified a number of Limiting Weaknesses and related issues. The recommendations summarized below show the relevant NSC traits as well as the relevant programs.
The table below crosswalks the recommendations with related assessments of Programs (discussed in Section 7.3) and NSC Traits (discussed in Section 7.4).
RECOMMENDATIONS INPO 12-012 TRAITS PROGRAMS Prioritize the engagement of external resources to assist in developing, delivering, implementing, and coaching sustainable actions.
- 1. Designate Operations and the I & C portion of the Aging Reactor Management section as priority groups and develop communications and team building strategies to better clarify and 7.4.1-LA address the quality of the safety culture environment and implement corrective actions. 7.3.5 ConOps During individual and focus group interviews, multiple individuals informed the assessors that a toxic 7.4.8-WE work environment between Operations and I & C may exist.
FINAL REPORT JUNE 2, 2023 90
INDEPENDENT THIRD-PARTY NUCLEAR SAFETY CULTURE ASSESSMENT NUREG-2165 /
RECOMMENDATIONS INPO 12-012 TRAITS PROGRAMS
- 2. NCNR-ROE to develop and deliver a competence-based understanding of Nuclear Safety Culture.
Develop and implement a formal NCNR-ROE Nuclear Safety Culture Program based upon INPO 12-012 Traits of a Healthy Nuclear Safety Culture / NUREG 2165 to understand Nuclear Safety Culture Overall NSC (including SCWE), Industrial Safety Culture, and the difference. 7.4.1-LA Develop and implement the vision, values, principles, and objectives that define the expected behaviors and attitudes needed to achieve and sustain a high-reliability Nuclear Safety Culture. 7.4.2-PI Enabling objectives need to focus on all NCNR-ROE leadership and staff owning and living nuclear 7.4.3-PA safety culture attributes and role-modeling behaviors to: 7.4.4-WP Build trust, 7.4.5-CL Breakdown silos, and 7.4.7-CO Improve daily communication behaviors.
Develop and implement NSC Training, and socialize expected NSC behaviors, teamwork and 7.4.8-WE interactions amongst departments and between people. 7.4.9-QA Engage external resources to assist in developing, delivering, implementing, and coaching sustainable 7.4.10-DM actions.
Interim: Re-implement daily use of the Gray Book INPO 12-012, Traits of a Healthy Nuclear Safety Culture.
- 3. Mitigate the staffing and resource shortage in the Operations Department:
Obtain and prioritize external resources (including funding) who can take on the Operations Department administrative workload (see LA.1, Section 7.4.1.2.2, last table entry).
7.4.1-LA 7.3.5 ConOps Prioritize the engagement of external resources to coach Operations Crew Chiefs, SROs, and non-licensed operators regarding shift Operational Focus.
Complete the planned selection and appointment of a Deputy Chief Reactor Operations.
FINAL REPORT JUNE 2, 2023 91
INDEPENDENT THIRD-PARTY NUCLEAR SAFETY CULTURE ASSESSMENT NUREG-2165 /
RECOMMENDATIONS INPO 12-012 TRAITS PROGRAMS
- 4. Develop, with Human Resources:
An effective and timely policy to attract and retain Operations staff. 7.3.2 CAP Build Operator ranks with full support to stand up a fifth shift with a prescribed minimum reserve 7.3.5 ConOps bench metric. Aggressively pursue increasing the number of licensed individuals.
Mitigate the resource shortage in the Aging Reactor Management (ARM) Group with external resources 7.4.2-PI 7.3.6 Training to: Program Take on the administrative tasks. 7.3.8 Causal Develop and implement organization-wide programs (e.g., CAP, Observation Program, et.al.). Analysis Coach and mentor ARM staff with regard to programmatic responsibilities.
- 5. Corrective Action Program: complete the PI&R Assessment as soon as practicable. Continue with 7.3.1 Mgmt plans to implement a right-sized PI&R process. In the interim, prioritize the following immediate Obs Program actions:
Task 02 RCA Establish and enforce an expectation for a reporting threshold across NCNR-ROE. 7.3.2 CAP Select external resources to assist with program implementation, administration, and coaching. Task 03 PI&R 7.3.3 Develop and implement an easy and personal input method for CAP. 7.4.2-PI Aud/Surv/QA Establish a Corrective Action Review Board (CARB) / Management Operations Review Group utilizing external resources to assist with development, charter, startup, administration, and 7.4.9-QA 7.3.8 Causal member coaching. Analysis (Implied)
- 6. Conduct behavioral assessments of the Operations Crew Chiefs to assess attitudes, alignment, leadership capabilities, and targeted development feedback.
7.4.1-LA Behavioral assessments, feedback and coaching have been shown to be cri cal to aligning leadership 7.3.5 ConOps behavior to obtain consistent performance improvements. 7.4.10 Consider extending these assessments to other key leaders as priorities allow.
- 7. Develop a behavior-based mentoring/coaching program focused on safety culture attributes, leadership, and process improvements.
Provide the top leadership team with a team coach at the Director/Chief level to promote organiza onal alignment and teamwork.
7.4.1-LA Iden fy key leaders in ROE and establish individual mentors to develop manager behaviors and their interface to the organiza onal recovery processes.
Conduct behavioral assessments of current organizational leadership starting at Crew Chiefs level and going to top level management (Director/Chiefs).
FINAL REPORT JUNE 2, 2023 92
INDEPENDENT THIRD-PARTY NUCLEAR SAFETY CULTURE ASSESSMENT NUREG-2165 /
RECOMMENDATIONS INPO 12-012 TRAITS PROGRAMS
- 8. Training and Procedures Programs: complete the Training and Procedures Assessments as Task 04 soon as practical.
Training Implement the Systematic Approach to Training (SAT).
Fully implement the PPA / INPO 11-003 requirements for procedures. Task 05 7.3.5 ConOps Procedures 7.3.6 Training 7.4.1-LA Program 7.4.4-WP 7.4.5-CL
- 9. Establish and maintain housekeeping/combustibles and gas cylinder loading that reflect OSHA standards in:
Guide Hall, 7.3.5 ConOps Confinement Building, Laboratories, and Other spaces.
- 10. Complete the planned development and implementation of an Employee Concerns Program (ECP).
Strongly recommend for the foreseeable future, establish and maintain a regular and consistent 7.4.6-RC 7.3.9 ECP onsite presence.
Include within the scope of follow-on Nuclear Safety Culture Assessments.
- 11. As an interim action, develop and implement a problem-solving approach for emergent issues to include:
Define the Problem Statement.
Conduct significance assessment.
Identify known consequences. 7.4.1-SA Perform extent of condition evaluation.
7.4.10-DM Pre-establish a Command-and-Control NCNR-ROE response structure.
Establish response teams with issue-specific roles & responsibilities.
Develop a written response plan.
Implement and revise response plan as necessary.
FINAL REPORT JUNE 2, 2023 93
INDEPENDENT THIRD-PARTY NUCLEAR SAFETY CULTURE ASSESSMENT NUREG-2165 /
RECOMMENDATIONS INPO 12-012 TRAITS PROGRAMS
- 12. As a function of development of the comprehensive improvement plan that results from planned assessments and the collective evaluation process, consider the following elements to address Cross-Cutting, Distributed Function Programs. 7.3.1 Mgmt Every line manager owns a piece of distributed functions; examples of distributed functions include, but Obs Program are not limited to: 7.3.2 CAP Nuclear Safety Culture and Roles, Responsibilities, Authorities, and Accountabilities Emergent Issue Management Process 7.3.3 Conduct of Operations Aud/Surv/QA Operational Focus 7.4.1-LA 7.3.4 Doc PI&R/Corrective Action Program Control Employee Concerns Program 7.4.3-PA 7.3.5 ConOps Training 7.4.5-CL Management Observation Program 7.3.6 Training 7.4.7-CO Program Work Planning, Management and Control Management of Change 7.3.7 QA and Oversight Function Procedures Risk Management 7.3.8 Causal Obtain and prioritize external resources (including funding) to support development and implementation Analysis of the programs listed above.
7.3.9 ECP Interim: Establish a daily framework: Morning meetings, daily schedule reviews, CAP Screening, Management Reviews, Plan of the Day, Break-out groups as needed.
FINAL REPORT JUNE 2, 2023 94
INDEPENDENT THIRD-PARTY NUCLEAR SAFETY CULTURE ASSESSMENT
8.0 Attachments
A. List of Acronyms B. People Contacted C. Documents Reviewed D. Meetings and Workplace Evolutions Observed E. Nuclear Safety Culture Assessment Plan F. Nuclear Safety Culture Traits Evaluation Summary FINAL REPORT JUNE 2, 2023 95
INDEPENDENT THIRD-PARTY NUCLEAR SAFETY CULTURE ASSESSMENT A. List of Acronyms NBSR National Bureau of Standards Reactor ADR Alternative Dispute Resolution [Process] NCNR NIST Center for Neutron Research AFI Area For Improvement NIST National Institute of ANA Area in Need of Attention Standards and Technology ANSI American National Standards NLO Non-Licensed Operator Institute NRC Nuclear Regulatory ARM Aging Reactor Management Commission AOS Area of Strength NSC Nuclear Safety Culture ATL, Inc. Advanced Technologies and PA Personal Accountability Laboratories International, Inc. P.E. Professional Engineer CAP Corrective Action Program PI Problem Identification and Resolution CHP Certified Health Physicist PII Personal Identifiable CL Continuous Learning Information CO Confirmatory Order PO Positive Observation CO Effective Safety PPA Procedure Professionals Communications Association CRE Chief of Reactor Engineering QA Quality Assurance CRO Chief of Reactor Operations QA Questioning Attitude CROE Chief of Reactor Operations RC Environment for Raising and Engineering Concerns DM Decision-making RCA Root Cause Analysis DOE Department of Energy ROE Reactor Operations and ECP Employee Concerns Engineering Programs SAC NCNR Safety Assessment IAEA International Atomic Energy Committee Agency SAT Systematic Approach to INPO Institute of Nuclear Power Training Operations SEC NCNR Safety Evaluation IP Inspection Procedure (NRC) Committee LA Leadership Safety Values SCWE Safety Conscious Work and Actions Environment LOI Line of Inquiry SIT Special Inspection Report MC Manual Chapter (NRC) SOW Statement of Work FINAL REPORT JUNE 2, 2023 96
INDEPENDENT THIRD-PARTY NUCLEAR SAFETY CULTURE ASSESSMENT SRO Senior Reactor Operator WE Respectful Work Environment WP Work Processes FINAL REPORT JUNE 2, 2023 97
INDEPENDENT THIRD-PARTY NUCLEAR SAFETY CULTURE ASSESSMENT B. People Contacted Last Name First Name Position Adams, James Director, NCNR (after 03/25/2023)
Almeida, Brian NLO, Crew 4 Arneson, Scott Crew 4 Ball, Lyndsey NLO, Crew 5 Bauman, Christopher NLO, Crew 2 Beach, Chris NLO, Crew 5 Berg, Chris Crew Chief, Crew 2 Bobik, Paul ARM, Lead Mechanical Brand, Paul Chief, Reactor Engineering Burge, Susan HP Tech Burmeister, Jeff Crew Chief, Crew 4 Burton, Jessie Health Physicist Campbell, Joshua Crew 1 Carattini, Dominic NLO, Crew 5 Celikten, Osman International Guest, Nuclear Systems Consani, Keith Health Physicist Dewey, Steve Chief, HP & Safety Dimeo, Robert Director, NCNR (prior to 03/25/2023)
Dudley, Michelle QA Lead (Reactor Engineering Department)
Fitt, Todd Crew 3 Frie, Brandy Scheduling, ARM Gahan, Andrew Chief, Aging Reactor Management Group Griffin, David Security Manager Gurgen, Anil International Guest, Nuclear Systems Halacsy, Attila Crew 1 Hix, David Engineer, Cold Source Jurns, John Lead, Cold Source Kelkay, Mitiku HP Tech Keyser, Dan Electronics, ARM Kingsbury, Lance NLO, Crew 3 Khan, Dan Technician, Cold Source Liposky, Paul Planning Lead (Reactor Engineering)
MacDavid, Sam Lead, Electronics, ARM Mackey, Elizabeth Chief Safety Officer, NIST Main, Andrew Lead, Systems Martinez, Dayana NLO, Crew 3 Mattes, Dan Mechanical Engineer, Planning Mengers, Tim Health Physicist Montalvo, David NLO, Crew 2 Neuman, Dan Director, ROE (Scientists)
Newby, Robert Engineer, Cold Source Deputy Director, NCNR, and Newton, Thomas Chief Reactor Operations and Engineering Nguyen, Ha Electronics, ARM Pierce, Don Supervisory Mechanical Engineer, Research Facility Ops FINAL REPORT JUNE 2, 2023 98
INDEPENDENT THIRD-PARTY NUCLEAR SAFETY CULTURE ASSESSMENT Last Name First Name Position Remley, Bryan Health Physicist Sahin, Dagistan Lead, Nuclear Systems Seiter, Jacob Training Manager (Acting)
Shaposhnik, Yaniv Guest, Nuclear Systems Shen, Joy Mechanical Engineer, Nuclear Systems Slaughter, Scott Crew Chief, Crew 1 Strader, Randy Chief, Reactor Operations Szakal, Andrea Chemist and Safety Program Coordinator Trossen, Jeffrey Drafter, Planning Vodopija, Andrew NLO, Crew 4 Walton, Avery HP Tech Weiss, Abdullah Nuclear Engineer, Nuclear Systems Whipple, James Electronics Engineer, Nuclear Systems Wright, Brian Electronics, ARM Zaker, Majd Electrical Engineer, Systems FINAL REPORT JUNE 2, 2023 99
INDEPENDENT THIRD-PARTY NUCLEAR SAFETY CULTURE ASSESSMENT C. Documents Reviewed DOCUMENTS REVIEWED REV 06 TYPE DOC. # REV DATE TO FROM TITLE/SUBJECT ANSI The Development of Technical Specifications ANSI/ANS-15.1-2007 04/20/2007 ANS Standard for Research Reactors ANSI Selection and Training of Personnel for ANSI/ANS-15.4-2016 04/19/2016 ANS Standard Research Reactors ANSI Quality Assurance Program Requirements for ANSI/ANS-15.8-1995 05/10/2013 ANS Standard Research Reactors ANSI Radiation Protection at Research Reactor ANSI/ANS-15.11-2016 05/13/2016 ANS Standard Facilities ANSI Format and Content for Safety Analysis ANSI/ANS-15.21-2012 04/03/2013 ANS Standard Reports for Research Reactors Benchmark Director, 05/26/2022 NCNR INL ATR Benchmarking Plan, 05/24-26/2022 Plan Benchmark Director, ORNL HFIR Benchmarking Plan, 06/15-06/16/2022 NCNR Plan 16/2022 Benchmarking Practices that Support a Healthy Benchmark Director, 05/26/2022 NCNR Nuclear Safety Culture at Idaho National Report Laboratory May 24 - 26, 2022 Benchmark Director, Benchmarking Practices that Support Nuclear 06/15/2022 NCNR Report Safety Culture at ORNL-HFIR Best Practices to Establish and Maintain a Best Practice ML040350487 Undated NRC Safety Conscious Work Environment Best Practice #181: Safety Culture Monitoring Best Practice 181 06/2015 DOE EFCOG Process/Panel FINAL REPORT JUNE 2, 2023 100
INDEPENDENT THIRD-PARTY NUCLEAR SAFETY CULTURE ASSESSMENT DOCUMENTS REVIEWED REV 06 TYPE DOC. # REV DATE TO FROM TITLE/SUBJECT Establishing Quality Assurance Programs for CFR 10 CFR Part 71 05/16/2013 Packaging Used in Transport of Radioactive Material EA-21-148 NRC Office National Institute of Standards and Technology, Confirmatory 08/01/2022 of Center for Neutron Research - Confirmatory Order ML22206A213 Enforcement Order Nuclear Regulatory Commission
[Docket No. 05000184; EA-21-148; NRC-2022-7590-01-P NRC Office Confirmatory 0150]
08/03/2022 of Order ML22202A423 Enforcement Confirmatory Order Modifying License of National Institute of Standards and Technology, Center for Neutron Research ATL Questions as a Source for NIST Survey Marathon Data Table None 12/28/2019 Consulting [Extract from 2019 ATR NSC Assessment Group 12/28/2019, prepared by Marathon Consulting Group]
Draft Order, Personnel Selection, Training, DOE Order DOE O 426.2A Undated DOE Qualification, and Certification Requirements for DOE Nuclear Facilities Federal Vol 76 No 114 34773 - 34778 06/14/2011 NRC Final Safety Culture Policy Statement Register Principles for a Strong Safety Culture Guide None None DOE
[32-page guide]
Safety Culture Practitioner Guide: A Guide to Improving Staffing Safety Culture Guide None 0 06/2019 DOE EFCOG Competencies in the DOE Community
[ISMS Working Group]
Management and Independent Assessments Guide DOE G 414.1-1C 03/27/2014 DOE Guide FINAL REPORT JUNE 2, 2023 101
INDEPENDENT THIRD-PARTY NUCLEAR SAFETY CULTURE ASSESSMENT DOCUMENTS REVIEWED REV 06 TYPE DOC. # REV DATE TO FROM TITLE/SUBJECT Integrated Safety Management System Guide, Guide DOE G-450.4-1C 09/20/2011 DOE Attachment 10 Safety Culture Self-Assessment of Safety Culture in Nuclear IAEA IAEA-TECDOC-1321 11/2002 IAEA Installations - Highlights and Good Practices Guidelines for Performance Improvement at INPO 05-005 08/2005 INPO Nuclear Power Stations Guidance for Excellence in Procedure and INPO 11-003 0 06/2011 INPO Work Instruction Use and Adherence INPO INPO 12-013 0 12/2012 INPO Performance Objectives and Criteria Traits of a Healthy Nuclear Safety Culture INPO 12-012P 01 04/2013 INPO (Pocket)
Nuclear Safety Culture - Industry Best INPO 20-004 01 06/2021 INPO Practices Inspection Chapter 0305 06/28/2021 Operating Reactor Assessment Program Manual Inspection Chapter 0308 01/01/2023 Reactor Oversight Process Basis Document Manual Inspection Chapter 0310 02/25/2019 Aspects within the Cross-cutting Areas Manual Inspection Operating Reactor Security Assessment Chapter 0320 04/04/2009 Manual Program Oversight of Reactor Facilities in a Shutdown Inspection Chapter 0350 04/01/2018 NRC Condition Due to Significant Performance Manual and/or Operational Concerns Inspection Research and Test Reactor Inspection Chapter 2545 06/01/2020 NRC Manual Program FINAL REPORT JUNE 2, 2023 102
INDEPENDENT THIRD-PARTY NUCLEAR SAFETY CULTURE ASSESSMENT DOCUMENTS REVIEWED REV 06 TYPE DOC. # REV DATE TO FROM TITLE/SUBJECT Inspection 42401 10/27/2010 NRC Part 52, Plant Procedures Procedure Inspection 42453 08/19/2008 NRC Part 52, Operating Procedures Inspection Procedure Inspection 42700 11/15/1995 NRC Plant Procedures Procedure Inspection 60705 07/27/1995 NRC Preparation for Refueling Procedure Inspection 69001 02/03/2004 NRC Class II Research and Test Reactors Procedure Class I Research and Test Reactor Operator Inspection 69003 02/03/2004 NRC Licenses, Requalification, and Medical Procedure Examinations Inspection Class I Research and Test Reactor Effluent 69004 02/03/2004 NRC Procedure and Environmental Monitoring Inspection Class I Research and Test Reactor 69005 02/03/2004 NRC Procedure Experiments Class I Research and Test Reactors Inspection 69006 02/03/2004 NRC Organization and Operations and Maintenance Procedure Activities Inspection Class I Research and Test Reactor Review and 69007 02/03/2004 NRC Procedure Audit and Design Change Functions Inspection 69008 02/03/2004 NRC Class I Research and Test Reactor Procedures Procedure Inspection Class I Research and Test Reactor Fuel 69009 02/03/2004 NRC Procedure Movement FINAL REPORT JUNE 2, 2023 103
INDEPENDENT THIRD-PARTY NUCLEAR SAFETY CULTURE ASSESSMENT DOCUMENTS REVIEWED REV 06 TYPE DOC. # REV DATE TO FROM TITLE/SUBJECT Inspection Class I Research and Test Reactor 69010 02/03/2004 NRC Procedure Surveillance Inspection Class I Research and Test Reactor Emergency 69011 02/03/2004 NRC Procedure Preparedness Inspection Class I Research and Test Reactors Radiation 69012 02/03/2004 NRC Procedure Protection Inspection Reactor Safety - Initiating Events, Mitigating 71111 01/01/2023 NRC Procedure Systems, Barrier Integrity Inspection 71152 12/14/2021 NRC Problem Identification and Resolution Procedure Inspection 71715 04/14/1992 NRC Sustained Control Room and Plant Observation Procedure Inspection 88005 09/05/2006 NRC Management Organization and Controls Procedure Inspection 88010 02/07/2014 NRC Training Procedure Inspection Supplemental Inspection Response to Action 95001 08/19/2021 NRC Procedure Matrix Column 2 (Regulatory Response) Inputs Supplemental Inspection Response to Action Inspection 95002 04/01/2021 NRC Matrix Column 3 (Degraded Performance)
Procedure Inputs Supplemental Inspection Response to Action Inspection 95003 06/07/2022 NRC Matrix Column 4 (Multiple/Repetitive Degraded Procedure Cornerstone) Inputs Inspection 95003.01 01/15/2009 NRC Emergency Preparedness Procedure FINAL REPORT JUNE 2, 2023 104
INDEPENDENT THIRD-PARTY NUCLEAR SAFETY CULTURE ASSESSMENT DOCUMENTS REVIEWED REV 06 TYPE DOC. # REV DATE TO FROM TITLE/SUBJECT Guidance for Conducting an Independent NRC Inspection 95003.02 04/01/2019 NRC Nuclear Safety Culture Assessment Procedure (Attachment 02 to IP 95003)
Inspection Procedure 12/20/2022 NRC Hyperlink list of NRC Inspection Procedures Hyperlink List National Institute of Standards and Technology Inspection - U S Nuclear Regulatory Commission Interim 05000184/2021201 04/14/2021 NRC NRR Report Special Inspection Report No.
05000184/2021201 EA-21-148 National Institute of Standards and Technology Inspection 03/16/2022 NRC NRR - U S Nuclear Regulatory Commission Special Report 05000184/2022201 Inspection Report No. 05000184/2022201 National Institute of Standards and Technology EA-21-148 Supplemental Inspections [Attachment to letter Letter Undated NRC ML22206A011 EA-21-148 of 08/01/2022] Supplemental inspection activity table.
National Institute of Standards and Technology EA-21-148 National Bureau of Standards Test Reactor -
Letter Undated NRC ML22206A012 Supplemental Inspection Plan [Enclosure to letter EA-21-148 of 08/01/2022]
NBSR report on February 3, 2021 fuel failure Director, Letter ML21274A027 10/01/2021 event, root causes, corrective actions, and NIST restart readiness National Institute of Standards and Technology
- Supplemental Information Needed for the Request to Restart the National Bureau of Letter ML21294A277 11/18/2021 NRR Standards Test Reactor Following Exceedance of the Cladding Temperature Safety Limit (EPID L2021LLN0000)
FINAL REPORT JUNE 2, 2023 105
INDEPENDENT THIRD-PARTY NUCLEAR SAFETY CULTURE ASSESSMENT DOCUMENTS REVIEWED REV 06 TYPE DOC. # REV DATE TO FROM TITLE/SUBJECT Subj: Unplanned Shutdown of the NIST Director, LANL Lab Reactor, February 3, 2021. Sections:
Letter 02/14/2022 NIST Director (T. Resources, Procedures and Training (Olthoff) Mason) Emergency Response NIST Institutional Risk Management EA-21-148 US Nuclear Regulatory Commission Letter 08/01/2022 NRC NRR Supplemental Inspection Plan for National ML22206A010 Institute of Standards and Technology Discussions:
- 1. Conditions that allowed Feb 3 incident Letter re: Julia M.
to occur February 2022 Phillips 2. NCNRs emergency response 2/3/21 event
- 3. NIST organizational response
- 4. Efficacy and completeness of proposed corrective actions Discussions:
Eric Kaler, 1. Conditions that allowed Feb 3 incident Director, President to occur Letter re:
02/15/2022 NIST Case 2. NCNRs emergency response 2/3/21 event Western (Olthoff) 3. NIST organizational response Reserve U 4. Efficacy and completeness of proposed corrective actions Discussions:
- 1. Conditions that allowed Feb 3 incident Director, to occur Letter re: Alexander 03/02/2022 NIST Adams Jr. 2. NCNRs emergency response 2/3/21 event (Olthoff) 3. NIST organizational response
- 4. Efficacy and completeness of proposed corrective actions FINAL REPORT JUNE 2, 2023 106
INDEPENDENT THIRD-PARTY NUCLEAR SAFETY CULTURE ASSESSMENT DOCUMENTS REVIEWED REV 06 TYPE DOC. # REV DATE TO FROM TITLE/SUBJECT National Institute of Standards and Technology Director, - Authorization to Restart Following Letter 03/09/2023 NRC NRR NCNR Exceedance of the Safety Limit (EPID L-2021-LLN-0000)
Subj: ERCAS Consideration of Safety Culture During Evaluation of Emergency Response and E. Mackey, NCNR Assessment of Technical Working Group Memo 04/18/2022 ERCAS Director Chair Causal Factors and Associated Corrective Actions. [Groups causes and observations under Safety Culture Traits]
Event NIST Chief, Memo ML21274A024 09/21/2021 Recovery Reactor Ops Memorandum on Recovery Items File & Eng NEI NEI 09-07 1 March 2014 NEI Fostering a Healthy Nuclear Safety Culture NUREG NUREG-2165 March 2014 NRC Safety Culture Common Language Operations 68 Operations Report No. 68 for the NBSR 04/25/2017 NRC NIST Report ML17123A366 (January 1 to December 31, 2016).
Operations 69 Operations Report No. 69 for the NBSR 04/02/2018 NRC NIST Report ML18103A027 (January 1 to December 31, 2017)
Operations 71 Operations Report No. 71 for the NBSR 04/25/2019 NRC NIST Report ML19126A272 (January 1 to December 31, 2018)
Operations 72 Operations Report No. 72 for the NBSR 04/02/2020 NRC NIST Report ML20093F261 (January 1 to December 31, 2019)
Operations 73 Operations Report No. 73 for the NBSR 04/2/2021 NRC NIST Report ML21123A168 (January 1 to December 31, 2020)
Operations 74 Operations Report No. 74 for the NBSR 04/04/2021 [sic] NRC NIST Report ML22094A108 (January 1 to December 31, 2021)
FINAL REPORT JUNE 2, 2023 107
INDEPENDENT THIRD-PARTY NUCLEAR SAFETY CULTURE ASSESSMENT DOCUMENTS REVIEWED REV 06 TYPE DOC. # REV DATE TO FROM TITLE/SUBJECT Reactor Operations & Engineering Org Chart Org Chart 01/20/2023 NIST
[excel document]
Org Chart 01/20/2023 NIST Safety & ROE Org Chart [excel document]
Not Corrective Action Plan - Corrective Actions Plan ML21274A025 09/30/2021 indicated Required Prior to Startup Corrective Action Plan - Corrective Actions Plan ML21274A026 09/30/2021 Not indicated Required Post-startup AR 5.4 The Observation Program PowerPoint AR 5.4 Undated (Observation Program Initial Training)
Safety Improvements to Address Deficiencies Presentation 11282022 11/28/2022 that Led to the 2021 Fuel Damage Event Press NRC Authorizes Restart of National Institute of 23-021 03/10/2023 NRC Release Standards and Technology Reactor Procedure Administrative Rule 5.0 02 Undated NBSR Procedure Use and Adherence NBSR Reactor Engineering Document Control Procedure NBSR-0001-DOC-03 03 05/12/2009 Plan Quality Assurance Program for Mods to NBSR Procedure NBSR-0002-DOC-02 02 08/2007 Reactor and NCNR Facility Guidelines for Completing Engineering Procedure NBSR-0003-DOC-06 06 09/07/2012 Changes Procedure NSBR-0004-DOC-05 05 03/04/13 NIST Packaging and Shipping QA Program NSBR Calibration Program for Measurement Procedure NSBR-0005-DOC-00 0 05/14/2019 and Test Equipment Engineering Change Control for NBSR Reactor Procedure NCNR-1000-DOC-00 0 07/10/2007 Operations and Engineering, NCNR Procedure Administrative Rule 1.0 02 05/31/2022 NBSR NBSR Conduct of Operations FINAL REPORT JUNE 2, 2023 108
INDEPENDENT THIRD-PARTY NUCLEAR SAFETY CULTURE ASSESSMENT DOCUMENTS REVIEWED REV 06 TYPE DOC. # REV DATE TO FROM TITLE/SUBJECT Procedure Administrative Rule 1.1 01 08/27/2021 NBSR Human Performance Tools Procedure Administrative Rule 5.0 01 08/12/2021 NBSR Procedure Use and Adherence Procedure Administrative Rule 5.0 02 08/12/2021 NBSR Procedure Use and Adherence [DRAFT]
Procedure Administrative Rule 5.1 02 11/15/2022 NBSR Procedure Writers Guide Procedure Administrative Rule 5.4 01 08/22/2022 NBSR Observation Program and Checklist System Health Summary Template and Procedure Administrative Rule 7.1.1 01 Undated NBSR Instructions Corrective Action Program Level 2 Procedure Administrative Rule 7.1.2 01 Undated NBSR Requirements and Workflows Procedure Administrative Rule 7.2 02 02/27/2023 NBSR Corrective Action Program Corrective Action Program Level 0 Procedure Administrative Rule 7.2.0 0 02/27/2023 NBSR Requirements and Workflows Corrective Action Program Level 1 Procedure Administrative Rule 7.2.1 0 02/27/2023 NBSR Requirements and Workflows Corrective Action Program Level 2 Procedure Administrative Rule 7.2.2 0 02/28/2023 NBSR Requirements and Workflows Corrective Action Program Level 3 Procedure Administrative Rule 7.2.3 0 03/03/2023 NBSR Requirements and Workflows Corrective Action Program Item Input Procedure Administrative Rule 7.2.4 01 02/22/2023 NBSR Information Procedure Administrative Rule 11.0 E 01/14/2020 NBSR Lockout/Tagout NIST Safety Culture Program Program NIST S 7101.06 10/25/2022 NIST
[Effective date TBD]
Not RC Response ML21274A022 01 09/20/2021 Root Cause Response Rev 1 indicated FINAL REPORT JUNE 2, 2023 109
INDEPENDENT THIRD-PARTY NUCLEAR SAFETY CULTURE ASSESSMENT DOCUMENTS REVIEWED REV 06 TYPE DOC. # REV DATE TO FROM TITLE/SUBJECT Addendum to Root Cause Investigation of February 2021 Fuel Failure (Addendum 5 of RCI ML21274A020 06/03/2021 NCNR TWG 18)
NCNR Technical Working Group (TWG)
NCNR Technical Root Cause Investigation of February 2021 RCI ML21274A019 02 09/13/2021 Working Fuel Failure Rev 2 Group S. Haber &
RSP-0060 AECB, M. Barriere Development of a Regulatory Organizational Report 01/20/1998 Ottowa, of Human AECB 2.341.2 Canada Performance and Management Review Method Analysis Assessment of Safety Culture Sustainment Report 06/2020 DOE Processes at U.S. Department of Energy Sites August 2018 - November 2019 National Institute of Standards and Technology
- Report on the Regulatory Audit re: Restart Report ML22322A218 11/28/2022 NRC NRR Request Following Exceedance of Cladding Temperature Safety Limit (EPID: L-2021-LLN-0000)
External Subject Matter Reports on the Feb 3, Report 09/23/2022 NIST 2021 Alert at the NCNR National Academy of An Assessment of the Center for Neutron Report 2022 Sciences, Research at the National Institute of Standards Engineering and technology: Fiscal Year 2021
& Medicine National Institute of Standards and Technology Center for Neutron Research SAC Report 12/10/2020 Safety Assessment Committee Report -
Calendar Year 2020 FINAL REPORT JUNE 2, 2023 110
INDEPENDENT THIRD-PARTY NUCLEAR SAFETY CULTURE ASSESSMENT DOCUMENTS REVIEWED REV 06 TYPE DOC. # REV DATE TO FROM TITLE/SUBJECT SAC Report 05 Undated NCNR 2021 SAC Report Rev 5 NCNR 2022 Safety Assessment Committee SAC Report 01 Undated (SAC) Report SAC Report Undated NCNR Response to the 2020 SAC Report
Response
SAC Report Undated NCNR Response to 2021 SAC Report
Response
Safety Not ML21274A023 09/21/2021 Latch Improvement Safety Analysis Analysis indicated NIST Center for Neutron Research & NBSR SAR ML041120205 SAR Chapter 1 Undated NIST Chapter 1 SAR ML041120206 SAR Chapter 2 Undated NIST Site Characteristics Design of Structures, Systems, and SAR ML041120207 SAR Chapter 3 Undated NIST Components SAR ML041120210 SAR Chapter 4 Undated NIST Reactor Description SAR ML041120237 SAR Chapter 13 Undated NIST Accident Analyses FINAL Report SEC Subcommittee Report:
Review of the NCNR Event Response and SEC Report ML21274A021 08/12/2021 NIST SEC Technical Working Group Cause Analysis and Corrective Action Plan SRP DOE-STD-3006-2010 2010 Planning and Conducting Readiness Reviews Not NBSR Safety Culture Survey 2021 Report Survey None Undated indicated (package) 2021 Climate Group Survey Results Survey (Engineering, Operations, Health Physics &
Safety)
FINAL REPORT JUNE 2, 2023 111
INDEPENDENT THIRD-PARTY NUCLEAR SAFETY CULTURE ASSESSMENT DOCUMENTS REVIEWED REV 06 TYPE DOC. # REV DATE TO FROM TITLE/SUBJECT Technical Specifications for Test Reactor Tech Spec License No. TR-5, Amendment 09 09/24/2013 (NSR)
Technical Specifications for Test Reactor Tech Spec License No. TR-5, Amendment 10 09/10/2015 (NSR)
Technical Specifications for Test Reactor Tech Spec License No. TR-5, Amendment 11 12/15/2017 (NSR)
FINAL REPORT JUNE 2, 2023 112
INDEPENDENT THIRD-PARTY NUCLEAR SAFETY CULTURE ASSESSMENT D. Meetings and Workplace Evolutions Observed Date Type Discussion Behaviors Impacting Traits 4/3 - Onsite Visit During the two-week onsite assessment period, startup beyond several LA, PI, WP, QA, DM - (ANA) 14/23 hundred kW was hampered by two predominant issues: tramp fuel particle fissioning that resulted in higher than estimated fission gases, and lack of effective routing of fission gas due to deficient ventilation system integrity.
Evaluation of the NCNR license change was not discussed to evaluate if this specific problem was addressed in the license change safety evaluation.
Trouble shooting activities were not part of an approved process.
Elevating reactor power levels appeared to be a trial and error without a specific plan.
4/5/23 Plant Tour During a plant tour, a question was asked about 2 outside gates that were PI, PA, RC, CO - (ANA)
Observation left open. A Radiation Technician stated they shouldnt be left open and called the control room. The next day in a focus group interview the observer provided a positive observation to the Technician concerning the way they took immediate action and called the control room. The Technician responded that the control room told her, You are not in Operations anymore so forget about it. This Technician had just left the Operations organization. (PS) for Radiation Technician and an (AFI) for Operations.
Tour Guides were very knowledgeable of the plant and provided detailed descriptions of various locations. The tour provided by Health Physics discussed several aspects of the facility such as Reactor Operations, Scientific Operations, Radiation Protection program responsibilities, Emergency Program responsibilities and the fuel latching issue. This was one of the best plant tours that this observer participated in while working at several nuclear facilities.
FINAL REPORT JUNE 2, 2023 113
INDEPENDENT THIRD-PARTY NUCLEAR SAFETY CULTURE ASSESSMENT Date Type Discussion Behaviors Impacting Traits 4/3 - Interviews During interviews, NCNR personnel were very open and discussed LA, PI, PA, DM, QA - (ANA) 4/14/23 numerous topics. One item that the interviewers asked Operators about was the ventilation Technical Specification surveillance and that it would not pass the surveillance and would be continued to be performed until a successful surveillance would be accomplished. When they were asked about the specific surveillance, they actually laughed, as if they knew the methodology that the site has implemented was wrong. When asked about their responsibility, they stated that they give the surveillance to their supervisor and whether it passes of fails, was up to the supervisor. When asked about if there were other examples, operators stated the shim voltage adjustments.
They stated that after testing, the voltage would have to be adjusted in order for the shims to properly operate.
4/4/23 Plan of the NIST Observation LA, PA, WP, CO, PI, DM Day (POD)
Observed Reactor startup in Control Room (ANA)
Operator at controls was trainee. Ex-Navy reactor operator. This was his first startup at the controls here under guidance of licensed SRO. Excellent OJT (PO-CL).
Pre-job brief was thorough. CRO was in attendance (PO-LA). Procedure use was good (PO). Issues were identified and discussed by CRO. One occasion of background noise by others in CR was addressed and use/expectation of ARPs was also communicated (ANI). The CRO also prompted the reperformance of the Estimated Critical Position due to a slight temperature change.
Reactor was brought critical and power was raised to 100kw to observe fission product gases.
FINAL REPORT JUNE 2, 2023 114
INDEPENDENT THIRD-PARTY NUCLEAR SAFETY CULTURE ASSESSMENT Date Type Discussion Behaviors Impacting Traits 4/6/23 Plan of the Detailed discussion of Reactor Fission Product generation and PI, LA - (ANA)
Day (POD) Contamination of Basement 1.
Verbal trouble shooting and floating of ideas around how the basement is LA, PI, WP - (ANA) becoming contaminated and protentional of isolating the basement to contain the contamination.
No Problem Statement development or agreement. Jumped to fixes based LA, PI, DM, QA - (ANA) on assumption that the ventilation system was faulty.
CRO volunteered to take the issue and work it. Actions from the meeting PA, LA -(ANA) were to leave the Reactor S/D and perform a smoke test the next day.
Fission product discussion was detailed and at this time there was concern that the FPs were being generated linearly to power level (tests at 200, 400, LA, QA - (PO) and 800 KW). Plan to monitor 4/7/23 Plan of the Called to order on time.
Day (POD)
Detailed discussion of Reactor Fission Product generation and PI, LA - (AFI)
Contamination of Basement 1. Still no Problem Statement and not in CAP.
Smoke test not performed, many leaks identified. Missed opportunity to find all the leaks.
Ops attempting to contact NIST Ventilation folks. Many misses in getting LA, CO - (ANA) them lined up.
Determination made outside the meeting that probable fix is to recoat the ventilation system with mastic to return system to design. Non-Q system and non-Q material. Application Skill of Craft by NIST OFPM group.
Good reminder from Training Manager that any temp repairs or changes QA -(PO) need an ECR/ECN.
Found many flow paths to the B1 including cable chases, etc.
Still no visible problem statement or logical plan PI, WP, LA -(ANA)
FINAL REPORT JUNE 2, 2023 115
INDEPENDENT THIRD-PARTY NUCLEAR SAFETY CULTURE ASSESSMENT Date Type Discussion Behaviors Impacting Traits 4/10/23 Plan of the FP/Ventilation issue continues. Decision to mastic the ventilation stands and PI, LA, WP - (ANA)
Day (POD) NIST should perform the work today. Still not in CAP and not documented in a Plan.
QA - (PO)
Good questions on impact of HE sweep leakage to C200 and potential of ODH. Safety estimated a 24-hour period for safety.
Engineering didnt weigh in to verify ODH Calc. PA - (ANA)
When questioned the NCNR was told the mastic would cure in 1-3 hours (Turns out not), when asked the acceptance criteria for restart the Reactor QA, WP, PI - (ANA) restart was addressed but not the criteria for ventilation restart.
After the meeting coached on CAP entry. Engineering entered CAP with a recommended extent of conditions for systems that werent checked out LA - (ANA) thoroughly prior to Reactor Restart.
FINAL REPORT JUNE 2, 2023 116
INDEPENDENT THIRD-PARTY NUCLEAR SAFETY CULTURE ASSESSMENT Date Type Discussion Behaviors Impacting Traits 4/11/23 Plan of the Yesterday and overnight - it turns out the Mastic cure time was 24-72 hours QA - (PO)
Day (POD) when questioning and reading the can.
The overnight discussion re-raised the ODH issue. CO, QA, DM (ANA)
Engineering again wasnt engaged to verify ODH Calc and help set up PA, LA, WE (ANA) mitigation.
Ventilation was restarted and the mastic repair failed, requiring rework. LA, CO, WP (ANA)
Rework Started.
The Airbourne Tritium was 1 DAC on C200 (Control Room) and when WP, (AFI) questioned the Chief HP thought C200 was fully posted (it turns out it wasnt.
QA (ANA)
The elevator was neither disabled nor posted).
Broken glass was found on a tour by EOs in training as reported by the PI (PO) training manager. When queried about CAP entry it was stated it was put into PA, PI, LA (ANA) an email. Entered into CAP with prompting (ANA - PA, PI, LA)
Good discussion about not slipping back into old performance behaviors lead by Chief HP, i.e., housekeeping, collaboration and robust pre-job briefs LA, QA, WE, WP - (PO)
(Issues from yesterdays and todays 5K Waste Tank pump-out). PJB issues entered into CAP.
An NCNR employee reported out on his IRIS inspection - Excellent catch -
aluminum filings in a trash can on top of a lithium 14-volt battery. Removed PA, LA, PI, QA - (PO) and isolated the hazard and documented. Good discussion. CRO (without prompting) coached to get it into CAP.
FINAL REPORT JUNE 2, 2023 117
INDEPENDENT THIRD-PARTY NUCLEAR SAFETY CULTURE ASSESSMENT Date Type Discussion Behaviors Impacting Traits 4/12/23 Plan of the Plan of the Day Meeting started at 1415.
Day (POD)
Chief of Operations provided a brief update on hole patching, linkage issue Part 1 near a damper; waiting on some parts; a grommet needs replacing.
(Observer note: There was no discussion on what happened last night; what occurred is mastic had been placed on leaking sections of the ventilation LA, QA - (ANA) system. Cure time: 24 hours2.777778e-4 days <br />0.00667 hours <br />3.968254e-5 weeks <br />9.132e-6 months <br />. At the four (4) hour point, Chief of Operations asked the ventilation system be turned on. The mastic was sucked in. This was apparently a solo decision, i.e., with no input from other departments).
ROE Chief followed with this statement - verbatim:
Please continue to raise issues. Think things through before acting. Involve LA, PI - (PO) all who should weigh in. Think things through. The Chief provided no context before or after.
No one asked about what ROE Chief said; the meeting went on. QA (ANA)
(Observer note: it was unclear that anyone knew what Tom was talking about
- though post-meeting it was apparent that most indeed knew what had occurred the previous evening and what Tom was referring to. YET it was not discussed,)
4/12/23 Plan of the 1418: Safety Officer asked, Will there be a CAP on this work planning QA, LA - (PO)
Day (POD) issue?
Part 2 ROE Chief (and CRO): Yes.
(Observer note: CAP had just been launched the week prior and there had QA - (ANA) been very little input into the CAP engine. Andrea was speaking from that perspective. No discussion, just a yes.)
FINAL REPORT JUNE 2, 2023 118
INDEPENDENT THIRD-PARTY NUCLEAR SAFETY CULTURE ASSESSMENT Date Type Discussion Behaviors Impacting Traits 4/12/23 Plan of the More updates on todays work, including RP support of the Vent System QA (ANA)
Day (POD) work.
Part 3 Thursday work list - leading off - showed Low Power Testing to 400kW. The discussion covered other Thursday work, skipping the reactor low power testing topic.
(Observer note: everyone seemed to know that the reactor would not likely startup on Thursday - let alone not get to 400kW, yet no one even mentioned the line item. No one mentioned anything.)
4/12/23 Plan of the Someone (unsure who) asked CRO if Low Power Testing was going to QA, LA - (PO)
Day (POD) happen Thursday.
Part 4 No. It wont happen Friday either.
1428: Safety Officer asked, Who will submit the CAP?
ROE Chief: I will.
(Observer note: Safety Officer had been simmering for nearly 10 minutes to ask about who was going to submit a CAP for the mastic mishap that occurred 18 hours2.083333e-4 days <br />0.005 hours <br />2.97619e-5 weeks <br />6.849e-6 months <br /> previously. The Director of ROE responded as though no one knew what to say).
1429: meeting closed.
4/13/23 Plan of the Mastic still curing. Found additional leaks. DM, WP - (ANA)
Day (POD)
Questions were starting to arise about the design criteria for the normal ventilation system and if it was meant to handle the FP inventory the facility was expecting.
FINAL REPORT JUNE 2, 2023 119
INDEPENDENT THIRD-PARTY NUCLEAR SAFETY CULTURE ASSESSMENT Date Type Discussion Behaviors Impacting Traits 4/14/23 All hands NCNR Program Director presented status of the reactor, restart schedule meeting and criteria and fielded questions.
When asked about other systems for readiness for reactor operations, was CO, LA - (ANA) optimistic rather than accepting the question as a good one and taking action.
Bought up the Fatality from last year and said there was going to be work to do in rewriting Pre-Planned Hazard Analysis that NCNR uses. Potential big LA, CO, (ANA) impact to Operations.
Negatives. Program Director wasnt wired and doesnt speak loudly. Many comments from within the conference room and TEAMS participants. CO - (ANA)
Program Director didnt repeat questions from the audience before answering CO, LA - (ANA)
Remarks were prepared but no visuals or take-aways.
The audience wants more frequent all hands (bi-Weekly). Program Director was committed to more frequent but shot down every other week.
LA - (ANA)
FINAL REPORT JUNE 2, 2023 120
INDEPENDENT THIRD-PARTY NUCLEAR SAFETY CULTURE ASSESSMENT Date Type Discussion Behaviors Impacting Traits 4/21/23 Plan of the Detailed discussion of Reactor Fission Product generation and ongoing Day (POD) contamination of Basement 1. Issue is in CAP Visible White board use from last week. DM - (PO)
Fission product discussion was detailed and at this time the concern that the FPs were being generated linearly to power level (tests at 200, 400, and 800 KW) seem moot. They were linear this week during operation.
Discussion of temporary modifications to reduce leakage from the reactor top QA - (PO) and ventilation changes to reduce the basement airborne and surface contamination.
Also, discussion of changing the normal ranges of HE sweep operation to WP - (ANA) absolute minimums, to reduce leakage to atmosphere.
All pretty squishy. Requires follow up.
Reactor is Shutdown pending contamination source control. DM - (ANA)
Assessment Scope (Trait) Section Trait Abbreviation Number 4.1 Leadership, Safety Values and Actions LA 4.2 Problem Identification and Resolution PI 4.3 Personal Accountability PA 4.4 Work Processes WP 4.5 Continuous Learning CL 4.6 Environment for Raising Concerns RC 4.7 Effective Safety Communications CO 4.8 Respectful Work Environment WE 4.9 Questioning Attitude QA 4.10 Decision Making DM FINAL REPORT JUNE 2, 2023 121
INDEPENDENT THIRD-PARTY NUCLEAR SAFETY CULTURE ASSESSMENT E. Nuclear Safety Culture Assessment Plan NATIONAL INSTITUTE OF STANDARDS AND TECHNOLOGY PLAN FOR THIRD PARTY NUCLEAR SAFETY CULTURE PERFORMANCE ASSESSMENT OF NCNR BACKGROUND The National Institute of Standards and Technology (NIST) Center for Neutron Research (NCNR) owns and operates a heavy water-moderated nuclear test reactor and associated neutron beam research facility. The NCNR's reactor is licensed by the Nuclear Regulatory Commission (NRG) under the name National Bureau of Standards Reactor (NBSR).
On February 3, 2021 , the Chief of NCNR Reactor Operations and Engineering notified the NRG (event notification EN 55094) of an alert concerning elevated radiation levels at the NBSR.
Pursuant to the event notification received from NCNR staff on February 3, 2021 , the NRG initiated a special inspection at the NBSR. On April 14, 2021, the NRG staff issued an interim special inspection report to provide an initial assessment of their understanding of the event sequence , consequences , and the NCNR's response to the event (ADAMS Accession No. ML21077A094).
On March 16, 2022, the NRG released a final report of its initial conclusion from its special inspection report. The NRC's final report confirms and expands on many aspects of NIST's analysis of the incident, pointing to deficiencies in policies, procedures, training, and safety culture as contributing to the incident.
On August 1, 2022, a Confirmatory Order Modifying License No. TR-5 was issued by the NRG.
The issued Confirmatory Order documents action completed and planned by NCNR as well as the commitments made by NIST to enable the safe operation of the NBSR.
NCNR has engaged an independent nuclear safety culture team to support recovery from this event and to enhance facility safety. Th is task-based contract covers a wide array of activities in six program areas. Task 1 is the conduct of an independent Third-Party Nuclear Safety Culture Performance Assessment (Assessment) of NCNR.
PURPOSE The independent Third Party Nuclear Safety Culture Assessment Team (Team) will evaluate the nuclear safety culture of NCNR against the nuclear safety culture traits defined by the Common Language Initiative as documented in NUREG 2165/INPO 12-012 and alluded to in NRG Manual Chapter 0350 ("Oversight of Reactor Facilities in a Shutdown Condition due to Significant Performance and/or Operational Concerns") and NRG Inspection Procedure 95003.02, ("Guidance for Conducting an Independent NRG Safety Culture Assessment") as revised to reflect NUREG-2165. Based upon this evaluation , the Team will identify areas for improvement in the NCNR nuclear safety culture.
FINAL FINAL REPORT JUNE 2, 2023 122
INDEPENDENT THIRD-PARTY NUCLEAR SAFETY CULTURE ASSESSMENT NCNR NUCLEAR SAFETY CULTURE ASSESSMENT PLAN REFERENCES US DOE G 414.1-1C, "Management and Independent Assessments Guide US NRC Inspection Procedure 95003 US NRC Inspection Procedure 95003 ATTACHMENT 02 US NRC Inspection Manual Chapter 0350, "Oversight of Reactor Facilities in a Shutdown Condition "
NEI 09-07, "Fostering a Strong Nuclear Safety Culture" NUREG 2165 , "Safety Culture Common Language" INPO 12-012, "Traits of a Healthy Nuclear Safety Culture" Research Report RSP-0060, "Development of a Regulatory Organizational and Management Review Method." Haber, S.B. and Barriere, M.T. , dated June 7, 2022 Task Order 01 dated January 31, 2023 (and additional documents referenced therein)
SCOPE AND AREAS FOR EVALUATION The Team will evaluate the following nuclear safety culture traits at NCNR (quoted from NRC NUREG-2165). In performing this evaluation , the Team will consider, as appropriate , the specific attributes and examples related to these traits described in NRC NUREG-2165 and INPO 12-012.
I. Leadership Safety Values and Actions (LA) - Leaders demonstrate a commitment to safety in their decisions and behaviors.
II. Problem Identification and Resolution (Pl) - Issues potentially impacting safety are promptly identified, fully evaluated, and promptly addressed and corrected commensurate with their significance.
Ill. Personal Accountability (PA) - All individuals take personal responsibility for safety.
IV. Work Processes (WP)- The process of planning and controlling work activities is implemented so that safety is maintained.
V. Continuous Learning (CL) - Opportunities to learn about ways to ensure safety are sought out and implemented.
VI. Environment for Raising Concerns (RC) - A safety-conscious work environment (SCWE) is maintained where personnel feel free to raise safety concerns without fear of retaliation , intimidation, harassment, or discrimination.
VII. Effective Safety Communication (CO) - Communications maintain a focus on safety.
VIII. Respectful Work Environment (WE) - Trust and respect permeate the organization.
IX. Questioning Attitude (QA) - Individuals avoid complacency and continuously challenge existing conditions and activities in order to identify discrepancies that might result in error or inappropriate action.
X. Decision making (DM) - Decisions that support or affect nuclear safety are systematic, rigorous, and thorough.
FINAL 2 FINAL REPORT JUNE 2, 2023 123
INDEPENDENT THIRD-PARTY NUCLEAR SAFETY CULTURE ASSESSMENT NCNR NUCLEAR SAFETY CULTURE ASSESSMENT PLAN In addition , the Team will review cause evaluations, inspection results , and information associated with NRC findings and observations made within the past three years to determine whether there are other particular issues related to safety culture that warrant examination.
METHODOLOGY The independent Team will perform its work in two phases:
Phase one: Development and administration of the 2023 NCNR Nuclear Safety Culture Survey. Results will be compared to output from the 2021 NBSR Nuclear Safety Culture Survey. The survey will provide insights into what NCNR personnel believe.
Phase two : In-depth observation and evaluation of workplace behaviors and interactions, as well as further exploration of survey results. The field observation/evaluation phase will establish what NCNR personnel do.
The general data collection methods to be considered include (but are not limited to):
- Functional analysis
- Structured interviews
- Focus Groups
- Observations
- Review archived data
- Behavioral Anchored Rating Scales (BARS)
- Behavioral Checklist
- Safety Culture Survey The Team and the Behavioral Scientist will collectively determine the specific collection methods that are appropriate for this assessment once the Nuclear Safety Culture Survey results have been received.
The Assessment will include several different assessment activities designed to evaluate the NCNR Nuclear Safety Culture . These include:
- Administration and evaluation of a confidential Nuclear Safety Culture Survey created by the Team , which will perform follow-up reviews and interviews as necessary to ensure that potential significant safety culture issues identified by the survey results and comments provided during the survey are understood. This will include specific focus on organizational elements that the survey identifies as having relatively weaker nuclear safety culture.
- Reviews of NCNR and NIST corporate policies, programs, procedures , and training as they relate to nuclear safety culture.
- Reviews of a sample of communications related to NCNR activities with emphasis on nuclear safety culture events and issues.
- Reviews of information regarding NCNR performance and issues, including , but not limited to, results of previous assessments , NRC inspections and findings , employee concerns, allegations , Human Resources information , Corrective Action Program (CAP) data , performance metrics, site assessment programs, organizational performance goals , resolution of significant issues, safety culture analyses in NCNR event causal evaluations, and site performance indicators. The confidentiality of sensitive security-and personnel-related information will be respected and protected.
FINAL 3 FINAL REPORT JUNE 2, 2023 124
INDEPENDENT THIRD-PARTY NUCLEAR SAFETY CULTURE ASSESSMENT NCN R NUCLEAR SAFETY CULTURE ASSESSMENT PLAN
- Observations of selected site activities and meetings to determine the extent to which the conduct and results of those activities and meetings are indicative of a strong nuclear safety culture.
- Receipt of information provided by NCNR management on selected key issues and areas for improvement.
- Selected interviews of site and supporting personnel external to NCNR. Focus Group interviews will be used to gather information on the nuclear safety culture in the various site organizations. In addition, individual interviews will be conducted to gather information on particular topics and issues.
The Team will compare the collected information with the NSC Traits, Attributes, and Examples cited in NUREG-2165 to evaluate the NCNR nuclear safety culture. The Team will cross-validate the results and identify nuclear safety culture issues, to the extent they exist.
NCNR management has informed the Team Leader that the Team will have full access to any NCNR personnel, documents, or other information necessary to support a complete and thorough assessment, with due respect for requirements to limit access to sensitive information.
Evaluation activities will be assigned to Team members having experience or expertise in the areas being evaluated. Team members will determine the specific evaluation activities they will perform in consultation with the Team and will consider the inspection elements described in NRC Inspection Procedure 95003 and the attributes and examples for the safety culture traits described in NUREG 2165/INPO 12-012 in developing their evaluation approach in each area .
Team members will prepare records of their specific evaluation activities.
The on-site portion of the assessment will span a two-week period currently scheduled for April 3rd through April 14th . The three-person sub-teams will be different for each week. A dedicated conference room has been reserved during that period for conducting interviews (individual or focus groups) and for processing data and information.
Interviews will generally be conducted by two team members while the third member performs observations of daily conduct of operations and work activities. Specific observations are subject to scheduled or emergent NCNR evolutions available for observation during that time-frame. Individual roles may change during the week.
Many of the Team members will have NIST badges and NCNR badges allowing unrestricted access to many parts of the NCNR building. Unbadged Team members will be escorted by badged Team members. Team members might require NCNR escort to enter restricted areas.
Access and escorts necessary for Team members to enter restricted areas will be coordinated through the COR (Don Pierce) or his designee.
Daily meetings will be held with the COR (Contracting Officer's Representative) and TPOC (contract Technical Point of Contact) to discuss progress and address logistical issues.
With respect to particular evaluation methods and Team assignments, the following are planned during onsite assessment activities:
- Week of April 3-7: a three-person team including Behavioral Scientist Dr. Rogers will be onsite.
- Week of April 10: a three-person team will be onsite.
- Bruce O'Brien will be onsite to bridge the two assessment weeks.
FINAL 4 FINAL REPORT JUNE 2, 2023 125
INDEPENDENT THIRD-PARTY NUCLEAR SAFETY CULTURE ASSESSMENT NCNR NUCLEAR SAFETY CULTURE ASSESSMENT PLAN
- The two sub-teams will interview a cross-section of site personnel, with a goal of approximately 25 individual interviews. Interviewees will include managers, supervisors, and individual contributors.
- Given that licensing exams are scheduled for several operators during the week of April 3, the onsite Team will focus more on non-operations staff during this week.
- During the week of April 10 the Team will interview/meet with the remaining operations staff population as well as other NCNR personnel.
- The Team will conduct Focus Group interviews that will cover a cross-section of site personnel, with a goal of 5 groups representing the significant site organizations (without inclusion of supervisory personnel), and one group of first line supervisors. Insofar as practical, Focus Groups will be structured to include between 5 and 8 personnel. Two Team members will participate in each Focus Group interview. Prepared question sets will be used when the purpose of interviews or Focus Groups is to gain an understanding of overall employee views of safety culture within an organization but may or may not be used for interviews on specific topics. Focus Group and individual interviews will include open-ended questions that allow interviewees to provide their views on safety culture issues they believe are important, whether or not covered in the prepared questions.
- Observations of site meetings and work activities will be conducted using observation outlines to record the safety culture traits or issues observed.
- The results of observations of activities, interviews, and Focus Groups will be compiled and evaluated against the nuclear safety culture traits identified in NUREG-2165. The purpose and results of each observation, interview, and Focus Group will be recorded ,
including why and how the observation was made and the individuals or Focus Group were selected and interviewed.
- A record of the documents reviewed during the Assessment will be kept.
- Many of the areas to be evaluated during the Assessment may have been subject to recent assessments, cause analyses, surveys, or NRC inspections. In cases where the Team concludes that the results of those previous efforts are reliable and currently relevant, the Team may rely upon those results as part of its evaluation. However, the Team also will perform independent evaluation activities to ensure that the Team's general conclusions are supported by the results of its own evaluation activities.
- Insofar as practical, Assessment records will be prepared, redacted , and/or maintained in such a way as to prevent the identity of personnel who provide specific pieces of information to the Team from being correlated , except as necessary to ensure that any safety conditions identified during the assessment are addressed .
- Whether or not selected for interview, NCNR personnel will be provided information on how to contact the Team to share their views on nuclear safety culture.
At least one week prior to each onsite visit, the Team will provide NCNR with requests for individual interviews and Focus Groups. NCNR is asked to provide the operators individual shift schedules no later than March 21 to coordinate operator interviews.
The Team will meet collectively during the assessment to review the information that has been obtained and develop observations, findings, and conclusions regarding the safety culture at NCNR. Conclusions regarding nuclear safety culture will be supported by at least two separate sources of observation or data and will reflect the consensus of the Team. Those findings and FINAL 5 FINAL REPORT JUNE 2, 2023 126
INDEPENDENT THIRD-PARTY NUCLEAR SAFETY CULTURE ASSESSMENT NCNR NUCLEAR SAFETY CULTURE ASSESSMENT PLAN conclusions will be documented in (1) a presentation of preliminary results to NCNR management; and (2) a subsequent written report.
EVALUATION TEAM MEMBERS The independent Assessment Team membership is as follows :
Dr. Michael D. Quinn , Task Leader, and Analyst Michael T. Coyle, RADM, USN (ret.), Analyst Steven K. Crowe, CAPT, USNR (ret.), Analyst Michael J. Fecht, SRO, Analyst Richard N. Swanson, P.E. (ret.), Analyst, and Assessment Report Leader This team of highly experienced nuclear professionals has a combined experience in the nuclear industry of over 240 years. Each member has participated in multiple "performance recoveries" of nuclear organizations , in both DOE and NRG-regulated facilities . Additionally, each member has operated nuclear reactors and managed and supervised nuclear personnel.
Dr. Mary Jo Rogers, Behavioral Scientist, will advise the team throughout the Assessment.
Additional independent individuals may be added as, and if, required.
MAJOR SCHEDULE MILESTONES EVENT DATE Final Nuclear Safety Culture Survey Questions Provided 02/24/2023 (Complete)
NSC Survey Launch 02/27/2023 (Complete 02/24)
NSC Survey Complete 03/10/2023 Draft NSC Assessment Plan to NCNR 03/10/2023 Reconcile NCNR/NRC Comments 03/21 - 28/2023 Final NSC Assessment Plan 03/28/2023 Onsite Entrance Meetinq NSC Assessment 04/03/2023 Pre-Assessment Kickoff (NIST/NCNR Team) 04/03/2023 Onsite Assessment Complete 04/14/2023 Exit Meetinq with Draft Summary 04/14/2023 Draft Assessment Report to NCNR for Comment 04/28/2023 Reconcile NCNR Comments 05/05 - 05/19/2023 Final Assessment Report Delivered 05/19/2023 FINAL 6 FINAL REPORT JUNE 2, 2023 127
INDEPENDENT THIRD-PARTY NUCLEAR SAFETY CULTURE ASSESSMENT NCNR NUCLEAR SAFETY CULTURE ASSESSMENT PLAN NUCLEAR SAFETY CULTURE ASSESSMENT REPORT OUTLINE The draft and final Assessment Reports will contain the following sections; the Team may re-order or add additional sections as necessary to communicate Assessment results and conclusions.
Executive Summary Assessment Scope Identification of Team Members Identification of Personnel Contacted Documents Reviewed Work Performance Observed Assessment Process and Criteria Assessment Results Areas of Strengths Areas in Need of Attention Areas for Improvement Identified issues of non-compliance (Findings}
2023 NSNR Nuclear Culture Survey Analysis FINAL 7 FINAL REPORT JUNE 2, 2023 128
INDEPENDENT THIRD-PARTY NUCLEAR SAFETY CULTURE ASSESSMENT F. Nuclear Safety Culture Traits Evaluation Summary NUCLEAR SAFETY CULTURE TRAITS EVALUATION
SUMMARY
ATTRIBUTE CATEGORY EVALUATION TREND LA.1 Resources AFI LA.2 Field Presence ANA LA.3 Incentives, PO Sanctions, Rewards LA.4 Strategic AFI Commitment La.5 Change ANA Management LA.6 Roles &
AFI Responsibilities LA.7 Constant AFI Examination LA.8 Leader Behaviors ANA Marginally
- 1. Leadership Trait (LA)
Effective Positive PI.1 Corrective Action AFI Program PI.2 Evaluation AFI PI.3 Resolution AFI PI.4 Trending AFI
- 2. PI&R Trait (PI) Not Effective Flat PA.1 Standards AFI PA.2 Job Ownership AFI PA.3 Teamwork AFI Marginally
- 3. Personal Accountability Trait (PA)
Effective Flat WP.1 Work Management PO WP.2 Design Margins AFI WP.3 Documentation AFI WP.4 Procedure AFI Adherence Marginally
- 4. Work Processes Trait (WP)
Effective Positive CL.1 Operating AFI Experience CL.2 Self Assessment ANA CL.3 Benchmarking ANA CL.4 Training AFI Marginally
- 5. Continuous Learning Trait (CL) Positive Effective FINAL REPORT JUNE 2, 2023 129
INDEPENDENT THIRD-PARTY NUCLEAR SAFETY CULTURE ASSESSMENT NUCLEAR SAFETY CULTURE TRAITS EVALUATION
SUMMARY
ATTRIBUTE CATEGORY EVALUATION TREND RC.1 SCWE PO RC.2 ECP AFI Marginally
- 6. Environment for Raising Concerns Trait (RC) Positive Effective CO.1 Work Process PO Communication CO.2 Basis for Decision ANA CO.3 Free Flow of ANA Information CO.4 Expectations ANA Marginally
- 7. Safety Communication Trait (CO)
Effective Positive WE.1 Respect is Evident AFI WE.2 Opinions are PO Valued WE.3 Trust AFI WE.4 Conflict Resolution ANA Marginally
- 8. Respectful Work Environment Trait (WE)
Effective Positive QA.1 Nuclear is Unique AFI QA.2 Challenge the ANA Unknown QA.3 Challenge ANA Assumptions QA.4 Avoid ANA Complacency
- 9. Questioning Attitude Trait (QA) Effective Positive DM.1 Consistent ANA Process DM.2 Conservative Bias ANA DM.3 Accountability for ANA Decisions Marginally
- 10. Decision Making Trait (DM)
Effective Positive FINAL REPORT JUNE 2, 2023 130