ML24240A067
| ML24240A067 | |
| Person / Time | |
|---|---|
| Issue date: | 08/27/2024 |
| From: | Sushil Birla, Dickerson K, Doug Eskins, Stephen Fleger, Brian Green, Natalee Green, Watkins H, Jing Xing NRC/NRR/DRO, NRC/RES/DRA/HFRB |
| To: | |
| Shared Package | |
| ML24240A064 | List: |
| References | |
| RIL 2024-10 | |
| Download: ML24240A067 (107) | |
Text
RIL 2024-10 Regulatory Significance of Halden Human-Technology-Organisation Project Research Activities for 2021-2023 Extended Summary of U.S. NRC Involvement Research Information Letter Office of Nuclear Regulatory Research 8/27/2024 Date Published: September 2024 Prepared by:
N. Hughes Green1 H. Watkins1 J. Xing1 B. Green2 S. Birla3 S. Fleger1 K. Dickerson1 D. Eskins3 1Office of Nuclear Regulatory Research/Division of Risk Analysis 2Office of Nuclear Reactor Regulation/Division of Reactor Oversight
i DISCLAIMER Legally binding regulatory requirements are stated only in laws, U.S. Nuclear Regulatory Commission (NRC) regulations, licenses (including technical specifications), or orders, not in research information letters (RILs). An RIL is not regulatory guidance, although the NRCs regulatory offices may consider the information in an RIL to determine whether any regulatory actions are warranted.
ii ABSTRACT This report summarizes the regulatory significance of the research undertaken during the 2021-2023 program period through the Halden Human-Technology-Organisation Project (Halden HTO), under the auspices of the Organisation for Economic Co-operation and Developments Nuclear Energy Agency. The report focuses on the collaboration between Halden HTO and the U.S. Nuclear Regulatory Commission (NRC). The research conducted by Halden HTO addresses critical gaps in the technical bases and regulatory criteria for new and advanced reactor designs. Key areas of focus include enhancing human-system interfaces, exploring cognitive performance in crew operations, and developing guidelines to support the integration of new technologies into regulatory frameworks and enabling performance-based safety assurance and evaluation of digital instrumentation and control systems. This report highlights specific research activities conducted by Halden HTO that have shaped NRC guidance documents and enhanced modern regulatory practices. Asterisked sections of the report are quoted from the report Halden HTO Program Achievements 2021-2023, which summarizes the 2021-2023 research program accomplishments and highlights key achievements (Halden HTO, 2024). By fostering dialogue and the exchange of innovative approaches, the collaboration between Halden HTO and the NRC has contributed significantly to efforts to update nuclear safety standards and to prepare for the challenges posed by future nuclear technologies.
iii TABLE OF CONTENTS DISCLAIMER................................................................................................................................. i ABSTRACT.................................................................................................................................. ii LIST OF TABLES........................................................................................................................ vi LIST OF FIGURES.................................................................................................................... viii ABBREVIATIONS AND ACRONYMS........................................................................................ ix 1 BACKGROUND.................................................................................................................... 1-1 1.1 Halden Human-Technology-Organisation................................................................... 1-1 1.2 The NRC Participation in Halden HTO........................................................................ 1-3 1.3 Research Areas Addressed by Halden HTO............................................................... 1-5 1.4 Research Capabilities Offered by Halden HTO........................................................... 1-5 1.4.1 Halden Man-Machine Laboratory (HAMMLAB)............................................ 1-5 1.4.2 Pressurized-Water Reactor Simulator.......................................................... 1-5 1.4.3 Halden Small Modular Reactor Simulator (HSMR)...................................... 1-5 1.4.4 Halden Human-Machine Satellite Lab (HAMMSAT)..................................... 1-6 1.4.5 Human Automation Lab (HA-Lab)................................................................ 1-6 1.4.6 Halden Virtual Reality Centre (HVRC).......................................................... 1-6 1.4.7 Cybersecurity Centre.................................................................................... 1-6 1.4.8 IFE FutureLab............................................................................................... 1-6 1.5 Regulatory Applications of Halden HTO Research..................................................... 1-7 1.5.1 Addressing Licensing Challenges................................................................ 1-7 1.5.2 Guidance Development for Advanced Reactors.......................................... 1-7 1.5.3 Human Factors Engineering Guidance Development.................................. 1-8 1.5.4 Safety Assurance, Digital Instrumentation and Control, and Cybersecurity Guidance Development....................................................... 1-10 2 PROJECT OVERVIEWS..................................................................................................... 2-11 2.1 Human Performance................................................................................................. 2-11 2.1.1 Operator Performance in Digital Control Rooms........................................ 2-12 2.1.2 Crew Factors, Teamwork, and Role Independence in Control Rooms...... 2-14 2.1.3 Decision-Making under Uncertainty............................................................ 2-16 2.1.4 Event Investigations: The Added Perspective of Successes...................... 2-18 2.1.5 The Human Performance Data Repository................................................ 2-20 2.2 Digital Instrumentation and ControlsSafety Assurance......................................... 2-22 2.2.1 Risk-Informed Safety Assurance................................................................ 2-23 2.2.2 Evidence Collection, Evaluation, and Combination for Safety Assurance................................................................................................... 2-25 2.3 Control Room Design and Evaluation....................................................................... 2-28 2.3.1 Lessons Learned on Control Room Validation........................................... 2-29 2.3.2 The Impact of Overview Displays on Human Performance........................ 2-32 2.3.3 Augmented Reality for On-Site Human Factors Control Room Assessment of Ergonomic and Regulatory Compliance............................ 2-34 2.4 Human-Automation Collaboration............................................................................. 2-37 2.4.1 Human Performance in Operation of Small Modular Reactors.................. 2-38 2.4.2 Operator Performance in Highly Automated Plants.................................... 2-40
iv 2.4.3 Effects of Adaptive Automation on Operator Performance in Future Plants.......................................................................................................... 2-42 2.5 Digital Systems for Operations and Maintenance..................................................... 2-44 2.5.1 Advanced Condition-Based Maintenance Using Digital Twins................... 2-45 2.5.2 Advanced Condition Monitoring for Decision Support................................ 2-47 2.5.3 Safety Awareness in Outage Organizations............................................... 2-49 2.6 Digital Transformation of Decommissioning.............................................................. 2-51 2.6.1 Spatial Computing and Augmented Reality for Hazard Mapping and Visualization............................................................................................... 2-52 2.6.2 Automated Assessment of Field Worker Performance Using VR-and AR-Based Simulator Training..................................................................... 2-54 2.6.3 Enabling Robotic and Remote Operation................................................... 2-57 2.7 Cybersecurity for Main Control Rooms..................................................................... 2-59 2.7.1 Digital Systems Architecture and Threat Landscape.................................. 2-60 2.7.2 Incident Detection and Response Using Simulation Modeling and Tools........................................................................................................... 2-62 2.7.3 Human Behavior during Cyber Incident Response.................................... 2-64 3 SELECTED PROJECTS..................................................................................................... 3-66 3.1 Operator Performance in Digital Control Rooms....................................................... 3-66 3.1.1 NRC Engagement...................................................................................... 3-66 3.1.2 Regulatory Use of Project Outcomes (2021-2023).................................... 3-67 3.2 Crew Factors, Teamwork, and Role Independence in Control Rooms..................... 3-69 3.2.1 NRC Engagement...................................................................................... 3-69 3.2.2 Regulatory Use of Project Outcomes (2021-2023).................................... 3-69 3.3 Decision-Making under Uncertainty.......................................................................... 3-71 3.3.1 NRC Engagement...................................................................................... 3-71 3.3.2 Regulatory Use of Project Outcomes (2021-2023).................................... 3-71 3.4 Performance-based Risk-Informed Safety Assurance and Evidence Collection, Evaluation, and Combination for Safety Assurance.................................................. 3-72 3.4.1 NRC Engagement...................................................................................... 3-72 3.4.2 Regulatory Use of Project Outcomes (2021-2023).................................... 3-72 3.5 Lessons Learned on Control Room Validation.......................................................... 3-73 3.5.1 NRC Engagement...................................................................................... 3-73 3.5.2 Regulatory Use of Project Outcomes (2021-2023).................................... 3-73 3.6 The Impact of Overview Displays on Human Performance....................................... 3-75 3.6.1 NRC Engagement...................................................................................... 3-75 3.6.2 Regulatory Use of Project Outcomes (2021-2023).................................... 3-75 3.7 Human Performance in Operation of Small Modular Reactors and Operator Performance in Highly Automated Plants.................................................................. 3-76 3.7.1 NRC Engagement...................................................................................... 3-76 3.7.2 Regulatory Use of Project Outcomes (2021-2023).................................... 3-77 3.8 Digital Systems Architecture and Threat Landscape................................................ 3-80 3.8.1 NRC Engagement...................................................................................... 3-80 3.8.2 Regulatory Use of Project Outcomes (2021-2023).................................... 3-80 4 FUTURE DIRECTIONS......................................................................................................... 4-1 4.1 Planned Topic Areas................................................................................................... 4-1 4.2 Regulatory Significance of Continued Collaboration................................................... 4-2 4.2.1 Human Factors and Human Reliability......................................................... 4-2 4.2.2 Digital Instrumentation and Control.............................................................. 4-3
v 4.2.3 Cybersecurity................................................................................................ 4-4 4.2.4 NRC Technical Training Center.................................................................... 4-5 4.3 Discussion................................................................................................................... 4-5 5 REFERENCES...................................................................................................................... 5-1 6 APPENDIX............................................................................................................................ 6-1 A-1 List of Halden HTO Reports for 2021-2023................................................................ 6-1 A-2 Infographic on the Significance of the NRC-Halden HTO Collaboration..................... 6-3
vi LIST OF TABLES Table 1-1 Percentage Contributions of Participating Member Countries to Halden HTO for the 2021-2023 Period................................................................................... 1-3 Table 2-1 Halden HTO Objective, Results, and References for Operator Performance in Digital Control Rooms*..................................................................................... 2-12 Table 2-2 Halden HTO Objective, Results, and References for Crew Factors, Teamwork, and Role Independence in Control Rooms*Halden HTO Objective................. 2-14 Table 2-3 Halden HTO Objective, Results, and References for Decision-Making under Uncertainty*...................................................................................................... 2-16 Table 2-4 Halden HTO Objective, Results, and References for Event Investigations: The Added Perspective of Successes*................................................................... 2-18 Table 2-5 Halden HTO Objective, Results, and References for the Human Performance Data Repository*.............................................................................................. 2-20 Table 2-6 Halden HTO Objective, Results, and References for Risk-Informed Safety Assurance*....................................................................................................... 2-23 Table 2-7 Halden HTO Objective, Results, and References for Evidence Collection, Evaluation, and Combination for Safety Assurance*........................................ 2-25 Table 2-8 Halden HTO Objective, Results, and References for Lessons Learned on Control Room Validation*................................................................................. 2-29 Table 2-9 Halden HTO Objective, Results, and References for the Impact of Overview Displays on Human Performance*................................................................... 2-32 Table 2-10 Halden HTO Objective, Results, and References for Augmented Reality for On Site Human Factors Control Room Assessment of Ergonomic and Regulatory Compliance*................................................................................... 2-34 Table 2-11 Halden HTO Objective, Results, and References for Human Performance in Operation of Small Modular Reactors*............................................................. 2-38 Table 2-12 Halden HTO Objective, Results, and References for Operator Performance in Highly Automated Plants*............................................................................. 2-40 Table 2-13 Halden HTO Objective, Results, and References for Effects of Adaptive Automation on Operator Performance in Future Plants*.................................. 2-42 Table 2-14 Halden HTO Objective, Results, and References for Advanced Condition-Based Maintenance Using Digital Twins*........................................ 2-45 Table 2-15 Halden HTO Objective, Results, and References for Advanced Condition Monitoring for Decision Support*...................................................................... 2-47 Table 2-16 Halden HTO Objective, Results, and References for Safety Awareness in Outage Organizations*..................................................................................... 2-49 Table 2-17 Halden HTO Objective, Results, and References for Spatial Computing and Augmented Reality for Hazard Mapping and Visualization*............................. 2-52
vii Table 2-18 Halden HTO Objective, Results, and References for Automated Assessment of Field Worker Performance Using VR-and AR-Based Simulator Training*........................................................................................................... 2-54 Table 2-19 Halden HTO Objective, Results, and References for Enabling Robotic and Remote Operation*........................................................................................... 2-57 Table 2-20 Halden HTO Objective, Results, and References for Digital System Architecture and Threat Landscape*................................................................ 2-60 Table 2-21 Halden HTO Objective, Results, and References for Incident Detection and Response Using Simulation Modeling and Tools*............................................ 2-62 Table 2-22 Halden HTO Objective, Results, and References for Human Behavior during Cyber Incident Response*................................................................................ 2-64
viii LIST OF FIGURES Figure 2-1 Hypothesized behavioral indicators patterns in relation to excessive cognitive workload (Braarud and Pignoni, 2023)............................................................. 2-30 Figure 2-2 Evaluation of sightlines on the large screen, on-site in HAMMLAB using AR........ 2-35 Figure 2-3 Image of HAMMSAT Laboratory............................................................................ 2-40 Figure 2-4 Graphical user interface visualizing data analysis and dataflow in a digital twin framework......................................................................................................... 2-47 Figure 2-5 Screenshot of the XR training scene integrating 3D textured mesh scan, simulated radiation, XSens Awinda avatar, SPOT avatar................................ 2-55 Figure 2-6 Integrated safety case components........................................................................ 2-57
ix ABBREVIATIONS AND ACRONYMS 3D three dimensional ACRS Advisory Committee on Reactor Safeguards ADAMS Agencywide Documents Access and Management System ADVANCE Accelerating Deployment of Versatile, Advanced Nuclear for Clean Energy AHFE Applied Human Factors and Ergonomics AI artificial intelligence AR augmented reality ATHEANA A Technique for Human Event Analysis BIM building information modeling BOP balance of plant CANDU Canada Deuterium Uranium CBP computer-based procedure CFR Code of Federal Regulations CNSC Canadian Nuclear Safety Commission COG CANDU Owners Group CONOPs concept of operations CRIEPI (Japan) Central Research Institute of Electric Power Industry CSE cognitive systems engineering DI&C digital instrumentation and controls DOE U.S. Department of Energy DT digital twin EHPRG Enlarged Halden HTO Programme Review Group EPRI Electric Power Research Institute FA Function Allocation F&M Fuels and Materials FANR Federal Authority for Nuclear Regulation FKA Forsmarks Kraftgrupp AB FLEX Diverse and Flexible Coping Strategies FRAM Functional Resonance Analysis Method GIS geographic information system GRS Gesellschaft für Anlagen-und Reaktorsicherheit GmbH GSN goal structuring notation GVD group-view display HAMMLAB Halden Man-Machine Laboratory HAMMSAT Halden Human-Machine Satellite Lab HAZOP hazard and operability study HEP human error probability HFE human factors engineering
x HOF human and organizational factors HPI human performance improvement HPWR Halden Pressurized Water Reactor HRA human reliability analysis HRP Halden Reactor Project HSF Halden Safety Fan HSI human-system interface HSMR Halden Small Modular Reactor HTO Human-Technology-Organisation HTOR Human-Technology-Organisation Project report HVRC Halden Virtual Reality Centre I&C instrumentation and controls ICS industrial control system ICT information and communication technology IDHEAS Integrated Human Event Analysis System IDHEAS-ECA Integrated Human Event Analysis System for Event Condition Assessment IDS intrusion detection system IEEE Institute of Electrical and Electronics Engineers IFE Institute for Energy Technology (Institutt for Energiteknikk)
INL Idaho National Laboratory IoT Internet of Things ISV integrated system validation KAERI Korea Atomic Energy Research Institute LPD large-panel display LSTM long short-term memory ML machine learning MSV multistage validation MTO Man-Technology-Organisation MWe megawatts electric NASA National Aeronautics and Space Administration NEA Nuclear Energy Agency NEI Nuclear Energy Institute NEIMA Nuclear Energy Innovation and Modernization Act NNL National Nuclear Laboratory NPP nuclear power plant NRA (Japan) Nuclear Regulatory Authority NRC U.S. Nuclear Regulatory Commission NRG Dutch Nuclear Research and Consultancy Group NRR Office of Nuclear Reactor Regulation NSIR Office of Nuclear Security and Incident Response
xi OECD Organisation for Economic Co-operation and Development OIP Office of International Programs OKG OKG Aktiebolag PRA probabilistic risk assessment PWR pressurized-water reactor PWROG Pressurized Water Reactor Owners Group RAB Ringhals AB RAMS Real-time Anomaly Monitoring System RES Office of Nuclear Regulatory Research RIL research information letter RO reactor operator RPS reactor protection system SA situational awareness SAMG severe accident management guidelines SMR small modular reactor SNERDI Shanghai Nuclear Engineering Research and Design Institute Co., Ltd.
SPAR-H Standardized Plant Analysis Risk Human Reliability Analysis SRO senior reactor operator SSA structured safety argumentation SSM Swedish Radiation Safety Authority STA shift technical advisor STPA System Theoretic Process Analysis SWaT scaled-down replica of a full-fledged water treatment plant TTC Technical Training Center UJV Rez Ustav Jaderneho Vyskumu Rez VR virtual reality WGHOF Working Group on Human and Organizational Factors XR extended reality
1-1 1 BACKGROUND This report summarizes the regulatory significance of research conducted during the 2021-2023 Programme at the Organisation for Economic Co-operation and Developments (OECDs)
Nuclear Energy Agencys (NEAs) Halden Human-Technology-Organisation Project (Halden HTO), focusing on its collaboration with the U.S. Nuclear Regulatory Commission (NRC). It includes an extended summary of the research undertaken during the 2021-2023 Programme period.
1.1 Halden Human-Technology-Organisation Halden HTO, based in Halden, Norway, is an ongoing international collaborative sponsored by the NEA and managed by the Institute for Energy Technology (Institutt for Energiteknikk, or IFE). Halden HTO is a continuation of the Man-Technology-Organisation (MTO) research program, which was in place for more than 40 years as part of the OECDs Halden Reactor Project (HRP). Halden HTO focuses on human performance, reliability, technology, and organization at the different stages of life of a nuclear power plant (NPP), including design, modernization, operation (particularly during accident situations), and dismantling. Halden HTO aims to improve the safety engineering capabilities of the nuclear industry through collaborative research.
Halden HTO is managed by the Halden Management Board, made up of leaders from each participating organization. The research program covers topics such as human-automation collaboration, control room design and evaluation, digital operations and maintenance systems, sustainable decommissioning, and cybersecurity of main control rooms.
Halden HTO plans and conducts its research over a 3-year program cycle. The initial cycle spanned from 2021 to 2023. The report Halden HTO Programme Achievements 2021-2023, released in January 2024, provides a comprehensive review of the findings from research activities during this period (Halden HTO, 2024). In the current cycle, from 2024 to 2026, Halden HTO aims to undertake further safety research related to human performance, reliability, and digital automation within the nuclear domain.
During the 2021-2023 period, Halden HTO researchers produced 67 reports, presented 57 papers at international conferences, and published 23 journal articles and book chapters.
They collected data in a variety of ways. For human factors research, licensed crews working in the Halden Man-Machine Laboratory (HAMMLAB) served as a primary source of data. In 2022, one crew participated in studies, and in 2023, seven crews participated. Data were also collected from training simulators at NPPs with three crews. For research related to safety assurance for digital instrumentation and controls (DI&C), workshops with experts from nonnuclear sectors provided valuable knowledge about the state of the art. From September 25 to 28, 2023, the Enlarged Halden HTO Programme Review Group (EHPRG) convened a meeting in Lillehammer, Norway, to share the results of work performed during the 2021-2023 cycle. This workshop showcased 67 presentations and drew approximately 120 attendees from participating organizations. Updated statistics on the achievements of Halden HTO can be found on its website (https://www.oecd-nea.org/jcms/pl_61937/halden-human-technology-organisation-hto-project).
1-2 The member organizations actively participating in Halden HTO, formerly referred to as the HRP,1 represent a cross-section of the nuclear community, including licensing and regulatory authorities, national research organizations, reactor and fuel vendors, and utilities.
The participating organizations include the following2:
CanadaCanadian Nuclear Safety Commission (CNSC)
CanadaCanada Deuterium Uranium (CANDU) Owners Group (COG)
Canada - Canadian Nuclear Laboratories (CNL)
ChinaShanghai Nuclear Engineering Research and Design Institute Co., Ltd.
(SNERDI)
CzechiaUstav Jaderneho Vyskumu Rez (UJV Rez)
FranceInstitute for Radiation Protection and Nuclear Safety (IRSN)
GermanyFramatome GmbH GermanyGesellschaft für Anlagen-und Reaktorsicherheit GmbH (GRS)
JapanCentral Research Institute of Electric Power Industry (CRIEPI)
JapanNuclear Regulation Authority (NRA)
KoreaKorea Atomic Energy Research Institute (KAERI)
NetherlandsDutch Nuclear Research and Consultancy Group (NRG)
NorwayInstitute for Energy Technology (IFE)
SwedenSwedish Radiation Safety Authority (SSM)
SwedenForsmarks Kraftgrupp AB - (FKA)
SwedenOKG Aktiebolag (OKG)
United Arab EmiratesFederal Authority for Nuclear Regulation (FANR) 1 The HRP formerly comprised two sectors of research: Fuels and Materials (F&M) and Man-Technology-Organisation (MTO). In 2020, the program was split in two, with HRP conducting only F&M research, while the previous MTO (including the research simulators) was renamed the Halden Human-Technology-Organisation Project (Halden HTO).
2 In 2024, CzechiaUstav Jaderneho Vyskumu Rez (UJV Rez) and JapanNuclear Regulation Authority (NRA) were added as members.
1-3 United KingdomNational Nuclear Laboratory (NNL)
United StatesU.S. Department of Energy (DOE)
United StatesU.S. Nuclear Regulatory Commission (NRC)
United StatesElectric Power Research Institute (EPRI)
Halden HTOs funding is provided by its member organizations. Table 1-1 shows the percentage contributed by each country for the 2021-2023 period. Note that the contributions from Canada and the United States are split among multiple member organizations as listed above.
Table 1-1 Percentage Contributions of Participating Member Countries to Halden HTO for the 2021-2023 Period Country Percentage Contribution Norway 45%
Canada*
5%
China 6%
France 3%
Germany GRS 3%
Japan CRIEPI 4%
Korea KAERI 3%
Netherlands NRG 3%
Sweden SSM 4%
UAE FANR 2%
United States*
24%
TOTAL 100%
Note: The contribution from Canada is split between CNL, CNSC and COG. The contribution from the United States is split between the NRC, the DOE, and EPRI.
1.2 The NRC Participation in Halden HTO U.S. membership in Halden HTO is divided between three members: the NRC, the Electric Power Research Institute, and the U.S. Department of Energy, including national laboratories.
The NRC and its predecessor, the U.S. Atomic Energy Commission, have participated in Halden HTO (formerly the HRP) since its inception in 1958. In advance of each 3-year program, Halden HTO prepares a research proposal detailing the areas of study that it will focus on for the upcoming period. The research topics for each cycle are selected based on trends in the industry, the needs expressed by the Halden Management Board, advice and recommendations given in HTO meetings, and direct discussions with the NRC staff and other Halden HTO members. Halden HTO member organizations review the proposal. Halden HTO also visits each member organization to present the proposal formally and to discuss ways to improve the proposal to better meet members needs.
1-4 NRC staff members from the Office of Nuclear Regulatory Research (RES), the Office of Nuclear Reactor Regulation (NRR), the Office of Nuclear Security and Incident Response (NSIR), and the Office of International Programs (OIP) participate in the day-long proposal discussions to ensure that Halden HTO receives a variety of agency perspectives and that the research produced meets agency needs. The NRC staff members who review, evaluate, and provide input on Halden HTO activities have expertise in a diverse range of technical disciplines, including human factors engineering (HFE), human reliability analysis (HRA), DI&C, and probabilistic risk assessment (PRA). Halden HTO members rank the proposed projects to communicate their preferences for the direction and approach of each project. At the NRC, RES staff members coordinate the agencys proposed project rankings with NRR counterparts before submitting final ranking sheets to Halden HTO. Halden HTO reconciles this feedback and uses it to prioritize its activities.
During the feedback period, the NRC staff may identify specific topic areas that align with mission needs and collaborate with Halden HTO leads to help inform and refine the research direction. These are referred to as Halden HTO coordination groups. Regular engagement on specific projects of interest, beyond the formal visit every 3 years, is mutually beneficial for the NRC and Halden HTO, creating more valuable products and research outcomes. Specifically, engaging in these coordination groups supports the NRC staff in developing guidance by addressing research gaps in areas significant to nuclear regulation. During meetings, NRC subject matter experts work together to help guide the direction of research aimed at addressing these gaps and to provide updates on project progress. These deliverables are intended to facilitate updates and modernization of regulatory guidance. For examples, see Section 1.5.3 Human Factors Engineering Guidance Development. As shown in Chapter 3, Selected Projects, of this report, research projects with greater NRC involvement resulted in better, more useful data that enhanced regulatory decision-making. For the upcoming 2024-2026 research cycle, NRC technical staff members with subject-matter expertise in HFE, HRA, DI&C, and PRA, as well as the more recent focus areas of artificial intelligence (AI), data science, and cybersecurity, will continue to engage with Halden HTO coordination groups on topics of specific interest and utility to the agencys mission.
The NRCs participation in Halden HTO helps optimize the use of the agencys research funds by pooling resources with other Halden HTO participants. Table 1-1, which shows the percentage of funding contributed by each member country, illustrates the multiplicative benefits of the NRCs investment. Furthermore, collaboration with Halden HTO allows for a more efficient research process, which is a key tenet of the NRCs philosophy on guidance development. For example, much of the research performed by Halden HTO contributes to the technical basis for the NRCs primary regulatory guidance on HFE. Finally, Halden HTO facilitates cooperation and technical information exchange among the participating countries.
The NRC staff will, therefore, continue to participate in Halden HTO during the upcoming agreement period, both to leverage the projects research benefits and to promote international cooperation and information exchange on regulatory research.
The NRC also leverages the coordination among the Halden HTO and other NEA working groups. For example, the Halden HTO is supported from NEA by the same technical secretary supporting the NEA Working Group for Human and Organizational Factors (WGHOF). Many of the active member organizations in WGHOF are also part of the Halden HTO, including NRC.
The NRC support for both the Halden HTO and WGHOF is coordinated out of the human factors and reliability branch (HFRB) in RES Division of Risk Analysis (DRA).
1-5 1.3 Research Areas Addressed by Halden HTO During the 2021-2023 program cycle, Halden HTO addressed seven primary research areas:
(1)
Human Performance (2)
DI&CSafety Assurance (3)
Control Room Design and Evaluation (4)
Human-Automation Collaboration (5)
Digital Systems for Operations and Maintenance (6)
Digital Transformation of Decommissioning (7)
Cybersecurity for Main Control Rooms The NRC derives direct, ongoing benefits from research in these areas.
1.4 Research Capabilities Offered by Halden HTO Halden HTO conducts a wide range of studies on the behavior and performance of control room operators. To conduct these studies, Halden researchers have access to an internationally recognized, state-of-the-art collection of research laboratories and equipment; a global pool of licensed reactor operators, mainly from the United States and Sweden, to participate in these studies; and a full-time, dedicated staff of simulator engineers and software developers to support their facilities. Some of the research capabilities of Halden HTO are described below.
1.4.1 Halden Man-Machine Laboratory (HAMMLAB)
HAMMLAB (https://ife.no/en/laboratory/hammlab/) is equipped with an experimenters gallery, which allows researchers to control simulation scenarios, record data, and simulate the roles of field operators or other plant personnel communicating with the main control room. HAMMLAB's data recording capabilities include microphones for audio recording, cameras for the control room and individual offices, screen capture for presentation displays, and event logs for all operator actions. Teams are typically trained for 1.5 days on study procedures. The scenarios are specifically designed to target factors of interest such as cognitive dependence, decision-making under uncertainty, and various human performance variables.
1.4.2 Pressurized-Water Reactor Simulator In HAMMLAB, Halden HTO uses a generic pressurized-water reactor (PWR) simulator based on a three-loop design, with an output of approximately 969 megawatts of electricity. The PWR simulator includes an individual operators desk and screens for the team leader and team technical advisor, facilitating complete control and monitoring.
1.4.3 Halden Small Modular Reactor Simulator (HSMR)
In addition to the PWR simulator, Halden HTO also operates a small modular reactor (SMR) simulator, known as the HSMR (https://ife.no/en/service/cutting-edge-smr-simulator-and-expert-services). Installed in summer 2022, the HSMR is a natural-circulation PWR design operating at a capacity of 50 megawatts of electricity. The HSMR can operate from 1 to 12 units at a time, although it is currently configured with 6 units. This simulator allows for the study of issues relevant to multi-unit installations, such as situational awareness or unit confusion.
1-6 1.4.4 Halden Human-Machine Satellite Lab (HAMMSAT)
Halden HTO also operates the Halden Human-Machine Satellite Lab (HAMMSAT), a smaller scale simulator with a single-operator station. This simulator allows for studies where a single operator controls a single unit. The HAMMSAT setup provides valuable insight into the dynamics and challenges of single-operator control.
1.4.5 Human Automation Lab (HA-Lab)
Established in 2019, the Human Automation Lab (HA-Lab) (https://ife.no/en/laboratory/ha-lab/)
brings together a diverse team to conduct applied research on the possibilities for interaction, collaboration, and teaming between humans and advanced technologies in digital and complex environments. The HA-Lab leverages over 30 years of experience in human factors and human-technology interaction research across various domains. The multidisciplinary group includes experts in electronics engineering, human factors, cognitive psychology, and machine learning. HA-Lab researchers explore how humans can effectively interact and collaborate with automated technologies in diverse scenarios and settings.
1.4.6 Halden Virtual Reality Centre (HVRC)
The Halden Virtual Reality Centre (HVRC)
(https://ife.no/en/laboratory/halden-virtual-reality-centre/) provides facilities for identifying and testing novel applications of emerging virtual reality (VR) and augmented reality (AR) technologies. The focus is on design, training, and optimization of safety-critical work in industrial environments. The proximity of HVRC to HAMMLAB enables joint experiments, with HAMMLAB used to simulate a control room while HVRC simulates activities in a digital twin of the facility being controlled.
1.4.7 Cybersecurity Centre The Cybersecurity Centre (https://ife.no/en/laboratory/cybersecurity-centre/), established in 2019, offers research, innovation, and consultancy services in cybersecurity. The facility draws upon decades of research and development expertise in operational safety and security for critical infrastructures. It brings together researchers and engineers from various disciplines, including security engineering, safety risk management, human factors, process engineering, and simulation, to develop holistic solutions for cybersecurity. Alongside HAMMLAB, the Cybersecurity Centre provides a unique environment for studying the potential for collaboration between security teams and control room staff in safeguarding organizations from cybersecurity threats. The Centre employs hardware-in-the-loop simulators for realistic testing, vulnerability assessment, incident response, and evaluation of human preparedness.
1.4.8 IFE FutureLab The IFE FutureLab (https://ife.no/laboratory/ife-fremtidslab/), established in 2012, serves as an arena for exploring future operational concepts within complex domains. Research at the FutureLab focuses on team organization, control environment layout, human-system interfaces (HSIs), and other support tools. Safety-critical industries worldwide are currently grappling with rapid technological advancements, including the development of increasingly capable systems to assist humans in various tasks. The FutureLab complements HAMMLAB by providing a flexible setup for exploratory research in this field. FutureLab activities include idea workshops, prototyping sessions, demonstrations, user feedback sessions, and evaluations. Equipped with
1-7 large, high-resolution multi-touch screens, portable devices, and a touch table, the lab can be reconfigured for diverse control environments. Research topics span human-automation collaboration and centralized control of multiple units, such as reactors or oil platforms.
Additionally, the lab employs a dedicated feasibility test methodology that evolves alongside research activities, with guidelines for prototypes, use cases, and feasibility studies.
In sum, Halden HTO has a variety of research capabilities, from large-scale simulators to smaller, single-operator configurations. These resources enable researchers to conduct comprehensive and focused studies on various aspects of reactor operation and control room design that are important to nuclear regulation.
1.5 Regulatory Applications of Halden HTO Research The research conducted under Halden HTO supports evidence-based, risk-informed regulatory decision-making in many ways, including informing guidance development efforts for new and advanced reactors and addressing challenging licensing issues. In addition to their facilities, the researchers at Halden HTO have access to a global pool of operators and have relationships with many plants, both in the United States and internationally. As a result, they have visibility into operator performance in both conventional and modernized plants that is otherwise unavailable to regulators. Appendix A-2, Infographic on the Significance of the NRC-Halden HTO Collaboration, summarizes the significance of the NRCs collaboration with Halden HTO.
1.5.1 Addressing Licensing Challenges The work conducted by Halden HTO helped the NRC staff understand and address some of the challenges in developing a multistage validation (MSV) process. The NRC conducts licensing reviews for operating reactor control rooms undergoing modernization, with analog components being replaced by new digital alarms, controls, and displays. These reviews have encountered unique challenges due to the limited availability of control room simulators for use in integrated system validation. Therefore, licensees and the NRC staff are now learning to use glass-top simulators during an MSV process. This tool makes the licensing process more flexible and helps develop regulatory confidence in the modified design earlier in the design process.
1.5.2 Guidance Development for Advanced Reactors In the United States, recent legislation has accelerated the need for performance-based, technology-independent regulatory practice. For instance, the Nuclear Energy Innovation and Modernization Act (NEIMA), enacted on January 14, 2019, requires the NRC to establish a regulatory framework supporting advanced nuclear reactors. Specifically, NEIMA section 103(a)(4) directs the NRC to create a technology-inclusive framework for new reactor license applications by December 31, 2027. The proposed rule, to appear as Title 10 of the Code of Federal Regulations (10 CFR) Part 53, Risk-Informed, Technology-Inclusive Regulatory Framework for Commercial Nuclear Plants (see SRM-SECY-23-0021, Proposed Rule:
Risk-Informed Technology-Inclusive Regulatory Framework for Advanced Reactors (RIN 3150-AK31), dated March 1, 2023; NRC 2023f), defines technology-inclusive, performance-based requirements that build on existing NRC regulations and risk-informed principles.
Even more recently, the Accelerating Deployment of Versatile, Advanced Nuclear for Clean Energy Act of 2023 (ADVANCE Act of 2023; U.S. Congress, 2023) requires the NRC to develop a process for timely licensing of nuclear production and utilization facilities and to coordinate
1-8 certain international nuclear activities. As previously mentioned, Halden HTO supports the NRCs efficiency goals related to preparedness for advanced reactors by leveraging international cooperation to develop the needed technical bases and guidance.
The Halden HTO research portfolio informed many of the NRC documents related to the proposed 10 CFR Part 53 rulemaking including the proposed 10 CFR 53.730 (c) requirement to submit, as part of combined and operating license applications, a facility concept of operations (SECY-23-0021; NRC, 2024c). The portfolio also informed the development of key guidance supporting human-system considerations for advanced reactors including the interim staff guidance (ISG) for staffing (DRO-ISG-2023-02: NRC, 2022a) and scalable HFE review (DRO-ISG-2023-03: NRC, 2022b). Specifically, the research performed by Halden HTO provides key technical inputs for the HFE reviewer guidance that the NRC is preparing for its reviews of advanced reactor applications. The NRCs HFE staff has incorporated numerous insights from Halden HTOs research into the development of 10 CFR Part 53 and supporting guidance for advanced reactors. Typically, advanced reactors that are designed to meet the Policy Statement on the Regulation of Advanced Reactors (NRC, 2008) may be smaller than conventional reactors and may rely on passive safety systems, rather than on active systems monitored and operated by humans. Use of passive safety features, such as natural circulation, may result in few operator actions, similar to advanced reactor designs described in the NRCs advanced reactor policies and guidance including the NRC Standard Review Plan 19.3, Regulatory Treatment of Nonsafety Systems for Passive Advanced Light Water Reactors (NRC, 2013), SECY-94-084, Policy and Technical Issues Associated with the Regulatory Treatment of Nonsafety Systems in Passive Plant Designs, (NRC, 1994a) and associated SRM-94-084 (NRC, 1994b). The applicable regulations and guidance must therefore be flexible enough to account for the changing role of operators, while still ensuring that operators have the appropriate HSIs, training, procedures, and tools needed for safe plant operation. The NRC staff is actively developing guidance on a range of human factors topics, including remote and autonomous operation. The Halden SMR simulator and the remote HAMMSAT capability were established during the 2021-2023 period and will inform these guidance development activities (see sections 1.4.3 and 1.4.4).
Furthermore, Halden HTOs research in DI&C safety assurance will enable performance-based assurance that is independent of the reactor technology. Many of the projects in this area focus on SMRs, AI, or remote and autonomous operations, which are all anticipated to be relevant to future advanced reactors. More broadly speaking, high levels of automation are anticipated both in advanced reactors and in the modernization of existing control rooms. Halden HTOs research on advanced automation and automation failure may contribute to the technical basis for reviewing automation technologies in these contexts.
1.5.3 Human Factors Engineering Guidance Development Over the years that the NRC has been a member organization, Halden HTOs research activities have played a crucial role in updating key NRC HFE regulatory guidance documents.
These include NUREG-0700, Human-System Interface Design Review Guidelines, Revision 3, issued July 2020, and NUREG-0711, Human Factors Engineering Program Review Model, Revision 3, issued November 2012. Revision 4 of NUREG-0700, which was under development during the 2021-2023 research cycle, is anticipated to be released in 2025. The research conducted during both the 2021-2023 and 2024-2026 periods is expected to inform the development of the next revision of NUREG-0711 (Rev. 4), with work on this beginning in 2024.
NUREG-0700 provides guidelines for reviews of detailed designs of the main control room and other HSIs in NPPs, to ensure that the HSIs accommodate human capabilities and limitations in
1-9 accordance with HFE guidelines. The criteria in NUREG-0700 address both physical and functional characteristics of HSIs (NRC, 2020b). NUREG-0711 contains guidelines for reviews of HFE programs, describing the 12 elements of an HFE program and the review criteria for each element. As part of the review process, NRC reviewers evaluate the interfaces between plant personnel and the plants systems and components for conformance with HFE guidelines (NRC, 2012).
NRC HFE subject-matter experts have confirmed that collaborations with Halden HTO supported updates made in developing the following sections of NUREG-0700, Revision 3:
Section 3, Analog Display and Control Devices Section 4, Alarm System Section 5, Safety Parameter Display System Section 9, Automation System o Section 9.7 Computerized Operator Support Systems (COSS)
Section 11, Workstation Design Section 12, Workplace Design Section 14, Degraded HSI and I&C Conditions Halden HTO activities are also informing the development of NUREG-0700, Revision 4, including updates to the following sections:
Section 1, Information Display Section 3, Analog Display and Control Devices Section 7, Soft Control System Section 10, Communication System Section 15, HSI Integration (a new section based on display functions)
Section 6, Group-View Displays (revised as part of the overall development of section 15; the guidelines in section 6 of Revision 3 focus on physical large-panel displays, so that section will be retitled to Large-Panel Display System to reflect the change)
Similarly, Halden HTO activities contributed to the ongoing updates of NUREG-0711. Previous updates saw contributions from Halden HTO, particularly in the following sections of NUREG-0711, Revision 3:
Section 4, Functional Requirements Analysis and Function Allocation Section 7, Treatment of Important Human Actions Section 8, Human-System Interface Design Section 11, Human Factors Verification and Validation Additional Information sections for some of the review criteria The next update to NUREG-0711, Rev 4, is beginning in 2024 and will benefit from the research activities of Halden HTO spanning 2021 to 2023 and beyond. See also section 1.5.2 for related human factors guidance development activities for advanced reactors (e.g., Scalable HFE Reviews (DRO-ISG-2023-03: NRC, 2022b))
1-10 1.5.4 Safety Assurance, Digital Instrumentation and Control, and Cybersecurity Guidance Development Halden HTOs research supports a multifaceted approach to addressing safety and security assurance of DI&C systems and devices in NPPs. Emphasis is placed on enhancing risk assessments, hazard analyses, and cybersecurity measures to maintain the safety and security of digital systems. Projects include applications of structured safety argumentation (SSA) to refine safety assurance processes, reviews of existing guidance and standards for terminology disambiguation, and the development of new methodologies for evidence evaluation.
Additionally, Halden HTO projects have addressed the integration of artificial intelligence (AI),
machine learning, and augmented reality (AR) into nuclear safety practices to support decision-making, advance techniques for human factors validation, and improve incident response capabilities. Research in these areas may also support the development of best practices and recommendations, which may inform both licensee guidance (e.g., regulatory guides) and guidance for the NRC staff (e.g., on conducting reviews using NUREG-0800, Standard Review Plan for the Review of Safety Analysis Reports for Nuclear Power Plants:
LWR Edition; NRC, 2023g). The collaboration between the NRC and Halden HTO is thus providing valuable insights for maintaining safety standards and updating regulatory frameworks as digital and automated systems become increasingly integrated into NPPs.
2-11 2 PROJECT OVERVIEWS This chapter details Halden HTOs 2021-2023 research activities in seven areas:
(1)
Human Performance (2)
DI&CSafety Assurance (3)
Control Room Design and Evaluation (4)
Human-Automation Collaboration (5)
Digital Systems for Operations and Maintenance (6)
Digital Transformation of Decommissioning (7)
Cybersecurity for Main Control Rooms For each activity, the subsections Halden HTO Objective, Halden HTO Results, and Halden HTO References are quoted from the report Halden HTO Program Achievements 2021-2023 (Halden HTO, 2024). The subsection, NRC Significance, describes the NRCs assessment of the regulatory significance of the research activity. Chapter 3 contains further details on research activities deemed highly beneficial to NRC regulatory guidance.
2.1 Human Performance Research on human performance in NPPs is intended to support safe operation, safety analysis, and design development by producing new knowledge about how technology and organizational structures affect human performance. For example, researchers may examine how computer-based procedures (CBPs) affect human reliability. Such research can help show whether and how HRA methods can be used to analyze modern systems and to guide new safe designs. Further study is still needed on general human factors issues such as teamwork and crew roles.
2-12 2.1.1 Operator Performance in Digital Control Rooms Table 2-1 Halden HTO Objective, Results, and References for Operator Performance in Digital Control Rooms*
Halden HTO Objective This activity studies the effects of industry-standard digital control room technologies, changes in operator roles, and new conducts of operation on operator performance during emergencies.
Halden HTO Results The main empirical activity during the reporting period was a data collection at a nuclear new build site, focusing on changes in crew roles from a reader-doer style of procedure use to a model where the reactor operator oversees the procedure. Two crews were observed in three emergency scenarios during recurrent training. Audio/video data was recorded and simplified workload and system usability ratings were collected. Participants expressed a strong preference for the work style where the reactor operator both reads the procedure and executes actions, as opposed to the reader-doer work style used in conventional control rooms. Contrary to our expectations, reactor operators did not express uncomfortable or excessive workload when the responsibility of both reading the procedure and executing the action is on them. Crew dynamics and work distribution between operators at the controls and supervisors therefore remains an important research topic for digital control rooms.
The concept of operations for a digital control room in China was summarized in HTOR-025.
We continued research coordination with HTO member organizations, including a task modeling workshop conducted in October 2023 to look at feasibility and requirements for modeling low-level task performance. An expert panel on the topic Human Factors in New Nuclear was organized at the 2023 Human Factors and Ergonomics Society conference.
Halden HTO References HTOR-025: Song, F., et al. (2022). The Concept of Operations in a Digital Control Room.
HTOR-068: Hildebrandt, M., McDonald, R. (2023). Operator performance in digital control rooms: Insights from on-site observation of simulator training.
HWR-1313: Hildebrand, M., McDonald, R. (2020). Procedure Use, Communication, and Teamwork in a Digital Control Room.
Bye, A. (2023). Future needs of human reliability analysis: The interaction between new technology, crew roles and performance. Safety Science 158:105962.
Note: Contents of this table have been excerpted from the report Halden HTO Program Achievements 2021-2023 (Halden HTO, 2024). For Halden HTO reports (HTORs) referenced in this report and published within the 2021-2023 program period, see appendix A-1.
2-13 2.1.1.1 NRC Significance The results of this work illustrate how new technologies may affect the conduct of operations at an NPP; they are of particular interest for human factors and reliability analysts. The activities performed provided an onsite demonstration of operator responses to the introduction of new technologies, such as CBPs, which allowed researchers to observe changes in operator roles and the conduct of operations. The observational data and resulting task modeling enabled the systematic exploration of operator behavior, providing empirical information on human reliability issues in control rooms that use DI&C. In addition, the task modeling performed created a basis for systematic HRA data collection in digital control rooms. Chapter 3 offers more details on the significance of this work to the NRC.
2-14 2.1.2 Crew Factors, Teamwork, and Role Independence in Control Rooms Table 2-2 Halden HTO Objective, Results, and References for Crew Factors, Teamwork, and Role Independence in Control Rooms*Halden HTO Objective In nuclear power plant control rooms, the operators work together as a team where the crew members hold specific roles and responsibilities. Conduct of operations is crucial for the performance of the control room team, where procedures, task distribution, crew roles, coordination and supervision are elements that guide the overall functions and tasks of plant operations. Despite different organizational mechanisms to manage and support teamwork, the roles of the crew members and the way the team works together interacts with many different factors which can affect and potentially counteract the benefits from collaborative work and decision-making in the control room.
The overall objective of this activity is to study crew factors and their impact on crew performance. The specific objective is to establish knowledge regarding group processes such as trust, groupthink, as well as to look further into the independency of the shift technical advisor (STA) role.
Halden HTO Results A review on the topic of trust was performed. It covers definitions of trust, trust in humans and trust in automation, how trust has been measured, and factors that often have been seen to correlate with trust (HTOR-010). A digital questionnaire was developed and distributed to operators on their views on remote supervision microreactors and their trust toward such technology. The findings were presented in a paper prepared for ESREL-2022. Groupthink has been an influential concept related to poor decision-making processes in teams. A review on groupthink has been conducted, exploring what groupthink is and how the phenomenon has been studied empirically to get input to how, if possible, groupthink can be avoided (HTOR-049). The HAMMLAB study investigating STA independency has been completed (HTOR-050).
Halden HTO References HTOR-010: Kaarstad, M. (2022). A review of theories, models and measures of human-automation trust.
Kaarstad, M., McDonald, R. (2022). Microreactors: Challenges and Opportunities as Perceived by Nuclear Operating Personnel. Proceedings of ESREL 2022.
HTOR-049: Kaarstad, M. (2023). Groupthinka literature review.
HTOR-050, Rev.2: Kaarstad, M., Nystad, E., McDonald, R., Odéen, D. (2023). Physical location of shift technical advisor in nuclear power plant scenarios - impact on performance?
Note: Contents of this table have been excerpted from the report Halden HTO Program Achievements 2021-2023 (Halden HTO, 2024). For HTORs referenced in this report and published within the 2021-2023 program period, see appendix A-1.
2-15 2.1.2.1 NRC Significance The results of this activity provide useful information on how new technologies (e.g., remote supervision of microreactors) may affect crew factors, the conduct of operations, and staffing, all of which are significant issues in licensing reviews for new, advanced, and modernized reactors.
Researchers reviewed crew factors such as trust, groupthink, and shift technical advisor (STA) independence. They obtained novel data on roles and responsibilities, which let them further examine the impact of new technologies on the conduct of operations and the role of STA. Their results also provided a technical basis for novel concepts of operations (CONOPs) in advanced reactors, where engineering expertise may play a different role than it does in currently operating plants or may be combined with other staffing roles. Chapter 3 offers more details on the significance of this work to the NRC.
2-16 2.1.3 Decision-Making under Uncertainty Table 2-3 Halden HTO Objective, Results, and References for Decision-Making under Uncertainty*
Halden HTO Objective This activity aims at providing a coherent description of the decision-making strategies used by the operators in situations of uncertainty. Control room operators make discretionary decisions under uncertainty, due to e.g., the meaning of procedure steps, the interpretation of indications, or the causes of malfunctions. The highest level of uncertainty occurs in unpredicted, unstructured, and ambiguous situations. The operators respond by adopting mental strategies (heuristics) that simplify and accelerate judgment and choice.
Halden HTO Results The literature review covered heuristics across different approaches in psychology (Judgment and Decision-Making, Ecological Rationality) and human factors (Naturalistic Decision-Making, Cognitive Systems Engineering (CSE). Following the review, we scoped the activity under the CSE approach, since it focuses on supporting discretionary decisions in real-work settings and explicitly considers heuristic decision-making. The HAMMLAB data collection was performed in August-September 2023 with three U.S. crews on a 150 minutes-long scenario with several malfunctions (including small feedwater steam line leak, steam generator tube rupture, and station blackout). This was analyzed in detail by a multidisciplinary team of two control room operation experts and a human factors specialist.
The operators heuristic decisions are described within the frame of an original information-flow map (a CSE way of modeling decision strategies) adapted to the studied domain. The model stresses the active nature of operators decision-making insofar the operators continually test their hypotheses about the situations through actions that create new data and expected effects on the plant. The model explicitly includes the procedures as an important source for hypothesis generation.
Halden HTO References HTOR-016: Massaiu, S. and Pignoni, G. (2022). Literature Review and Study Plan for the HTO-1.3 Activity Decision-Making under Uncertainty.
Massaiu, S. (2023). Operator Decision Strategies in Nuclear Control Centres: A Domain-Specific Information Flow Map. Proceedings of ESREL 2023.
HTOR-039: Massaiu, S. (2023). Adaptive decision-making in control room emergencies:
Results of a HAMMLAB study with the HPWR3 simulator.
Note: Contents of this table have been excerpted from the report Halden HTO Program Achievements 2021-2023 (Halden HTO, 2024). For HTORs referenced in this report and published within the 2021-2023 program period, see appendix A-1.
2-17 2.1.3.1 NRC Significance This work contributes to the NRC staffs understanding of the behaviors, challenges, and strategies of operators faced with uncertainty. Scenario data captured operators deviations from standard procedures in circumstances in which the information available to the operators was incomplete or degraded. Such data are particularly relevant as input for the NRCs development of ways to evaluate risk during emergency events (e.g., station blackouts resulting in loss of CBP systems). Based on their data, the researchers created a model of decision-making strategies that captured these operators heuristic decisions in relation to procedures in an NPP. Chapter 3 offers more details on the significance of this work to the NRC.
2-18 2.1.4 Event Investigations: The Added Perspective of Successes Table 2-4 Halden HTO Objective, Results, and References for Event Investigations: The Added Perspective of Successes*
Halden HTO Objective Learning from events is important for upholding and improving safety in nuclear facilities. The methods used to investigate and learn from events focus on analyzing and correcting the failures that caused the events, based on a long history of research recognizing that failures generate rich insights for learning. However, more recent research indicates the potential to learn from successful performance in nuclear facilities, i.e., actions that achieved their intended or desired objectives. This raises the question, can learning be enhanced by examining both failures and successful performance in nuclear facility event investigations?
Halden HTO Results Research conducted in the activity supports the idea that closer analysis of the successful performance displayed during the progression of an event could contribute to new knowledge and learning from event investigations. It provides practical guidance to help event investigation teams better identify and analyze successful performance displayed during the progression of an event. Additionally, it examines the potential of more systematic interventions to help bring successful performance into the scope of an event analysis.
Researchers in this activity also co-led a survey study for the NEA-CNSI WGHOF collaborative project, Good practices for investigators on identifying human and organizational factor (HOF) issues from event analysis processes, seeking to identify the practices used to investigate HOF-related events.
Halden HTO References HTOR-014: Solberg, E. and Bisio, R. (2022). Event Investigations: The Added Perspective of Success.
Solberg, E. and Bisio, R. (2022). Improving learning by adding the perspective of success to event investigations (ESREL 2022).
HTOR-031: Solberg, E., Bisio, R., Kwei-Narh, P. (2023). Learning from Successful Performance in Failure Events - A Case-Based Study.
HTOR-046: Solberg, E., Kwei-Narh, P., Bisio, R. (2023). Including successful performance in the scope of the event analysis.
Solberg, E., Kwei-Narh, P., Bisio, R. (Under review for ESREL 2024) Including successful performance in an events causal analysis: test of an instructional intervention.
Park. J. and Solberg, E. (2023). Development of detailed questions for investigating the status of human and organizational factor (HOF) issue identifications from event investigation processes (ESREL 2023).
2-19 Note: Contents of this table have been excerpted from the report Halden HTO Program Achievements 2021-2023 (Halden HTO, 2024). For HTORs referenced in this report and published within the 2021-2023 program period, see appendix A-1.
2.1.4.1 NRC Significance The results of this work may be useful for identifying HFE-related safety issues and for developing recommendations on implementing the conduct of operations in a control room. The analysis methodology used by the event investigation teams focuses on both successes and failures (Solberg and Bisio, 2022). This enhanced viewpoint may help the NRC obtain empirical evidence on operators recovery from failures. It may also support NRC evaluations of inspection criteria and corrective actions as part of future improvements to the Reactor Oversight Process. In addition, this work may be relevant to the updating of the evaluation criteria in NUREG-0711 to support licensing activities for advanced reactors. For instance, the NUREG-0711 section on operating experience currently focuses on identifying HFE-related safety issues and preventing the recurrence of past negative events, with links to prescriptive documents based on the extensive operating experience available for large light-water reactors (NRC, 2012).
2-20 2.1.5 The Human Performance Data Repository Table 2-5 Halden HTO Objective, Results, and References for the Human Performance Data Repository*
Halden HTO Objective The overarching goal of this activity is to improve sharing and dissemination of empirical findings and data among the Halden Project members. The main activity is to develop a data repository of human performance data and information produced by the Halden Reactor and Halden HTO projects. A second related activity is to support international collaborative projects with project members for improving exchange of human performance and reliability data.
Halden HTO Results The scope of the project evolved along the years, eventually abandoning the initial ambitious goal of populating the repository with human performance data sets. After exploring technical solutions including data organization (ontology), database population, text analysis, search engine implementation, user interface development, and data security integration) the activity has developed a web-based portal for searching and retrieving research reports from two Halden Project series: the Man-Technology-Organization Halden Work Reports (MTO-HWRs) from 1988 until 2020, and the Halden Human-Technology-Organisation Project reports (HTORs) from 2021.
The activity has also supported the following international collaborative initiatives: (a) the EPRI led development of a taxonomy and meta-analysis template for operators/control room features, (b) the CANDU Owners Group plant-wide data and knowledge management, and (c) the international collaboration on human reliability analysis (HRA) for SMRs with KAERI, Idaho National Laboratory (INL) and the U.S. NRC.
Halden HTO References Presley, M. Boring, R., Ulrich, T., Medema, H., Mohon, J., Delvecchio, M., Massaiu, S., Bye, A., Park, J., Kim, Y and Julius, J. (2021). A Taxonomy and Meta-Analysis Template for Combining Disparate Data to Understand the Effect of Digital Environments on Human Reliability. Proceedings of the ANS 2021 International Topical Meeting on Probabilistic Safety Assessment and Analysis (PSA 2021).
Bisio R., Massaiu S. (2021). An Approach to Sharing Human Performance Data and Findings in the International Nuclear Research Community. Proceedings of ESREL 2021.
HTOR-057: Bisio, R. E. (2023). Human Performance Repository: Lessons Learned and Experience in Publishing the site for Halden Reports search.
Note: Contents of this table have been excerpted from the report Halden HTO Program Achievements 2021-2023 (Halden HTO, 2024). For HTORs referenced in this report and published within the 2021-2023 program period, see appendix A-1.
2-21 2.1.5.1 NRC Significance This work is relevant to HRA and human factors research. The web-based portal is a great first step in making the Halden human performance data more accessible to human factors, HRA, and PRA experts at the NRC. The portal provides nuclear-specific quantitative data, which complement the more readily available information arising from engineering judgment and research in nonnuclear domains. This activity may be useful for validating new technologies and improving HRA. The collection of microtask data serves as a point of interest in collaborations between Halden HTO, KAERI, and the INL to examine HRA in the context of emerging technologies and SMRs. The HFE and HRA staff at the NRCs RES are participating periodically in these international collaborations to leverage opportunities for research and data collection on topics of common interest.
2-22 2.2 Digital Instrumentation and ControlsSafety Assurance DI&C research is focused on ensuring the security of digital systems used in NPPs, which in turn supports plant safety. DI&C systems provide vital data to operators and help manage safety protocols, among other functions. Researchers in this area conduct risk assessments and hazard analyses to establish robust cybersecurity measures to protect against cyber threats. As digital technology evolves, DI&C research will become increasingly vital in supporting licensing reviews.
2-23 2.2.1 Risk-Informed Safety Assurance Table 2-6 Halden HTO Objective, Results, and References for Risk-Informed Safety Assurance*
Halden HTO Objective To deliver effective and practical support for DI&C safety assurance efforts. The target audience is the practitioners and researchers involved in safety assurance activities or research. The work intends to improve their shared understanding and thus improving the validity of characterizations and reducing engineering deficiencies.
Halden HTO Results Research activities of the period focused on the practical application of Structured Safety Argumentation (SSA). One activity applied the guidance for the SSA approach on the Halden Safety Fans (HSF) system specification which resulted in revision for both the SSA guidance and the HSF specification. Another activity investigated how implicit safety related information in the Institute of Electrical and Electronics Engineers (IEEE) 603 standard could be made explicit. This activity explored the use of ontology, risk-insight augmentation, and SSAs to reveal the safety intent behind the clauses of the standard. The third activity focused on the assessors view on safety assurance, i.e., knowing what the assessor looks for when evaluating the safety of an application, which resulted in an SSA case study using the original design control document of the APR1400 application at U.S. NRC. These activities were performed in close collaboration with U.S. NRC and led to a U.S. NRC call for further research about a domain-specific framework for a performance-based SSA approach. The fifth and last activity applied System Theoretic Process Analysis (STPA) on the HSF design and compared it to the HAZOP4 of HSF for a more complete hazard landscape.
Halden HTO References HTOR-004: Gao, X. & Karpati.P. (2022). A study on the application of the Structured Safety Argumentation Approach Guidance on the Halden Safety Fan.
HTOR-005: Karpati. P., Gao, X., Carte, N.C. (2023). Connecting safety standards to safety.
HTOR-008: Gran, B.A. & Gao, X. (2022). Halden Safety FanHAZOP and revised system specification.
HTOR-032: Shukla, A. & Gao, X. (2023). Halden Safety Fanapplication of STPA.
Note: Contents of this table have been excerpted from the report Halden HTO Program Achievements 2021-2023 (Halden HTO, 2024). For HTORs referenced in this report and published within the 2021-2023 program period, see appendix A-1.
4 HAZOP stands for hazard and operability study, which is a systematic method for identifying potential hazards and problems in complex industrial processes.
2-24 2.2.1.1 NRC Significance To enable migration from the current compliance-oriented practice of safety evaluation to a performance-based approach, the hazardous conditions implicit in NRC-referenced standards must be made explicit. Halden HTOs case study of IEEE 603, IEEE Standard Criteria for Safety Systems for Nuclear Power Generating Stations, helped the NRC understand the challenges in this effort (e.g., the need to disambiguate the vocabulary). Chapter 3 offers more details on the significance of this work to the NRC. The experience gained in this activity about STPA and safety assurance cases was useful in activities for safety analysis and assurance in the Digital Transformation of Decommissioning activities. For details on the collaboration between NRC and the Halden HTO on this topic, see section 3.4 for a description of the workshops NRC supported in collaboration with the Halden HTO that resulted in the development of a domain-specific framework for a performance-based SSA and enhanced state of the art safety assurance knowledge. This experience will also be useful in future research on applying robotic automation for hazardous tasks in NPPs. Chapter 3 offers more details on the significance of this work to the NRC.
2-25 2.2.2 Evidence Collection, Evaluation, and Combination for Safety Assurance Table 2-7 Halden HTO Objective, Results, and References for Evidence Collection, Evaluation, and Combination for Safety Assurance*
Halden HTO Objective The main objectives are to identify the state-of-practice support within the nuclear industry for classifying, categorizing, combining, and evaluating evidence suitability to support main safety or security claims in DI&C systems, and to identify state-of-the art related to classifying, categorizing, combining, and evaluating evidence as support for claims and requirements.
One aim is also to develop guidance and technically sound basis for producing or reviewing evidence to support an assurance claim. The guidance and technical basis should provide general recommendations on e.g., how to collect evidence, main quality characteristics, conditions for unambiguous information exchange and correct integration, and how to evaluate and communicate the safety or security assurance.
Halden HTO Results A literature review initiated a list of parameters and best practices to produce evidence and documentation for a safety demonstration. The observation was made that there is no clear guidance or template to follow in regards of the how to best produce and present your evidence to demonstrate your safety claim. Some best-practice and good habits were described to improve the safety demonstration documentations, including planning time and resources needed to produce the evidence, communication, and early dialogue with assessors. An evaluation of what requirements are sufficient for producing evidence was done in the analysis of the HARDENS case, a safety demonstration report based on a reactor protection system (RPS). The results discussed criteria to assess the evidence in a possible safety demonstration based on literature review, former research, and standards on safety demonstrations.
Workshop sessions on state-of-the-art of DI&C safety assurance were performed with leading international experts. A list of gaps and updated literature utilized in DI&C system is addressed in the proceedings.
Halden HTO References HTOR-006: Ayele, Y.Z., Esnoul, C. (2022). Independently Verifiable Evidence for Safety Assurance: Results of Expert Survey and Literature Review.
HTOR-023: Esnoul, C., Simensen, J.E., Ayele, Y. Z., Sechi, F., Gao, X. (2022). HTO Workshop proceedingsRiskand EvidenceInformed Safety Assurance for Digital Instrumentation and Control (DI&C).
HTOR-033: Sarshar, S., Esnoul, C., Gao, X., Karpati, P., Mikkelsplass, S.A. (2023).
Proceeding of HTO Workshop series - Safety Assurance through Correct-by-Construction Techniques.
2-26 Note: Contents of this table have been excerpted from the report Halden HTO Program Achievements 2021-2023 (Halden HTO, 2024). For HTORs referenced in this report and published within the 2021-2023 program period, see appendix A-1.
2.2.2.1 NRC Significance In a performance-based paradigm for safety evaluation, a broader range of evidence would be admissible compared to conventional evaluations, and a logical organization would be needed to enable the reviewer to reach the safety conclusion (known as a safety assurance case or a safety case). Halden HTOs case study has helped the NRC understand the challenges it faces in developing a performance-based evaluation standard. Furthermore, Halden HTO-conducted workshops have shown the NRC staff the state of the art and the gaps to be addressed before performance-based evaluation is feasible. This understanding led to the formulation of a follow-on NRC research project to acquire state of the art safety assurance knowledge through a series of workshops conducted in collaboration with the Halden HTO. Chapter 3 offers more details on the significance of this work to the NRC.
HTOR-034: Gran, B.A., Esnoul, C., Karpati, P. (2023). Evidences for the HARDENS Case.
2-27
2-28 2.3 Control Room Design and Evaluation HFE is pivotal for ensuring safe and efficient operations at nuclear facilities. Current practices, as outlined in design guidance and regulatory documents such as NUREG-0711, provide a framework for integrating HFE considerations into both modernizations and new builds within the industry. However, updates to such documents are needed as technology advances. In view of the changing role of operators, the new draft rule 10 CFR Part 53 and supporting guidance establish regulations that are more flexible while still ensuring that operators have the HSIs, training, procedures, and tools needed for safe plant operation. The NRCs HFE staff is developing guidance on a range of human factors topics, including remote and autonomous operation.
For example, research is now being conducted on the use of large-panel displays (LPDs) and group-view displays (GVDs) in control rooms. Researchers are studying how LPDs and GVDs can affect human performance and situational awareness, particularly in multi-unit environments such as SMRs, as well as how they can mitigate the challenges of managing multiple units simultaneously. This work may lead to the development of new guidelines for optimizing operator support through advanced overview displays.
2-29 2.3.1 Lessons Learned on Control Room Validation Table 2-8 Halden HTO Objective, Results, and References for Lessons Learned on Control Room Validation*
2-30 Halden HTO Objective The objective is to improve human factors validation guidance by providing principles and insights from HRP/MTO research from the period 2000-2020 and from nuclear industry experts. The research addresses main validation challenges reported by the nuclear industry.
Halden HTO Results The HTOR-020 revealed several challenges of using currently available measures of situation awareness (SA), teamwork, and cognitive workload. Industry practice relies on expert observer assessment supported by task performance checklists. The HTOR-021 identified eleven types of workload measures utilized in nuclear control room settings. The review revealed limited focus on defining and measuring limits of workload (e.g., acceptable zones).
The HTOR-035 found limited support for the validity of common situation awareness measures and proposed to improve this situation by nuclear specific research.
Correspondingly, the HTOR-071 showed that state of science provides limited support for assessing cognitive overload and loss of situation awareness and points to ways forward.
Figure 2-1 Hypothesized behavioral indicators patterns in relation to excessive cognitive workload (Braarud and Pignoni, 2023)
2-31 Halden HTO References HTOR-020: Braarud, P.Ø. and Pignoni, G. (2022). Human factors validation of control roomsinsights and experiences from nuclear industry projects.
HTOR-021: Braarud, P.Ø. (2022). Measuring Cognitive Workload in the Nuclear Control Room: A Review.
HTOR-035: Braarud, P.Ø. & Pignoni, G. (2023). Review of Situation Awareness Measures Applied in the Halden Human-Machine Laboratory.
HTOR-071: Braarud, P.Ø. & Drøivoldsmo, A. (2023). The assessment of cognitive workload and situation awareness in Human Factors Validation.
Note: Contents of this table have been excerpted from the report Halden HTO Program Achievements 2021-2023 (Halden HTO, 2024). For HTORs referenced in this report and published within the 2021-2023 program period, see appendix A-1.
2.3.1.1 NRC Significance This work provides useful insights for developing integrated system validation criteria. The researchers identified gaps and limitations in existing measures of workload and situational awareness, and they proposed future directions for nuclear-specific research. Chapter 3 offers more details on the significance of this work to the NRC.
2-32 2.3.2 The Impact of Overview Displays on Human Performance Table 2-9 Halden HTO Objective, Results, and References for the Impact of Overview Displays on Human Performance*
Halden HTO Objective Assumed advantages of overview displays include providing an instant overview of the plant state, guiding operators to detailed process displays, and supporting crew coordination.
Despite these presumed advantages, there is a concern that such displays might add complexity by introducing an extra layer of process information. These concerns, unaddressed in previous research, highlight the need for a systematic evaluation of the impact of overview displays on human performance, motivating our empirical investigation.
Halden HTO Results The research comprises five user studies with 28 participants, including certified nuclear operators. Within this activity, we enhanced the framework for identifying best practices in implementing overview displays, including an interview guide. Additionally, we proposed a structure and classification describing different types of overview displays, and their utilization in the nuclear industry. This structure is expected to enhance comparison and understanding of the variability of findings in empirical research.
The results from our study, which gathered objective measures of human performance, highlight the relevance of interface consistency in the control room to promote better performance. Based on the consolidated findings from a survey, along with previously collected data, we established a link between the perceived usefulness and usability of the overview displays in accordance with the Technology Acceptance Model predictions. Such models can guide further studies on human performance impact of overview displays.
Halden HTO References HTOR-001 rev.1: Braseth, A.O., Fernandes, A., and Bloch, M. (2023). A Framework for studying Overview Displays Best Practices.
HTOR-022: Bloch, M., Braseth, A.O., and McDonald, R. (2022). Overview Displays - Best Practices and Evaluation of Framework.
HTOR-030: Fernandes, A., Braseth, A.O. (2023). Overview Displays in Nuclear Control Rooms: A State of Practice Review HTOR-042: Fernandes, A., Bloch, M., and Braseth, A.O. (2023). Perceived usefulness and usability of overview displays in nuclear control rooms.
Note: Contents of this table have been excerpted from the report Halden HTO Program Achievements 2021-2023 (Halden HTO, 2024). For HTORs referenced in this report and published within the 2021-2023 program period, see appendix A-1.
2-33 2.3.2.1 NRC Significance This work provides a basis for guidance on large-panel displays (LPDs) and group-view displays (GVDs). The results of the research include a classification system for what Halden HTO refers to as overview displays. Additionally, the researchers recommend best practices for implementation. Halden HTOs work on this topic has informed the latest update to one of the NRCs primary HFE guidance documents, NUREG-0700 (e.g., Revision 4. See section 1.5.3 for details of planned revisions in this update), and is likely to contribute to the technical basis of future revisions to NUREG-0700 (NRC, 2020b). Chapter 3 offers more details on the significance of this work to the NRC.
2-34 2.3.3 Augmented Reality for On-Site Human Factors Control Room Assessment of Ergonomic and Regulatory Compliance Table 2-10 Halden HTO Objective, Results, and References for Augmented Reality for On Site Human Factors Control Room Assessment of Ergonomic and Regulatory Compliance*
2-35 Halden HTO Objective The objective is to provide an assessment and guidance on the use of handheld Augmented Reality (AR) technology to support on-site human factors control room design review and evaluation, including: 1) reviewing the state-of-the-art of AR technologies for mapping and measurement of nuclear control rooms, and 2) demonstrating how AR technologies can be used for human factors (HF) evaluation of control rooms using recommendations and design review guidelines in line with industry practice.
Halden HTO Results A research testbed utilizing AR technology has been developed to conduct feasibility and usability testing of new concepts related to human-centered design and HFE guidelines review of nuclear control rooms. It was aligned with typical use cases for designers and evaluators for HFE information collection. Four use cases were tested, including evaluating lines of sight, comparing control room layouts, optimizing control panel designs, and evaluating physical movement and access issues. The findings indicate that AR is a cost-effective alternative for HFE evaluations in control rooms, replacing the need for virtual or physical measurements with minimal or no preparation. This makes AR a valuable tool for control room modernization projects in a stepwise or multi-stage approach. Furthermore, AR's 2D interaction capability proved valuable for high-accuracy measurements, increasing precision in evaluating control room displays and panels. However, careful consideration of AR utilization is essential, weighing it against the flexibility needs, as other technologies like virtual reality offer more flexibility as it is not limited by physical boundaries.
Figure 2-2 Evaluation of sightlines on the large screen, on-site in HAMMLAB using AR Halden HTO References HTOR-040: Gustavsen, M., Louka, M.N., & Bach, P-C. (2023). Augmented Reality for On-Site Human Factors Control Room Assessment Note: Contents of this table have been excerpted from the report Halden HTO Program Achievements 2021-2023 (Halden HTO, 2024). For HTORs referenced in this report and published within the 2021-2023 program period, see appendix A-1.
2-36 2.3.3.1 NRC Significance While this work could support designers, especially in improving ergonomics assessments of control room environments, it was not initially identified as useful for the development of regulations or guidance. However, RES performed research on a related topic as part of the external crowdsourcing pilot of the National Aeronautics and Space Administration (NASA)
Tournament Labs and concluded that future research into the use of AR (but not VR) in the nuclear domain would be useful to the NRC. Thus, the NRC should consider the results of this Halden HTO activity in the event that it pursues follow-on work on human factors issues related to AR. In addition, some of this work dovetails with the ongoing development of guidance on alternative forms of validation for advanced reactors. It should, therefore, be considered in developing review guidance on AR validation techniques.
2-37 2.4 Human-Automation Collaboration Future reactor designs are likely to require significant changes to operational concepts, as these designs are expected to use advanced DI&C technologies, especially automation, as described in NRC (2008), NRC(1994a), and NRC (1994b), and build on new types of plant designs, such as SMRs. The introduction of highly automated technologies into the control room will create both opportunities and challenges related to safety and reliability. Thus, in both creating and evaluating future control room designs, it will be vital to understand the impact of advanced technologies on human performance. The activities described in this section, which are continuations of ongoing HTO activities on interlinked topics, explore increasingly sophisticated levels of control room automation and new plant designs in line with technology trends already evident in the industry worldwide.
2-38 2.4.1 Human Performance in Operation of Small Modular Reactors Table 2-11 Halden HTO Objective, Results, and References for Human Performance in Operation of Small Modular Reactors*
5 MWe stands for megawatts electric.
Halden HTO Objective Small Modular Reactors (SMRs) are nuclear reactors that generate 300 MWe5 or less and are intended to have a simpler modular design with inherent safety features, higher levels of automation, and a lower minimum staff needed to operate the plant. This activity intends to understand and evaluate the possible impacts of SMR control room design concepts on human performance.
Halden HTO Results HTOR-011 provides an update on recent developments in SMR concepts and research needs identified among Halden HTO Project members. Some of the main issues identified are staffing requirements for different SMR designs, the extent of automated functions and passive safety systems, and their implications for operation of the plants.
The Halden SMR simulator was established in 2022. This is an integral pressurized water reactor (iPWR) simulator capable of simulating up to twelve units with a maximum output power of 50Mwe each. Currently, six units are set up in HAMMLAB with two screens per unit and a large screen overview display.
Three subsequent data collections were conducted with licensed operators from the USA and Sweden (HTOR-029, HTOR-065). The purpose of the studies was to develop an understanding of operator monitoring strategies, prioritization of taskwork, and response to multi-unit failures. The participants were tested in teams of two and three in the SMR simulator.
The scenarios were designed to be increasingly demanding. The research team observed the ability to handle multiple planned tasks and disturbances at several units. The findings suggest that the participants relied on the overview display for cross-unit monitoring and preferred to reduce simultaneous tasks on multiple units whenever feasible. However, the operator roles and supervision strategies for multi-unit operation need to be further developed.
Halden HTO References HTOR-011: Blackett, C., Bloch, M., Pichancourt, I., Poret, C. & Jeffroy, F. (2022). Small Modular ReactorsState of the Art Review 2021.
HTOR-029: Blackett, C., Arigi, A., Eitrheim, M. H. R. & Fernandes, A. (2023). Human Performance in Operation of SMRs: Results from 2022 and 2023 Small Studies.
2-39 Note: Contents of this table have been excerpted from the report Halden HTO Program Achievements 2021-2023 (Halden HTO, 2024). For HTORs referenced in this report and published within the 2021-2023 program period, see appendix A-1.
2.4.1.1 NRC Significance This project demonstrated the possible effects of certain SMR control room features on human performance. The researchers characterized operator monitoring strategies and task prioritization under increasing workloads arising from multi-unit failures. The results of this work may be relevant to several ongoing NRC research activities, including experimental studies on remote and autonomous operations, as well as projects of the NEA's newly formed Working Group on Human and Organizational Factors, which is co-led by the NRC. Insights from the work may also support the guidance in NUREG-1791, Guidance for Assessing Exemption Requests from the Nuclear Power Plant Licensed Operator Staffing Requirements Specified in 10 CFR 50.54(m), issued July 2005, and in DRO-ISG-2023-02, Draft Interim Staff Guidance Augmenting NUREG-1791, Guidance for Assessing Exemption Requests from the Nuclear Power Plant Licensed Operator Staffing Requirements Specified in 10 CFR 50.54(m), for Licensing Commercial Nuclear Plants under 10 CFR Part 53, issued September 2022, which addresses advanced reactors (NRC, 2005; NRC, 2022a). Chapter 3 offers more details on the significance of this work to the NRC.
HTOR-065: Arigi, A. & Blackett, C. (2023). Effects of staffing on human performance in a multi-unit SMR simulator.
Four conference papers at PSAM, NPIC&HMIT, and AHFE.
2-40 2.4.2 Operator Performance in Highly Automated Plants Table 2-12 Halden HTO Objective, Results, and References for Operator Performance in Highly Automated Plants*
Halden HTO Objective We suspect that the advantages of effective and reliable automation will stimulate the nuclear industry to embrace the global autonomy trend. The Halden HTO Project has anticipated this development through proactive knowledge-building and human-automation interaction experiments. The purpose of this research activity is to investigate how new forms of automation may affect operator performance in highly automated plants.
Halden HTO Results We have established a laboratory dedicated to human-automation interaction studies (HAMMSAT)see the picture below. In this compact test environment, we have installed a single-unit variant of our SMR research simulator, which is ideal for realistic small-scale experiments with professional nuclear operators as participants.
Figure 2-3 Image of HAMMSAT Laboratory In a first experiment on operator responses to automation failure, the participants were individually exposed to a challenging scenario where a small steam header break was masked by aggressive turbine control automation, which overpowered the reactor and put the plant in a vulnerable state. That vulnerability was unmasked by the introduction of a steam generator tube rupture. The experiment demonstrated how intricate automation failures may overwhelm operators, as in the Boeing 737 MAX accidents.
Halden HTO References HTOR-052: Skraaning, G., Odéen, D & Jamieson, G. A. (2023). Operator Responses to Systemic Automation Failure: 1st HAMMSAT Experiment.
2-41 Note: Contents of this table have been excerpted from the report Halden HTO Program Achievements 2021-2023 (Halden HTO, 2024). For HTORs referenced in this report and published within the 2021-2023 program period, see appendix A-1.
2.4.2.1 NRC Significance This work demonstrates how a lack of available information affects the performance of personnel operating advanced autonomous systems. Further exploration of this topic could yield useful guidance on information transparency requirements, including insights into how information transparency affects operators perception of information, situational understanding, and response planning. The lessons learned from the researchers approximation (in the nuclear domain) of the automation challenges that occurred during the Boeing 737 MAX accidents could also be helpful. Chapter 3 offers more details on the significance of this work to the NRC.
HTOR-053: Jamieson, G. A. & Skraaning, G. (2023). Getting a Grip on Systemic Automation Failure.
2-42 2.4.3 Effects of Adaptive Automation on Operator Performance in Future Plants Table 2-13 Halden HTO Objective, Results, and References for Effects of Adaptive Automation on Operator Performance in Future Plants*
Halden HTO Objective As technological advances improve automation capabilities, it opens opportunities to introduce flexible automated agents in a control loop for operating a nuclear power plant. A change in working modalities can have big impacts on operator performance, posing new challenges and suggesting new ways of cooperating between human and machine. The objective with this activity is to understand potential benefits and risks of these changes and empirically evaluate the impact on human performance.
Halden HTO Results Potential capabilities offered by technology were explored, conceptualizing adaptive automation as artificial agents cooperating with operators (HTOR-007). The report included discussions on how effects on human performance can be empirically evaluated in our labs.
A workshop to share perspectives on adaptive automation was conducted (HTOR-024).
A prototype of adaptive automation was connected to the Halden PWR simulator to perform empirical studies. The prototype can support dynamic task allocation, perform on-demand actions required by a turbine ramp-up procedure, and monitor relevant conditions and anomalies. Explorative studies were carried out with U.S. and Swedish operators. Then a more structured data collection focusing on situation awareness was performed. The relatively simple scenario and prototype did not significantly affect the situation awareness.
We found that operators wanted to be in control, which could be supported by the flexibility to control, when necessary, the automation at rather low granularity of action. In addition, operators appreciated the ability of the automation to warn and suspend its activity when anomalies arise, helping to give a context to the alarm occurring in the control room (HTOR-051).
Halden HTO References HTOR-007: Bisio, R., Bloch, M. & Hurlen, L. (2023). Controlling a Plant with Adaptive Automation: Perspectives and Challenges.
HTOR-024: Bloch, M. & Bisio, R. (2022). Summary of the Halden Workshop on Adaptive automation.
Bisio, R. Bloch, M. (2023). Impact of Adaptive Automation for Supporting Operation in Nuclear Power Plant, an Exploratory Study. Proceedings of ESREL 2023.
HTOR-051: Bisio, R. & Bloch, M. (2023). Adaptive Automation - Studies Report.
2-43 Note: Contents of this table have been excerpted from the report Halden HTO Program Achievements 2021-2023 (Halden HTO, 2024). For HTORs referenced in this report and published within the 2021-2023 program period, see appendix A-1.
2.4.3.1 NRC Significance This research activity was not identified as useful for the development of regulations or guidance. Preliminary NRC research in this area indicates that adaptive automation is not likely to be implemented in the nuclear industry; see Research Information Letter (RIL) 2020-05, Adaptive Automation: Current Status and Challenges, issued November 2020, and RIL 2020-06, Safety Evaluations of Adaptive AutomationSuitability of Existing Review Guidance, issued January 2022 (NRC, 2020a; NRC, 2022d). However, the NRC is monitoring the technology landscape; if it changes, the agency should consider input from this research activity in future guidance development.
2-44 2.5 Digital Systems for Operations and Maintenance The nuclear industry is expected to undergo a digital transformation, albeit more gradually than many other industries. Digital systems offer tremendous benefits for operations and maintenance in NPPs; however, they also carry potential safety risks. Thus, innovations and solutions from other industries are not directly transferable to the nuclear industry; before they can be implemented, new research is necessary. For example, the use of AI technologies in an NPP would require extensive research on trustworthiness and explainability.
Halden HTOs projects in this area show how digital systems can support operators analysis and decision-making by providing insight into data from complex processes. Researchers studied the use of digital twins for condition-based maintenance; the use of AI models for condition monitoring; and the effects of certain human, technological, and organizational factors on safety awareness during outages.
2-45 2.5.1 Advanced Condition-Based Maintenance Using Digital Twins Table 2-14 Halden HTO Objective, Results, and References for Advanced Condition-Based Maintenance Using Digital Twins*
6 IoT stands for Internet of Things.
Halden HTO Objective Successful maintenance of a process plant requires up-to-date information about every aspect of the process, as well as possessing the diagnostic ability to accurately determine the current state and the predictive ability to evaluate possible future states of the process.
Reliable evaluations of the current condition and estimated aging of physical components would also be essential elements in a condition-based maintenance support system.
A digital twin may prove an invaluable tool by providing the necessary functionality to extract and process relevant information and visualizing it in an unambiguous and interactive manner to the operator. Based on data collected in real-time by traditional sensors and/or IoT6 sensors, a digital twin may replicate process components and their connections, thus enabling simulations, testing, modeling, and monitoring without having to be close to the actual physical process. The objective of the activity is to develop a modular framework to facilitate the creation of digital twins and thus enable investigation of potential benefits of using a digital twin for condition-based maintenance.
Halden HTO Results The HTOR-017 provides an overview of definitions relevant to digital twins, as well as an overview of tools previously developed at Halden and evaluation of their suitability as potential modules for a digital twin framework. The report gives an overview of relevant open-source technologies and standards for modularity, data handling, and data visualization for the selected development languages. Some of the identified technologies were used to implement a demonstrator.
The HTOR-058 focuses on providing an overview of how digital twins are being applied for operations & maintenance in the nuclear energy sector. One challenge identified was that much of this work is not published in scientific journals and conferences, but rather on the companies websites in a popularized format, where the information provided is limited to superficial descriptions of Artificial Intelligence (AI) / Machine Learning (ML) and Augmented Reality (AR) / Virtual Reality (VR) being applied to make digital twins more accurate.
Halden HTO References HTOR-017: Thunem, H. P-J. (2022). Towards a Development Framework for Digital Twins.
HTOR-058: Misra, S., Thunem, H. P-J. (2023). Digital Twins for Operation and Maintenance in the Nuclear Energy Sector: An Overview.
2-46 Note: Contents of this table have been excerpted from the report Halden HTO Program Achievements 2021-2023 (Halden HTO, 2024). For HTORs referenced in this report and published within the 2021-2023 program period, see appendix A-1.
2.5.1.1 NRC Significance At the time of evaluation, this research activity was not identified as useful for the development of regulations or guidance. However, the NRC has since begun considering possible AI use, as discussed in NUREG-2261, Artificial Intelligence Strategic Plan: Fiscal Years 20232027 issued May 2023 (NRC, 2023b). The results of this work may thus provide useful input for ongoing regulatory research on AI and ML use in the nuclear sector, including AI-enabled autonomy. Section 2.5.2.4 contains additional information.
2-47 2.5.2 Advanced Condition Monitoring for Decision Support Table 2-15 Halden HTO Objective, Results, and References for Advanced Condition Monitoring for Decision Support*
Halden HTO Objective The costs of operation and maintenance for a nuclear power plant is considerably higher compared to other energy production sources. This disadvantage must be met with innovation in digital technology and data analysis. A main obstacle in the implementation of AI at NPPs is the difficulty in explaining the underlying methodology and get regulatory acceptance. The objective of this activity is to deliver techniques and methods for implementing data analytics for improved decision support in operation and maintenance, as well as guidelines on how to introduce these at the utilities to ensure continued safe and efficient operation.
Halden HTO Results Several new AI/ML architectures for data analysis were explored to test how they perform a signal validation task. The models used were an autoencoder, Long Short-Term Memory (LSTM), Dual-Stage Attention-Based Transformer and an Informer model. All the models showed convergence, but added complexity did not necessarily mean better results and the models need to be tested further.
A technology demo was implemented in collaboration with activity 5.1 on Digital Twins (DT),
implementing the signal validation models in a DT-framework. ProcSee was used to implement a graphical user interface visualizing the data analysis, as well as the dataflow in the DT-framework.
HTOR-059 provides an overview of the survey conducted at the start of the three-year period to assess the use of data-driven systems in the nuclear domain. The explored AI/ML architectures and their performance on test data are described. In addition, the technology used for the implementation of the demo and the graphical interface are documented.
Figure 2-4 Graphical user interface visualizing data analysis and dataflow in a digital twin framework Halden HTO References
2-48 Note: Contents of this table have been excerpted from the report Halden HTO Program Achievements 2021-2023 (Halden HTO, 2024). For HTORs referenced in this report and published within the 2021-2023 program period, see appendix A-1.
2.5.2.1 NRC Significance At the time of evaluation, this research activity (like the previous activity) was not identified as useful for the development of regulations or guidance; however, the NRC has since begun considering possible AI use, as discussed in NUREG-2261 (NRC, 2023b). The results of this work may provide useful input for ongoing regulatory research on the use of AI and ML in the nuclear sector, including AI-enabled autonomy. Furthermore, the Pressurized Water Reactor Owners Group (PWROG) has expressed interest to the NRC in pursuing a broader range of condition-based online monitoring techniques, including ML techniques. The NRC is anticipating that the PWROG will submit a topical report for approval and is planning commensurate research. It would be useful to monitor these activities in relation to the related research conducted under this project.
HTOR-059: Hoffmann, M., Tran, C., Randem, H.O., Bodal, T., Thunem, H. P-J. (2023).
Advanced Condition Monitoring for Decision Support.
2-49 2.5.3 Safety Awareness in Outage Organizations Table 2-16 Halden HTO Objective, Results, and References for Safety Awareness in Outage Organizations*
Halden HTO Objective The main objective of this activity is to derive knowledge of the human, technological, and organizational factors that influence situation awareness of safety elements and potential hazards in outage work, and of how it can be supported by the technologies, tools, and work practices used in outages. Previous work has indicated how different ways of organizing outage work can have implications for situation awareness. Thus, to fully understand how safety awareness in outage may be influenced and best supported by new technologies, tools, and work practices, it is also important to consider the different ways of organizing this work.
Halden HTO Results Research conducted in this activity contributes to a better understanding of the different situation awareness errors that occur in outage work and of the human, technological, and organizational factors that contribute to these errors. It also contributes to a better theoretical understanding of how human performance improvement (HPI) tools could prevent situation awareness errors in outage work. Moreover, research in this activity examined the relevance and applicability of transferring experience from the Integrated Operations (IO) initiative developed for Norwegian Continental Shelf petroleum organizations to nuclear organizations.
Findings support that principles from the IO initiative are relevant and applicable to the nuclear industry.
Halden HTO References HTOR-015: Solberg, E., Nystad, E., McDonald, R. (2022). Safety Awareness in Outage Work
- A Study of U.S. Licensee Event Reports between 2016-2020.
Solberg, E., Nystad, E., McDonald, R. (2022). Situation awareness in outage work - A study of events occurring in U.S. nuclear power plants between 2016-2020. Safety Science, 158.
HTOR-037: Kwei-Narh, P. and Nystad, E. (2023). Situation awareness (SA) in outage work:
An agentic perspective on SA errors, recognition, and mitigation.
HTOR-036: Solberg, E. and Kwei-Narh, P. (2023). Human performance improvement tools and situation awareness in outage work.
Solberg, E. and Kwei-Narh, P. (2023). Human performance improvement tools and situation awareness in nuclear power plant outage work. Proceedings of ESREL 2023.
2-50 Note: Contents of this table have been excerpted from the report Halden HTO Program Achievements 2021-2023 (Halden HTO, 2024). For HTORs referenced in this report and published within the 2021-2023 program period, see appendix A-1.
2.5.3.1 NRC Significance Situational awareness is important for operators not only during regular system operations, but also during outages and maintenance. However, outages (including both planned outages for maintenance or refueling, and unplanned outages, such as those due to weather) present many challenges that may impede situational awareness, such as increased staffing, time pressure, and physical demands. This work, which identifies factors that contribute to situational awareness errors during outages, can help reduce performance error and increase adherence to appropriate work procedures. While the direct regulatory value of the work is marginal, further study of outages could yield broader experience about what can go wrong outside of normal operations.
HTOR-038: Drøivoldsmo, A. (2023). Comparing nuclear and offshore petroleum organisations.
2-51 2.6 Digital Transformation of Decommissioning Recently, technologies such as building information modeling (BIM), mobile robots, three-dimensional (3D) scanning sensors, digital twins, AI, computer vision, and extended reality (XR) (an umbrella term for AR and VR) have matured enough to enable holistic 3D hazard-aware simulation. These advances, in combination with recent standardization efforts, have made it possible to develop holistic simulation-aided facility and site models, which could support decommissioning of existing nuclear facilities as well as life-cycle management of new nuclear assets, including advanced reactors. Research has shown, however, that while these techniques offer huge benefits for optimizing processes and preparing workers for safety-critical jobs, they need to be integrated with facility-wide information systems and sensor technologies to be maximally useful. Such integration poses many challenges; in particular, pilot studies and evaluations are needed to demonstrate how it can be accomplished while meeting the specific requirements (including safety requirements) of the nuclear sector. The Halden HTO activities described in this section support the development of a holistic digital approach to commissioning and decommissioning, at both existing and future nuclear sites, with focus on safety and sustainability. This goal is pursued through pilot testing and evaluation of new methods for real or realistic use cases.
2-52 2.6.1 Spatial Computing and Augmented Reality for Hazard Mapping and Visualization Table 2-17 Halden HTO Objective, Results, and References for Spatial Computing and Augmented Reality for Hazard Mapping and Visualization*
7 BIM is a technology providing a digital representation of a facilitys physical and functional characteristics.
8 GIS stands for geographic information system. A GIS is a computer system that analyzes and displays geographically referenced information.
9 XR stands for extended reality; it is an umbrella term encompassing AR, VR, and mixed reality.
Halden HTO Objective Improve basis for risk-informed decision-making by studying how best to acquire adequate environmental data and to provide field workers and supporting staff with pertinent information on hazards in the work environment, including visually conveying information quality and uncertainty associated with radiological and other hazards.
Develop guidance and recommendations on nuclear safety related aspects of site-or wide-area digital twins based on BIM7, GIS8, and open AR cloud standards to enable interoperability and long-term use for location-specific data, while ensuring consistency, security, privacy, and reliability of shared data.
Halden HTO Results Taking advantage of the experience gained with our contributions to the development of XR9 standards, we created a radiation visualization platform that enables field workers to intuitively map and avoid hazards in complex physical environments. A standards-based spatial computing solution that greatly facilitates the decision-making required to conduct operations in a safe and secure manner, while also providing mechanisms to collaborate with the support staff in real-time.
After successfully applying this technology to implement a study on radiological risk visualization and perception using wearable AR (HTOR-060), we recorded the knowledge we gained in the process in a comprehensive list of recommendations that can be used to build safer and more interoperable Digital Twins for the nuclear sector.
Halden HTO References HTOR-060: Louka, M.N., Salazar, M. (in press). A Study on Radiological Risk Visualisation and Perception using Wearable Augmented Reality.
HTO-061: Salazar, M., Louka, M.N. (2023). Spatial Computing and Augmented Reality:
Lessons learned from using and developing open standards to enable safety-oriented decision-support for field workers.
HTOR-066: Salazar, M. (2023). Spatial Computing and Augmented Reality for Hazard Mapping and Visualisation.
2-53 Note: Contents of this table have been excerpted from the report Halden HTO Program Achievements 2021-2023 (Halden HTO, 2024). For HTORs referenced in this report and published within the 2021-2023 program period, see appendix A-1.
2.6.1.1 NRC Significance This research activity was not identified as useful for the development of regulations or guidance for reactors. However, RES performed research on a related topic as part of the external crowdsourcing pilot of the NASA Tournament Labs, and it concluded that future research into the use of AR (but not VR) in the nuclear domain would be useful to the NRC. In addition, broadly speaking, the NRC is interested in learning more about the use of digital twins, ML, and AI in the nuclear sector. Lessons from the use of these technologies in decommissioning could be generalizable to safety-related areas. For example, several advanced reactor vendors have proposed the use of XR integration with the plant operating environment or simulation resources. One open issue is that in a plant managed using a digital twin, the locus of human factors related safety concerns will shift from the operation and maintenance of the physical plant to the creation and sustenance of the digital twin. Regulatory research is needed on this topic to help resolve some of these open issues.
2-54 2.6.2 Automated Assessment of Field Worker Performance Using VR-and AR-Based Simulator Training Table 2-18 Halden HTO Objective, Results, and References for Automated Assessment of Field Worker Performance Using VR-and AR-Based Simulator Training*
2-55 Halden HTO Objective Develop automated training assessment techniques to enable effective and cost-efficient Virtual Reality (VR), Augmented Reality (AR), and simulator-based decommissioning training for field workers to practice for jobs that require complex motor skills.
Halden HTO Results Various essential technologies for training assessment automation were integrated in a training scenario that involved work in a radiological environment. This included use of machine learning for both avatars and real robots, 3D motion capture and replay, and use of 3D scans. A VR immersive scenario created early in the project was successfully transferred into extended reality (XR) configuration where the participants could interact with a Spot robot.
User tests have been performed and results analyzed. These are presented in HTOR-062.
Figure 2-5 Screenshot of the XR training scene integrating 3D textured mesh scan, simulated radiation, XSens Awinda avatar, SPOT avatar Halden HTO References HTOR-026: Stephane, L.et al. (2023). Automated Assessment of Field Worker Performance:
Theoretical Framework and Performance Indicators in XR Training Simulators.
HTOR-062: Stephane, L.et al. (2023). Automated Assessment of Decommissioning Field Workers Performance in XR traininguser tests and results.
Note: Contents of this table have been excerpted from the report Halden HTO Program Achievements 2021-2023 (Halden HTO, 2024). For HTORs referenced in this report and published within the 2021-2023 program period, see appendix A-1.
2-56 2.6.2.1 NRC Significance This work, which incorporated XR virtual environments, ML, and 3D scanning, provides new insights into the use of simulator--based training for field workers. In particular, the work shows how such technologies can provide immersive practice scenarios through which workers performance on tasks that require complex motor skills can be assessed. Lessons from this activity can be applied to the NRC staffs understanding of how these technologies might apply to training more broadly, even outside of the decommissioning space. For example, proposed Part 53 proposes the potential use of alternative simulation facilities (e.g., partial scope simulators) would be allowed provided that all associated requirements could be demonstrated to be met using alternative approaches and methods including the use of augmented reality technology to provide training and examination value while avoiding the operation of actual plant components (SRM SECY-23-0021: NRC, 2024c). They can also be applied to other technologies, such as AI and digital twins. As the nuclear industry pursues the automation of tasks that are currently performed manually, it will be important for designers and regulators to understand how the new technologies affect the workload of field workers, in order to verify that tasks are allocated appropriately and safely between humans and automated systems. Function allocation (FA) is a method for distributing tasks and functions across people and technology and is addressed in the guidelines found in NUREG-0711. FA is also a proposed requirement § 53.730(d) in 10 CFR Part 53 (NRC, 2024c). In 2024, the NRC is beginning work to update guidance related to FA for advanced reactors and will continue to monitor any ongoing Halden HTO work on this topic for relevant insights.
2-57 2.6.3 Enabling Robotic and Remote Operation Table 2-19 Halden HTO Objective, Results, and References for Enabling Robotic and Remote Operation*
Halden HTO Objective The goal is to provide an integrated framework for human-robot and robot-robot collaboration in decommissioning missions that leverages safety assurance. The framework will identify industry needs and requirements, high impact use cases, and typical scenarios for using robots to support those use cases. It will include recommendations on available technology and related training, and insights from evaluations. It will also include guidelines and best practices for using modular robotic systems for decommissioning.
Halden HTO Results A state-of-the-art of standards for mobile robots (updated in December 2023), safety assurance standards, and safety analysis tools has been performed. Robotic system safety was modeled with both Functional Resonance Analysis Method (FRAM) and System Theoretic Process Analysis (STPA). IFEs InstrucT software has been extended with the goal structuring notation (GSN), and the NASA AdvoCATE software has been used with several robotic systems for identifying safety requirements. The proposed safety case structure verification & validation, safety analysis, and digital twinwas instanced with several use cases and digital twin solutions.
Robotic configurations and software solutions were identified, instanced, and tested successfully for remote operations.
Figure 2-6 Integrated safety case components Halden HTO References HTOR-063: Stephane, L. et al. (2023). Enabling Robotic and Remote Operations in Decommissioning: Safety-Centered Framework.
2-58 Note: Contents of this table have been excerpted from the report Halden HTO Program Achievements 2021-2023 (Halden HTO, 2024). For HTORs referenced in this report and published within the 2021-2023 program period, see appendix A-1.
2.6.3.1 NRC Significance This project helped researchers identify promising technologies and formulate several follow-on activities for Halden HTOs 2024-2026 research program, including System Theoretic-Process Analysis (STPA) as a hazard analysis method (Topic 2.3); the safety assurance case approach for organizing, integrating, and evaluating safety-relevant evidence (Topic 2.1); and regulatory assessment issues with AI-assisted robotic automation of operational and maintenance activities in a nuclear facility (Topic 6.2). STPA and assurance case knowledge gained through research in DI&C was helpful in these activities. As with other activities described in this section, insights from this research may have significant implications for the remote operation of future facilities, which is a high-priority area for NRC research and guidance development. The nuclear industry has proposed the use of remote and robotic-enabled activities within operating plants.
Given the potential safety impacts, the NRC should continue to monitor how these developments might be applied. Robotics, like any other plant system or activity, must at a minimum demonstrate non-interference with plant safety functions and systems. For example, if a robot is operating within a plant, it must be ensured that it does not accidentally collide with safety-related equipment such as pumps. If a robot were to perform a safety function (though none have been proposed in operating plants to date), it would need to provide assurance that the safety function is reliably performed, which would require NRC approval. Non-interference is typically assessed through changes, tests, and experiments under 10 CFR 50.59 and may or may not necessitate NRC approval depending on the plants screening process. While this research was initially planned in the context of decommissioning, its results may also be relevant to the automation of hazardous tasks in operating environments, such as inspections and contamination measurements in high-radiation areas. The findings could be beneficial for future research on applying robotic automation for hazardous tasks in nuclear power plants, and thus, is an area of significance to the NRC.
Stephane, L., Edvardsen, S.T. (2022). Enabling Robotic & Remote Operations in Decommissioning. HPRG presentation, October 2022.
2-59 2.7 Cybersecurity for Main Control Rooms Energy facilities, including NPPs, are considered critical infrastructure---that is, infrastructure that is essential for maintaining vital societal functions. Damage or destruction of such infrastructure through cyberattacks, natural disasters, terrorism, or criminal activity may jeopardize the security and safety of society at large. As energy facilities rely increasingly on information and communication technology (ICT), they are vulnerable to cyber threats and other unconventional forms of attack, which are not always addressed by traditional risk assessment and measures for protection. For example, NPPs are exposed to cybersecurity threats inherited from the ICT sector. Improving the protection of NPPs has, therefore, become a high priority for utilities and authorities. Of course, some threats cannot be foreseen, and it may not be cost effective to minimize all possible risks completely. The research presented in this section focuses on identifying plausible threat scenarios with cascading security risks, reducing the chances that such scenarios will occur, and addressing attacks during which the main control room is in operation.
2-60 2.7.1 Digital Systems Architecture and Threat Landscape Table 2-20 Halden HTO Objective, Results, and References for Digital System Architecture and Threat Landscape*
Halden HTO Objective The objectives are to identify the threat landscape for NPPs, identify cascading security risks with influence on the safety of NPPs, model selected feasible cyberattack scenarios and facilitate understanding of the NPP digital systems architecture and interdependencies.
Halden HTO Results The HTO workshop Operationalizing threats and vulnerabilities in relations to system specific effects was arranged with twenty participants from HTO member organization and invited organizations, with external presentations from petroleum and maritime domains. A literature review on cyber security threat landscapes for nuclear presented a synthesis of 43 studies.
The results indicate a lack of both methods and information supporting organizations to perform their own cyber threat intelligence on a higher level, and that nuclear is facing the same threats and challenges as other domains. The fact that the same tools, analysis methodologies, and threat repositories are used across domains, contributes toward similar threat pictures for nuclear and other domains sharing similar operational technologies. In addition to the review, a set of more than 200 threat information sources, and threat and vulnerability repositories have been collected. To support organizations in performing cyber security assessments of existing systems, an evaluation of a safety-oriented high-level system specification diagram from the point of view of security experts resulted in a ranked lists of vulnerabilities and potential threats. Finally, a security analysis technique concept based on the Lockheed Martin 7-step kill chain model was proposed. The technique was piloted on high-level NPP system diagrams.
Halden HTO References HTOR-002: Esnoul, C., Simensen, J. E., and Chockalingam, S. (2021). Halden HTO Project Cyber Security Workshop proceedings.
Simensen, J. E., Jørgensen, P-A., Toppe, A. L. (2022). Cybersecurity Centrecapabilities and insights to a technical cyber security experiment on critical infrastructure. Paper at EHPRG, March 2022.
HTOR-043: Simensen, J. E., Shukla, A. (2023). Summary of the HTO Workshop on:
Operationalizing threats and vulnerabilities in relations to system specific effects.
HTOR-054: Simensen, J. E., Randem, H. O. (2023). Systematic Literature Review - Cyber security threat landscape for nuclear.
2-61 Note: Contents of this table have been excerpted from the report Halden HTO Program Achievements 2021-2023 (Halden HTO, 2024). For HTORs referenced in this report and published within the 2021-2023 program period, see appendix A-1.
2.7.1.1 NRC Significance This work establishes an appropriate scope for research on NPP cyberattacks by better defining potential problems for regulatory focus. It provides a system for prioritization and importance-based classification of threats against digital assets. As a starting point, researchers focused on the interfaces and interactions between information technology (which is primarily used to process information) and operational technology (which transforms the physical states of energy and machinery). Their findings shed light on the potential capabilities of cyberattacks.
Furthermore, the development of a plant structure model is highly beneficial to the simulation of cyberattacks and contributes to the understanding of plant element relationships important to cybersecurity outcomes. The results of this research activity may inform updates to guidance on identifying the critical digit assets that licensees need to protect from cyberattack. Further research could also yield information that would be useful in reviewing licensees cybersecurity plans. For example, conducting research in this field helps build a foundational understanding of asset relationships and potential attack pathways, which is essential for the effective approval and oversight of these plans. Chapter 3 offers more details on the significance of this work to the NRC.
HTOR-055: Shukla, A., Simensen, J. E. (2023). System modelling for security - appending cyber security threat and vulnerability information to existing system models.
HTOR-056: Simensen, J. E., Mikkelsplass, S. A., Sechi, F. (2023). Concept for operationalizing system threat assessment through adversary thinking.
2-62 2.7.2 Incident Detection and Response Using Simulation Modeling and Tools Table 2-21 Halden HTO Objective, Results, and References for Incident Detection and Response Using Simulation Modeling and Tools*
10 RAMS stands for real-time anomaly monitoring system.
11 SWaT stands for scaled-down replica of a full-fledged water treatment plant.
Halden HTO Objective The objectives are to investigate the usability of simulation methods for high-fidelity representation of NPP systems for cybersecurity and RAMS10 incident detection and response, and to identify the relevant early signals of threats for timely detection and response. We will propose approaches to better communicate and visualize the cyber security risk picture of the plant to facilitate effective incident response.
Halden HTO Results A literature review provided an overview of cyberattack monitoring and detection methods and tools for Operational Technology environments, identifying 38 cyberattack monitoring and detection methods, and eleven tools for monitoring and detection of cyber-attacks. Based on the analysis, it was identified key characteristics and applications of the methods and tools, such as testbeds like SWaT11 and Asherah NPP play an important role in application, demonstration, and evaluation of the identified methods. We also identified sixteen studies that addressed testbeds for cybersecurity of digital I&C systems in NPPs (USA, South Korea, Brazil, and Austria). Comprehensive evaluation of state-of-the-art Intrusion Detection Systems (IDS) using appropriate testbeds is lacking. Therefore, we adapted a realistic Industrial Control System (ICS) testbed to align with the ISA/IEC 62443 standard, where the results that we obtain through experiments on the adapted ICS testbed can be generalized to different domains. Finally, we performed experiments utilizing diverse attack scenarios on the adapted testbed to compare selected IDS including Elastic Stack with security functionality, Omicron (StationGuard), and Suricata. Our evaluation revealed both strengths and limitations of the IDSs evaluated and why both network-and host agent-based IDSs are needed to get a complete picture of what is happening in a system.
Halden HTO References HTOR-013: Chockalingam, S., Mikkelsplass, S.A. (2022). A Systematic Review of Monitoring and Detection Methods and Tools for Cyber Security in Operational Technology (OT)
Environments.
HTOR-045: Mikkelsplass, S.A., Toppe, A.L., Chockalingam, S. (2023). A Comparative Analysis of State-of-the-Practice Intrusion Detection Systems.
Chockalingam, S., Pieters, W., Teixeira, A. M., van Gelder, P. (2023). Probability Elicitation for Bayesian Networks to Distinguish between Intentional Attacks and Accidental
2-63 Note: Contents of this table have been excerpted from the report Halden HTO Program Achievements 2021-2023 (Halden HTO, 2024). For HTORs referenced in this report and published within the 2021-2023 program period, see appendix A-1.
2.7.2.1 NRC Significance This work addresses incident detection and responsein particular, the responses of operators to cyberattacksand may thus contribute to the technical basis for regulations on cybersecurity.
Researchers identified the ways in which operators may detect (or fail to detect) cyberattacks, the ways in which they respond to cyberattacks, and the problems they may encounter in responding to cyberattacks. As advanced reactor licensees are likely to adopt novel CONOPs such as remote facility operation and increased reliance on automation, the security of plant communication networks is of critical importance for advanced reactors. This research could be particularly useful to the NRC if synergized with the cybersecurity research performed at Sandia National Laboratories.
Technical Failures. Journal of Information Security and Applications, vol. 75, no.
103497.
HTOR-044: Chockalingam, S., Shukla, A., Sechi, F. (2023). Testbeds for Cyber Security of Nuclear Power Plants: Key Characteristics and Architecture Design.
2-64 2.7.3 Human Behavior during Cyber Incident Response Table 2-22 Halden HTO Objective, Results, and References for Human Behavior during Cyber Incident Response*
Halden HTO Objective An important aspect of cybersecurity is the response capacity when an incident occurs.
However, the largest investments seem to be made in tools and systems to fight cyber-attacks rather than addressing human behavior. Research has begun to tap into the human aspects of mitigating and responding to incidents, pointing to a further need to better understand how control room operators can be supported in managing the high level of uncertainty they face during incidents, to gain further knowledge on human behavior in cybersecurity incident response and use this knowledge to strengthen the response capacity.
Halden HTO Results A cross-industry systematic literature review about human behavior during cybersecurity incident response has been conducted. The review provided findings on operator behaviors and competence needs during cyber-attacks, how tools and organizational measures can provide support and how to perform and evaluate cyber security simulator training. Following this, a set of research activities investigated the role and responsibilities of control room operators during cyber incidents that impact plant systems:
- i.
A simulator study of cybersecurity awareness and the effect of providing cyber warnings, eight licensed operators participated.
ii.
Interviews with twenty licensed control room operators and operational managers about their cyber security training, their knowledge and understanding of cyber-attacks, and their view of their own role and responsibility during such incidents.
iii.
A review of known cyber-attacks on nuclear organizations, including their attack vectors and consequences.
The results from these activities indicate that there are gaps in operators cybersecurity awareness and perceived responsibility for cybersecurity incident detection and mitigation.
We proposed a framework for enhancing operator cybersecurity awareness using simulation exercises.
Halden HTO References HTOR-048: Nystad, E., Kwei-Nahr, P.A., Esnoul, C., & Nyusti, L. (2023). The role of control room operators during a cybersecurity incident and cybersecurity awareness training needs.
HTOR-012: Nystad, E., & Olsen, S. A. (2022). Human behaviour during cyber security incident response: A systematic literature review.
2-65 Note: Contents of this table have been excerpted from the report Halden HTO Program Achievements 2021-2023 (Halden HTO, 2024). For HTORs referenced in this report and published within the 2021-2023 program period, see appendix A-1.
2.7.3.1 NRC Significance For this work, researchers simulated realistic attack scenarios at the Cybersecurity Centre and HAMMLAB. Their simulator study, interviews, and literature review identified gaps in cybersecurity awareness. The simulated scenarios helped them assess the behavioral aspects of attack responses in a controlled environment. This research contributes empirical evidence and data on cyberattack responses in NPPs, a topic that it is crucial to understand as the industry moves increasingly towards remote and automated operation, which may rely on digital communication networks. Additionally, it may provide information relevant to oversight of NPP response to cyberattacks. This research could be particularly useful to the NRC if synergized with the cybersecurity research performed at Sandia National Laboratories.
3-66 3 SELECTED PROJECTS This chapter offers a comprehensive analysis of the Halden HTO research activities identified as having the most significant regulatory impact. For each activity, the subsection NRC Engagement outlines the involvement of the NRC staff and its past work in related areas. The section Regulatory Use of Project Outcomes (2021-2023) elaborates on how the activity has contributed to regulatory activities such as guidance development.
3.1 Operator Performance in Digital Control Rooms 3.1.1 NRC Engagement Since 2015, the NRC has actively engaged in this project, primarily in an advisory role but at times collaboratively, shaping the aim and design of the studies conducted. HFE staff members from RES have identified possible research questions, while HFE staff members from NRR have provided key regulatory insights, significantly influencing the direction and outputs of this activity.
The NRC staffs involvement in this activity reflects the significance of the research conducted.
In particular, the NRC staff recommended that Halden HTO pursue the topic of operator use of Computer-Based Procedure (CBP) systems. To facilitate, the NRC coordinated some of the initial data collection activities at U.S. plants with participating U.S. operators. NRC staff members also coauthored reports with Halden researchers (Taylor et al., 2017; Taylor et al.,
2018). Supporting project development, the NRC staff recommended an emphasis on ways to adapt the conduct of operations to new designs. This focus would provide a foundation for best practices in developing and modifying the conduct of operations in control rooms. Research on the conduct of operations would, in turn, inform the development of a framework for creating a facilitywide CONOPs.
The NRC has conducted many experimental studies of operator performance in both analog and digital main control room environments, including studies in collaboration with Halden HTO on operator use of CBP systems (e.g., Hildebrandt et al., 2023). In a conventional main control room, the conduct of operations is based on a reader-doer framework, in which the senior reactor operator (SRO) reads the (paper-based) procedures and instructs the reactor operator (RO) and balance-of-plant (BOP) operator to carry them out. By contrast, in a main control room with CBPs, the RO owns the procedures and implements the actions, while the SRO and BOP operator follow along on their own desktop CBP displays. This adjustment results in a quieter control room, because fewer communications must be made aloud. It may also allow for the SRO to maintain greater situational awareness and a better overall view of the operational scenario, as they are no longer in charge of place-keeping in a paper-based procedure. The work already conducted under this activity shed light on these possibilities and provided insights that informed safety-related regulatory decisions for a number of license applications for designs using CBPs, such as the Westinghouse AP1000 and NuScale SMR designs.
Research on digital control rooms remains ongoing, as industry advancements continue to drive designs toward new technologies. At the 2023 Human Factors and Ergonomics Society conference, the NRC staff in coordination with Halden researchers convened an expert panel entitled Human Factors in New Nuclear. This collaboration serves to inform ongoing NRC human factors research for advanced reactors.
3-67 3.1.2 Regulatory Use of Project Outcomes (2021-2023)
Table 2-1 in section 2.1.1 provides project details and Halden HTO references.
Novel technologies and advanced CONOPs appear within efforts to modernize the current reactor fleet (e.g., by changing from analog instrumentation and controls (I&C) to DI&C), as well as in designs for new reactors (e.g., the AP1000), SMRs (e.g., NuScale), and advanced reactors. The results of this work illustrate how new control room technologies may affect the conduct of operations at an NPP. Insights from the work have informed regulatory guidance on future, novel, and advanced CONOPs, such as remote operation, as well as guidance on the more highly automated facilities anticipated under the draft 10 CFR Part 53. Researchers focused on how digital control room technologies, operator role changes, and new operational methods affected operator performance during emergencies. Data were collected at a nuclear new build site, where researchers observed two crews during three emergency scenarios.
These data provided empirical evidence of one way in which new technologies can change the conduct of operations (e.g., by shifting away from a reader-doer model as described above).
Operators did not find the dual responsibility of overseeing and executing procedure steps excessively burdensome. This highlights the importance of additional research into crew dynamics and work distribution in digital control rooms.
Onsite research activities demonstrated operator responses to the introduction of new technologies, such as CBPs. Researchers were able to observe how crew roles changed when the reader-doer model of carrying out procedures was changed to a model in which the RO oversaw procedures. One aspect studied was how operators responded when they lost access to CBPs and had to transition back to paper-based procedures. This work answered questions posed by the NRC staff about CBP use. It also provided input for the NRCs development of HRA methods and risk evaluation during emergency events (e.g., station blackouts resulting in loss of CBP systems). The observational, usability, and workload data collected, along with the resulting task modeling, provided valuable avenues for the NRC staff to systematically explore operator behavior during changes to the conduct of operations. Continuations of this work may have implications for risk evaluation, HRA method development, and HFE concerns related to potential communications losses within a remote operation paradigm.
This work also increased the NRC staffs understanding of and confidence in operator use of CBPs, such as those introduced by Westinghouse for its new build AP1000. This contributed to the successful licensing and safe commercial operation of two new plants in the U.S. power grid. These plantsthe first new nuclear units built in the United States in more than three decades, constituting the first U.S. deployment of the Westinghouse AP1000 Generation III+
reactor technologyentered commercial operation on July 31, 2023, and April 29, 2024.
In October 2023, the NRC hosted a task modeling workshop focused on operator performance, with the aim of developing a systematic approach to HRA or risk-informed human factors data collection. The method was developed with input from Halden HTO and INL researchers, to be used in planned simulator data collections. Workshop participants proposed a bottom-up approach, based on systematic collection and analysis of data from each of the individual steps of a procedure. These steps would then be combined to form human failure events, and the data collected for individual steps would be integrated to represent the data associated with each event. Data collection would take place in three steps: (1) collection of baseline performance information from procedural steps and observed steps taken, (2) collection of data on performance expectations and observed performance, and (3) collection of operator performance data on individual procedure steps for a given event and conditions, which would
3-68 yield numerical data on rates of human error. Workshop participants also discussed a possible approach to HRA for SMRs, which involves identifying key procedures, then performing a multifaceted analysis on selected items (e.g.., HRA using IDHEAS-ECA software; see RIL 2020-02, Integrated Human Event Analysis System for Event and Condition Assessment (IDHEAS ECA), issued January 2020) to quantify key operator actions (NRC, 2020c).
This project broadly supports the development of technical bases and review guidance as the industry transitions to digital control rooms. As well as providing a new framework for the conduct of operations, it offers insights into changing crew roles that may inform guidance on role division. The findings from this work and related research not only illuminate how current designs influence the conduct of operations, but they are also likely to be useful when new digital functions or systems are introduced in the future. Likewise, this work may inform guidance development for the NRC review of future remote operation concepts that may employ novel staffing concepts to achieve remote operation.
Both the research conducted in this area from 2021 to 2023 and the research planned for future program cycles may help refine the systematic approach to HRA described above. For instance, as previously noted, research in this area provided insights for the NRCs development of HRA methods and risk evaluation for emergency scenarios (e.g., station blackouts) that could apply to operating, new, and advanced reactor facilities (Wright and Bye, 2022). The data collected provided empirical information useful for HRA applications for control rooms using DI&C.
The methodology, introduced and tested in the October 2023 task modeling workshop, supports regulatory decisions by offering a clear and structured way to analyze how operators perform. Using a structured approach helps staff compare results from different studies and more easily identify common patterns in how operators behave during plant operations. The methodology provides a more solid foundation for making informed and risk-aware decisions.
Research conducted in this area has yielded empirical evidence informing several NRC guidance development activities. These include future updates to NUREG-0700 and NUREG-0711, which are to be used for HFE reviews of new and advanced reactor license applications as well as license amendment requests for existing plants that are modernizing by replacing analog I&C with DI&C.
Research in this area has also contributed to advanced reactor regulation, informing the development of a facility-specific scaled HFE review plan that focuses appropriately on the human role in safe operations. The NRC is developing regulatory guidance to aid compliance with the proposed requirement in 10 CFR 53.730(c) for applicants for combined and operating licenses to submit a facility-wide CONOPs. The NRC will also need to update NUREG-0700 with guidance on advanced functionality for CBPs, including review guidance for automation of plant control functions, multiple-user CBPs, and CBPs for multiple units.
In addition, research in this area has informed the development of standards, an activity in which the NRC is heavily involved. Specifically, it has provided input for standards on interface design and guides for CBP use developed with IEEE Subcommittee 5, Human Factors, Control Facilities, and Human Reliability. The work has also informed the development of guidance on human factors reviews of novel approaches to reactor operations, including remote facility operation and increased reliance on automation for key safety-significant structures, systems, and components.
Finally, the work performed in this area has informed research conducted in the NRCs Human Performance Test Facility, which contains the agencys in-house human factors simulator (NRC
3-69 2023a; NRC 2023c; NRC 2023d; NRC 2023e; Dickerson et al., 2024; Dickerson et al., 2023; Lin et al., 2023; Lin et al., 2022).
3.2 Crew Factors, Teamwork, and Role Independence in Control Rooms 3.2.1 NRC Engagement The NRC has maintained ongoing awareness of the research conducted as part of this activity.
In particular, HFE and HRA staff members from RES and NRR provided suggestions for research directions during the planning stages of the project. They recommended that researchers first review operating experience of crew factors and teamwork, then enrich their findings with information from the literature and past Halden HTO studies. Earlier work conducted in this area (e.g., Braarud, 2020) informed the NRC staffs approval and design certification of the NuScale 12-unit design, which was the first SMR design to be certified by the NRC (see section 3.2.2 for details). The NRC staff also recommended that future work under this activity include investigations of cognitive dependency (see section 4.2.1).
The work performed under this activity is related to another topic of NRC interest, namely, lessons learned about crew performance during the 1991 incident at the Crystal River Unit 3 Nuclear Generating Plant. The incident provides a significant case study of operator performance in failure events, underscoring the importance of crew factors, teamwork, and role independence in plant operations. An event analysis was documented in the NRCs initial technical report, EGG-HFRU-10085, On-Site Analysis of the Human Factors of an Event at Crystal River Unit 3 December 8, 1991 (Pressurizer Spray Valve Failure), issued January 1992 (NRC, 1992). The incident demonstrated the importance of accurate information, adherence to procedural guidance, effective communication, and leadership. It also highlighted the consequences of errors of commission and the importance of situation assessment in preventing such errors. Post-event HRA for this event and others led to the development of a new HRA method, described in NUREG-1624, Technical Basis and Implementation Guidelines for a Technique for Human Event Analysis (ATHEANA), issued 2000 (NRC, 2000).
3.2.2 Regulatory Use of Project Outcomes (2021-2023)
Table 2-2 in section 2.1.2 gives project details and Halden HTO references.
New technologies may introduce challenges to the conduct of operations and crew roles. For instance, research indicates that main control room operators can often detect and recover from (correct) the errors they make in detecting information and executing procedures. However, new technologies may affect workers error detection and recovery capabilities. For example, they may introduce unintended changes in resource management (e.g., changes to who owns a procedure). Role independence may also affect error detection and recovery, in that the lack of an STA may reduce the chances that errors will be detected independently. Operators may have difficulty maintaining awareness of the dynamic environment, which may result in poorer decision-making during error recovery efforts. This project examined crew factors and their impact on crew performance, providing insight into group processes such as trust, groupthink, and the independence of the STA role.
During this period, Halden HTO researchers completed a HAMMLAB study investigating STA independence (HTOR-050). This work supports the NRC staffs understanding of teamwork, the roles of the STAs and other operators, and future CONOPs in the context of new control room technologies and advanced control room designs. The understanding gained of the STAs role
3-70 supports the technical basis for licensing activities related to advanced reactors by enhancing NRC staff knowledge about the role of on-shift engineering expertise (see, for example, DRO-ISG-2023-02; NRC, 2022a). Advanced CONOPs, such as remote operation of multi-unit sites, combined with economic drives to employ fewer human workers at a facility, introduce the potential for new staffing models, including new roles to be shared between human and automated agents. Under such CONOPs, the role of personnel with engineering expertise may be different from the STA role in plant operation, or it may be combined with other staffing roles. Halden HTOs work in this area may inform the technical basis and review guidance for the review of these novel concepts, as well as the development of guidance for validation of designs for advanced control rooms and modifications to control rooms.
Work performed under this topic in prior research cycles was applied in regulatory activities related to staffing levels and the STA position; as the final decision to allow the elimination of the STA position was design-specific, the NRC may encounter this decision again and the results of the 2021-2023 and ongoing research on this topic may support future data-driven regulatory decisionmaking. On March 16, 2021, the Advisory Committee on Reactor Safeguards (ACRS) held a meeting and discussed the proposed staffing plan for NuScales 12-module power plant (as described in the ACRS letter, NuScale Topical ReportControl Room Staffing Plan, dated May 21, 2021;NRC 2021c). NuScale had provided many technical arguments to justify reducing staffing levels and eliminating the STA position. The NRC staff approved this plan and discussed the topic of the independence of the STA in SECY-21-0039, Elimination of the Shift Technical Advisor for the NuScale Design, issued April 2021 (NRC, 2021a). NuScales first-of-its-kind approach to control room CONOPs represented a shift from existing agency and industry practices, with implications for Commission policy. Ultimately, the ACRS confirmed the staffs position (with caveats) in a letter to the Commission (ML21139A226; NRC 2021c).
Halden HTOs research in this area provided crucial insights about the STA role, which informed the NRC staffs conclusion that NuScale had given sufficient technical justification to provide reasonable assurance that its proposed staffing plan would be adequate to ensure safe plant operation. It should be noted, however, that elimination of the STA position has been determined acceptable only for the specific operational characteristics of the NuScale plant, including use of extended timeframes, substantially improved HSIs, and the availability of engineering expertise in the form of an additional SRO should abnormal conditions occur.
For a different design, however, independence may be important; thus, when examining future requests to eliminate the STA role, the NRC should exercise caution (Bley, 2021). In particular, the agency should consider Halden HTOs ongoing and future research on this topic, especially its work on groupthink, which is acknowledged as a major cause of poor decision-making in teams. DRO-ISG-2023-02 contains review guidance, drawn largely from the NRCs experience with NuScale, on the use of available engineering expertise in lieu of a dedicated STA (NRC, 2022b).
Halden HTOs work on the STA role and groupthink is helping the nuclear industry benefit from innovative technologies while maintaining reasonable assurance of safe operation. As described above, this work has supported regulatory decision-making for past, ongoing, and future license applications proposing multi-unit facilities.
Human performance in teams, especially in settings that involve automation, depends heavily on trust. Understanding the relationship between trust and safety is important. Halden HTOs work under this activity included a review of definitions of trust, trust in humans and in automation, ways of measuring trust, and factors observed to be correlated with trust. The
3-71 findings provide technical insights about how novel technologies may influence crew dynamics, workers trust in technology, and human-automation interaction.
Specifically, researchers developed and distributed to operators a digital questionnaire about their views on remote supervision of microreactors and their level of trust toward such technology (Kaarstad and McDonald, 2022). Researchers also studied the challenges and opportunities associated with remote operation of nuclear facilities by exploring automation and remote control both in surrogate industries (Drøivoldsmo et al., 2022) and for nuclear microreactors (Kaarstad and McDonald, 2022). This work is related to ongoing NRC research on human factors considerations for remote and autonomous control of nuclear facilities.
Finally, work on this topic also supports further refinement of HRA techniques, which may be useful for future revisions to previously developed analysis methods such as ATHEANA (detailed in NUREG-1624; NRC, 2000), SPAR-H 12(see NUREG/CR-6883, The SPAR-H Human Reliability Analysis Method, issued August 2005; NRC, 2005), and IDHEAS. Future versions of these techniques may incorporate the impact of new technologies on performance during accident response and human failure events.
3.3 Decision-Making under Uncertainty 3.3.1 NRC Engagement NRC staff members from the HFE, HRA, and DI&C disciplines in RES and NRR participated in this research activity in an advisory capacity. As recommended by the NRC staff, researchers first conducted a literature review, then scoping to focus on real-life situations and decisions.
The work is especially relevant to the NRC because U.S. crews participated in the study.
This activity evolved from previous work pertaining to NRC staff interests that investigated decision-making with severe accident management guidelines (SAMGs), analyzing the strengths and weaknesses of SAMGs and current approaches to severe accident management.
3.3.2 Regulatory Use of Project Outcomes (2021-2023)
Table 2-3 in section 2.1.3 gives project details and Halden HTO references.
This work supports the NRCs assessment and development of a technical basis for risk evaluation during emergency events (e.g., station blackouts resulting in loss of CBP systems).
In particular, the nuclear-specific empirical data collected is essential for developing human error probabilities (HEPs) as input for the NRCs IDHEAS-ECA13 method (NUREG-2256: NRC, 2022c). The literature review documented heuristics across various approaches in psychology and human factors. Based on this review, the research team decided to perform its data collection using one of the human factors approaches, cognitive systems engineering (CSE), as it was most applicable to a real-world nuclear control room setting. The researchers adapted the CSE way of modeling decision strategies to the nuclear domain by describing operators heuristic decisions in terms of an original information-flow map. This approach emphasizes the 12 SPAH-H stands for Standardized Plant Analysis Risk (SPAR) HRA and is a type of HRA method developed by the NRC.
13 IDHEAS-ECA stands for Integrated Human Event Analysis System for Event Condition Assessment and is a type of HRA method developed by the NRC.
3-72 active aspects of decision-making, showing how operators continuously test their hypotheses about situations by taking actions, which in turn create new data and changes to the plant state.
In this model, procedures are an important source of hypothesis generation. Plant operations are extremely procedure-driven; thus, to produce realistic and meaningful results, it is critical to include procedures as a part of the investigation.
Scenario data captured operator deviations from standard procedures in situations where information was missing or degraded. The experimental scenarios were realistic and dynamic; they included a small feedwater steam line leak, a steam generator tube rupture, and a station blackout. The data were analyzed by a multidisciplinary team comprising two main control room operations experts and a human factors specialist. This collaboration resulted in a robust analysis that effectively balanced practical operational knowledge with laboratory expertise in human factors.
Understanding operator behaviors, challenges, and strategies when faced with degraded information (e.g., failure of CBP systems) is relevant to the human factors review of new technologies in the main control room. It may also be of regulatory use in the development of guidance related to the Nuclear Energy Institute (NEI) document NEI 12-06, Diverse and Flexible Coping Strategies (FLEX) Implementation Guide, issued December 2016 (NEI, 2016).
The findings may illuminate questions related to the tradeoff between procedures and training and the challenges of including FLEX training in the systematic-approach-to-training process (since doing so has a high cost in operator time, and licensees may wish to receive credit for good procedures in lieu of training). Furthermore, the insights obtained on operator decision-making during emergencies with degraded information may support updates to guidance for reviewing existing, new, and advanced reactor applications.
3.4 Performance-based Risk-Informed Safety Assurance and Evidence Collection, Evaluation, and Combination for Safety Assurance 3.4.1 NRC Engagement DI&C staff members from RES and NRR actively participated in multi-year research activities under these projects. They plan to collaborate with Halden HTO to document the work performed in a series of NUREG/IA reports. In particular, the NRCs DI&C staff has collaborated with the Halden HTO staff in conducting workshops to acquire state-of-the-art safety assurance knowledge from experts in other sectors.
Previous work under this topic helped to inform the technical basis for NRCs DI&C safety evaluation and HFE licensing activities (see section 3.4.2 for details). The NRCs DI&C staff continues to engage with Halden HTO on this topic and is exploring opportunities to involve the NRCs HFE staff in the ongoing dialogue to explore further applications of the work.
3.4.2 Regulatory Use of Project Outcomes (2021-2023)
Table 2-1 in section 2.1.1 gives project details and Halden HTO references.
In general, DI&C safety research supports the development of the technical basis for performance-based safety evaluation. Halden HTOs work in this area produced case-based findings on the safety assurance case approach.
3-73 The case study of interest to the NRC investigated how implicit safety-related information in IEEE Std 603'-2018 (IEEE 2018) could be made explicit. This case study explored the use of ontology, risk-insight augmentation, and a safety assurance case structure to reveal the safety intent implicit in the provisions of the standard. The study revealed the need to identify the implicit hazardous conditions underlying each requirement, as well as the challenges in doing so. It also highlighted a need to remove ambiguity from the vocabulary used in IEEE 603 and related documents. Earlier work conducted under this topic, documented in HWR-1193, Improving Safety Arguments for Better ComprehensionLessons Learned from a Case Study in the Nuclear Field, (Karpati et al, 2016) provided concrete examples of logically weak formulations in licensee submittals and showed how to express the intent in a logically sound way.
Another case study focused on what the assessor looks for when performing a safety evaluation of an application, using the original design control document of the APR1400 application submitted to the NRC. These case studies were performed in close collaboration with the NRC and led to an internal NRC research project to develop a domain-specific framework for a performance-based safety assurance case approach.
Halden HTO participants, including NRC staff members, expressed particular appreciation for an HTO workshop series held to assess the state of the art in safety assurance for high-criticality digital systems (such as reactor protection systems in operating NPPs). The workshop clarified the technical assertions under examination, their limitations, and conditions under which they would hold. It also provided examples of industry applications, issues experienced in current practice, obstacles to the adoption of better approaches, knowledge gaps, and opportunities for future collaborative research.
3.5 Lessons Learned on Control Room Validation 3.5.1 NRC Engagement HFE staff members from RES and NRR participated in research activities in this area, and the results will inform and improve the HFE parts of license amendment requests related to modernization using DI&C.
Previous and ongoing work in this area informed a number of significant RES and NRR HFE guidance development and licensing activities (see section 3.5.2 for details).
3.5.2 Regulatory Use of Project Outcomes (2021-2023)
Table 2-8 in section 2.3.1 contains project details and Halden HTO references.
This work is relevant to the NRCs efforts to develop technical bases, increase reviewer knowledge of measures of workload and situational awareness, and determine integrated system validation (ISV) criteria. Specifically, researchers generated four reports on limitations of existing measures, examples of measures used in nuclear control rooms, and future directions for nuclear-specific lines of inquiry.
The first report (HWOR-020) revealed the limitations of the available measures of situational awareness, teamwork, and cognitive workload. The second (HTOR-021) identified 11 types of workload measures used in the nuclear industry and found that there was little knowledge of acceptable zones of workload for different types of tasks. Greater knowledge on this topic would
3-74 be of interest for NRC reviewers. The remaining reports (HTOR-035 and HTOR-071) documented limited support for the validity of common measures of situational awareness and for assessing cognitive overload and loss of situational awareness. They recommended ways to address this gap with nuclear-specific research.
The research outcomes described are relevant to a number of NRC activities. Many of the 12 review elements identified in NUREG-0711, Revision 3, highlight the importance of workload, situational awareness, and teamwork. (NUREG/CR-7190, Workload, Situation Awareness, and Teamwork, issued March 2015, gives a knowledge base for these constructs; NRC, 2015.)
NRC license applicants have proposed a variety of metrics to measure these constructs in order to demonstrate their successful implementation in HFE program designs. However, many of the metrics used by applicants have significant limitations because they were developed for use in nonnuclear domains, with nonnuclear participants, and under specific conditions (e.g., they were administered after a task). Halden HTOs research clarifies these limitations and provides details on the validity of metrics for use within the nuclear domain.
NUREG-0711, Revision 3, currently focuses on ISV as the means of conducting human factors validation of control room designs and modifications. ISV is an evaluation, using performance-based tests, to determine whether an integrated systems design (i.e., hardware, software, and personnel elements) meets performance requirements and supports the plants safe operation (NRC, 2012). The NRC is developing review guidance for validation activities conducted in the design process before ISV; for validations conducted at multiple stages, including ISV; and for assessing when incremental changes in design warrant validation, through a process known as multistage validation (MSV). MSV is a specific approach for validating systems through a series of successive, coordinated activities performed at multiple points or periods during the development or modification of a control room design.
Halden HTOs work in this area has informed several significant NRC HFE guidance development and licensing activities that are likely to benefit the entire nuclear industry, not just advanced reactor applicants. These activities include the following:
development of DRO-ISG-2023-03, Draft Interim Staff Guidance: Development of Scalable Human Factors Engineering Review Plans, issued September 2022 (NRC, 2022b) development of review guidance and criteria for validation activities (e.g., ISV and MSV of control room designs and modifications for operating, new, and advanced reactors) future updates to NUREG/CR-7190 (NRC, 2015)
In previous work conducted in this area, Halden HTO supported an NRC-led activity for the NEAs Working Group on Human and Organizational Factors that led to the development of the NEA report Multi-Stage Validation of Control Room Designs and Modifications, issued December 2019. This report was instrumental to breaking logjams related to digital modernization efforts, which represent both a high priority for industry and a highly challenging licensing issue for the NRC (NEA, 2017). The work from these workshops contributed to the development of IEEE 2411TM-2021, IEEE Guide for Human Factors Engineering for the Validation of System Designs and Integrated Systems Operations at Nuclear Facilities (IEEE, 2021).
3-75 Halden HTOs ongoing research on MSV serves to inform licensing reviews and activities related to the modernization of existing plants (e.g., the use of MSV for digital modifications).
This research has increased the clarity and reliability of the DI&C regulatory infrastructure, permitting the NRC to implement it to enable the safe use of advanced digital technologies in new reactor designs and operating plants (refer to SECY-23-0092, Annual Update on Activities to Modernize the U.S. Nuclear Regulatory Commissions Digital Instrumentation and Controls Regulatory Infrastructure and License Amendment Requests, dated November 1, 2023; NRC, 2023a).
3.6 The Impact of Overview Displays on Human Performance 3.6.1 NRC Engagement HFE staff members from RES and NRR have remained aware of the work conducted in this area, which has informed ongoing updates to the NRCs primary guidance document, NUREG-0700 (NRC, 2020b).
3.6.2 Regulatory Use of Project Outcomes (2021-2023)
Table 2-9 in section 2.3.2 contains project details and Halden HTO references.
This work provides a basis for guidance on LPDs and GVDs. It contributes to the state of knowledge with a classification system for what Halden HTO refers to as overview displays (a term that includes LPDs and GVDs) and recommendations for best practices for implementation, which have informed proposed updates to NUREG-0700.
As recommended by the NRC staff, Halden HTO researchers introduced an enhanced framework for identifying best practices for implementing overview displays (e.g., an interview guide). They also developed a classification system for the various types of overview displays, organized by how they are used in the nuclear industry. This classification may shed light on the variability of findings in existing and future empirical research on overview displays. The researchers collected data based on objective measures of human performance, which highlighted how interface consistency in the control room can influence performance. They linked their findings to the technology acceptance model; such models may be useful in guiding further studies on overview displays.
The NRCs HFE staff updates NUREG-0700 to maintain consistency with state-of-the-art human factors principles. NUREG-0700 was the first HFE guidance document to address the functionality of overview displays (i.e., their role in operations and their relationship to workstation displays). As reactor technology advances, so too do the display technology and HSIs used by the control room crew to maintain safe operations. Much of the current guidance was originally developed in the early 1990s; since then, the NRC has made ongoing revisions.
Because of rapid developments in computer technology, in recent years the staff has updated NUREG-0700 incrementally, publishing Revision 3 in 2020 and planning for release of Revision 4 in 2025. The NRC is currently updating NUREG-0700 to reflect recent changes in the definitions of LPDs and GVDs, which were partly informed by Halden HTOs work. An update to the section on GVDs is planned for a future revision to NUREG-0700 (Revision 5).
Halden HTOs work on overview displays is helping to ensure that NRC guidance remains state-of-the-art and technically defensible to support the NRCs HFE safety reviews of existing,
3-76 new, and advanced reactor applications. Because Halden HTO already has an extensive portfolio on this topic, future updates will mainly require gathering and synthesis that can be performed in-house, resulting in valuable cost savings to the NRC.
3.7 Human Performance in Operation of Small Modular Reactors and Operator Performance in Highly Automated Plants 3.7.1 NRC Engagement HFE and HRA staff members in RES and NRR participated in refining the research scope for these project areas. This included considerations of human performance for SMRs and highly automated plants due to advanced reactor guidance development needs. For example, NRC emphasized a need to focus on factors supporting updates to staffing review guidance for these designs. This included the need for new staffing criteria for SMRs, operator roles in multi-unit disturbances, use of large overview displays, team integration, errors of commission, and new concepts of operations. The NRC staff also emphasized the importance of examining automation issues related to plant safety for various SMR designs anticipated to enter the market. The NRC staff recommended the research be designed to test the concepts that are generic to multi-unit operation, rather than concepts that are specific to a given design. This helps to ensure the results can be generalized across diverse design concepts. Work performed in this area during the 2018-2020 research cycle was of great significance to the NRC and laid a strong foundation for the work performed during the 2021-2023 cycle. Specifically, the 2018 white paper Small-Scale Simulator Studies on Multi-Unit Operation helped resolve several of the practical issues faced by the NRC staff in reviews of applications involving multi-unit operation (Eitrheim et al., 2018). The paper successfully summarized the salient SMR concepts and related human performance issues in a way that may continue to inform NRC guidance development for advanced reactors, including remote and highly automated nuclear facilities.
This earlier work informed several past and more recent licensing activities. The NRC has completed several design certification reviews for large light-water reactors, including the AP1000. In 2023, the construction of the first AP1000 facility in the United States, at Vogtle Electric Generating Plant, Unit 3, was completed, and all the inspections, tests, analyses, and acceptance criteria were closed. This was the first plant licensed, constructed, and operated under 10 CFR Part 52, Licenses, Certifications, and Approvals for Nuclear Power Plants. In early 2024, Vogtle Unit 4 became the second such plant; both are now fully operational and producing power to the grid.
In 2022, NRC completed the design certification rulemaking for the NuScale Power design, which was the first SMR design certified in the United States. This design uses a unique CONOPs in which 12 units are controlled by a single crew. (By contrast, in the operating fleet, a separate crew controls each reactor.) Halden HTOs research also informs staff review of applications such as the NuScale standard design approval application for its six-unit design (https://www.nrc.gov/reactors/new-reactors/smr/licensing-activities/current-licensing-reviews/nuscale-us460.html; NRC 2024b).
Additional work performed by the NRC includes a characterization of automation based on task analysis and the conduct of operations (OHara & Higgins, 2010). In earlier research cycles, the NRC staff encouraged Halden HTO researchers to distinguish between the concept of operations (CONOPs) and conduct of operations. The CONOPs describes how a licensees or applicants organizational structure, staffing, and management framework relate to the systems, design, and operational characteristics of the plant (see NUREG-1791; NRC, 2005). The
3-77 conduct of operations is the set of responsibilities of, and practices used by, operators to ensure safe plant operation (e.g., the use of three-way communication when senior operators direct operators to perform tasks) (see NUREG-0800, Chapter 13, Conduct of Operations; NRC 2023g). Novel designs, including new and advanced reactors, introduce the need for both novel CONOPs and novel approaches to conduct of operations. The proposed regulation in the draft 10 CFR 53.730(c) requires the submittal of a concept of operations that is of sufficient scope and detail to appropriately inform the staff (NRC, 2024c). Halden HTOs research gives the NRC staff valuable information about these CONOPs, which will become increasingly important for the implementation of 10 CFR Part 53. Furthermore, the research maintains an important balance in focusing on both CONOPs and the conduct of operations, which is important for the evaluation of novel designs.
In 2023, the RES HFE staff sponsored a seminar called Automation in the Nuclear Industry and invited Halden HTO researchers to speak to the agency on the topics of automation, autonomy, and their potential effects on human performance.
The work performed under Operator Performance in Highly Automated Plants (e.g., HTOR-052 and HTOR-053) was also published in the scientific literature, under the title The Failure to Grasp Automation Failure (Skraaning and Jamieson, 2023). Later, in 2024, RES HFE and HRA staff members published an invited commentary on the latter paper, entitled A Regulatory Perspective: Have We Done Enough on Grasping Automation Failure? (Xing and Hughes Green, 2024). This commentary acknowledged the important contributions of the human factors community to automation research and evaluated the NRCs framework of human factors, HRA, and DI&C against the automation failure taxonomy of Skraaning and Jamieson (2023).
3.7.2 Regulatory Use of Project Outcomes (2021-2023)
Table 2-11 in section 2.4.1 and Table 2-12 in section 2.4.2 contains project details and Halden HTO references.
These projects demonstrated the possible effects of certain SMR control room features (e.g.,
multiunit and highly automated designs) on human performance. The Halden SMR simulator, which represents an integral PWR, was established during this research period and is capable of simulating up to 12 units with a maximum output power of 50 MWe each. This capability enables the systematic examination of human performance during multi-unit operations, which permits research to strengthen the technical basis for the use of these and similar SMR features.
For example, empirical data collected using the simulator documented recent developments in staffing concepts for different SMR designs, the extent of automated functions and passive safety systems, and their implications for plant operation (HTOR-011). The studies add to an understanding of operator monitoring strategies, task prioritization, and response to scenarios with multi-unit failures, all questions of great interest to the NRC staff.
At present, the NRC is preparing for reviews of advanced reactor applications, which it anticipates will propose novel approaches to reactor operations (NRC, 2021b, NRC, 2024c). As discussed in the NRCs 2021 white paper Ground Rules for Regulatory Feasibility of Remote Operations of Nuclear Power Plants and in SRM-SECY-23-0021, dated March 4, 2024, these novel approaches may include remote facility operation; increased reliance on autonomous operations for safety-significant structures, systems, and components; and management of
3-78 multiple units from a single control room (Hughes Green and Morrow, 2024). Halden HTOs work in this area is useful because it has (1) focused on automation in relation to known CONOPs of SMRs, (2) investigated the relationship between design concepts and conduct of operations, and (3) identified areas of safety significance in design concepts and conduct of operations, including implications for human performance related to advanced automation and automation failures.
Specifically, Halden HTOs work on CONOPs and conduct of operations provided insights to which the NRC staff would not otherwise have access, except during the ISV step of the verification-and-validation HFE element of the licensing review process. Halden HTOs findings about how both the design and the CONOPs influence the conduct of operations were part of the reason for the proposed requirement in 10 CFR 53.730(c) to submit a facility CONOPs as part of combined and operating license applications. Unlike applications submitted under 10 CFR Part 50, Domestic Licensing of Production and Utilization Facilities, or 10 CFR Part 52, applications for advanced reactors under 10 CFR Part 53 must include seven CONOPs dimensions; these are expanded upon in appendix A to DRO-ISG-2023-03 (NRC, 2022b). Outgrowths of this work have informed the ongoing development of supplementary guidance on scalable HFE reviews and CONOPs submittals under 10 CFR Part 53.
NRC anticipates higher levels of automation for advanced NPP designs and in the modernization of existing control rooms. Halden HTOs examination of operator use of advanced automation and instances of automation failure contributes to the technical basis on human-automation interactions for complex process control. From a regulatory perspective, Halden HTO has made several important contributions. First, it has provided a comprehensive analysis of automation failure events in complex systems, not just in a traditional laboratory setting. Second, its work addressed a broad range of complex cognitive processes, beyond the slice of processes traditionally represented in the human information processing model. Third, its analysis focused on the causes and mechanisms of automation failure. Fourth, its conclusions incorporated the effects of a variety of factors, including automation systems, design logic and components, DI&C, human-automation integration, and human and organizational factors. Finally, it produced a taxonomy that may inspire experimental research scenarios that are more relevant to the nuclear industry. Halden HTOs study has also inspired the RES staff to review the relevant regulatory guidance on addressing automation failure (Xing and Hughes Green, 2024).
Broadly speaking, Halden HTOs work in this area can be used to support a risk-informed path in NRC regulatory reviews. It could help identify which human actions have safety significance; whether human operators are using an unexpected conduct of operations when faced with novel technologies, designs, or CONOPs; and whether operators are performing unplanned, unintended actions (i.e., actions the designer did not intend) that could have safety-significant consequences. The work provides a good platform for studying unexpected human behavior and errors of commission.
Similarly, this work supports methods used in HRA and PRA. For example, the base human error probabilities (HEP) values used in existing HRA methods were derived from the designs and conduct of operations for traditional plants, which are entirely different from those of operating SMRs. Similarly, the HEP values currently used in PRA were estimated for well-trained operators; the basis for HEPs for nonlicensed persons is unknown. Halden HTOs data could help quantify the differences in HEPs between licensed and nonlicensed individuals.
Other topics of interest related to multi-unit CONOPs include teamwork, the potential impact of removing three-way communications, and human error in multi-unit operations. Broadly
3-79 speaking, Halden HTOs data serve as an independent study of HEPs for the types of CONOPs found in SMRs.
This work may also support cognitive task analysis and workload assessment for highly automated SMR and advanced reactor concepts. Many of the review criteria in the task analysis element of NUREG-0711 are not applicable to highly automated SMRs (NRC, 2020b). With SMR CONOPs, many operator tasks are shifted from observable, time-consuming actions to unobserved, invisible actions that still require cognitive work. For example, it takes only half a second for an operator to perceive a display parameter. In such contexts, it is not appropriate to assess the operator's workload through traditional measures of time use; instead, one should use a half-second time sampling strategy. Halden HTOs work in this area can help to identify better approaches to task analysis for highly automated settings, as a basis for designing HSIs, procedures, staffing, and training.
Cognitive task analysis also considers workload assessment and staffing levels. Workload is influenced by the conduct of operations; within the same plant design, operators will experience different levels of workload if they conduct the work in different ways. For example, in Halden's early automation experiments, some crews worked with and even moved ahead of the automation, rather than being driven out of the loop by the automation. As a result, the automation design did not reduce the operators workload as expected. A confirmatory study on the dependence of workload on the conduct of operations would provide a technical basis for the NRC staff to evaluate applicants workload assessments.
With respect to staffing basis, emergent SMR designs will have more diverse possibilities for staffing plans than traditional plant designs. A wide range of concepts is being considered for remotely monitored and operated, highly automated reactors (Hughes Green and Morrow, 2024). The Halden HTO experimental setting can be used to test these concepts, providing input for the NRCs ongoing revisions of the existing guidance on reviewing staffing exemptions through the review of CONOPs (DRO-ISG-2023-02; NRC, 2022a). Furthermore, insights from Halden HTOs work may suggest improvements to NUREG-1791 and DRO-ISG-2023-03, and they may support the identification of gaps in the existing guidance related to staffing for advanced operational concepts, such as remote facility operation (NRC, 2005; NRC, 2022b).
The design and operational characteristics of advanced reactors may rely on high levels of automation, potentially including a fully autonomous mode and requiring little, if any, human monitoring, control, or intervention; some advanced designs may not even include a control room in the traditional sense. The roles and responsibilities for personnel and automation are also expected to vary greatly in the next generation of reactors, thanks to the use of automation, passive safety features, and inherent safety characteristics. These changes will introduce human factors complexities related to the role of the operator, situational awareness, workload, response time, communication, teamwork, and vigilance (Hughes Green and Morrow, 2024).
Halden HTOs work in this area (see, e.g., Blackett et al., 2022a; Blackett et al., 2022b; Blackett et al., 2023) has provided useful input for a number of high-priority NRC research projects. For example, the NRC is currently collaborating with INL to identify human factors considerations for remote and highly automated operation of nuclear facilities. As part of this project, NRC hosted a Workshop on Human Factors for Remote Operation of Nuclear Facilities on January 31 -
February 1, 2024 (ML24061A181; NRC, 2024d). In addition, a member of the RES HFE staff is serving as a task lead in an international collaboration, under the auspices of the NEA, to study human and organizational factors in SMRs. The NRC is also updating review guidance on minimum staffing levels, such as NUREG/CR-6838, Technical Basis for Regulatory Guidance for Assessing Exemption Requests from the Nuclear Power Plant Licensed Operator Staffing
3-80 Requirements Specified in 10 CFR 50.54(m) (NRC, 2004). Halden HTOs work supports the development of scalable HFE review plans that will enable risk-informed HFE design reviews.
3.8 Digital Systems Architecture and Threat Landscape 3.8.1 NRC Engagement DI&C staff members from RES and NRR and cybersecurity specialists from NSIR provided a significant amount of direction for the research activities in this area from 2021 through 2023.
They provided initial subject-matter expertise for the technical basis and guided the review and analysis portions of the investigation. The RES and NRR HFE staff are also remaining aware of work conducted in this area.
3.8.2 Regulatory Use of Project Outcomes (2021-2023)
Table 2-12 in section 2.4.2 gives project details and Halden HTO reference.
During the program period, a Halden HTO workshop brought together individuals from HTO member organizations and external organizations, including organizations in surrogate domains (e.g., petroleum and maritime), to discuss the problems of operationalizing threat assessment and identifying system-specific vulnerabilities. Halden HTO researchers also compiled a literature review on cybersecurity threat landscapes in the nuclear sector, where they identified challenges similar to those experienced in other domains. They documented a set of over 200 sources of information on threats and vulnerabilities. A security analysis technique was piloted on high-level NPP system diagrams.
This work supported the NRCs development of best practices and recommendations for safety assurance. It informed the development of guidance for licensees (e.g., regulatory guides) and for the NRC staff (e.g., supplements to existing staff guidance such as NUREG-0800; NRC, 2023g). It also supported the development of HFE review guidance related to remote and autonomous operations. In a remotely operated facility, it is crucial to guarantee the reliability of communication networks and protect them from cyberattacks to ensure safe and secure operation.
4-1 4 FUTURE DIRECTIONS Research discussed in this chapter has been identified as necessary to inform the NRCs regulatory direction. Halden HTOs objective for 2024-2026 is to continue studying the safety aspects of digital automation, human performance, reliability, and organization in various stages of the plant lifecycle, including design, modernization, operation (especially during accident situations), and decommissioning. A focus on SMRs through the lens of human and organizational performance is anticipated. In the 2024-2026 period, Halden HTO will carry out research in seven high-priority topic areas covering human factors and digital systems in both existing and new reactors.
4.1 Planned Topic Areas Halden HTO has selected the topic areas for 2024-2026 based on industry trends, the interests of the Halden Board of Management, and direct discussions with Halden HTO members. The list of topic areas below is quoted from the organizations website (https://www.oecd-nea.org/jcms/pl_61937/halden-human-technology-organisation-hto-project),
where each area is discussed in greater detail.
Topic 1 on Human Performance addresses four thematic areas:
Cognitive failure modes in digital control rooms Crew dynamics and cognitive dependency in digital control rooms Decision-making under uncertainty with decision aids Learning from success and failure to understand event recurrence Topic 2 on Digital I&CSafety Assurance addresses three thematic areas:
State-of-the-art to assure safety of digital systems Addressing safety assurance through use cases State-of-the-art in hazard analysis Topic 3 on Control Room Design and Evaluation addresses five thematic areas:
Human factors validation of new designs The assessment of situation awareness for modern plant designs The use of overview displays Display designs supporting multi-unit supervision Virtual reality use cases for NPP control room operator training and early system validation
4-2 Topic 4 on Human-Automation Collaboration and Multi-Unit Operation addresses five thematic areas:
Human performance challenges in operation of SMRs Work design in advanced reactors Operator performance in highly automated plants Cooperating with adaptive procedures HRA for advanced reactors Topic 5 on Digital Systems for Operation and Maintenance addresses three thematic areas:
Effects of adopting digital twins for operational support Maintenance optimisation for nuclear power plants Fostering a safety culture in outage organisations Topic 6 on Sustainable Decommissioning and Asset Lifecycle Management addresses two thematic areas:
A holistic digital approach to safety at legacy and decommissioning sites Impact of AI, data, and robotics on regulatory acceptance Topic 7 on Cybersecurity for Main Control Rooms addresses three thematic areas:
Cybersecurity of black-box solutions and technology Distinguishing attacks and faults to support effective response Cyber situational awareness 4.2 Regulatory Significance of Continued Collaboration The NRC staff has reviewed Halden HTOs planned and ongoing work for the 2024-2026 program period. During the most recent program proposal meeting, the NRC staff ranked the proposed research activities to communicate preferences for the direction and approach of each project. Selected activities identified as having particular regulatory significance are discussed below.
4.2.1 Human Factors and Human Reliability Halden HTOs research proposal for 2024-2026 includes four primary areas of study related to human factors and human reliability: human performance, control room design and evaluation, human-automation collaboration, and simulator-based training. Work in these areas will leverage existing NRC resources, such as the Human Performance Test Facility. It will also inform the development of the technical bases for high-priority sections of proposed 10 CFR Part 53. The NRC has expressed particular interest in the following activities within these research areas: (1) cognitive failure modes in digital control rooms, (2) human factors validation of new designs, (3) human performance challenges in operation of SMRs, and (4) XR use cases for NPP control room operator training. The staff has ranked these activities as having a high priority in meetings and interactions with Halden HTO researchers.
4-3 Research on cognitive failure modes will enhance systematic HRA data collection methods and help build datasets across plants. The project is expected to support the NRCs efforts related to control room modernization, new reactors, and 10 CFR Part 53 by providing empirical data on human failure modes and failure mechanisms in environments using DI&C. It will also provide insights for enhancing HFE regulatory guidance.
Research on human factors validation of new designs will illuminate how new design assessment techniques may support design analysis and validation. Current guidelines for NPPs require high-fidelity simulators for ISV, because of the complexity of human-system interactions. However, ISV may not be appropriate for new designs (such as advanced reactors, fuel cycle facilities, or research reactors), necessitating alternative approaches. One possibility is to use MSV, collecting evidence across the design and operation stages, to assess whether a design effectively meets the intended use, with criteria varying on a case-by-case basis.
Ongoing research is investigating ways to validate advanced reactor designs without relying solely on full-scale simulators, by integrating human factors into the process.
Continued collaboration around human performance challenges in the operation of SMRs will support the NRCs licensing and regulatory activities for small modular reactors and advanced reactors, within the scope of 10 CFR Part 53 and regulations on scalable HFEs. In particular, it will inform several high-priority HFE guidance development activities for advanced reactors, including the development of technical bases for remotely monitored and controlled reactors with high levels of automation, function allocation, modernized risk assessments for reviewing human actions, and guidance for CONOPs submittals. Ongoing work should consider the changing role of the human given the anticipated reliance on passive safety features (e.g., NRC, 2008).
Research on XR use cases for NPP control room operator training may support licensing activities with respect to 10 CFR Part 53 ISGs and usefulness of extended reality for design validation. The results of this work may have implications for operator licensing activities related to the level of simulator fidelity used for operator training. Research in this area could inform the NRC staffs understanding of the effectiveness of extended reality to replace some aspects of simulator-based training for training and skill development. This training may focus on hybrid or fully remote operations and may have integrations with other advanced technologies such as digital twin systems, as they may support remote operations where personnel are not located on site. Ongoing work in this area may be referenced to inform the technical basis modernization of reactor operator training.
4.2.2 Digital Instrumentation and Control Halden HTOs research proposal for 2024-2026 includes four primary areas of study related to DI&C: D&C safety assurance, digital systems for operations and maintenance, digital transformation of decommissioning, and cybersecurity for main control rooms. The NRC is especially interested in (1) state-of-the-art to assure digital safety systems, (2) cyber situational awareness, and (3) a framework for efficient development of digital twins, which may be addressed under Topic 5.
Collaboration on the state-of-the-art to assure digital safety systems will contribute to the continued safe operation of operating reactors and support for the regulation of new and advanced reactors. New work in this area will support the modernization of existing guidance.
For instance, the NRCs design review guide on instrumentation and controls is high level and relies on the future development of detailed safety case guidance. This research activity is
4-4 expected to help move the guidance from a deterministic and prescriptive approach to the performance-based approach envisioned in the design review guide. The activity is also expected to support updates to staff review guidance for new license applications from non-light-water reactors.
The NRC should also monitor the work conducted in the Halden Cybersecurity Centre, which may provide useful insights for realistic testing, vulnerability assessment, incident response, and evaluation of human preparedness. Specifically, the research activity cyber situational awareness is expected to support regulatory activities related to inspecting and assessing licensees cyberattack response programs. This work is also expected to contribute to the development of an assessment framework for operators and licensees cybersecurity awareness in various reactor operation scenarios, including current-fleet reactor operation, autonomous operation, and remote operation. A joint study of remote operation, involving the NRC, Halden HTO, and DOE national laboratory partners, is already underway. Ultimately, this research is expected to support the development of a technical basis for licensees cybersecurity training requirements related to remote operation.
Work toward a framework for efficient development of digital twins is needed to provide context on where and how digital twins should be deployed. This work will support the identification of measures, approaches, and use-case demonstrations for regulatory assessments of digital twin use. It will also inform the NRC staff about the safety implications of digital twin technologies.
Further anticipated outcomes of this work are proposed regulatory use cases (such as understanding the capabilities of digital twins), characterizations of safety aspects of new technologies (including digital twins), and a comprehensive evaluation of the limitations of digital-twin-based tools. The Halden HTO experience gained in the Digital Transformation of Decommissioning activities will also be useful in future research on applying robotic automation for hazardous tasks in NPPs.
4.2.3 Cybersecurity Halden HTOs research proposal for 2024-2026 includes three primary areas of study related to cybersecurity: cybersecurity of black-box solutions and technology, distinguishing attacks and faults to support effective response, and cyber situational awareness. From a regulatory point of view, the NRC is particularly interested in cybersecurity considerations related to nuclear applications of remote and autonomous systems, characterizing nuclear cybersecurity states with AI and ML, and AI/ML technologies generally.
Haldens research on cyber situational awareness may support the NRCs research on cybersecurity considerations for nuclear applications of remote and autonomous systems by providing methodologies and tools for identifying novel cybersecurity concerns affecting such systems. For example, cyberattacks on remote or autonomous systems may be manifested in ways not obvious to plant staff; technical bases are needed to evaluate whether new approaches to establishing operator awareness of plant states resulting from cyberattacks are sufficient. This research also has implications for ongoing NRC work on human factors considerations for remote and autonomous operations.
Halden HTOs research on black-box solutions may support the NRCs research on AI/ML systems, because some AI/ML systems feature black-box elements that cannot be assessed through traditional approaches. Novel approaches to verifying the cybersecurity performance of AI/ML systems may support the NRCs development of technical bases for these technologies.
4-5 Finally, Halden HTOs research on distinguishing cyberattacks from faults may support ongoing NRC research on the utility and feasible of characterizing nuclear cybersecurity states using AI/ML. States resulting from malfunctions and states resulting from cyberattacks may have similar manifestations; technical bases are needed to assess approaches that leverage AI/ML technologies for these types of state classifications.
4.2.4 NRC Technical Training Center The NRCs Technical Training Center (TTC) is an in-house training facility offering classroom and simulator courses for NRC personnel, including the reactor technology courses required for qualification of NRR technical staff. The TTC houses and manages simulator platforms for several reactor technologies, including PWRs, boiling-water reactors, Combustion Engineering, and AP1000. Insights from Halden HTOs work may inform the future development of the TTC simulator capabilities for advanced reactor designs. For example, research under Topic 4, Human-Automation Collaboration and Multi-unit Operation, addressing SMRs and other advanced reactor designs may inform modeling decisions for future TTC platforms. In addition, research on the conduct of operations in advanced reactors could inform the modeling of the expected conduct of operations from each vendor. Data and insights from Halden HTOs work on applications of AR, VR, and XR for operator training and simulation may also be applied, particularly if the TTC pursues digital control systems or plant computer modifications in the future.
4.3 Discussion In addition to the topics given above, the NRC staff has actively exchanged ideas with Halden HTO to address research gaps and to support areas where a technical basis is lacking for new and advanced reactor designs. Halden HTO has expressed interest in collaborating with the NRC staff on these areas. By focusing on gaps in the technical basis and regulatory criteria, Halden HTOs research will support the agencys mission of modernizing its regulatory activities.
For instance, Halden HTO has proposed to examine the still unresolved issues related to cognitive dependency in crew performance. The NRC staff has conveyed its enthusiasm for this research anticipated for the 2024-2026 research cycle and has shared with Halden HTO a list of needed deliverables. These deliverables are expected to support updates and modernization of regulatory guidance in accordance with 10 CFR Part 53.
In the 2024-2026 program period, the NRC staff will continue its collaborations with Halden HTO researchers. To facilitate interaction and support timely engagement between NRC subject-matter experts and Halden HTO researchers, the staff will establish intra-agency project-based coordination groups. Regular meetings will be held to enable close collaboration.
For the projects identified as relevant to supporting regulatory activities, the NRC staff will actively review research methodologies and experimental designs to ensure that they will yield the expected benefits. The staff will also actively integrate research findings from internal projects with new Halden HTO discoveries, fostering research synergy and facilitating knowledge exchange. NRC subject-matter experts will remain engaged in projects, stay up-to-date with research progress, provide reviews for reports, and share new findings with agency staff.
5-1 5 REFERENCES14 10 CFR Part 2, Rules of Practice for Domestic Licensing Proceedings and Procedural Changes in the Hearing Process.
10 CFR Part 50, Domestic Licensing of Production and Utilization Facilities.
10 CFR 50.59, Changes, tests and experiments.
10 CFR Part 52, Licenses, Certifications, and Approvals for Nuclear Power Plants.
Bisio, R., and Bloch, M. (2023). Impact of Adaptive Automation for Supporting Operation in Nuclear Power Plant, an Exploratory Study. In Proceedings of ESREL 2023.
Bisio, R., and Massaiu, S. (2021). An Approach to Sharing Human Performance Data and Findings in the International Nuclear Research Community. In Proceedings of ESREL 2021.
Blackett, C., Eitrheim, M., and Bye, A. (2022a). The Challenge of Assessing Human Performance and Human Reliability for First-of-a-Kind Technologies. In Proceedings of the Probabilistic Safety Assessment and Management (PSAM 16), Honolulu, Hawaii.
Blackett, C., Eitrheim, M., McDonald, R., and Block, M. (2022b). Human Performance in Operation of Small Modular Reactors. In Proceedings of the Probabilistic Safety Assessment and Management (PSAM 16), Honolulu, Hawaii.
Blackett, C., Skraaning, G., Kaarstad, M., and Eitrheim, M.H.R. (2023). Human Factors Considerations for Remote Operation of Small Modular Reactors. 13th NPIC and HMIT, Knoxville, Tennessee.
Bley, D.C. (2021). Perspective on an Independent Shift Technical Advisor. U.S. Nuclear Regulatory Commission, Washington, DC.
Braarud, P.O. (2020). An Efficient Screening Technique for Acceptable Mental Workload Based on the NASA Task Load IndexDevelopment and Application to Control Room Validation. International Journal of Industrial Ergonomics, 76, 102904.
Bye, A. (2023). Future Needs of Human Reliability Analysis: The Interaction between New Technology, Crew Roles and Performance. Safety Science, 158, 105962.
Chockalingam, S., Pieters, W., Teixeira, A.M., and van Gelder, P. (2023). Probability Elicitation for Bayesian Networks to Distinguish between Intentional Attacks and Accidental Technical Failures. Journal of Information Security and Applications, 75, 103497.
Dickerson, K., Grasso, J., Watkins, H., and Green, N.H. (2023). Characterizing Complexity: A Multidimensional Approach to Digital Control Room Display Research. Human Factors and Simulation, 67.
14 For a list of Halden HTO Reports for the 2021-2023 program period, see appendix A-1.
5-2 Dickerson, K., Hildebrandt, M., Green, N.H., Watkins, H., Wang, I., and Gerhardstein, P. (2024).
Technology Affinity in the Nuclear Domain: An Initial Assessment of Operator Attitudes towards Technology. Human Factors and Simulation.
Drøivoldsmo, A., Nystad, E., and Lunde-Hanssen, L. (2022). Analytical Estimation of Operator Workload in Control Rooms: How Much Time Should Be Available for Surveillance and Control? In Proceedings of ESREL 2022.
Eitrheim, M., Fernandes, A., Hurlen, L., and Skraaning, G. (2018). Small-Scale Simulator Studies on Multi-unit Operation, Report No. HWhP-069, Issue 1. Halden, Norway:
OECD Halden Reactor Project.
Hildebrandt, M., Hughes Green, N.M., Blackett, C., Boring, R., Egli, S., Green, B., and Zhang, S. (2023). Human Factors and New Nuclear: Advancements and Future Possibilities. Proceedings of the Human Factors and Ergonomics Society Annual Meeting, 67, 1002-1006, https://doi.org/10.1177/21695067231192555.
Hughes Green, N. and Morrow, S. (2024). Exploring Human Factors Considerations of Remote and Automated Operations for the Nuclear Domain. In Advances in Reliability, Safety, and Security: ESREL 2024 Collection of Extended Abstracts. Krakow, Poland: Polish Safety and Reliability Association.
Institute of Electrical and Electronics Engineers (2018). IEEE Standard Criteria for Safety Systems for Nuclear Power Generating Stations, IEEE 603'-2018, Piscataway, New Jersey, doi: 10.1109/IEEESTD.2018.8573235.
Institute of Electrical and Electronics Engineers (2021). IEEE Guide for Human Factors Engineering for the Validation of System Designs and Integrated Systems Operations at Nuclear Facilities, IEEE 2411'-2021, Piscataway, New Jersey, https://standards.ieee.org/ieee/2411/10357.
Kaarstad, M., and McDonald, R. (2022). Microreactors: Challenges and Opportunities as Perceived by Nuclear Operating Personnel. In Proceedings of ESREL 2022.
Karpati, P. Attwood, S.N., Vikash, K, Raspotnig, C. (2016). Improving Safety Arguments for Better Comprehension - Lessons Learned from a Case Study in the Nuclear Field.
HWR-1193. Halden, Norway: OECD Halden Reactor Project.Lin, J., Matthews, G.,
Hughes, N., and Dickerson, K. (2022). Novices as Models of Expert Operators:
Evidence from the NRC Human Performance Test Facility. In Human Factors and Simulation: The 13th International Conference on Applied Human Factors and Ergonomics (AHFE 2022), eds. J. Wright and D. Barber.
Lin, J., Matthews, G., Schreck, J., Dickerson, K., and Green, N.H. (2023). Evolution of Workload Demands of the Control Room with Plant Technology. Human Factors and Simulation, 57.
Massaiu, S. (2023). Operator Decision Strategies in Nuclear Control Centres: A Domain-Specific Information Flow Map. In Proceedings of ESREL 2023.
5-3 NRC (1992). On Site Analysis of the Human Factors of an Event at Crystal River Unit 3 December 8, 1991 (Pressurizer Spray Valve Failure) (EGG-HFRU-10085). Washington, D.C.: U.S. Nuclear Regulatory Commission. Access and Management System (ADAMS)
Accession Number: ML20092G489.
NRC (1994a). SECY-94-084, Policy and Technical Issues Associated with the Regulatory Treatment of Nonsafety Systems in Passive Plant Designs, dated March 28, 1994.
Washington, D.C.: U.S. Nuclear Regulatory Commission. ADAMS Accession No:
ML003708068.NRC (1994b). SRM-94-084. Staff Requirements Memorandum (SRM),
June 30, 1994. Washington, D.C.: U.S. Nuclear Regulatory Commission. ADAMS Accession No: ML003708098.
NRC (2000). Technical Basis and Implementation Guidelines for A Technique for Human Event Analysis (ATHEANA) (NUREG-1624, Revision 1). Washington, D.C.: U.S. Nuclear Regulatory Commission. ADAMS Accession No: ML003719212.
NRC (2004). Technical Basis for Regulatory Guidance for Assessing Exemption Requests from the Nuclear Power Plant Licensed Operator Staffing Requirements Specified in 10 CFR 50.54(m) (NUREG/CR-6838). Washington, D.C.: U.S. Nuclear Regulatory Commission.
ADAMS Accession No: ML040580289.
NRC (2005). Guidance for Assessing Exemption Requests from the Nuclear Power Plant Licensed Operator Staffing Requirements Specified in 10 CFR 50.54(m) (NUREG-1791).
Washington, D.C.: U.S. Nuclear Regulatory Commission. ADAMS Accession No:
NRC (2008). Policy Statement on the Regulation of Advanced Reactors (73 FR 60612; October 14, 2008). Washington, D.C.: U.S. Nuclear Regulatory Commission. ADAMS Accession No: ML082750370.
NRC (2012). Human Factors Engineering Program Review Model (NUREG-0711, Revision 3).
Washington, D.C.: U.S. Nuclear Regulatory Commission. ADAMS Accession No:
NRC (2013). Standard Review Plan for the Review of Safety Analysis Reports for Nuclear Power Plants: LWR Edition (NUREG-0800, Formerly issued as NUREG-75/087)
(NUREG-0800). Washington, D.C.: U.S. Nuclear Regulatory Commission. ADAMS Accession No: ML070810350.
NRC (2015). Workload, Situation Awareness, and Teamwork (NUREG/CR-7190). Washington, D.C.: U.S. Nuclear Regulatory Commission. ADAMS Accession No: ML15078A397.
NRC (2017). An Integrated Human Event Analysis System (IDEAS) for Nuclear Power Plant Internal Events At Power Application (NUREG-2199, Volume 1). Washington, D.C.: U.S.
Nuclear Regulatory Commission. ADAMS Accession No: ML17073A041.
NRC (2020a). Adaptive Automation: Current Status and Challenges (RIL 2020-05). Washington, D.C.: U.S. Nuclear Regulatory Commission. ADAMS Accession No: ML20176A199.
NRC (2020b). Human System Interface Design Review Guidelines (NUREG-0700, Revision 3).
Washington, D.C.: U.S. Nuclear Regulatory Commission. ADAMS Accession No:
5-4 NRC (2020c). Integrated Human Event Analysis System for Event and Condition Assessment (IDHEAS ECA) (RIL 2020-02). Washington, D.C.: U.S. Nuclear Regulatory Commission.
ADAMS Accession No: ML20016A480.
NRC (2020d). Proposed Rule: Rulemaking Plan on Risk Informed, Technology Inclusive Regulatory Framework for Advanced Reactors (RIN 3150-AK31; NRC 2019-0062)
(SRM SECY-20-0032). Washington, D.C.: U.S. Nuclear Regulatory Commission.
ADAMS Accession No: ML20276A293.
NRC (2021a). Elimination of the Shift Technical Advisor for the NuScale Design (SECY 0039). Washington, D.C.: U.S. Nuclear Regulatory Commission. ADAMS Accession No:
NRC (2021b). Ground Rules for Regulatory Feasibility of Remote Operations of Nuclear Power Plants. Washington, D.C.: U.S. Nuclear Regulatory Commission. ADAMS Accession No:
NRC (2021c). NuScale Topical Report - Control Room Staffing Plan. Washington, D.C.: U.S.
Nuclear Regulatory Commission. ADAMS Accession No: ML21139A226.
NRC (2022a). Draft Interim Staff Guidance Augmenting NUREG-1791, Guidance for Assessing Exemption Requests from the Nuclear Power Plant Licensed Operator Staffing Requirements Specified in 10 CFR 50.54(m), for Licensing Commercial Nuclear Plants under 10 CFR Part 53 (DRO-ISG-2023-02). Washington, D.C.: U.S. Nuclear Regulatory Commission. ADAMS Accession No: ML22272A049.
NRC (2022b). Draft Interim Staff Guidance: Development of Scalable Human Factors Engineering Review Plans (DRO-ISG-2023-03). Washington, D.C.: U.S. Nuclear Regulatory Commission. ADAMS Accession No: ML22272A051.
NRC (2022c). Integrated Main Event Analysis System for Event and Condition Assessment (NUREG-2256). Washington, D.C.: U.S. Nuclear Regulatory Commission. ADAMS Accession No: ML22165A282.
NRC (2022d). Safety Evaluations of Adaptive AutomationSuitability of Existing Review Guidance (RIL 2020-06). Washington, D.C.: U.S. Nuclear Regulatory Commission.
ADAMS Accession No: ML22006A019.
NRC (2023a). Annual Update on Activities to Modernize the U.S. Nuclear Regulatory Commissions Digital Instrumentation and Controls Regulatory Infrastructure and License Amendment Requests (SECY-23-0092). Washington, D.C.: U.S. Nuclear Regulatory Commission. ADAMS Accession No: ML23228A226.
NRC (2023b). Artificial Intelligence Strategic Plan (FY 2023-2027) (NUREG-2261). Washington, D.C.: U.S. Nuclear Regulatory Commission. ADAMS Accession No: ML23132A305.
NRC (2023c). Human Performance Test Facility (HPTF), Volume 1: Systematic Human Performance Data Collection Using Nuclear Power Plant Simulator: A Methodology (RIL 2022-11). Washington, D.C.: U.S. Nuclear Regulatory Commission. ADAMS Accession No: ML22356A207.
5-5 NRC (2023d). Human Performance Test Facility (HPTF), Volume 2: Comparing Operator Workload and Performance between Digitized and Analog Simulated Environments (RIL 2022-11). Washington, D.C.: U.S. Nuclear Regulatory Commission. ADAMS Accession No: ML22356A208.
NRC (2023e) Human Performance Test Facility (HPTF), Volume 3: Supplemental Exploratory Analyses of Sensitivity of Workload Measures (RIL 2022-11). Washington, D.C.: U.S.
Nuclear Regulatory Commission. ADAMS Accession No: ML22356A209.
NRC (2023f). SECY-23-0021: Proposed Rule: Risk-Informed, Technology-Inclusive Regulatory Framework for Advanced Reactors (RIN 3150-AK31). Washington, D.C.: U.S. Nuclear Regulatory Commission. ADAMS Accession No: ML21162A093.
NRC (2023g). Standard Review Plan for the Review of Safety Analysis Reports for Nuclear Power Plants (NUREG-0800). Washington, D.C.: U.S. Nuclear Regulatory Commission.
ADAMS Accession No: ML092330826.
NRC (2024a). Micro Reactor Licensing and Deployment Considerations: Fuel Loading and Operational Testing at a Factory (SECY-24-008). Washington, D.C.: U.S. Nuclear Regulatory Commission. ADAMS Accession No: ML23207A250.
NRC (2024b). NuScale US460. Washington, D.C.: U.S. Nuclear Regulatory Commission.
Retrieved August 21, 2024, from https://www.nrc.gov/reactors/new-reactors/smr/licensing-activities/current-licensing-reviews/nuscale-us460.html.
NRC (2024c). Proposed Rule: Risk Informed Technology Inclusive Regulatory Framework for Advanced Reactors (RIN 3150-AK31) (SRM SECY-23-0021). Washington, D.C.: U.S.
Nuclear Regulatory Commission. ADAMS Accession No: ML24064A047.
NRC (2024d). Public Meeting Summary of Workshop on Human Factors for Remote Operation of Nuclear Facilities. Washington, DC: U.S. Nuclear Regulatory Commission. ADAMS Accession No: ML24061A181.
Nuclear Energy Agency (2017). Human Factors Validation of Nuclear Power Plant Control Room Designs and Modifications: Proceedings of the Expert Workshop, NEA/CSNI/R(2016)17, OECD Publishing, Paris, France.
Nuclear Energy Agency (2019). Multi-Stage Validation of Control Room Designs and Modifications, December 2019.
Nuclear Energy Institute (2016). Diverse and Flexible Coping Strategies (FLEX)
Implementation Guide, NEI 12-06, December 2016.
OECD NEA Halden HTO Project (2024). "Halden HTO Programme Achievements 2021-2023, HTO-096, May 2024. (Halden HTO, 2024)
OHara, J.M, and Higgins, J.C. (2010). "Human-system Interfaces to Automatic Systems: Review Guidance and Technical Basis". United States. https://doi.org/10.2172/1013461.
https://www.osti.gov/servlets/purl/1013461. ADAMS Accession No: ML102720251.pdf.
5-6 Park, J. and Solberg, E. (2023). Development of Detailed Questions for Investigating the Status of Human and Organizational Factor (HOF) Issue Identifications from Event Investigation Processes. In Proceedings of ESREL 2023.
Presley, M., Boring, R., Ulrich, T., Medema, H., Mohon, J., Delvecchio, M., Massaiu, S., Bye, A.,
Park, J., Kim, Y., and Julius, J. (2021). A Taxonomy and Meta-analysis Template for Combining Disparate Data to Understand the Effect of Digital Environments on Human Reliability. In Proceedings of the ANS 2021 International Topical Meeting on Probabilistic Safety Assessment and Analysis (PSA 2021).
Schraagen, J.M., and Roth, E. (2024). Introduction to the Special Issue on Automation Failure.
Journal of Cognitive Engineering and Decision Making, https://doi.org/10.1177/15553434241262865.
Simensen, J. E., Jørgensen, P-A., Toppe, A. L. (2022). Cybersecurity Centre - capabilities and insights to a technical cyber security experiment on critical infrastructure. Paper at EHPRG, March 2022.
Skraaning G., Jamieson G. A. (2023). The failure to grasp automation failure. Journal of Cognitive Engineering and Decision Making. https://doi.org/10.1177/15553434231189375 Solberg, E., and Bisio, R. (2022). Improving Learning by Adding the Perspective of Success to Event Investigations. In Proceedings of ESREL 2022.
Solberg, E., Kwei-Narh, P., and Bisio, R. (2024). Including Successful Performance in an Events Causal Analysis: Test of an Instructional Intervention. Under review for ESREL 2024.
Solberg, E., Nystad, E., and McDonald, R. (2022). Situation Awareness in Outage Work: A Study of Events Occurring in U.S. Nuclear Power Plants between 2016-2020. Safety Science, 158.
Solberg, E., Nystad, E., McDonald, R. (2022). Situation awareness in outage work - A study of events occurring in U.S. nuclear power plants between 2016-2020. Safety Science, 158.
Stephane, L., and Edvardsen, S.T. (2022). Enabling Robotic and Remote Operations in Decommissioning. HPRG presentation, October 2022.
Taylor, C., Hildebrandt, M., Hughes, N., and McDonald, R. (2018). Operator Response to Failure of a Computerized Procedure System. In Proceedings of the 1st International Conference on Intelligent Human Systems Integration: Integrating People and Intelligent Systems (iHSI 2018).
Taylor, C., Hildebrandt, M., McDonald, R., and Hughes, N. (2017). Operator Response to Failures of a Computerised Procedure System: Results from a Training Simulator Study, Halden Work Report No. 1198. Institute for Energy Technology.
U.S. Congress (2023). S.1111, ADVANCE Act of 2023, https://www.congress.gov/bill/118th-congress/senate-bill/1111.
5-7 U.S. Congress (2019). S.512, Nuclear Energy Innovation and Modernization Act, https://www.congress.gov/bill/115th-congress/senate-bill/512.
Wright, A., and Bye, A. (2022). Do Modern Control Rooms Pertain New Error Mechanisms? In Proceedings of ESREL 2022.
Xing, J., and Hughes Green, N. (2024). A Regulatory Perspective: Have We Done Enough on Grasping Automation Failure? Journal of Cognitive Engineering and Decision Making, https://doi.org/10.1177/15553434241231056.
6-1 6 APPENDIX A-1 List of Halden HTO Reports for 2021-2023 Report Number Report Title HTOR-001, Rev. 1 A Framework for Studying Overview Displays: Best Practices HTOR-002 Halden HTO Project Cyber Security Workshop proceedings HTOR-004 A Study on the Application of the Structured Safety Argumentation Approach Guidance on the Halden Safety Fan HTOR-005 Connecting Safety Standards to Safety HTOR-006 Independently Verifiable Evidence for Safety Assurance: Results of Expert Survey and Literature Review HTOR-007 Controlling a Plant with Adaptive Automation: Perspectives and Challenges HTOR-008 Halden Safety FanHAZOP and Revised System Specification HTOR-010 A Review of Theories, Models, and Measures of Human-Automation Trust HTOR-011 Small Modular ReactorsState of the Art Review 2021 HTOR-013 A Systematic Review of Monitoring and Detection Methods and Tools for Cyber Security in Operational Technology (OT) Environments HTOR-014 Event Investigations: The Added Perspective of Success HTOR-016 Literature Review and Study Plan for the HTO-1.3 Activity Decision-Making under Uncertainty HTOR-017 Towards a Development Framework for Digital Twins HTOR-020 Human Factors Validation of Control RoomsInsights and Experiences from Nuclear Industry Projects HTOR-021 Measuring Cognitive Workload in the Nuclear Control Room: A Review HTOR-022 Overview DisplaysBest Practices and Evaluation of Framework HTOR-023 HTO Workshop ProceedingsRisk-and Evidence-Informed Safety Assurance for Digital Instrumentation and Control (DI&C)
HTOR-024 Summary of Haldens Workshop on Adaptive Automation HTOR-025 The Concept of Operations in a Digital Control Room HTOR-026 Automated Assessment of Field Worker Performance: Theoretical Framework and Performance Indicators in XR Training Simulators HTOR-029 Human Performance in Operation of SMRs: Results from 2022 and 2023 Small Studies HTOR-030 Overview Displays in Nuclear Control Rooms: A State of Practice Review HTOR-031 Learning from Successful Performance in Failure EventsA Case-Based Study HTOR-032 Halden Safety FanApplication of STPA HTOR-033 Proceeding of HTO Workshop SeriesSafety Assurance through Correct-by-Construction Techniques HTOR-034 Evidences for the HARDENS Case HTOR-035 Review of Situation Awareness Measures Applied in the Halden Human-Machine Laboratory
6-2 HTOR-039 Adaptive Decision-Making in Control Room Emergencies: Results of a HAMMLAB Study with the HPWR Simulator HTOR-040 Augmented Reality for On-Site Human Factors Control Room Assessment HTOR-042 Perceived Usefulness and Usability of Overview Displays in Nuclear Control Rooms HTOR-043 Summary of the HTO Workshop on: Operationalizing Threats and Vulnerabilities in Relations to System Specific Effects HTOR-044 Testbeds for Cyber Security of Nuclear Power Plants: Key Characteristics and Architecture Design HTOR-045 A Comparative Analysis of State-of-the-Practice Intrusion Detection Systems HTOR-046 Including Successful Performance in the Scope of the Event Analysis HTOR-049 Groupthinka Literature Review HTOR-050, Rev. 2 Physical Location of Shift Technical Advisor in Nuclear Power Plant ScenariosImpact on Performance?
HTOR-052 Operator Responses to Systemic Automation Failure: 1st HAMMSAT Experiment HTOR-053 Getting a Grip on Systemic Automation Failure HTOR-054 Systematic Literature ReviewCyber Security Threat Landscape for Nuclear HTOR-057 Human Performance Repository: Lessons Learned and Experience in Publishing the Site for Halden Reports Search HTOR-058 Digital Twins for Operation and Maintenance in the Nuclear Energy Sector: An Overview HTOR-059 Advanced Condition Monitoring for Decision Support HTOR-060 A Study on Radiological Risk Visualization and Perception Using Wearable Augmented Reality HTOR-061 Spatial Computing and Augmented Reality: Lessons Learned from Using and Developing Open Standards to Enable Safety-Oriented Decision-Support for Field Workers HTOR-062 Automated Assessment of Decommissioning Field Workers Performance in XR trainingUser Tests and Results HTOR-063 Enabling Robotic and Remote Operations in Decommissioning:
Safety-Centered Framework HTOR-065 Effects of Staffing on Human Performance in a Multi-unit SMR Simulator HTOR-066 Spatial Computing and Augmented Reality for Hazard Mapping and Visualization HTOR-068 Operator Performance in Digital Control Rooms: Insights from On-Site Observation of Simulator Training HTOR-071 The Assessment of Cognitive Workload and Situation Awareness in Human Factors Validation HWR-1313 Procedure Use, Communication, and Teamwork in a Digital Control Room
6-3 A-2 Infographic on the Significance of the NRC-Halden HTO Collaboration