ML23279A093
| ML23279A093 | |
| Person / Time | |
|---|---|
| Issue date: | 09/25/2023 |
| From: | Natalee Green, Jing Xing NRC/RES/DRA/HFRB |
| To: | |
| References | |
| Download: ML23279A093 (1) | |
Text
Failure Modes in Human-Automation Integration Jing Xing, Niav H. Green U.S. Nuclear Regulatory Commission Presented by Jing Xing to EHPRG, Sep-2023
Outline I.
Overview of human-automation integration II.
A framework of integrating digital instrumentation & control (DI&C) and human factors engineering (HFE)
III.
Insights on grasping automation failure 2
DI&C in nuclear power plants
- Control room modernization, new reactors, advanced reactors 3
Sensors /
signals Control logic Software Soft control Network DI&C elements Information Displays Alarm Systems Decision Support Systems Computer-based Procedures
Automation systems Autonomy DI&C systems
Differences in analog and digital I&C Analog - hardware degradation
- Limited failure modes
- Able to be pre-determined
- Mostly traceable Digital - programmable hardware and software faults
- Unlimited failure modes
- Not able to be pre-determined
- Often untraceable
Our understanding of DI&C failure
- DI&C failure databases such as COMPSIS.
- Event reports and literature on automation failure Boeing 737 accidents Failure to grasp automation failure (Skraaning & Jamieson, 2023)
- The revealed failures are merely the tip of the iceberg.
- Most DI&C failures are not failure in DI&C, but the failure in human operators using DI&C systems or responding to unexpected DI&C behaviors.
NRC regulatory and licensing activities in Human-automation integration 6
NUREG-0800 - Standard Review Plan, Chapter 7 on I&C, Chapter 18 on human factors engineering (HFE)
NUREG-0711 - Guidance for HFE review NUREG-0700 - Guidance for human-system-interface review DI&C-ISG Guidance for DI&C review DRO-ISG-2022 Guidance for Development of Scalable HFE review plans for submittals under proposed 10 CFR Part 53 for advanced reactors Control room design
NRC regulatory and licensing activities in Human-automation integration 7
continue to improve integration and communication among DI&C technical reviews, HFE reviews continue to improve its oversight programs for DI&C modifications develop guidance for assessing systems engineering approaches for the DI&C design and human factors life-cycle evaluation, which are important for ensuring that approved DI&C designs are appropriately integrated to maintain safety functionality.
explore potential avenues for increasing the collection and communication of DI&C operating experience In 2022, an interdisciplinary team of NRC staff working in DI&C, HFE, and risk analysis systematically evaluated the findings from investigative reports of BOEING 737 crashes.
The NRC team recommended focusing on the following areas to continue to improve DI&C licensing and regulatory oversight:
Enhancing human-automation integration to ensure safety functionality and safety operation Purpose of the study:
Build a technical basis for enhancing / developing guidance on human-automation integration to ensure safety functionality and safety operation Approaches:
Review NPP DI&C events, automation failure events / accidents, and simulator studies of human-automation integration in the literature Understand DI&C and human failure modes Identify human-automation integration deficiencies in the events /
accidents Develop a framework to incorporating the findings into the NRCs regulatory guidance and review process.
Outline I.
Overview of human-automation integration II.
A framework of integrating digital instrumentation & control (DI&C) and human factors engineering (HFE)
III.
Insights on grasping automation failure 9
10 Sensors /
signals Control logic Software Soft control Network DI&C elements Functional Requirements Analysis and Function Allocation Staffing and Qualifications Task Analysis Treatment of Important Human Actions Human-System Interface Design Procedure Development Training Program Development Human Factors Verification and Validation Human-System integration Detection Understanding Decisionmaking Action execution Interteam coordination Human cognitive system A framework of integrating DI&C and HFE
Types of failures in DI&C elements 11 DI&C element Failure modes Sensors / signals Sensor failure Unreliable signals Control logic Boundary conditions not clearly defined Problems in control strategies or algorithms Failure of automation disengagement Unclear or unexpected change in automation modes Dependency between systems or subsystems Software Errors or inappropriate use of the database Poor Software standardization, e.g., similarity of terms and units may be superficial, software versions may proliferate Unclear or unidentified software failures Network Network congested Soft control Failed execution Unauthorized execution
Functional Requirement Analysis deficiencies 12 Authority - describes the ability of the automated system to override or block human input, and vice-versa o
operators have responsibility but lack authority o
mode transitions may be uncommanded o
communication between automation and other systems is unsupervised o
control authority may be diffused o
automation puts operators out of loop
- Complexity
- Information requirements
- Functionality definitions
- System Coupling or dependency
- Observability or feedback
- Autonomy
Human-System-Interface deficiencies 13
- Human-centered design - the degree to which human characteristics, capabilities, limitations, and preferences are considered in automation design.
o operational knowledge may be lacking in design process o
cultural differences may not be considered Displays and controls o
cognitively demanding information integration may be required to use the automation o
data access may be difficult o
critical data may not be directly visible to operators o
controls of automation may be poorly designed o
lack of data entry verification o
flaw in automation state indication o
automation reset variables are not known to operators o
inadequate feedback from system to operators.
o lack of confirmation of actions execution
System Verification and Validation deficiencies 14 NUREG-0711 states, Verification and validation (V&V) evaluations comprehensively determine that the final HFE design conforms to accepted design principles and enables personnel to successfully and safely perform their tasks to achieve operational goals.
- Not identified or consolidated types of deficiencies in this element
- Lack of information on how V&V was performed or why V&V did not reveal the problems in event / accident reports reviewed.
- event reports made brief notes that human factors validation failed to capture the problems,
- Future research and guidance enhancement needed on Grasping Automation Failure in V&V process
Human failure modes using automation 15 The NRCs human reliability method. the Integrated Human Event Analysis System (IDHEAS), uses cognition-based failure modes to model human errors; All the human failures in the events / accidents reviewed can be represented with IDHEAS cognitive failure modes.
Failure of Detection (D)
D1. Fail to establish the correct mental model or to initiate detection D2. Fail to select, identify, or attend to sources of information D3. Incorrectly perceive or classify information D4. Fail to verify perceived information D5. Fail to retain, record, or communicate the acquired information Failure of Understanding (U)
Failure of Decisionmaking (DM)
Failure of Action Execution (E)
Failure of Interteam Communication (T)
Insights gained from DI&C / automation events and accidents 16
- Most human-automation failure events involved one or more deficiencies in the HFE element of functional requirements analysis and function allocation (FRA/FA) making the review and consideration of FRA/FA of great important for facilities with high levels of automation.
- Future research is needed to study human-automation integration deficiencies in system validation and understand how a multi-staged system validation approach can support identification of automation failure.
- The current HFE process described in NUREG-0711 focuses on ensuring the state-of-art human factors principles are incorporated into the design and implementation. Grasping automation failure requires risk-focused considerations of failure modes and failure causes.
Welcome feedback!
17 Something old something new Data primarily based on pre-2011 events / accidents Newly developed framework of integrating DI&C and HFE Welcome feedback Whether this framework useful, how it may be useful for your organization Any advice on data we could use to enrich and update the work