ML20083J316
| ML20083J316 | |
| Person / Time | |
|---|---|
| Issue date: | 03/12/2020 |
| From: | James Chang, Dejesus J, Jing Xing Office of Nuclear Regulatory Research |
| To: | |
| J. Chang | |
| Shared Package | |
| ML20083J304 | List: |
| References | |
| Download: ML20083J316 (5) | |
Text
Use of human error data for calculation of human error probabilities in IDHEAS-ECA method Jing Xing, James Chang, Jonathan DeJesus U.S. Nuclear Regulatory Commission, Washington DC, U.S.A.
INTRODUCTION The Integrated Human Event Analysis System for Event and Condition Assessment (IDHEAS-ECA) is a human reliability analysis (HRA) method developed by the U.S. Nuclear Regulatory Commission (NRC) staff to support risk-informed decisionmaking. IDHEAS-ECA analyzes human events and calculates human error probabilities (HEPs) for use in probabilistic risk assessment (PRA) applications. The method is based on the General Methodology of an Integrated Human Event Analysis System (IDHEAS-G) (NUREG-2198) [1].
IDHEAS-ECA calculates HEPs using an HEP model with a set of base HEPs and numeric values of the effects of performance influencing factors (PIFs) on HEPs. This presentation describes how we integrated the human error data available in the literature and human performance databases to estimate the base HEPs and numerical effects of PIFs on HEP (i.e., PIF weights)s in IDHEAS-ECA.
Overview of IDEHAS-ECA HEP model IDHEAS-ECA models human actions in a PRA (i.e., human failure events) using five macrocognitive functions: detection, understanding, decisionmaking, action execution, and teamwork. These macrocognitive functions are based on the cognitive basis for HRA, which was published as NUREG-2114 [2] and are described as follows:
- Detection (D) is noticing cues or gathering information in the work environment.
- Understanding (U) is the integration of pieces of information with a persons mental model to make sense of the scenario or situation.
- Decisionmaking (DM) includes selecting strategies, planning, adapting plans, evaluating options, and making judgments on qualitative information or quantitative parameters.
- Action execution (E) is the implementation of the decision or plan to change some physical component or system.
- Teamwork (T) focuses on how various teams interact and collaborate on an action.
The first four macrocognitive functions (D, U, DM, and E) may be performed by an individual or a team, and teamwork is performed by multiple groups or teams. In general, a human failure event (HFE) occurs due to the failure of any macrocognitive function. In IDHEAS-ECA, the failure of a macrocognitive function is defined as the cognitive failure mode (CFM).
The probability of an HFE (i.e., human error probability) is affected by the scenario context in which the action occurs. The context describes the conditions that challenge or facilitate human performance, IDHEAS-ECA uses PIFs to model the context. Table 1 shows the 20 PIFs used in IDHEAS-ECA in four context categories. Every PIF is associated with a set of attributes, each describing one mechanism that the PIF can increase the HEPs. The effect of an attribute on HEP is measured as the PIF weight, defined as the HEP when the attribute has an impact divided by the HEP when the attribute does not have an impact.
Table 1. PIFs in IDHEAS-ECA Environment and System Personnel Task situation
- Work location
- System and
- Staffing
- Information accessibility I&C
- Procedures, availability and and habitability transparency guidelines, and reliability
- Workplace visibility to personnel instructions
- Scenario familiarity
- Noise in workplace
- Human-
- Training
- Multi-tasking, and system
- Teamwork and interruptions and communication interfaces organizational distractions pathways
- Equipment factors
- Task complexity
- Cold/heat/humidity and tools
- Work
- Mental fatigue
- Resistance to processes
- Time pressure and physical stress movement
- Physical demands IDHEAS-ECA quantifies the HEP of an HFE in two parts: the error probabilities attributed to the CFMs (Pc) and the error probability attributed to the uncertainties and variability in the time available and time required for performing the HFE (Pt). The total HEP is the probabilistic sum of Pc and Pt:
- Pt can also be viewed as the probability that the time required to perform an action exceeds the time available for that action, as determined by the success criteria. Pt assumes that actions are performed at a normal pace and does not account for the increased likelihood of a human error due to time pressure. Time pressure is treated as a PIF and contributes to Pc.
- Pc assumes that the time to perform the HFE is sufficient. Sufficient time means that the HFE can be successfully performed within the time window that the system allows. If operators responses are as trained, then the time available to complete the action is sufficient. Pc captures the probability that the human action does not meet the success criteria due to human errors made in the problem-solving process.
IDHEAS-ECA uses an HEP model to calculate Pc, where the HEP of a CFM is calculated as a base HEP multiplied by the sum of the weights of all applicable PIF attributes. The base HEP is determined by the applicable attributes of three base PIFs: Information availability and reliability, Scenario familiarity, and Task complexity. Using this model to calculate HEPs requires knowing the base HEPs of every CFM at every nominal (i.e., no impact) PIF attribute as well as the attribute weights of the rest of the PIFs for every CFM. We used available human error data to estimate the base HEPs and attributes weights.
METHOD Along with the development of IDHEAS-G, we generalized human error data documented in various sources into IDHEAS-G Human Error Tables, which map human error data to the CFMs and PIF attributes. In developing IDHEAS-ECA, we integrated the available data as of July 2019 in the Human Error Tables to estimate the base HEPs and PIF weights for every CFM and PIF attribute in IDHEAS-ECA. Because of the limited amount of data, the integration involves interpolation, reasoning, and engineering judgment. Below are some general strategies we used in the integration:
The human error data are first evaluated for their uncertainties and practicality in the source documents. We considered that the NPP operational data that were systematically collected for HRA had the highest practicality while cognitive experiments performed in research laboratories with students had the least practicality. We used high practicality data to anchor a base HEP or PIF weight and used other data points to adjust the uncertainties in the high-practicality data points. For the multiple data points that have about the same level of practicality and certainty, we used the median of the data points as the base HEP or PIF weight.
Even if there are multiple data points for a base HEP or PIF weight, judgment and reasoning are still needed in generalizing and integrating the human error data because of uncertainties and complications in the data sources. The data sources as well as the process and considerations in generating the base HEPs and PIF weights should be documented.
- 2) Data points on the combined effects of several CFMs and/or PIF attributes When there were multiple data points with combined effects of two or three CFMs or PIF attributes, we performed data fitting to get the best-fit base HEP or PIF weight. When there were only a few data points or a variety of CFMs and PIFs involved in the data points, we combined the data points to estimate the range and then use the middle of the range as the base HEP or PIF weight.
- 3) No data point for a PIF weight The IDHEAS-G Human Error Tables do not have numeric human error information for many attributes in PIFs such as Work Process and Teamwork and Organizational Factors. Yet, there have been studies demonstrating that those PIFs impact human performance in measures other than human error rates, such as increasing personnel workload or reducing situational awareness. We assigned the PIF weight as 1.1 or 1.2 for those attributes, pending for future updates as relevant human error data become available.
- 4) Consistence checking and adjustment with benchmark values After the initial base HEPs and PIF weights are developed, they are checked for internal consistency against the literature that ranks the likelihood of certain types of human errors and the contribution of various PIFs. We also used reported rates of human events and estimated HEPs from the NRC 2018 FLEX HRA expert elicitation as benchmarks to check and adjust some base HEPs and PIF weights within their uncertainty ranges.
RESULTS The results consist of three base HEP tables and 17 PIF attribute weight tables. Each table is for one PIF. Each row in a table is for one attribute, with the first row for the No impact state of a PIF. The first column in a table is an identifier assigned for a PIF attribute. For example, the attributes for PIF Scenario Familiarity, shown in Table 2, have the identifiers SF1, SF2, SF3. The second column is the description of every PIF attribute. The remaining five columns contain the base HEP of a CFM or the PIF weight on the CFM imposed by the PIF attribute of the row. These five columns are for failure of Detection (D), Understanding (U),
Decisionmaking (DM), Action Execution (E), and Teamwork (T). The symbol NA in these columns means that the attribute is not applicable to the CFM there it has no impact on the CFM.
Table 2. Base HEP for Scenario Familiarity
PIF Attribute D U DM E T SF1 Unpredictable dynamics in known scenarios 6.6E-4 6.6E-3 6.6E-3 6.6E-4 NA
- Shifting task objectives,
- Infrequently performed tasks,
- unlearn a technique and apply one that requires an opposing philosophy SF3 Scenarios trained on but infrequently E-3 E-2 E-2 E-3 NA performed Scenario is unfamiliar, rarely performed 1.2E-2 E-1 E-1 3.3E-2 NA
- notice adverse indicators that are not part of the task at hands
- notice incorrect status that is not a part of the routine tasks Extremely rarely performed 3.3E-2 3E-1 3E-1 3.5E-1 NA
- Lack of plans, policies and procedures to address the situation
- No existing mental model for the situation
- Extremely rare events SF4 Bias or preference for wrong strategies exists, NA 2.6E-2 2.6E-2 NA NA mismatched mental models
SUMMARY
The base HEPs and PIF weights in the present version of IDHEAS-ECA are the first effort of integrating the data generalized using the IDHEAS-G Human Error Tables. Because of the limited amount of data available, we used interpolation, judgment, and benchmarking to develop the full set of base HEPs and PIF weights. In the long-term, generalizing human error data as new data become available should be a continuous effort, and there should be periodic integration and updates of the base HEPs and PIF weights based on the up-to-date available data in the Human Error Tables.
The NRC has the Scenario Authoring, Characterization, and Debriefing Application (SACADA) program collecting operator simulator training data [3]. The SACADA program continuously generates operator performance data classified as satisfactory, unsatisfactory, or deviated from the training objectives. The frequency of unsatisfactory performance is considered as human errors. There are also on-going operator simulator data collection programs in other organizations such as the Korea Atomic Energy Research Institute.
Several research organizations such as the Organisation for Economic Co-operation and Development Halden Reactor Project have been conducting human performance experiments with nuclear reactor simulators. The experimental results provide human error data relevant to NPP operations. The NRC also has plans to reach out to other sources of human performance data. We intend to periodically generalize human error data to the IDHEAS-G Human Error Tables and integrate the data to update IDHEAS-ECA.
REFERENCES
[1] J. Xing, Y. J. Chang, and J. DeJesus, The General Methodology of an Integrated Human Event Analysis System (IDHEAS-G) Draft Report, U.S. Nuclear Regulatory Commission, NUREG-2198 (ADAMS Accession No. ML19235A161), Aug.
2019.
[2] A. M. Whaley et al., Cognitive Basis for Human Reliability Analysis, U.S. Nuclear Regulatory Commission, NUREG-2114 (ADAMS Accession No. ML16014A045), Jan.
2016.
[3] Y. J. Chang, D. Bley, L. Criscione, B. Kirwan, A. Mosleh, T. Madary, R. Nowell, R.
Richards, E. M. Roth, S. Sieben, and A. Zoulis, The SACADA database for human reliability and human performance, Reliab. Eng. Syst. Saf., vol. 125, pp. 117-133, May 2014.