ML21096A181

Item 2 & 3 IDHEAS-G and IDHEAS-DATA by J Xing - 2021-04-08 Public Meeting

ML21096A181
Person / Time
Issue date:	04/08/2021
From:	Chang Y, Dejesus-Segarra J, Jing Xing NRC/RES/DRA/HFRB
To:
Xing, Jang - 301 415 2410
Shared Package
ML21096A176	List: ML21096A180 ML21096A181 ML21096A182 ML21096A183
References
Download: ML21096A181 (22)

Text

IDHEAS - An Integrated Human Event Analysis System Jing Xing, Y. James Chang, Jonathan DeJesus Segarra, U.S. Nuclear Regulatory Commission Presented by Jing Xing to the public meeting on IDHEAS April-08-2021

Outline I. Overview of IDHEAS II. IDHEAS General Methodology (IDHEAS-G)

III. Generalization of Human Error Data - IDHEAS-DATA 2

Where we started Scope

- Ex-CR actions HRA Use of Human

- Shutdown Variability Performance

- Severe accidents Data Uncertainties Analysts HRA in the scenario practices method Qualitative Qualitative to PIFs - explicit Cognitive and analysis guidance quantification description data basis 3

What we have achieved

• Expanded scope - IDHEAS is an HRA method suite for all nuclear HRA applications

• Use of human performance data - Human error data were explicitly used in IDHEAS

- The method and data structure are based on the same cognitive basis model such that data can be generalized and used by the method.

• HRA variability - IDHEAS improves HRA method variability by enhancing the four areas (identified in HRA benchmarking studies)

- Systematic qualitative analysis guidance

- Links between qualitative analysis outcomes and quantification of human error probabilities (HEPs)

- Explicit attributes for every performance influencing factor (PIF)

- Cognitive and data basis that links PIF attributes to cognitive failure modes (CFMs) 4

Development of IDHEAS

- An Integrated Human Event Analysis System Scientific Cognitive Basis for HRA (NUREG-2114)

SACADA and other Literature data sources

Research, IDHEAS General Methodology IDHEAS-operation experience (IDHEAS-G) (NUREG-2198) DATA IDHEAS Internal At- IDHEAS-ECA (RIL-2020-02) power Application (NUREG-2199)

Testing the Event FLEX HRA Evaluation SDP/ASP method analysis Expert of FLEX analysis Elicitation actions (RIL-2020-13) 5

Development of IDHEAS

- An Integrated Human Event Analysis System Scientific Cognitive Basis for HRA (NUREG-2114)

SACADA and all Literature data sources

Research, IDHEAS General Methodology IDHEAS-operation experience (IDHEAS-G) (NUREG-2198) DATA IDHEAS Internal At- IDHEAS-ECA (RIL-2020-02) power Application (NUREG-2199)

Testing the Event FLEX HRA Evaluation SDP/ASP method analysis Expert of FLEX analysis Elicitation actions (RIL-2020-13) 6

Outline I. Overview of IDHEAS II. IDHEAS General Methodology (IDHEAS-G)

III. Generalization of Human Error Data - IDHEAS-DATA 7

What is IDHEAS-G

• A methodology for developing application-specific HRA methods

• A platform for generalizing and integrating human error data to support HEP estimation

• A general HRA method for human event analysis and human error root causal analysis 8

Overview of IDHEAS-G IDHEAS-G consists of a cognition model as the framework for HRA, its implementation in an HRA process, and detailed guidance for HRA applications.

Cognitive Basis Structure Cognition Model PIF Structure Stage 1 Stage 2 Stage 3 Stage 4 Scenario Modeling of HEP Integrative analysis important quantification analysis human actions 9

Whats in IDHEAS-G report (NUREG-2198)

Model Guidance

• Cognitive model

• Collecting data and information for HRA

- Cognitive basis

• Analyzing scenario and searching for context structure

• Identifying and defining human failure events

- PIF structure

• Analyzing and characterizing tasks

• Time uncertainty model

• Understanding and selecting applicable

• HEP quantification model cognitive failure modes (CFMs)

• Dependency model

• Representing (mapping) context with performance influencing factors (PIFs)

• Structure for generalizing human error data

• Analyzing and documenting uncertainties

• Developing application-specific IDHEAS methods from IDHEAS-G A step-by-step HRA process Three full examples demonstrating IDHEAS integrating all the models and process guidance 10

Cognitive Basis Structure Macrocognitive Cognitive PIFs Processors functions mechanisms Processor - D1 Cognitive Detection mechanism PIF 1 Processor - D5 Cognitive Processor - U1 PIF 2 Understanding mechanism Processor - U5 Cognitive PIF 3 mechanism Human Decision- Processor - DM1 task Cognitive making Processor - DM6 PIF 17 mechanism Action Processor - E1 PIF 18 Cognitive execution mechanism Processor - E5 PIF 19 Interteam Processor - T1 Cognitive coordination mechanism Processor - T7 11

PIF Structure Environment Personnel and System Task Context and Situation organization

• Information

• Staffing availability and

• Accessibility/habitabi *Procedures, reliability

• System and I&C lity of workplace and transparency to guidance, and *Scenario familiarity PIF travel paths personnel instructions *Multitasking,

• Workplace visibility *Training interruptions, and

• Human system

• Workplace Noise *Team and distractions interface Cold/heat/humidity

• Equipment and organizational *Task complexity

• Resistance to physical factors *Mental fatigue tools movement *Work processes *Time pressure and stress

• Physical demands PIF *Poor lighting in

• Tools are difficult to

• Procedure is inadequate *Sustained high-attributes workplace use

• Procedure is difficult to demand cognitive

• Tools are unfamiliar to Note: The PIF *Glare or reflection use activities personnel attributes shown are on physical structure *Procedure is available, *Long working hours

• Tools do not work examples and *Smoke or fog- but does not fit the *Sleep deprivation

• Tools or parts are correspond to the PIFs induced low visibility situation highlighted in red. unavailable 12

IDHEAS-G HRA process PRA model PRA scenario Human Failure Event (HFE) HFE1 HFE2 HFE3 Critical tasks Critical task 1 Critical task 2 Critical task 3 Macrocognitive functions and Action Interteam Detection Understanding Decisionmaking cognitive failure execution coordination modes (CFMs)

Performance influencing PIFs PIFs PIFs PIFs PIFs factors (PIFs)

Human error probability (HEP) Pc - HEP of a CFM = f(PIFs); Pt - HEP of time uncertainty Integration to PRA Dependency analysis and Uncertainty analysis 13

IDHEAS-G HRA Process PIF attributes of every CFM Scenario for every CT PRA context and Calculate model list of (3.3.2) applicable Assess attributes of Analyze scenario context PIFs every applicable PIF (3.1.2)

(3.2.3)

List of Develop scenario narrative HFE and its applicable Calculate Develop scenario timeline definition CFM(s) for overall HEP (3.1.1) the CT(s) (3.3)

HFE and its List of definition Break down CT(s) Characterize the CT(s) and Identify and define HFE (3.1.3) HFE into CT(s) select applicable CFMs (3.2.1) (3.2.1 and 3.2.2)

HFE and its definition Estimate parameters and of distribution Analyze HFE timeline (subset of scenario timeline, if there (3.3.1)

Calculate are multiple HFEs in the scenario) Estimate parameters (3.3.1) of distribution (3.3.1) and Uncertainty and dependency analysis

= time required and documentation

= time available CFM = cognitive failure mode PIF = performance-influencing factor and = mean and standard deviation of CT = critical task PRA = probabilistic risk assessment

= error probability due to CFMs and = mean and standard deviation of HEP = human error probability 14 HFE = human failure event = error probability due variability in and

HEP QuantificationPc

• Probability of CFM, , can be estimated in one or a combination of the following three ways:

- Calculation from the number of errors divided by number of occurrences

- Expert judgment

- HEP quantification model

• IDHEAS-G provides a data structure of generalizing human error data to support the three ways.

15

Outline I. Overview of IDHEAS II. IDHEAS General Methodology (IDHEAS-G)

III. Generalization of Human Error Data - IDHEAS-DATA 16

Generalizing human error data to inform HEPs HEP = f(states of performance influencing factors)

• Human error data exist from various domains, in different formats, varying context and levels of detail.

Data source 1 Data source 2 Tasks Context Tasks Context Failure PIFs Failure PIFs modes modes A generic, adaptable set of failure modes and PIFs 17

Use human error data to inform HEPs

1. Evaluation - 2. Generalization - 3. Integration -

Assess data Represent source data Integrate the source with the CFMs and PIFs generalized data for HEP calculation

• Context and Human tasks -> Error rates - Base HEPs task Cognitive failure modes (CFMs) Change of error rates -

• Variables and PIF weights ( )

Measurements Context ->

Performance Others (e.g., PIF

• Uncertainties influencing interaction, time factors (PIFs) distribution, dependency) 18

Data sources A. Nuclear simulator data and operational data (e.g., SACADA, HuREX, German NPP maintenance database analysis)

B. Operation performance data from other domains (e.g., transportation, off-shore oil, military operations, manufacturing)

C. Experimental studies in the literature (e.g., cognitive and behavior science, human factors, neuroscience)

D. Expert judgment of human reliability in the nuclear domain E. Unspecified context (e.g., statistical data, ranking, frequencies of errors or causal analysis) 19

IDHEAS-DATA Structure

• IDHEAS-DATA has 27 tables (IDTABLEs) documenting generalized human error data and empirical evidence

• Human error data are generalized to IDHEAS-G CFMs and PIF attributes IDHEAS-DATA IDTABLE IDTABLE 1-3 Base HEPs IDTABLE-21 Lowest HEPs of CFMs IDTABLE-1 Scenario Familiarity IDTABLE-22 PIF Interaction IDTABLE-2 Information IDTABLE-23 Distribution of Task Needed IDTABLE-3 Task Complexity IDTABLE-24 Modification to Time Needed IDTABLE 4--20 PIF Weights IDTABLE-25 Dependency of Human IDTABLE 4-8 Environment PIFs Actions IDTABLE 9-11 System PIFs IDTABLE-26 Recovery of Human Actions IDTABLE 11-16 Personnel PIFs IDTABLE-27 Main drivers to human events IDTABLE 17-20 Task PIFs 20 20

Example datapoints from SACADA database

• SACADA collects operators task performance data in simulator training.

• The unsatisfactory performance rates (UNSAT) for training objective tasks were calculated from the SACADA data before April 2019 and used for IDHEAS-ECA.

• For example, SACADA characterizes operators scenario familiarity as three options: Standard, Novel, and Anomaly. The datapoints are used for the base HEPs of PIF Scenario Familiarity (SF3.1) for CFMs Failure of Understanding (U) and Failure of Decisionmaking (DM), as shown in the table below SACADA data IDHEAS-DATA Situation Task (Training Objectives) Error rates CFM PIF Uncertainties factors Operators diagnose in Anomaly 1.2E-1 U SF3.1 Other PIFs simulator training scenario (8/69) may exist Operators make decisions Anomaly 1.1E-2 DM SF3.1 Other PIFs in simulator training scenario (1/92) may exist 21

Summary of IDHEAS-DATA By 2020:

• Use of nuclear operation/simulation data (SACADA, HuREX, Halden studies)

• ~300+ literature generalized; another 200+ evaluated and selected for generalization

• The generalized data were independently verified and reviewed

• The generalized data were integrated for HEP calculation in IDHEAS-ECA In the future:

• Human error data needed in teamwork and organizational factors

• Data generalization is an on-going, continuous effort; Data integration should be periodically updated.

22