ML20016A481

Enclosuintegrated Human Event Analysis System for Event and Condition Assessment (IDHEAS-ECA)

ML20016A481
Person / Time
Issue date:	02/07/2020
From:	James Chang, Dejesus J, Jing Xing Office of Nuclear Regulatory Research
To:
J.Xing 415-2410
Shared Package
ML20016A480	List: ML20016A481
References
RIL-2020-02
Download: ML20016A481 (128)

Text

RIL-2020-02 Integrated Human Event Analysis System for Event and Condition Assessment (IDHEAS-ECA)

Date Published: February 2020 Prepared by:

Jing Xing James Chang Jonathan DeJesus Research Information Letter Office of Nuclear Regulatory Research

DISCLAIMER Legally binding regulatory requirements are stated only in laws, NRC regulations, licenses, including technical specifications, or orders; not in Research Information Letters (RILs). A RIL is not regulatory guidance, although NRCs regulatory offices may consider the information in a RIL to determine whether any regulatory actions are warranted.

ABSTRACT This report describes a human reliability analysis (HRA) method developed by the U.S. Nuclear Regulatory Commission (NRC) staff, which is referred to as the Integrated Human Event Analysis System for Event and Condition Assessment (IDHEAS-ECA). It is based on the General Methodology of an Integrated Human Event Analysis System (NUREG-2198).

IDHEAS-ECA supports risk-informed decisionmaking by providing an HRA method to be used in probabilistic risk assessment (PRA) applications. PRAs are used in the review of risk-informed license amendment requests, and evaluations of Notices of Enforcement Discretion, operational events (e.g., Management Directive 8.3, NRC Incident Investigation Program, and the Accident Sequence Precursor Program), and inspection findings (i.e., the Significance Determination Process). IDHEAS-ECA was developed because, in recent years, the scope of application of HRA has expanded into situations beyond the scope of existing HRA methods.

The intent of IDHEAS-ECA is to be applicable to the same situations that existing HRA methods model (e.g., nuclear power plant internal events while at-power) and beyond (e.g., external events, low power and shutdown events, and events where flexible and coping strategies (FLEX) equipment are used). The IDHEAS-ECA method provides step-by-step guidance for analyzing a human action and its context, and models a human action using five macrocognitive functions: detection, understanding, decisionmaking, action execution, and interteam coordination. The failure of a human action is modeled with a set of cognitive failure modes and performance-influencing factors, which are then used to calculate the human error probability (HEP). The IDHEAS-ECA method includes a software package that facilitates the documentation of the analysis of a human action and its context and uses the results of the analysis as input to calculate the HEP.

The report also provides additional information in the appendices, which include (1) a set of worksheets needed for analyzing and modeling human actions and its context, (2) three examples that demonstrate the use of the IDHEAS-ECA method, and (3) the human error data needed to calculate the HEPs.

iii

EXECUTIVE

SUMMARY

The Integrated Human Event Analysis System for Event and Condition Assessment (IDHEAS-ECA) is a human reliability analysis (HRA) method developed by the U.S. Nuclear Regulatory Commission (NRC) staff to support risk-informed decisionmaking. IDHEAS-ECA analyzes human events and estimates human error probabilities (HEPs) for use in probabilistic risk assessment (PRA) applications. The method is based on the General Methodology of an Integrated Human Event Analysis System (IDHEAS-G) (NUREG-2198) [1]. IDHEAS-G and IDHEAS-ECA were developed because, in recent years, the scope of application of HRA has expanded into situations beyond the scope of existing HRA methods. Also, they were developed, in part, in response to the Staff Requirements Memorandum M061020 [2] in which the Commission directed the Advisory Committee on Reactor Safeguards to work with the

[NRC] staff and external stakeholders to evaluate different Human Reliability models in an effort to propose either a single model for the agency to use or guidance on which model(s) should to

[sic] be used in specific circumstances.

IDHEAS-ECA models human actions in a PRA (i.e., human failure events) using five macrocognitive functions: detection, understanding, decisionmaking, action execution, and interteam coordination. These macrocognitive functions are based on the cognitive basis for HRA, which was published as NUREG-2114 [3] and are described as follows:

• Detection (D) is noticing cues or gathering information in the work environment.

• Understanding (U) is the integration of pieces of information with a persons mental model to make sense of the scenario or situation.

• Decisionmaking (DM) includes selecting strategies, planning, adapting plans, evaluating options, and making judgments on qualitative information or quantitative parameters.

• Action execution (E) is the implementation of the decision or plan to change some physical component or system.

• Interteam coordination (T) focuses on how various teams interact and collaborate on an action.

The first four macrocognitive functions (D, U, DM, and E) may be performed by an individual or a team, and interteam coordination is performed by multiple groups or teams. In general, a human failure event (HFE) occurs due to the failure of any macrocognitive function. In IDHEAS-ECA, the failure of a macrocognitive function is defined as the cognitive failure mode (CFM).

The probability of an HFE (i.e., human error probability) is affected by the scenario context in which the action occurs. The context describes the conditions that challenge or facilitate human performance, IDHEAS-ECA uses performance-influencing factors (PIFs) to model the context.

Table ES-1 shows the 20 PIFs used in IDHEAS-ECA in four context categories.

Table ES-1 PIFs in IDHEAS-ECA Environment and situation System Personnel Task

• Work location

• System and I&C

• Staffing

• Information availability and accessibility and transparency to

• Procedures, reliability habitability personnel guidelines, and

• Scenario familiarity

• Workplace visibility

• Human-system instructions

• Multi-tasking, interruption,

• Noise in workplace and interfaces

• Training and distraction communication

• Equipment and

• Teamwork and

• Task complexity pathways tools organizational

• Mental fatigue

• Cold/heat/humidity factors

• Time pressure and stress

• Resistance to physical

• Work processes

• Physical demands movement v

IDHEAS-ECA also provides a process to implement an HRA. An overview of the IDHEAS-ECA HRA process is shown in Figure ES-1. For HRA applications in nuclear power plants, the HRA process typically starts with a PRA model. Then, the HRA process consists of eight steps, which are described below. The NRC staff developed a software package (i.e., the IDHEAS-ECA software) to facilitate the documentation of the HRA process and calculate the HEP.

PIF attributes of every CFM Scenario for every CT PRA context and Step 5: Calculate model list of applicable Step 4: Assess PIFs Step 1: Determine PIFs applicable to every scenario context CFM List of Step 1: HFE and its applicable Step 7:

Develop scenario narrative definition CFM(s) for Calculate Develop scenario timeline the CT(s) overall HEP HFE and its Step 2: List of CT(s) Step 3: Characterize the Step 1: Identify HFE definition Analyze tasks CT(s) and select applicable Step 2: Define HFE and identify CFMs CT(s) in HFE HFE and its definition and Step 6: Estimate parameters Step 6: Analyze HFE timeline of distribution (subset of scenario timeline, if there are Step 6: Calculate multiple HFEs in the scenario)

Step 6: Estimate parameters of distribution and CFM = cognitive failure mode = error probability due to CFMs CT = critical task = error probability due to uncertainty in and HEP = human error probability = time available HFE = human failure event = time required PIF = performance-influencing factor and = mean and standard deviation of PRA = probabilistic risk assessment and = mean and standard deviation of Figure ES-1 IDHEAS-ECA HRA Process Step 1: Analyze the event scenario. Analyzing an event includes developing the scenario narrative and timeline, determining the scenario context, and identifying the HFEs. The scenario narrative is a storytelling-style representation that specifies the initial conditions, initiating event, boundary conditions of the event, and the scenario progression and end state.

The scenario timeline documents the system responses (to the initiating event) and HFEs in chronological order. Together the scenario narrative and timeline are the operational narrative.

Determining the scenario context refers to the search for the conditions that challenge or facilitate human performance in the scenario and results in a list of applicable PIFs. The HFEs are usually identified in the PRA model and are the analysis units of an HRA.

Step 2: Analyze the HFE. This includes developing the definition of the HFE, analyzing the tasks within the HFE, and identifying the critical tasks for HEP quantification. The definition of the HFE describes the failure of the human action and its link to the affected systems in the PRA model. Analyzing the tasks within an HFE provides a representation of how the HFE can occur and aids in the identification of critical tasks, which are those that are essential to the success of the HFE. Failure of any critical task will result in the occurrence of the HFE.

Step 3: Model the failure of critical tasks in an HFE. This includes characterizing the critical task and selecting the applicable CFMs of the critical task. Characterization of a critical task is to specify the conditions relevant to the critical task that can challenge or facilitate human performance of it. Any critical task can be achieved through one to all five macrocognitive functions. The cognitive failure of a critical task is the result of failure of any macrocognitive vi

function it demands. Thus, the CFMs are the classifications of the various ways that a critical task may fail.

Step 4: Assess the PIFs applicable to every CFM. This step uses the results of the scenario context (Step 1), HFE definition (Step 2), and task characterization (Step 3) to assess the PIFs, which results in a list of PIF attributes of every CFM for every critical task. The PIFs represent the context of the HFE and facilitate quantification of the HEP. A PIF attribute is an assessable characteristic of a PIF and describes a way the PIF challenges the macrocognitive functions of a critical task and, therefore, increase the likelihood of error in the macrocognitive functions.

Step 5: Calculate . is the probability of failure due to the CFMs and is calculated as the probabilistic sum of the HEPs of all the CFMs of the critical tasks, which are based on the PIF attributes assessed in Step 4. can be computed using the IDHEAS-ECA software or manually using the data in Appendix B.

Step 6: Analyze HFE timeline and calculate . is the probability of failure due to the uncertainty in time available and time needed (or required) to perform the HFE. Using the HFE definition, the timeline for the HFE is analyzed to obtain an estimate of the parameters of the probability distributions of time available and time needed. Then, the IDHEAS-ECA software is used to calculate .

Step 7: Calculate the overall HEP. The overall HEP is the probabilistic sum of and . That is, = 1 (1 )(1 ).

Step 8 (not shown in Figure ES-1): Analyze uncertainties in the HRA results and perform sensitivity analysis if needed.

Appendix A of this report provides a set of worksheets to document the analysis and modeling of human actions and its context. Appendix B contains the human error data needed to calculate the HEPs. Appendix C provides three examples that demonstrate the use of the IDHEAS-ECA method. Appendix D introduces the IDHEAS-ECA software.

IDHEAS-ECA improves existing HRA methods by (1) providing a systematic process and guidelines to analyze and model human actions and the associated scenario context, (2) using a human error database to calculate HEPs, and (3) including an extensive set of PIFs to represent the context of scenarios under various operational conditions, such as using flexible and coping strategies (FLEX) equipment. IDHEAS-G (and, therefore, IDHEAS-ECA) provides a platform to incorporate and generalize human error data from various sources to inform HEPs.

Data from the Scenario Authoring, Characterization, and Debriefing Application (i.e., SACADA) and operator simulator performance in other countries will be used to update the HEPs used in IDHEAS-ECA.

IDHEAS-ECA is envisioned to be used by NRC staff involved PRA applications, such as the review of risk-informed license amendment requests, and evaluations of Notices of Enforcement Discretion, operational events (e.g., Management Directive 8.3, NRC Incident Investigation Program, and Accident Sequence Precursor Program), and inspection findings (i.e., the Significance Determination Process). The intent of the IDHEAS-ECA is to be applicable to the same situations that existing HRA methods model (e.g., nuclear power plant internal events while at-power) and beyond (e.g., external events, low power and shutdown events, and events where FLEX equipment are used).

vii

TABLE OF CONTENTS ABSTRACT .................................................................................................................................. iii EXECUTIVE

SUMMARY

.............................................................................................................. v TABLE OF CONTENTS ............................................................................................................... ix LIST OF FIGURES ...................................................................................................................... xi LIST OF TABLES ......................................................................................................................... xi ACRONYMS AND TERMS .........................................................................................................xiii 1 INTRODUCTION TO IDHEAS-ECA ...................................................................................1-1 1.1. Intended Use ...............................................................................................................1-1 1.2. Scope of Application ....................................................................................................1-1 1.3. Intended Users ............................................................................................................1-1 1.4. Available Tools for Using IDHEAS-ECA ......................................................................1-1 1.5. Organization of this Report ..........................................................................................1-1 2 IDHEAS-ECA BASICS ........................................................................................................2-1 2.1. Overview of the Cognitive Basis for IDHEAS-ECA ......................................................2-1 2.2. Overview of the Cognition Model for IDHEAS-ECA.....................................................2-1 2.3. Overview of the PIF Structure for IDHEAS-ECA .........................................................2-2 3 GUIDANCE FOR THE IDHEAS-ECA PROCESS...............................................................3-1 3.1. Step 1 - Scenario Analysis ..........................................................................................3-2 3.1.1. Develop the Operational Narrative .......................................................................3-3 3.1.2. Identify and Define the Human Failure Events .....................................................3-6 3.1.3. Identify the Scenario/Event Context .....................................................................3-7 3.2. Step 2 - Analyzing Human Failure Events ................................................................3-11 3.2.1. Defining HFEs ....................................................................................................3-11 3.2.2. Task Analysis and Identification of Critical Tasks ..............................................3-11 3.3. Step 3 - Modeling Failure of Critical Tasks ...............................................................3-13 3.3.1. Characterization of a Critical Task .....................................................................3-13 3.3.2. Identification of Applicable Cognitive Failure Modes ..........................................3-14 3.4. Step 4 - Assessing PIF Attributes Applicable to CFMs .............................................3-18 3.5. Step 5 - Estimation of Pc - the Sum of Human Error Probabilities of Cognitive Failure Modes ...................................................................................................................................3-20 3.5.1. Estimation of Pc ..................................................................................................3-20 3.6. Step 6 - Estimation of Pt - the Convolution of the Distributions of Time Available and Time Required ......................................................................................................................3-23 3.7. Step 7: Calculate the Overall Human Error Probability .............................................3-30 3.8. Step 8: Analyze HRA Uncertainties and perform sensitivity analysis ........................3-30 ix

3.9. Summary of IDHEAS-ECA ........................................................................................3-31 4 DISCUSSION AND CONCLUDING REMARKS .................................................................4-1 4.1. From IDHEAS-G to IDHEAS-ECA ...............................................................................4-1 4.2. Integration of Human Error Data for IDHEAS-ECA .....................................................4-3 4.3. Future Development and Improvement .......................................................................4-4 4.4. Concluding Remarks ...................................................................................................4-6 5 REFERENCES ...................................................................................................................5-1 Appendix A IDHEAS-ECA WORKSHEETS ................................................................. A-1 Appendix B BASE HUMAN ERROR PROBABILITIES AND PERFORMANCE-INFLUENCING FACTOR WEIGHTS ............................................................................. B-1 Appendix C EXAMPLES .............................................................................................. C-1 Appendix D INTRODUCTION TO THE IDHEAS-ECA SOFTWARE ........................... D-1 x

LIST OF FIGURES Figure 2-1 Overview of the Cognitive Basis for IDHEAS-ECA...............................................2-1 Figure 2-2 IDHEAS-ECA Hierarchy for Modeling an Event ...................................................2-1 Figure 3-1 Overview of the IDHEAS-ECA HRA Process .......................................................3-2 Figure 3-2 Composition of the Operational Narrative ............................................................3-3 Figure 3-3 Cognitive Activities and Processors for Detection ..............................................3-16 Figure 3-4 Cognitive Activities and Processors for Understanding ......................................3-16 Figure 3-5 Cognitive Activities and Processors for Decisionmaking ....................................3-17 Figure 3-6 Cognitive Activities and Processors for Action Execution ..................................3-17 Figure 3-7 Activities and Processors for Interteam coordination .........................................3-17 Figure 3-8 Graphical Representation of Pt ...........................................................................3-30 Figure D-1 The structure of calculating a human failure events human error probability in the IDHEAS-ECA software ............................................................................................................. D-2 LIST OF TABLES Table 2-1 PIFs in IDHEAS-ECA ...........................................................................................2-3 Table 2-2 Environment- and Situation-related PIFs .............................................................2-3 Table 2-3 System-related PIFs .............................................................................................2-4 Table 2-4 Personnel-related PIFs .........................................................................................2-4 Table 2-5 Task-related PIFs .................................................................................................2-5 Table 3-1 Narrative Information Coverage of a scenario analysis ........................................3-5 Table 3-2 Critical Task Characterization for HRA ...............................................................3-13 Table 3-3 Taxonomy of Cognitive Activities .......................................................................3-15 Table 3-4 Typical Factors Contributing to Treqd ...................................................................3-26 Table 3-5 Uncertainty Factors that Modify the Distribution of Treqd .....................................3-27 Table 3-6 Example Implementation of Equation (3.7) using OpenBUGS [9] ......................3-29 Table 3-7 Summary of IDHEAS-ECA Worksheets .............................................................3-32 Table 3-8 Summary of the IDHEAS-ECA Process .............................................................3-34 Table 4-1 Summary of IDHEAS-ECA Development from IDHEAS-G ..................................4-2 Table B-1 Base HEP for Scenario Familiarity ...................................................................... B-1 Table B-2 Base HEP for Information Availability and Reliability .......................................... B-3 Table B-3 Base HEPs for Task complexity .......................................................................... B-5 Table B-4 PIF Weights for Environmental PIFs ................................................................. B-10 Table B-5 PIF Weights for System and I&C Transparency ................................................ B-11 Table B-6 PIF Weights for Human-System Interface ......................................................... B-11 Table B-7 PIF Weights for Equipment and Tools ............................................................... B-13 Table B-8 PIF Weights for Staffing .................................................................................... B-14 Table B-9 PIF Weights for Procedures, Guidance, and Instructions ................................. B-15 Table B-10 PIF Weights for Training ................................................................................ B-16 Table B-11 PIF Weights for Teamwork and Organizational Factors ................................ B-18 Table B-12 PIF Weights for Work Processes ................................................................... B-19 Table B-13 PIF Weights for Multitasking, Interruption, and Distraction ............................ B-20 Table B-14 PIF Weights for Mental Fatigue and Time Pressure and Stress .................... B-22 Table B-15 PIF Weights for Physical Demands ............................................................... B-23 Table D-1 The Operation, Calculation, and Display for Calculation of Pc ............................ D-4 Table D-2 The Operation, Calculation, and Display for Calculation of Pt ............................. D-5 xi

ACRONYMS AND TERMS AC alternating current ADAMS Agency wide Documents Access and Management System ASP accident sequence precursor (program)

CFM cognitive failure mode CT critical task D detection (one of the five macrocognitive functions)

DM decisionmaking (one of the five macrocognitive functions)

E action execution (one of the five macrocognitive functions)

ECA event and condition assessment ELAP extended loss of AC power EOC error of commission EOL end of life EOO error of omission EOP emergency operating procedure FLEX flexible and coping strategies FSG FLEX support guideline HEP human error probability HFE human failure event HRA human reliability analysis HSI human-system interface IDHEAS Integrated Human Event Analysis System IDHEAS-DATA Human Error Data Generalized with Integrated Human Event Analysis System IDHEAS-ECA Integrated Human Event Analysis System for Event and Condition Assessment IDHEAS-G General Methodology of an Integrated Human Event Analysis System IHA important human action I&C instrumentation and control LOCA loss-of-coolant accident NPP nuclear power plant NRC U.S. Nuclear Regulatory Commission PDP positive displacement pump xiii

PIF performance-influencing factor PRA probabilistic risk assessment psig pounds per square inch gauge RCP reactor coolant pump RCS reactor coolant system RIL Research Information Letter RO reactor operator SACADA Scenario Authoring, Characterization, and Debriefing Application SDP significance determination process SSCs structures, systems, and components T interteam coordination (one of the five macrocognitive functions)

TSC technical support center U understanding (one of the five macrocognitive functions) error probability due to CFMs error probability due variability in and time available time required mean of standard deviation of mean of standard deviation of xiv

1 INTRODUCTION TO IDHEAS-ECA 1.1. Intended Use The human reliability analysis (HRA) method presented in this report is based on the General Methodology of an Integrated Human Event Analysis System (IDHEAS-G). Details about IDHEAS-G can be found in NUREG-2198 [1]. The method is intended to be used in event and condition assessment (ECA) of nuclear power plants (NPPs) and it is referred to as IDHEAS-ECA.

1.2. Scope of Application IDHEAS-ECA supports probabilistic risk assessment (PRA) applications by analyzing human events and estimating human error probabilities (HEPs). The application scope of IDHEAS-ECA is broad because the performance-influencing factor (PIF) structure (see Section 2.3), which models the context of a human failure event (HFE), is comprehensive. The method covers all the PIFs in existing HRA methods and the factors reported in the broad literature and nuclear-specific human events. Because of the comprehensiveness of the PIF structure, IDHEAS-ECA can model the context of HFEs inside and outside an NPP control room including the use of flexible and coping strategies (FLEX) equipmentand during different plant operating states (i.e., at-power and shutdown). IDHEAS-ECA can be used in PRA applications, such as the review of risk-informed license amendment requests, and evaluations of Notices of Enforcement Discretion, operational events (e.g., Management Directive 8.3, NRC Incident Investigation Program, and Accident Sequence Precursor Program), and inspection findings (i.e., the Significance Determination Process).

1.3. Intended Users The intended users of IDHEAS-ECA are U.S. Nuclear Regulatory Commission (NRC) staff involved in PRA applications. Specifically, familiarity with probability, statistics, and PRA is expected as demonstrated by understanding the concepts discussed in the following NRC courses:

• P-105, PRA Basics for Regulatory Applications;

• P-200, System Modeling Techniques for PRA;

• P-203, Human Reliability Analysis; and

• P-102, Bayesian Inference in Risk Assessment.

1.4. Available Tools for Using IDHEAS-ECA To facilitate the use of IDHEAS-ECA, the NRC staff developed the following:

1. A set of worksheets (see Appendix A) that allows the documentation of the IDHEAS-ECA process, which supports the calculation of the HEP estimates.

2. A software tool that, based on user inputs of the results in the worksheets, calculates the HEP estimates.

1.5. Organization of this Report This report is organized as follows:

• Chapter 1 is a high-level introduction to IDHEAS-ECA.

• Chapter 2 introduces the basic concepts of IDHEAS-ECA. It is intended to help the HRA analysts to gain an overview and build the mental model of IDHEAS-ECA without diving 1-1

into the details. The chapter can also serve as a Who-is-Who list when the analysts are not familiar with the method. The downside of Chapter 2 is that some concepts introduced will only become clear to the readers after reading how the concepts are used in the IDHEAS-ECA process for conducting an HRA described in Chapter 3.

• Chapter 3 is the step-by-step guidance for the IDHEAS-ECA process. The guidance focuses on WHAT needs to be done for each IDHEAS-ECA step and HOW to perform each step. The guidance does not describe the technical basis on WHY the method is as it is. The technical basis is described in IDHEAS-G [1].

• Chapter 4 discusses the method and provides concluding remarks, including areas for future improvement.

• Chapter 5 lists the references used in this report.

• Appendix A contains all the worksheets for analysts to document the results of their step-by-step analysis.

• Appendix B has 15 tables containing the base HEPs and PIF weights needed to calculate HEPs.

• Appendix C provides three full examples demonstrating the IDHEAS-ECA process and documentation of the results.

• Appendix D introduces the IDHEAS-ECA software.

1-2

2 IDHEAS-ECA BASICS 2.1. Overview of the Cognitive Basis for IDHEAS-ECA IDHEAS-ECA uses the cognitive basis in IDHEAS-G, which consists of a macrocognition model and a PIF structure. An HFE is analyzed for the given scenario context, which are the conditions that affect human performance. IDHEAS-ECA uses the five macrocognitive functions in the macrocondition model to model failure of critical tasks in an HFE, and it uses the 20 PIFs in IDHEAS-G to model the context. Figure 2-1 outlines an overview of the cognitive basis for IDHEAS-ECA. This chapter will briefly describe the cognitive basis and the details can be found in the IDHEAS-G report (NUREG-2198) [1].

Scenario Context Critical 5 Macrocognitive functions and HFEs Tasks 20 PIFs (in 4 context categories)

Figure 2-1 Overview of the Cognitive Basis for IDHEAS-ECA 2.2. Overview of the Cognition Model for IDHEAS-ECA Figure 2-2 shows the IDHEAS-ECA hierarchy for modeling human actions in a scenario. The method identifies HFEs in the scenario and subsequently identifies critical tasks in an HFE. The failure of a critical task is modeled with the failure of the five macrocognitive functions in the IDHEAS-G macrocognition model. Several terms used in the IDHEAS-ECA hierarchy for modeling human actions in a scenario are described below.

Scenario Other human HFE 1 HFE 2 HFE actions Other tasks Critical Task 1 Critical Task Under- Decision- Action Interteam Detection standing making execution coordination Figure 2-2 IDHEAS-ECA Hierarchy for Modeling an Event Critical taskThe human action defined in an HFE may be decomposed into a set of discrete tasks for modeling. A critical task is essential to the success of the HFE; failure of any critical task in an HFE will result in the occurrence of the HFE. The critical tasks are the ones for which the HEPs will be calculated.

Cognitive activities and Macrocognitive functionsAny critical task involves performing cognitive activities, which demand brain resources. IDHEAS-ECA models the cognitive 2-1

demands of a critical task using five macrocognitive functions, which are the high-level brain functions that must be successfully accomplished to achieve the cognitive activities demanded by a critical task. IDHEAS-ECA uses the following macrocognitive functions:

• Detection (D) is noticing cues or gathering information in the work environment.

• Understanding (U) is the integration of pieces of information with a persons mental model to make sense of the scenario or situation.

• Decisionmaking (DM) includes selecting strategies, planning, adapting plans, evaluating options, and making judgments on qualitative information or quantitative parameters.

• Action execution (E) is the implementation of the decision or plan to change some physical component or system.

• Interteam coordination (T) focuses on how various teams interact and collaborate on a critical task.

The first four macrocognitive functions (D, U, DM, and E) may be performed by an individual or a team, and interteam coordination is performed by multiple groups or teams.

Notice that Action Execution is considered as a macrocognitive function. Some HRA methods classify human failure as cognition failure, or action or execution failure. In that sense, the failure of detection (CFM1), failure of understanding (CFM2), and failure of decisionmaking (CFM3) in IDHEAS-ECA are equivalent cognition failure and the failure of action execution (CFM4) is equivalent to action execution failure in other HRA methods. The failure of interteam coordination (CFM5) is not explicitly modeled in existing HRA methods.

Cognitive failure modesIDHEAS-ECA provides a set of five cognitive failure modes (CFMs) to model failure of a critical task. Each CFM represents the failure of a macrocognitive function demanded to accomplish the critical task, which are defined as follows:

• CFM1 - Failure of Detection

• CFM2 - Failure of Understanding

• CFM3 - Failure of Decisionmaking

• CFM4 - Failure of Action execution

• CFM5 - Failure of Interteam coordination Probability of an HFEThe probability of an HFE, , (i.e., the overall HEP) has two parts, and and is calculated as = 1 (1 )(1 ). is the HEP attributing to cognitive failures assuming that the time available for performing the human action of the HFE is adequate. is calculated as the probabilistic sum of the HEPs of the CFMs of all the critical tasks in an HFE. is the HEP attributing to the uncertainty in the time available and time needed to perform an action. It is calculated as the convolution of the probability distributions of time available and time needed.

2.3. Overview of the PIF Structure for IDHEAS-ECA The IDHEAS-ECA process begins with analyzing a scenario and searching for the context that challenges or facilitate human performance. The method provides a PIF structure that is composed of the following: (1) PIF category, (2) PIFs, and (3) PIF attributes and uses 20 PIFs and the associated attributes to model the scenario context. Several terms related to the IDHEAS-ECA PIF structure are described below.

Scenario context and PIF categoryThe context of a scenario are the conditions that challenge or facilitate human performance. Scenario context is documented in four categories:

environment and situation, system, personnel, and task, which are described as follows:

2-2

1) Environment and situation context This consists of conditions in personnels work environment and the situation in which actions are performed. It includes the weather, radiation or chemicals in the workplace, and any extreme operating conditions.

2) System context Systems are the objects of the HFEs, through which the actions are achieved. Systems include operational systems, supporting systems, instrumentation and control (I&C), physical structures, human-system interface (HSI), and equipment and tools.

3) Personnel context Personnel are the people who perform the action and includes individuals, teams, and organizations. The personnel context describes who the personnel are; their qualifications, skills, knowledge, abilities, and fitness to perform the action; how they work together; and the organizational measures that help personnel work effectively.

4) Task context The task context describes the cognitive and physical task demands for personnel and special conditions in the scenario that make tasks difficult to perform. An action may consist of one or more discrete tasks.

PIFsOnce the context of an event is identified, the context can be modeled with the PIFs.

IDHEAS-ECA has 20 PIFs in the four context categories as shown in Table 2-1. This list of PIFs covers all PIFs in the reviewed HRA methods and factors reported in the literature and nuclear-specific human event databases. The PIFs in each of the context categories are summarized in Table 2-2, Table 2-3, Table 2-4, and Table 2-5, respectively.

Table 2-1 PIFs in IDHEAS-ECA Environment and System Personnel Task situation

• Work location

• System and I&C

• Staffing

• Information availability accessibility and transparency to

• Procedures, and reliability habitability personnel guidelines, and

• Scenario familiarity

• Workplace visibility

• Human-system instructions

• Multi-tasking,

• Noise in workplace interfaces

• Training interruption and and communication

• Equipment and

• Teamwork and distraction pathways tools organizational

• Task complexity

• Cold/heat/humidity factors

• Mental fatigue

• Resistance to

• Work processes

• Time pressure and stress physical movement

• Physical demands Table 2-2 Environment- and Situation-related PIFs PIF Description Work location This PIF models the accessibility to and habitability of work accessibility and places where critical tasks are performed. Work places that habitability become inaccessible or uninhabitable negatively affect personnel performance of the critical tasks.

Workplace visibility This PIF models the visibility in the work place. Limited visibility may affect personnel performance of critical tasks.

Noise in workplace and This PIF models the ways communication of information communication required for critical tasks is affected by noise. Excessive noise pathways can negatively affect the communication of information that is required to perform a critical task.

2-3

Table 2-2 Environment and Situation-related PIFs (continued)

PIF Description Cold/heat/humidity This PIF models cold, heat, and humidity with respect to the performance of critical tasks. Extreme cold or heat and high humidity may affect personnel performance of critical tasks.

Resistance to physical This PIF models the ways resistance to movement affects the movement performance of critical tasks. Required protective clothes, obstructions, and slippery surfaces may negatively affect movement required to perform critical tasks.

Table 2-3 System-related PIFs PIF Description System and I&C This PIF models the impact of the design logic of systems and transparency to I&C on human performance. If the operation of the system or personnel I&C is not transparent to personnel, or personnel are unclear about system interdependency, they can make errors because of not understanding the systems in unusual scenarios.

Human-system This PIF models the impact of the HSI on human performance.

interface Poorly designed HSIs can impede task performance in unusual event scenarios. Even a well-designed HSI may not support human performance in specific scenarios that designers or operational personnel did not anticipate. HSIs may also become unavailable or unreliable in hazardous scenarios.

Equipment and tools This PIF models the availability and usability of equipment (including parts and portable equipment) and tools that are needed for the performance of critical tasks.

Table 2-4 Personnel-related PIFs PIF Description Staffing This PIF models that there is adequate and qualified staff to perform the required critical tasks. This includes the number of personnel, their skill sets, job qualifications (including fitness for duty), staffing structure (individual and team roles and responsibilities).

Procedures, guidelines, This PIF models the availability and usefulness of operating and instructions procedures, guidance, and instructions. Following procedures should lead to the success of the critical task. However, there may be situations in which procedures give incorrect or inadequate guidance or may not apply to the scenario.

Training This PIF models the training that personnel receive to perform critical tasks. Included in this consideration are personnels work-related experience and whether they have been trained on the type of the event, the amount of time passed since training, and training on the specific systems involved in the event. However, training may not address all possible event scenarios.

2-4

Table 2-4 Personnel-related PIFs (continued)

PIF Description Teamwork and This PIF models everything affecting team communication, organizational factors coordination, and cooperation.

Work processes This PIF models the aspects of doing work, supervision, management support, policies, and safetyconscious work environment at the organizational level.

Table 2-5 Task-related PIFs PIF Description Information availability This PIF is one of the three base PIFs and models whether the and reliability information needed for personnel to perform critical tasks is available to be perceived. If the information is perceived, this PIF also models whether that information is reliable and perceived in a timely manner. Cues and instrumentation readings are of interest in the modeling of this PIF.

Scenario familiarity This PIF is one of the three base PIFs and models the challenges to personnel in understanding the situation and making decisions. If the scenario is familiar, personnel are more likely to understand what is happening. In unfamiliar scenarios, personnel are more likely to perform situation-specific actions not identified in the procedures.

Multitasking, This PIF models performing concurrent and intermingled critical interruption, and tasks and things that interfere with personnels performance of distraction their critical tasks. Multitasking requires switching between critical tasks, and interruption and distraction keep personnel away from performing the tasks, which can make errors more likely.

Task complexity This PIF is one of the three base PIFs and models the task demand for cognitive resources (e.g., working memory, attention, executive control). The task complexity has two parts: (1) the complexity in processing the information to achieve the macrocognitive functions of the critical task, and (2) the complexity in developing and representing the outcomes to meet the task criteria. Complexity is characterized by the quantity, variety, and relation of the items to be processed or represented in a critical task.

Mental fatigue This PIF models the personnels vigilance and abilities to perform complex cognitive tasks. Mental fatigue can result from performing a task for an extended period of time, nonroutine tasks, and cognitively demanding tasks. Mental fatigue leads to loss of vigilance, difficulty in maintaining attention, reduced working memory capacity, and use of shortcuts in diagnosing problems or making decisions.

2-5

Table 2-5 Task-related PIFs (continued)

PIF Description Time pressure and This PIF models the personnels sense of time urgency to stress complete a task. Because time pressure is based on personnels perception and understanding of the situation, it may not reflect the actual situation. Other stresses and anxieties, such as concern for families in emergency conditions, fear of potential consequences of the event, and worrying about personal safety, can also increase the level of psychological stress and affect performance.

Physical demands This PIF models required extraordinary physical efforts, such as twisting, reaching, dexterity, or strong force to complete a critical task.

PIF attributeA PIF attribute is an assessable characteristic of a PIF and describes a way the PIF increases the likelihood of error in the macrocognitive functions. A PIF is characterized with a set of attributes, each describing one aspect of the PIF that challenges the macrocognitive functions demanded by a critical task. For example, one of the attributes of the PIF Human-system interface is the salience of indicators. Therefore, HEP estimation of a CFM is based on the assessment of PIF attributes applicable to the CFM. The PIF attributes were identified from cognitive and behavioral studies, as well as human error data from various sources. PIF attributes have the capability to link to existing human error data for HEP quantification.

Appendix B lists the attributes for all the PIFs.

2-6

3 GUIDANCE FOR THE IDHEAS-ECA PROCESS The HRA process with IDHEAS-ECA is composed of eight steps, which are briefly described below. The steps are described in more detail in the following subsections. Figure 3-1 presents an overview of the IDHEAS-ECA process and the flow of information. Each box represents a to-do item of a step in the process. The arrows represent the input(s) and output(s) from each of the items. To perform a step, all the inputs (information) for the step need to be available.

• Step 1: Analyze the event scenario. Analyzing an event includes developing the scenario narrative and timeline, determining the scenario context, and identifying the HFEs to be modeled (if not given in the PRA model).

• Step 2: Analyze the HFE. This includes defining the HFE, analyzing the tasks in the HFE with a task diagram and/or timeline, and identifying critical tasks for HEP quantification.

• Step 3: Model the failure of the critical tasks in an HFE. This includes characterizing the critical task, identifying cognitive activities required to achieve the critical task and subsequently identifying CFMs applicable to the critical task.

• Step 4: Assess the PIFs applicable to every CFM. This step uses the results of the scenario context (Step 1), HFE definition (Step 2), and task characterization (Step 3) to select the applicable PIF attributes for every CFM.

• Step 5: Calculate of an HFE. is the probabilistic sum of the HEPs of all the CFMs of the critical tasks. The HEP of a CFM can be computed using the IDHEAS-ECA software or manually calculated using the data in Appendix B.

• Step 6: Analyze HFE timeline and calculate of an HFE. is the HEP attributing to uncertainty in time available and time needed to perform the HFE and can be computed with the IDHEAS-ECA software.

• Step 7: Calculate the overall HEP. The overall HEP is the probabilistic sum of and .

That is, = 1 (1 )(1 ).

• Step 8 (not shown in Figure 3-1): Analyze uncertainties in the HRA results and perform sensitivity analysis if needed.

3-1

PIF attributes of every CFM Scenario for every CT PRA context and Step 5: Calculate model list of applicable Step 4: Assess PIFs Step 1: Determine PIFs applicable to every scenario context CFM List of Step 1: HFE and its applicable Step 7:

Develop scenario narrative definition CFM(s) for Calculate Develop scenario timeline the CT(s) overall HEP HFE and its Step 2: List of CT(s) Step 3: Characterize the Step 1: Identify HFE definition Analyze tasks CT(s) and select applicable Step 2: Define HFE and identify CFMs CT(s) in HFE HFE and its definition and Step 6: Estimate parameters Step 6: Analyze HFE timeline of distribution (subset of scenario timeline, if there are Step 6: Calculate multiple HFEs in the scenario)

Step 6: Estimate parameters of distribution and CFM = cognitive failure mode = error probability due to CFMs CT = critical task = error probability due to uncertainty in and HEP = human error probability = time available HFE = human failure event = time required PIF = performance-influencing factor and = mean and standard deviation of PRA = probabilistic risk assessment and = mean and standard deviation of Figure 3-1 Overview of the IDHEAS-ECA HRA Process The subsections below are structured such that a brief overview of the step (i.e., the what) is presented first, followed by where the information obtained from that step is documented in Appendix A, and ending with the guidance on how to perform the step.

3.1. Step 1 - Scenario Analysis The purpose of this step is to understand human performance in the event and collect information for quantification. This step includes developing operational narratives, identifying HFEs, and assessing the scenario/event context that affects human performance and HFEs in the scenario. The information obtained and generated from the analysis of Step 1 is documented in Worksheet A of Appendix A.

A human performance model may be initially sketched to serve as a framework to develop the operational narrative (Section 3.1.1) and assess the scenario context (Section 3.1.3). A human performance model for an HRA scenario consists of the following elements:

1) The goal of the scenario HRA focuses on safety; therefore, the goal of a scenario must relate to safety. For NPP events, the mission is to safely operate the plant or mitigate an unsafe condition in the plant. Specifically, the goal is to protect the fuel cladding, reactor coolant system (RCS), and containment.

2) The objectives and functions The objectives represent the desired outcomes of the scenario in achieving the goal. Examples of the objectives in NPP operation are restoring electrical power, initiating feed and bleed, and evacuating personnel. To achieve the objectives, a set of functions must be performed. The functions could be performed by systems, personnel, or a combination of both.

3-2

3) The systems IDHEAS-ECA uses the term systems to broadly refer to structures, systems, and components, as well as sensors, equipment, I&C, and HSIs. Systems are all the aspects that are necessary to achieve the objectives.

4) The personnel Personnel include all the people who perform the tasks in an event.

Personnel may work in various structures: (a) as individuals with roles, responsibilities, and tasks; (b) as teams working collaboratively for common goals; and (c) as an organization, which is a framework to outline authority and communication processes of individuals and teams.

3.1.1. Develop the Operational Narrative The operational narrative provides a detailed account of the scenario, which includes a scenario narrative and a scenario timeline. The scenario narrative is a storytelling-style representation that specifies the initial conditions, initiating event, boundary conditions of the event, and the scenario progression and end state. The initial conditions describe the beginning status of systems and personnel that have implications for the scenario progression, which are generally defined by the PRA. The initiating event originates from an internal or external hazard and causes abnormalities, which may require automatic system interventions, human interventions, or both, to protect safety. The boundary conditions describe the expected systems, site, and personnel status immediately after the initiating event and specify the scope and the assumptions applied to the HRA. The scenario progression describes the expected system and personnel responses and end state (or consequence). The scenario timeline documents the system responses and HFEs in chronological order and records the timing of system status changes and the cues for the HFEs. Figure 3-2 shows the composition of the operational narrative.

Scenario narrative

• Scenario overview

• Beginning of the scenario

• Initial condition

• Initiating event

• Boundary conditions

• Progression and end state Operational narrative Scenario timeline

• Date/time

• System automatic response, cues, human response, and notes Figure 3-2 Composition of the Operational Narrative For the purposes of IDHEAS-ECA, the operational narrative (scenario narrative and scenario timeline) should be developed based on the PRA (i.e., the event tree where the HFE is being credited) and documented in Section A.1 of Worksheet A found in Appendix A.

Guidance for Developing the Scenario Narrative The scenario narrative should first provide an overview of the scenario and highlight the safety considerations of the scenario. At a high-level, the scenario narrative covers the beginning of the scenario, the scenario progression, and the end state (or consequence).

Scenario Overview: The scenario overview documentation includes a title and a scenario summary. The title should be descriptive and provide a clue for the readers to predict the content. Therefore, the title should highlight the key safety considerations and consequence.

3-3

The scenario summary should cover when, where, and how the event occurred; the safety considerations; how the safety considerations were mitigated; and the consequence.

Beginning of the Scenario: The beginning of the scenario includes the initial condition, initiating event, and boundary conditions.

Initial Condition The initial condition describes the initial system and human conditions that have implications for the scenario progression and safety. The discussion should include information about the environment, system, personnel, and, task contexts.

Important aspects that should be identified include:

• Structures, systems, and components (SSCs) with latent failures, that are unavailable (tagged out), or have historically unreliable performance (especially the ones that would affect operators decisions and the scenario).

• The facility operating modes (e.g., at-power, low-power, and shutdown).

• Special or temporary system alignment.

• Workers not in their normal locations.

• Operating team not in normal configuration (e.g., temporarily having one individual performing dual responsibilities for a missing team member).

• Personnel substitution (e.g., temporary substitution of the individual familiar with the tasks to another individual who does not normally perform the tasks is likely to affect human performance).

• Other ongoing activities performed at the same time of the initiating event that can have effects on the scenario.

Initiating Event An initiating event could be triggered by a system failure or a human error. The initiating event narrative should be described at a level of specificity such that knowledgeable readers conversant with the design of the facilities in general, but not familiar with the details of the specific facility, can have a general understanding of the scenario (e.g., a small loss-of-coolant accident (LOCA) at a hot leg, a loss of offsite power event due to grid failure, and the loss of an essential electric bus causing reactor trip due to human error in maintenance).

Boundary Conditions The boundary conditions specify the analysis scope and the assumptions applied to the analysis. This could include limiting the analysis scope to focus on the primary considerations and to make simplified assumptions such as making deterministic assumptions about the status of systems (e.g., damage associated with the initiating event) and personnel (e.g., personnel availability).

Scenario Progression and End State: The scenario progression documents the scenario development following the given initial condition, initiating event, and boundary conditions. The scenario progression should be documented from the eyes of the human in the scenario. HRA analysts need to understand the mindset of the operators in different steps of the scenario (e.g.,

their view of the situation, task priorities, concerns, and locations, etc.). The scenario progression should describe the safety consideration and the responses of systems and humans to the safety consideration. At a high-level, these responses can be summarized using an analogy to the following macrocognitive functions:

• Cues for Detection

• Diagnostic Information for Understanding and Decisionmaking

• Physical actions for Action Execution

• Interteam interaction for Interteam coordination 3-4

The cues are the information that gets the attention of the human for Detection and triggers a persons cognitive process. The diagnostic information required to make a diagnosis and define the situation awareness are part of Understanding. Decisionmaking refers to making a decision based on the situation and diagnosis. Action execution refers to the tasks required to implement the decision. For each of the bullets above, describe the scenario progression from the environment, system, personnel, and task contexts. Table 3-1 provides guidelines about the content of the scenario progression.

Table 3-1 Narrative Information Coverage of a scenario analysis Safety consideration:

- What are the consequences?

- What are the needed system functions and human actions to prevent the consequences from happening given the initiating events and boundary conditions?

- What is the consequences safety significance?

Cues:

- What are the cues?

- How are the cues generated?

- What are the means to detect the cues?

Diagnosis and decisionmaking:

- What is the information needed for diagnosis?

- How are the diagnosis and decisionmaking performed? What are the bases and constraints of diagnosis and decisionmaking?

- What is the information that could mislead the human to a wrong diagnosis?

Physical actions:

- What are the automatic system responses to prevent the consequence from happening or to mitigate the severity of the consequence?

- What are the manual actions needed to mitigate the safety consideration? How are the actions performed? What are the constraints of performing the actions?

Interteam coordination:

- What kinds of communication, coordination, and collaboration among different entities are required?

- What are the considerations that could have significant effects on team responses?

The description of the scenario progression should include the end state of the system after the successes and failures of the responses of systems and humans to the safety consideration.

Guidance for Developing the Scenario Timeline The scenario timeline describes the scenario in chronological order. The documentation of the scenario timeline should use a two-column structure with the first column showing the date and time, and the second column showing all other information. It is recommended to add symbols in front of each statement in the second column to distinguish the type of information.

Column 1 Date and Time: For predictive (hypothetical) event analysis, the initiating event occurs at time zero. For retrospective (actual) event analysis, the initiating event starts at the local date and time that the actual event occurred. The actual local date and time has hidden information for assessing human performance. For example, if an event happens on a Sunday night, it could imply a reduced staffing level. If incidents occurred before the initiating event, the incidents should be indicated in the timeline. In this case, these events are placed before the initiating event as part of the background information.

3-5

Column 2 All Other Information System automatic responses, cues, human responses, and notes: The information in the second column is classified into four types to improve the understanding of the human-system interactions. Each information type is denoted by a bold letter as described below:

• System automatic responses (S): The S indicates that the information is a system automatic response based on the set points or logic of the automatic component actuations or that a system failed to perform its designed function. An example is S:

safety injection injected coolant into the RCS at 1,600 pounds per square inch gauge (psig).

• Information needed for human responses (I): The I indicates the information generated from a system or other source that is available for the human to diagnose the situation or make decisions. Examples are the alarms that trigger operator notification about a system abnormality.

• Human responses (H): The H indicates important human cognitive activities that include detecting the cue, making a diagnosis, entering/exiting procedures, making decisions important to the scenario, and performing actions. The actions could be either physical interference with a system to change the scenario progression or the actions that should be performed but are not performed that allows safety degradation of the scenario. Each human response should include the task and the individual who performs the task. For example, a reactor operators (RO) action can be denoted as H(RO). If every crew member could perform the action, the action can be denoted as H(Crew).

• Notes (N): The N indicates background, explanatory, context, or supplemental information to the system automatic responses (S), human response (H), and information (I). For example, an H(RO) is depressurize the reactor pressure vessel (RPV) to a certain pressure range at a rate less than 100 °F/hr. The (N) could be the task takes about two hours by periodically manually opening and closing a safety relief valve to provide additional information about the ROs action to depressurize the RPV.

Realizing that constructing a detailed timeline is resource intensive and may be impractical to include all human activities, the analysis should be done at the proper level of detail that is technically justifiable to capture human actions that are important to the scenario.

3.1.2. Identify and Define the Human Failure Events The purpose of this part of Step 1 is to identify HFEs as the analysis units of an HRA and define them at a high level. The PRA should provide the HFEs that need to be analyzed. HFEs include pre-initiator, initiator, and post-initiator actions. Typically, pre-initiator and initiator HFEs are not explicitly modeled in PRA because the human error contribution is included as part of the component reliability estimates and initiating event frequencies, respectively. Real events, such as those analyzed by the SDP and ASP program, may involve actions that are not included in the basic PRA models. If that is the case, additional HFEs may be identified, defined, and analyzed.

For the purposes of IDHEAS-ECA, the identified HFEs and their definitions are documented in Section A.2 of Worksheet A found in Appendix A.

Guidance for the Identification of HFEs As mentioned above, the PRA should already identify the HFEs that need to be analyzed. In the case that a new HFE needs to be identified to analyze a real event, the identification is based on how the real event deviated from the scenario modeled in the PRA. HFEs can be 3-6

identified by searching for human actions in which there is an interaction of humans with mission-critical systems as well as noncritical systems. Manipulations of noncritical systems may impact mission-critical system functions and personnel performing key actions with mission-critical systems. Generally, HFEs are modeled as errors of omission (EOOs).

However, the search process should also identify errors of commission (EOCs) that impact mission-critical system functions. With respect to EOCs, the following is a summary of the discussion in NUREG-1624, Rev. 1 [4] and NUREG-1921 [5] regarding the identification of EOCs:

• The action directly disables the system, sub-system or component needed to provide the system function required in the scenario.

• There is a rational justification to indicate that the EOC is well-intentioned. The common situations are: (1) existence of competing goals, and (2) personnel cannot fully evaluate the consequences of the decided action, or personnel do not understand the systems and consequences of the decided action.

• The unintended (slips type) human errors have EOO and EOC considerations that need to be analyzed separately. For example, switching off a wrong pump due to the close vicinity of the pump switches. First, the intended pump was not switched off (an EOO),

and second, an unintended pump was switched off (an EOC). Whether the EOC should be explicitly modeled depends on the EOCs impact on the scenario. The EOC should be modeled explicitly if it has cascading effect on the scenario course. If the EOC only affects the workers performance (e.g., increase workload) then the EOC does not need to be explicitly modeled.

3.1.3. Identify the Scenario/Event Context Identification of scenario context refers to the search for the conditions that challenge or facilitate human performance in the scenario. The process of searching for scenario context should focus on the conditions that can affect the macrocognitive functions and lead to undesirable consequences. Context affects human performance by directly impacting systems and personnel or mitigating the adverse effects of other conditions. Scenario context is documented in four categories: environment and situation, system, personnel, and task. The four context categories are not intended to represent an exhaustive classification system.

Rather, they are intended to guide the search. Scenario context serves as the high-level guidance for defining and analyzing HFEs and provides a basis for estimating the HEPs in the scenario. In HEP estimation, the context is represented by the PIF attributes.

The NRC staff developed several probing questions and considerations to identify the context that can affect the macrocognitive functions in each of the context categories. The probing questions and considerations are provided below. HRA analysts may develop additional questions and considerations to probe the possible conditions that can lead to impacts on human performance.

For the purposes of IDHEAS-ECA, the scenario context (identified using the probing questions and their answers) and the list of applicable PIFs are documented in Sections A.3 and A.4 of Worksheet A, respectively, found in Appendix A. The list of all the potentially applicable PIFs for each of the context categories is provided in Table 2-2 through Table 2-5, respectively.

Guidance for Assessing the Environment and Situation Context The environmental and situation context specifies the performance-challenging conditions in the personnels work environment and the situation in which the HFEs are performed. It includes weather, radiation or chemical materials in the workplace, and any extreme operating 3-7

conditions. Hazards such as steam, fire, toxic gas, seismic events, or flooding can introduce environmental conditions that impede personnel performance.

Questions for probing the environment and situation context that could affect human reliability are:

• Where do personnel perform the actions? Are there environmental considerations adverse to the action reliability?

• Are there things affecting accessibility or habitability of the workplace, including travel paths?

• Does the workplace have good visibility needed for human actions?

• Are the noise in the workplace and communication pathways expected to affect the reliability of completing the actions?

• Is the work environment very cold, hot, or humid?

• Is there resistance to personal or vehicle physical movement, such as strong wind, still or moving water?

Below are some considerations for the environment and situation context:

• Noise, smoke, and precipitation can affect information detection.

• Harsh environmental conditions, such as extreme heat or cold, may lead to early termination of situation assessment because personnel are unwilling to seek additional data to reconcile conflicts in the information.

• Harsh environmental conditions can adversely affect decisionmaking (e.g., reducing decisionmakers ability and effort in evaluating available strategies, thoroughly deliberating decisions, or mentally simulating action plans).

• Environmental conditions on travel paths and at worksites can restrict personnels motor movement, reduce their motor skills, or limit the time that they can steadily perform motor activities. Examples of these conditions are wearing heavy protective clothes, high water on travel paths, high winds, extreme heat or cold, earthquake aftershocks, and chemical or other toxic contamination.

• Environmental conditions such as noise or smoke can impede interteam collaboration.

Guidance for Assessing the System Context The system context specifies the conditions affecting the systems needed to perform design functions that can subsequently lead to human failures. Identification of system context should focus on conditions that create conflicting priorities, confusion, and distractions to human performance.

Questions for probing the system context:

• What are the consequence and the causes (e.g., core damage caused by a LOCA)?

• What are the system automatic responses expected to be actuated (e.g., reactor trip and safety injection actuation)?

• What are the SSCs needed to mitigate the event? What are the constraints of implementing their use?

• What are the system and human responses required to bring the system to a safe state or to mitigate the event? What are the set points for the automatic system responses?

3-8

Below are some considerations for the system context:

• Systems may become unavailable or behave abnormally due to accidents, incidents, hazards, maintenance, repairs, aging, or concurrent activities to protect workers or major equipment. For example, computer systems may become temporally unavailable due to network congestion; some sensors of NPP systems may become unreliable due to an electric fault; operational system components or equipment may be disabled due to problems in related systems (such as other reactor units in multi-unit NPPs).

• Electrical faults may reset systems or components to an undesirable status.

• The designed operational range of the SSC could be exceeded, and functions needed to support the component or instrument operation may be inadequate.

• Structures may have degraded environmental conditions or be inaccessible due to hazards or construction activities.

• Automated systems could be intentionally turned off based on a well-intentioned, but incorrect, beliefs by the crew.

Guidance for Assessing the Personnel Context The personnel context specifies the conditions that challenge or facilitate humans (e.g.,

individuals, teams, or organizations) to perform the tasks. The context affects personnels task performance in detecting information, understanding the situation, making decisions, executing planned actions, and interteam coordination.

Questions for probing the personnel context:

• What is the command and control structure?

• What are the key concepts of operation (e.g., staffing, training, validation, etc.)?

• Are there perceived potential fitness-for-duty (fatigue, substance abuse, or illness) issues?

• What are the manpower and skillsets needed in the scenario?

• What are the potential considerations that could adversely affect teamwork and communication?

Below are some considerations for the personnel context:

• Availability of personnelConsider the amount and types of personnel available to respond to the event relative to the personnel needed. Personnel may become unavailable due to reasons such as multiple simultaneous events, environmental effects, or duties unrelated to the event.

• Operational limitations of personnelPersonnel may not perform work as expected due to reasons such as physical limitations, not being prepared or trained for the type of events, or conformation to special safety or regulatory requirements.

• Organizations may not have adequate infrastructure to support teamwork due to reasons such as safety culture, authorization restrictions, conflict of interest or goals, or lines of communications.

• Availability of personnel supportPersonnel may lack necessary support such as training, tools, procedures or protocols, expertise due to reasons such as hazards, surprise of the event, beyond-design-basis accidents, lack of experience using the supporting items, and needs for sharing the limited supporting items.

• Environmental conditions (such as fire, smoke, flood, earthquake, noise, illumination, temperature extremes, and high radiation) that directly impact human performance may change during the evolution of the scenario.

3-9

Guidance for Assessing the Task Context The task context specifies special conditions about tasks that need to be performed, how these tasks are expected to be performed, the demands of the tasks, and the success criteria of the tasks. The conditions may change what human tasks are required, the task requirements, or the task difficulty. Task difficulty refers to the demand for personnel cognitive resources and collaboration. The characterization of the human-system interactions and the conduct of operations specify how tasks should be performed. Some aspects such as burden and pace of the tasks may be better understood from the perspective of the conduct of operations and operational experience.

Questions for probing the task context:

• What are the constraints in implementing the tasks?

• What is the potential task interference (e.g., sharing the same resource with the other concurrent tasks) and task dependency (e.g., tasks have to be performed in sequential order, such as obtaining external permission to perform the task)?

• Cues for Detection: This refers to cues that would lead an operator to notice the safety consideration.

o What are the credible cues that point to the system problem?

o How are the cues generated?

o How are the cues detected (by whom, where, and timing)?

o What training is related to the cues in the scenario?

o What are the key factors affecting cue detection?

• Diagnosis and situation awareness for Understanding: This refers to the information and mechanisms for the operator to understand the situation and diagnose the problem.

o What information is needed for the situation diagnosis? How is each individual piece of information generated and obtained (by whom, where, and timing)?

o What is the basis (e.g., which procedure) for making the diagnosis and situation awareness and by whom and where is it implemented?

o What is the operator training related to the diagnosis?

o What are the key factors affecting the diagnosis?

• Decisionmaking: This uses the information based on the understanding of the situation to make decisions about how to respond to the situation.

o What are the criteria or rules for making the decisions?

o How is the decision made and decision basis (e.g., which procedure, by whom, where, and timing)?

o What are the competing goals and alternative options when making the decision?

o What are the key factors affecting the decision?

• Action: This refers to implementing the decision by interacting with the system to change the scenario direction.

o What is the basis for performing the tasks (e.g., which procedure), and how the tasks expected to be performed (by whom, where, and timing)?

o What are the success criteria of the actions?

o What are the key factors affecting reliability of completing the actions?

o Action execution - Are the manual actions physically strenuous?

• Interteam coordination: This refers to interactions between multiple entities (individuals, teams, and/or organizations) involved in the event.

o What decisionmaking authorities are involved (and other organizational factors/interactions that might come into play?

3-10

o How are communications, resource allocations, information, and knowledge managed?

Below are some considerations for the task context:

• Use of computerized HSIs and supporting systems add additional work to personnel.

• Multiple, simultaneous events may lead to multitasking, interruption, and distraction.

• Failure or unavailability of operational system components may make event progression unpredictable.

• Unusual event evolution may reduce time available for required human actions.

• Complex events often require personnel to perform tasks in distributed locations.

• Personnel may need to perform additional tasks upon failures of automated systems.

• Personnel may make non-required changes to system status or interfere with system automation with good intentions, yet the changes may lead to undesirable consequences.

3.2. Step 2 - Analyzing Human Failure Events The purpose of this step is to model the challenges to human performance of an HFE and identify failure opportunities for HEP quantification. It includes defining HFEs and identifying critical tasks in an HFE. The information obtained and generated from the analysis of Step 2 is documented in Worksheet B of Appendix A.

3.2.1. Defining HFEs The purpose of defining HFEs is to define the scope of analysis for an HFE. HFEs are the human actions defined in PRAs human basic events. Thus, the HFEs should have been defined in a PRA model. Yet, HRA analysts should verify the definition and may add additional specifications for HFEs in the event being analyzed under the given conditions (described in Worksheet A). The HFE definitions are documented in Section B.1 of Worksheet B found in Appendix A.

Guidance for the Definition of HFEs The HFE is defined at a level that describes the failure of the human action and links it to the affected systems. The definition of the HFE should include, but not be limited to, the following items:

• success criteria that define the desired end states or outcomes of the systems with the success of the HFE,

• consequence of the HFE occurrence,

• beginning and ending points of the HFE,

• relevant procedural guidance for the HFE,

• cues and indications for initiating the HFE and their timing, and

• available time to perform the HFE (whether the HFE is time critical).

3.2.2. Task Analysis and Identification of Critical Tasks The purpose of task analysis is to identify potential failure opportunities in an HFE for HEP quantification. The potential failure opportunities in an HFE are represented by critical tasks.

An HFE can be divided into tasks and critical tasks. A critical task is essential to the success of the HFE and failure of any critical task will result in the occurrence of the HFE. A task is not essential to the success of the HFE; however, tasks may interfere with the 3-11

performance of critical tasks. Therefore, tasks should be considered when analyzing critical tasks.

Performing task analysis for an HFE is to understand what it takes for operators to succeed or fail the human action. A task diagram and/or timeline is a useful tool to graphically represent the tasks and critical tasks needed for the HFE. Moreover, a task diagram can depict the relationship between the critical tasks and the success or failure of the HFE, while an HFE timeline helps to assess the time available and time needed for the HFE. In addition, if an HRA credits recovery of failure of the critical tasks, a task diagram should identify the credible recovery opportunities.

An important aspect in identifying critical tasks is the level of breaking down an HFE into critical tasks. Guidance for this important aspect is provided below.

For the purposes of IDHEAS-ECA, the task diagram/timeline and critical tasks are documented in Section B.2 of Worksheet B found in Appendix A.

Guidance for Identifying Critical Tasks and Breaking Down an HFE into Critical Tasks Reviewing existing documentation is usually the first step in identifying critical tasks. The critical tasks for an HFE may have already been defined in training programs, quality assurance documents, fault tree analysis, etc. Identification of critical tasks may also consider the error recovery opportunities. Because there may be opportunities for the operating personnel to recover from an error within the time window, the task analysis may also identify opportunities for such error recoveries. Examples of error recovery include additional cues and monitoring system feedback (i.e., indications that the system is not responding as would be expected if the intended action had been completed correctly).

Additional guidance for identifying critical tasks is as follows:

• What is a critical task:

o A critical task constitutes a recognizable and consequential unit of human activities.

o A critical task needs to be performed by humans to achieve a desired plant status; failure of a critical task leads to the HFE.

o Successful performance of the execution portion of a critical task will alter the scenario progression towards a safer plant status.

• Boundaries between critical tasks can be distinguished by any of the following:

o Clearly defined goal.

o Clearly defined initial or entry state.

o Clearly defined ending or exit state (i.e., consequences or outputs).

• Scope of a critical task o A critical task may be represented with one or several macrocognitive functions.

A critical task usually includes physical actions to change the scenario progression, but human physical actions are not a necessity. In some situations, a critical task could be any of the macrocognitive functions.

Breaking an HFE into too many detailed critical tasks tends to hide the context and results in the tedious work of quantifying HEPs for all the critical tasks. Because the critical tasks identified for an HFE are just one way to model the HFE, there are no universally applicable rules on the level at which an action should break down into tasks. After all, the purpose of representing an HFE with critical tasks is to facilitate PIF assessment and HEP estimation. Following are the guidelines for breaking down an HFE into critical tasks:

3-12

• Use as few critical tasks as possible to represent the HFE.

• Further break down the HFE only when the PIF attributes vary for different portions of the HFE.

• An HFE should be broken into critical tasks at a level that retains the context of the HFE and can be represented with macrocognitive functions.

• Stop breaking down the tasks at the level where there are performance indications or empirical data available to inform HEPs. For example, expert judgment has been a prevalent way to estimate HEPs; if expert judgment is used, the HFE should be broken down to critical tasks at the level with which experts are familiar enough to make judgment.

More detailed guidance on task analysis can be found in Chapter 4 of NUREG-2199, Vol.1 [6],

which offers explicit guidelines on developing task diagrams, identifying recovery paths, and developing timelines. Detailed guidance on identifying critical tasks can also be found in Chapter 4 and Appendix G of NUREG-2198 [1].

3.3. Step 3 - Modeling Failure of Critical Tasks The purpose of this step is to model the failure of critical tasks in order to quantify . This is performed for every critical task in an HFE. It includes characterizing a critical task and determining CFMs applicable to the critical task. The information obtained and generated from the analysis of Step 3 is documented in Worksheet C of Appendix A.

3.3.1. Characterization of a Critical Task The characterization of a critical task is to specify the conditions relevant to the critical task that can affect the reliability of performing the critical task. In Step 3, the high-level information about task characterization has been collected and documented in Worksheet A for the entire scenario and Worksheet B for the whole HFE. This step specifies and refines the information for a given critical task. For example, while the HFE definition may include all the procedures needed for the HFE, every critical task may have its own procedure, or a critical task may not have a procedure. The characterization of a critical task is one of the inputs for assessing PIFs, especially the task-related PIFs, and is documented in Section C.2 of Worksheet C found in Appendix A Guidance for the Characterization of Critical Tasks Characterization of a critical task should include, but is not limited to, assessment of the characteristics listed in Table 3-2.

Table 3-2 Critical Task Characterization for HRA Critical Task Description Characteristics Critical task goal The expected outcome of the critical task with respect to the desired system states (e.g., reach hot shutdown within 3 hours3.472222e-5 days <br />8.333333e-4 hours <br />4.960317e-6 weeks <br />1.1415e-6 months <br />, flee the building).

Specific requirements Specifications for the critical task goal such as timing requirements or how the critical task goal should be achieved (e.g., monitoring parameters at a certain time interval, using secondary cues when the primary cues are not available, cooling down the RCS within a certain rate).

3-13

Table 3-2 Critical Task Characterization for HRA (continued)

Critical Task Characteristics Description Cues and supporting The cues to initiate the critical task and key information needed to information perform the task. A cue could be an alarm, an indication, a procedure instruction, or others (e.g., onsite report). The supporting information is in addition to the cue required to perform the task.

Procedures Available procedures, guidance, or instructions designed for the critical task.

Personnel Types of personnel needed for the critical task, minimum staffing required, special skillsets required.

Task support Job aids and reference materials needed, and tools and equipment needed.

Location Where the task is performed, special environmental factors at the location.

Cognitive activities Cognitive activities involved in the task that place demands on their corresponding macrocognitive functions.

Concurrent tasks Concurrent tasks that compete for personnels cognition and resources (e.g., tools, job aids).

Interteam Interteam collaborative activities required for the task and coordination requirements for communication facilities (e.g., equipment, tools, considerations devices).

3.3.2. Identification of Applicable Cognitive Failure Modes Any critical task can be achieved through one to all five macrocognitive functions. The cognitive failure of a critical task is the result of failure of any macrocognitive function it demands. Thus, the CFMs are the classifications of the various ways that a critical task may fail.

For the purposes of IDHEAS-ECA, the applicable CFMs are documented in Section C.1 of Worksheet C found in Appendix A.

Guidance for Identifying the Applicable CFMs The five CFMs used in IDHEAS-ECA are the failure of the macrocognitive functions (the high-level CFMs described in IDHEAS-G [1]). Therefore, once the macrocognitive functions demanded by a critical task are identified, the failure of the identified macrocognitive function(s) is (are) the applicable CFM(s). The CFMs are defined as follows:

• CFM1 - Failure of Detection

• CFM2 - Failure of Understanding

• CFM3 - Failure of Decisionmaking

• CFM4 - Failure of Action execution

• CFM5 - Failure of Interteam coordination Any critical task consists of cognitive activities such as monitoring parameters or executing procedure steps. The cognitive activities determine the macrocognitive functions required for the critical task and are the basis for identifying the CFMs that are applicable to the critical task. The analysts should have a clear understanding and documentation of the actual human activities included in each CFM.

3-14

The cognitive activities of a critical task are assessed using the taxonomy of cognitive activities, which is summarized in Table 3-3. The macrocognitive function demanded by the critical task is identified using the assessment of the type of cognitive activities (i.e., the second column of Table 3-3).

Table 3-3 Taxonomy of Cognitive Activities Macrocognitive Types of cognitive activities function Detection

• Detect cues (through carefully monitoring, searching, inspecting, or comparing, etc.)

• Acquire information (checking, reading, communicating/chatting, computing, etc.)

Understanding

• Maintain situational awareness

• Assess status based on indirect information

• Diagnose problems and resolve conflicts in information

• Make predictions or form expectations for the upcoming situation development Decisionmaking

• Make a go/no-go decision for a prespecified action

• Select among multiple options or strategies

• Change or add to a preexisting plan or strategy (e.g., changes of personnel, criteria, subgoals)

• Develop a new strategy or plan Action

• Execute cognitively simple actions Execution

• Execute cognitively complex actions

• Execute long-lasting actions

• Execute control actions

• Execute fine motor actions

• Execute physically strenuous actions Interteam

• Communicate between different groups, teams, or organizations coordination

• Cooperate between different groups, teams, or organizations

• Coordinate (including command and control) between different groups, teams, or organizations While the selection of CFMs applicable to a critical task is determined by the cognitive activities involved in the critical task, there may be situations in which the boundary between CFM1 Failure of Detection and CFM2 Failure of Understanding is ambiguous, and so is the boundary between CFM2 Failure of Understanding and CFM3 Failure of Decisionmaking. For example, if a critical task involves cognitive activities acquiring multiple pieces of information through checking or reading indicators, then CFM1 Failure of Detection is applicable. However, after the pieces of information are correctly acquired, operators still could not form a satisfactory understanding of the situation, diagnose the problem, or resolve conflicts in the information; thus, they need to seek additional information. This activity of seeking additional information demands the macrocognitive function of understanding because it requires operators to integrate multiple pieces of information to form a coherent mental representation of the situation.

IDHEAS-G [1] explains the process of achieving each macrocognitive function, and the elements of the process are referred to as processors. Each processor represents a way that the macrocognitive function fails. It is recommended that HRA analysts use the processors to verify the selection of the applicable CFMs and distinguish between the CFMs of a critical task.

3-15

Figure 3-3 through Figure 3-7 show the cognitive activities and processors associated with each macrocognitive function, respectively.

Detection cognitive activities Detection processors D1. Initiate detection - Establish the mental model for information to be detected D2. Select, identify, and attend to

• Detect cues sources of information

• Acquire (gather) D3. Perceive, recognize, and information classify information D4. Verify and modify the outcomes of detection D5. Retain, document/record, or communicate the outcomes Figure 3-3 Cognitive Activities and Processors for Detection Understanding cognitive activities Understanding processors U1. Assess/select data

• Maintain situational awareness U2. Select/adapt/develop the mental model

• Assess status based on indirect information U3. Integrate data with the mental model to generate the

• Diagnose problems and outcome of understanding resolve conflicts in (situational awareness, information diagnosis, resolving conflicts)

• Make predictions or form U4. Verify and revise the outcome expectations for the through iteration of U1, U2, upcoming situation and U3 development U5. Export the outcome Figure 3-4 Cognitive Activities and Processors for Understanding 3-16

Decisionmaking cognitive activities Decisionmaking processors DM1. Adapt the infrastructure of decisionmaking

• Make a go/no-go decision DM2. Manage the goals and for a pre-specified action decision criteria

• Select among multiple DM3. Acquire and select data for options or strategies decisionmaking

• Change or add to a pre- DM4. Make decision (judgment, existing plan or strategy strategies, plans)

• Develop a new strategy or DM5. Simulate or evaluate the plan decision or plan DM6. Communicate and authorize the decision Figure 3-5 Cognitive Activities and Processors for Decisionmaking Action Execution cognitive activities Action Execution processors

• Execution of a cognitively simple action E1. Assess action plan and criteria

• Execution of a cognitively E2. Develop or modify action complex action scripts

• Long-lasting action E3. Prepare or adapt infrastructure for action implementation

• Control action E4. Implement action scripts

• Fine motor action E5. Verify and adjust execution

• Physically strenuous outcomes action Figure 3-6 Cognitive Activities and Processors for Action Execution Interteam coordination activities Interteam coordination processors T1. Establish or adapt teamwork infrastructure T2. Manage information T3. Maintain shared situational awareness

• Communication T4. Manage resources

• Cooperation T5. Plan interteam collaborative

• Coordination activities T6. Implement decisions and commands T7. Verify, modify, and control the implementation Figure 3-7 Activities and Processors for Interteam coordination 3-17

Below is some supplementary guidance to assist HRA analysts to determine applicable CFMs:

• Whether a CFM should be selected for a critical task depends on the nature of the task, not the PIFs. The required macrocognitive function is critical to accomplish the critical task. For example, if collecting information or detecting is necessary to achieve the goal of the critical task, then CFM1, failure of detection, is applicable.

• CFM2 through CFM5 are conditional assuming that the preceding macrocognitive functions are successful. For example, CFM2 is the failure of personnel to understand the situation assuming that personnel have correctly detected the cues needed to start the critical task.

• CFM1 is applicable if detection is required for understanding the situation, making a decision, or executing an action.

• CFM2 is under the assumption that information/cues are correctly detected, and personnel needs to integrate pieces of information with their mental model of the situation to make sense of the situation.

• Similarly, CFM3 is under the assumption that personnel already detected the information and made the right understanding of the situation. If a procedure directs operators' response without uncertainty (e.g., if... then ...), then CFM3 is negligible.

• CFM4 is under the assumption that personnel correctly detected the cues, made the right diagnosis/correct understanding of the situation, had the right decision or response plan, and all the personnel needs to do is execute the decision/plan by manipulating the systems. As long as there is manipulation required, CFM4 cannot be neglected.

• CFM5 is exclusively for failure of interteam coordination, communication, and cooperation. For example, the technical support center (TSC) fails to coordinate with the emergency response center for allocating some equipment. CFM5 is under the assumption that the individual personnel or teams correctly performed the other macrocognitive functions.

3.4. Step 4 - Assessing PIF Attributes Applicable to CFMs The PIFs represent the context of the HFE and facilitate quantification of the HEP. A PIF attribute is an assessable characteristic of a PIF and describes a way the PIF challenges the macrocognitive functions of a critical task and, therefore, increases the likelihood of error in the macrocognitive functions. Each PIF has a no impact attribute, which means that the PIF has no observable impact on the HEP. A PIF attribute represents a negative impact on human performance, which increases the HEP. Positive impacts to human performance are represented by the PIF having no impact on the HEP. The assessment of PIF attributes is based on the scenario context and list of applicable PIFs (Section 3.1.3), the definition of the HFE (Section 3.1.2), and the characterization of the critical task and applicable CFMs (Sections 3.3.1 and 3.3.2). Appendix B provides the full list of PIF attributes.

For the purposes of IDHEAS-ECA, the assessment of PIF attributes is documented in Section C.3 of Worksheet C found in Appendix A. Note that Worksheet C documents the results of both Step 3 and Step 4. This is for the convenience of documenting applicable PIF attributes of every CFM right after the CFM selection.

3-18

Guidance for Assessing PIF Attributes Identifying the applicable PIF attributes uses the scenario context, list of applicable PIFs, definition of the HFE, and list of applicable CFMs for every critical task, the assessment of PIF attributes involves the following steps:

(1) Select PIFs within the boundary conditions context of the scenario and definition of the HFE The IDHEAS-G PIF structure provides 20 PIFs (as listed in Table 2.1 and described in Table 2-2 through Table 2-5), which should be the starting point. Based on the scenario context, many PIFs may not be relevant; therefore, they are not selected. If a PIF is not selected, a rationale should be given for why it is not relevant.

(2) Select PIF attributes relevant to the CFMs Elimination of some PIF attributes may be necessary so that the total number of PIF attributes associated with a CFM is manageable for the purposes of HEP estimation. The PIF attributes that do not contribute significantly to the CFM may be eliminated. If eliminating a PIF attribute is not obvious, a rationale should be provided for the elimination.

(3) Represent contexts that positively affect human performance While IDHEAS-G [1]

defines context as the conditions that challenge or positively affect human performance, the PIFs are defined as neutral. The PIF attributes all have a negative impact on HEPs except the attribute of no impact. The contexts that positively affect human performance are represented by alleviating some PIF attributes. For example, the no impact attribute of training means that training in the aspect of responding to the scenario being analyzed is good enough and would not increase HEPs. The context that training is better than the baseline and specific to the scenario of analysis means that the training may alleviate some PIF attributes such as in familiarity and teamwork and organizational factors.

(4) Assess the level of multi-scale PIF attributes The effect of an attribute on HEP can vary continuously with the quantitative measure of the attribute. Multiple discrete scales are used to model those attributes, which are referred as multi-scale attributes because they have multiple scales instead of being just present versus absent. The PIF tables in Appendix B present several measures for such attributes. The IDHEAS-ECA software uses those measures as benchmark scales and allows HRA analysts to select a scale value between one to ten, with one being the lower limit and ten being the upper limit of the attribute being modeled. For each scale selected, the software assigns the corresponding base HEP or PIF weight (i.e., a multiplier) based on a linear interpolation between the benchmarks.

Below is some supplementary guidance that may assist HRA analysts to determine applicable PIFs and attributes:

• Defining the boundary condition in Step 1 is to define the scope of the HRA being performed. Some factors, e.g., staffing, can be of concern, but there is no information on the average staffing level. The analysts believe that plants follow the minimal adequate staffing rules. Thus, the boundary conditions would include the reasonable assumption that the Staffing PIF has No Impact on the HEPs for this HRA.

• Assessment of PIFs should begin with the base PIFs: Scenario Familiarity, Information Completeness and Reliability, and Task Complexity. These three PIFs model the overall scenario characteristics and the specific task characteristics. When assessing the remaining PIFs, analysts do not need to select the attributes if those are already represented in the selected base PIFs.

3-19

• Table 2-2 through Table 2-5 describe the PIFs in their No-Impact state. The description is generic. For NPP control room operation, a simple way to think about No Impact PIFs is the following context: Experienced crews perform emergency operating procedures (EOPs) in control room simulators on routinely trained scenarios without complications, such as the SGTR example in Appendix C. With this context, all the PIFs are No Impact except the Task Complexity which is specific to a critical task.

• PIFs model the context that can increase or decrease the likelihood of human errors.

They do not model personnels uncomfortableness in performing the action unless the uncomfortableness exceeds some threshold of leading to human errors. For example, the working room may be out of ventilation and it is hot and humid, but not to the extent causing personnel to make more errors compared to what they would be in a ventilated room. In this case the Coldness/Heat/Humility PIF is considered as No Impact. In other words, the PIF is negligible.

3.5. Step 5 - Estimation of Pc - the Sum of Human Error Probabilities of Cognitive Failure Modes The purpose of this step is to estimate the probability of an HFE (i.e., the HEP) attributing to the CFMs of the critical tasks. The estimation of the overall HEP of an HFE has two parts:

estimating the error probabilities attributed to the CFMs ( ) and estimating the error probability attributed to the uncertainties and variability in the time available and time required to perform the HFE ( ). The estimation of the HEP is the probabilistic sum of and  :

= 1 (1 )(1 ) (3.1)

In Equation (3.1), is the probability of the HFE being analyzed (i.e., the HEP), and and have already been defined. Note the following:

• can also be viewed as the probability that the time required to perform an action exceeds the time available for that action, as determined by the success criteria. Pt assumes that actions are performed at a normal pace without complications and does not account for the increased likelihood of a human error due to time pressure. Time pressure is treated as a PIF and contributes to .

• assumes that the time to perform the HFE is sufficient. Sufficient time means that the HFE can be successfully performed within the time window that the system allows. If operators responses are as trained, then the time available to complete the action is sufficient. captures the probability that the human action does not meet the success criteria due to human errors made in the problem-solving process.

The information obtained and generated from the analysis of Step 5 is documented in Worksheet D of Appendix A. HRA analysts may choose to use the IDHEAS-ECA software to perform this step and document the results in lieu of Worksheet D.

3.5.1. Estimation of Pc is the probabilistic sum of the error probabilities of every critical task and is estimated as:

= 1 1 = 1 1 1 1 2 1 (3.2)

=1 where is the total number of critical tasks and is the error probability of the th critical task.

The error probability of th critical task ( ) is the probabilistic sum of the probabilities of all its applicable CFMs and is estimated as:

3-20

= 1 1 = 1 1 1 1 2 1 (3.3)

=1 where is the total number of CFMs applicable to the critical task, and is the probability of the th CFM applicable to the critical task. The probability of a CFM applicable to the critical task is a function of the PIF attributes associated with the critical task. The calculation of the probability of a CFM for any given set of PIF attributes, provided that all the PIF impact weights and base HEPs are obtained, is estimated as:

1

= 1 + ( 1)

=1 (3.4)

(1 + (1 1) + (2 1) + + ( 1))

=

The terms in Equation (3.4) are defined as follows:

• is the base HEP of a CFM for the given attributes of the following three PIFs:

information availability and reliability, scenario familiarity, and task complexity.

is also calculated as the probabilistic sum of the base HEPs for the three PIFs:

= 1 [(1 )(1 )(1 )] (3.5) where , , and are the base HEPs for information availability and reliability, scenario familiarity, and task complexity, respectively. The base HEPs for the information availability and reliability, scenario familiarity, and task complexity are provided in Table B-1 through Table B-3, respectively.

• is the PIF impact weight for the given attributes of the remaining 17 PIFs and is calculated as:

= (3.6) where is the human error rate at the given PIF attribute and is the human error rate when the PIF attribute has no impact. The human error rates used in Equation (3.6) are obtained from empirical studies in the literature or operational databases that measured the human error rates while varying the PIF attributes of one or more PIFs. Appendix B provides the values of the ratio for different PIFs in Table B-4 through B-15. It is noted that Table B-4 contains all the PIFs in the environment and situation context category and Table B-14 contains the PIFs mental fatigue, and stress and time pressure.

• is a factor that accounts for the potential recovery from failure of a critical task, and it is set to 1 by default. IDHEAS-ECA allow analysts to determine the value based on their judgment on the chance of recovering a critical task for the given CFM.

For the purposes of IDHEAS-ECA, the selected PIF attributes and the estimation of are documented in Worksheet D found in Appendix A.

3-21

Guidance for Estimating is calculated using the equations in this section and the tables in Appendix B. Alternatively, can be calculated using the IDHEAS-ECA software.

Note that the current version of IDHEAS-ECA and the software only provide the mean values of the base HEPs and PIF weights without giving the information of the main body and range of the distribution of those values. If an HRA requires the inclusion of the HEP distribution, the analysts need to make their own judgment of the distribution of .

Guidance for crediting recovery effect in PRA defines recovery actions as human actions that, on an as-need basis, provide a more realistic evaluation of significant accident sequences. Operator actions can be credited to restore functions, systems or components; to do this, operator recovery actions should restore failed equipment or find alternative equipment or configurations within the time period required.

Significant recovery actions may be evaluated through the same process as all other HFEs when it is considered important to do so to provide additional justification for the credit assumed.

Repair of components, meaning the restoration of a failed SSC by correcting the failure and returning the component to operability is typically quantified using empirical data (if credited at all) and is not treated using HRA techniques. These actions to restore functions, systems or components are new basic events that would be added to the PRA, not to be confused with the recovery of an HFE which is credited in the HEP of the HFE.

IDHEAS-ECA credits recovery in the HEP of the critical tasks of an HFE. The task diagram of an HFE shows a success path on which one or several critical tasks are performed to achieve the success of the human action being modeled. No matter what the reason for failure of a critical task, the assumption is made that following the failure, the operators continue other critical tasks. Consequently, operators have opportunities to detect the failure and correct the errors made. Such recovery mechanisms are typically credited in the evaluation of the HEP for the HFE, and not modeled explicitly as separate basic events in the PRA model. A recovery opportunity viewed in isolation is essentially another way of getting success. The opportunities for recovery can come from a number of sources. The error correction opportunities refer to the potential for placing the crew on an alternative success path or acting as additional cues to perform the correct task. In addition, plant conditions may evolve and generate new alarms or key parameter changes that crews would normally be monitoring, and which would serve as cues for identifying the need for a different response.

Crediting recovery should first assess the feasibility of recovering, e.g., whether the recovery opportunity occurs sufficiently early to allow time for the appropriate response to be executed. If the cues that could be used to correct the error would not occur before the end point of the HFE, then there is no opportunity for recovery. However, if the recovery is clearly feasible in that the cues for recovery would occur in time for diagnosis and recovery to the correct path, and time for the remaining tasks would also still remain available (e.g., any additional decisions or response execution activities), then there is an opportunity for recovery. The following criteria are used to assess the feasibility of crediting recovery in the HEP of an HFE:

1) A recovery path exists. It should be demonstrated that the event progression allows personnel to go back to the failure point to correctly perform the failed critical task. Some critical tasks may be irreversible and thus cannot be credited for recovery.

3-22

2) There are cues or indicators available to personnel for them to recognize the faiure and need for recovery.

3) There is at least one crew member responsible for monitoring the plant status and detecting the cues of the failure.

4) The time of the cue or the time taken to reach a procedural step that indicates the need for recovery is early enough to allow adequate time for recovery.

Recovery is feasible if all the criteria are met. If a critical task is recoverable, IDHEAS-ECA allows analysts to assign a recovery factor specific to each CFM of the critical task because the potential for recovery is dependent on the failure mode. For example, the error correction opportunities of manipulation tasks will primarily arise from a monitoring activity that is capable of detecting that the plant is not responding as would be expected if the intended action had been completed correctly. These opportunities focus on the crews assessment of the plant feedback.

The recovery factor, , in HEP calculation varies from 1 to any positive number, with 1 being no potential for recovery. IDHEAS-ECA does not provide reference values of the recovery factor mainly because recovery potential is situation specific. The potential for recovery can be quite different for well-practiced procedural tasks performed in a control room than for rarely performed tasks outside the control room. Below are some recovery mechanisms that can influence recovery potential:

Procedure design - late procedure steps requires operators to check and verify the correct performance of important earlier steps.

Training, work process, and conduct of operation (e.g., plant status check performed for shift turnover).

Unexpected instrument responses to an action.

New alarms that provide cues to indicate potential errors.

Multiple, diverse cues for recognition of the deed for recovery.

Finally, anaysts should consider the dependency between the error made and recovery. If the recovery relies on the same context as that for the early faiure of the critical task, then the recovery potential is reduced because of the dependence. In reaity, there are no truly independent opportunities to correct the errors. To actually credit recovery and especially the recovery in multiple CFMs and critical tasks, analysts shoud carefully review the timeline of the specific recovery paths and identify opportunities for recovery that are sufficiently independent.

3.6. Step 6 - Estimation of Pt - the Convolution of the Distributions of Time Available and Time Required For time-critical HFEs, uses the time available ( ) and time required ( ) to perform the HFE. To calculate , is represented by its cumulative distribution function (),

and is represented by its probability density function () and is estimated as the convolution of the two probability distributions, that is [7]

3-23

= = 1 () () (3.7) 0 HRA analysts need to estimate the probability distribution 1 (central tendency and dispersion) of and , for which guidance is provided below.

For the purposes of IDHEAS-ECA, the estimation and justification of the probability distributions for and are documented in Worksheet E found in Appendix A, and the calculation of is to be performed with IDHEAS-ECA Software.

Guidance for Estimating the Distribution of Time Available Estimating may require reference to engineering calculations [7]. For NPPs, is typically generated by thermal-hydraulic studies or computer simulations. It represents the time lapse from time zero to the time that a selected key parameter would exceed its safety threshold without human intervention. The nuclear industry has been developing computer codes to simulate plant behaviors in various conditions and scenarios. Performing many simulations that include various combinations of plant and equipment conditions with use of high-fidelity simulation programs can be very resource demanding and thus is not practical. On the other hand, many questions concerning event sequence timing are thermal-hydraulic problems.

Often low-cost, relatively simple calculations would have adequately answered the question at hand (e.g., the time taken to boil dry the steam generators in a loss of feedwater event). The analytic approach starts by reviewing the preliminary risk analysis results to identify the dominant risk contributors. The calculations can help analysts identify areas where uncertainty analysis is needed and where more sophisticated analyses should be performed to better define the success criteria. This phased approach makes uncertainty analysis feasible. Traditional engineering analyses tend to use point estimates (e.g., the best estimate) and deterministic analysis, but there are physical and analytical uncertainties and operational variability for .

Sensitivity studies allow analysts to evaluate the effects of the uncertainties and the variability associated with plant operation.

Estimation of the distribution should also consider the effect of human performance, which is the time dependency between important human actions in a PRA scenario. Studies show that there is significant crew-to-crew variability in performance time. Some crews moved through the response efficiently, resulting in more time available for subsequent actions. Other crews responded less efficiently than expected, resulting in less time available for subsequent actions. Therefore, any time dependency between the actions in an event may substantially affect the distribution of .

The IDHEAS-ECA software offers five options to represent  :

1) Normal distribution. When thermal-hydraulic simulation data suggest that the system time available for the required human action can be roughly modeled as a normal distribution, HRA analysts can calculate or estimate the mean and standard deviation of 1 Special caution when the probability distributions of and are assumed to be normal (Gaussian):

Since a normally distributed [random variable] can take on a value from the (, +) range, it has limited applications in reliability problems that involve time-to-failure estimations because time cannot take on negative values. However, for cases where the mean µ is positive and is larger than [i.e., the standard deviation] by several folds, the probability that the [random variable] takes negative values can be negligible. For cases where the probability that [random variable] takes negative values is not negligible, the respective truncated normal distribution can be used. [8]

3-24

the time available and enter these parameters to the IDHEAS-ECA software. Features of the normal distribution include:

• A normal distribution has a symmetric bell shape, the mean and median are equal, both located at the center of the distribution. The assumption of normality means that the data roughly fits a bell curve shape.

• A normal distribution is represented by the mean (µ) and standard deviation (). The probabilities below µ, µ + , µ + 2, and µ + 3 are about 50%, 84%, 98%, and 99.9%, respectively.

2) Gamma distribution. Gamma distribution is a two-parameter family of continuous probability distributions. The two parameters specify the shape and scale of a distribution. It is widely used to model continuous variables that are always positive and have skewed distributions.

3) Weibull distribution. It also uses two parameters, Shape and Scale, to model almost any kind of data distribution. It is a commonly used distribution for modeling reliability data and is often used to model the useful life of products.

4) Five-point estimation of probability distribution. Often the system time available for an HFE does not fall into a normal distribution. HRA analysts can estimate five points of the time distribution at 5th, 25th, 50th, 75th, and 95th percentile. The IDHEAS-ECA software interpolates the full distribution based on these estimates using the step function. In the IDHEAS-ECA software, the probability density functions of between zero to 5th percentile is specified as a half of the probability density function of between 5th and 25th percentiles, and the probability density function between 95th and 100th percentile is specified as a half of the probability density functions of between 75th and 95th percentiles.

5) Single-value threshold. The IDHEAS-ECA software allows analysts to enter a single value for that is usually calculated by using thermal-hydraulic simulation assuming no human intervention.

Guidance for Estimating the Distribution of Time Required is the probability that personnel could not complete the required human action within the available time. Human actions in HRA are assumed being performed as trained. The distribution of the time-required to complete a trained action can be caused by many factors.

Estimating the distribution of should consider three key aspects: nominal contributors, uncertainty factors, and bias factors. The following process is recommended for estimating the probability distribution of time required:

• Obtain an initial distribution of time needed including the central tendency and range.

This information can be obtained by reviewing operational and simulator data and interviewing operators. HRA analysts should collect a range of times (using multiple independent estimates to the extent possible). Average crew response time should be obtained, as well as an estimate of the time by which the slowest operating crews would be expected to complete the actions.

• Calibrate the initial estimation by reviewing the factors contributing to (see Table 3-4).

3-25

• Modify the distribution by identifying and reviewing uncertainty factors that may change (see Table 3-5).

Table 3-4 Typical Factors Contributing to Treqd Macrocognitive Factors Contributing to Time Required Function Detection Travel to the location to obtain the information.

Prepare and calibrate equipment needed for detection.

Detect and attend to an indication.

Confirm and verify the indicators.

Record and communicate the detected information.

Understanding Assess the information needed for diagnosis, such as knowledge and status of equipment.

Integrate low-level information to create and/or determine high-level information.

Identify plant status and/or conditions based on several parameters, symptoms and the associated knowledge; collect information and delineate complex information such as a mass and/or energy flow with which two or more systems interact with each other.

Delineate conflicting information and unstable trends of parameters (e.g.,

interpret SG pressure trends when one train has failed).

Wait for continuous or dynamic information from the system to complete diagnosis.

Verify the diagnosis results or reach a team consensus.

Decisionmaking Prioritize goals; establish decision criteria; collect, interpret, and integrate data to reach a satisfying decision.

Make decision based on parameters, choose strategies, or develop a plan.

Coordinate the decisionmakers (especially with hierarchy of decisionmaking or distributed decisionmaking team), achieve consensus needed for the decision, or wait for certain information to make a decision.

Simulate or evaluate the outcome of the decision.

Action Execution Evaluate the action plan and coordinate staff.

Travel and gain access to the action site.

Acquire (deploy, install, calibrate) the tools and equipment (e.g., put on gloves) to perform the actions.

Implement the action steps or continuous action and required timing of steps.

Confirm completion of the actions and wait for system feedback.

3-26

Table 3-5 Uncertainty Factors that Modify the Distribution of Treqd Uncertainty Considerations Factors Environmental Environmental factors affect allowable time for work.

factors Delay in personnel and equipment movement because of external hazards (e.g., bad weather makes it longer than usual to move personnel and equipment).

Limited continuous habitation (e.g., high radiation and dose exposure limits or external hazards reduce the habitable duration a worker can spend in the work area).

Plant condition Simultaneous multiple events that demand the same set of resources.

Multiunit events (e.g., an external hazard impacts multiple units in the same site).

Work site Different travel paths to worksite (e.g., the shortest path traveling to the accessibility work area may not be not available such that workers need to take alternative paths).

Hurdles to access the worksite (e.g., security system denies access).

Information Visibility of information availability Familiarity with the sources of information Procedures/ Applicability of procedures or instructions instructions Recency of training applicability and training Decisionmakers Variability of decisionmakers Variability in decision infrastructure Communication in distributed decisionmaking Staff Staff adequacy (e.g., whether concurrent activities would reduce the staff available for the action or whether tasks can be performed concurrently with more than adequate staff). Certain skill requirements may apply to the staff.

Command and control structure Staff experience (e.g., whether less trained, nonregular staff is used)

Equipment, Familiarity about setting up and operating the equipment tools, parts, and The availability and the time needed to obtain the needed parts, fuel, and keys keys to setup and operate the equipment Scenario Familiarity with the scenario familiarity Fatigue (mental Time of day and physical) Duration of having been on shift Crew-to-crew Crew-to crew-variability in time needed to perform the same actions; variability different crews may take different procedure paths, which leads to variability in time needed.

3-27

The IDHEAS-ECA software offers four options to represent  :

1) Normal distribution. Normal distribution is often used to represent the uncertainties in the time that it takes humans to perform an action. With the guidance above on estimating time required, HRA analysts can calculate or estimate the mean and standard deviation of the time required and enter these parameters to the IDHEAS-ECA software.

2) Gamma distribution. The analysts enter two parameters, Alpha and Beta, to specify the distribution.

3) Weibull distribution. The analysts enter two parameters, Shape and Scale, to specify the distribution.

4) Five-point estimation of probability distribution. If operational data are not adequate for confident estimation of the mean and standard deviation of the assumed normal distribution, or if evidence suggests that normal distribution is not appropriate for the situation (for example, the personal modeled fall into two distinctive groups), HRA analysts can estimate five points of the time distribution at 5th, 25th, 50th, 75th, and 95th percentile. The IDHEAS-ECA software interpolates the full distribution based on the five-point estimates.

Note that the IDHEAS-ECA software does not provide the option of entering a single number for

. This is because the time needed to perform an action can vary with a variety of time contributing factors such as those listed in Table 3-4 and inherit uncertainties with a variety of time uncertainty factors such as those listed in Table 3-5.

Guidance on Selecting a Time Distribution

• A normal distribution is for data that have one or a uniform set of factors driving the distribution. For example, the time it takes for individuals or crews to perform a well-trained task in the same way as trained mostly likely would fit to a normal distribution because the variability mainly comes from individual differences. However, if the individuals fall into two categories: well trained and less experienced with little training, then the performance time will not fit to a single normal distribution.

• Gamma distribution is often used to model biological data of which multiple heterogeneous factors drive the distribution. By manipulating the two parameters, a Gamma function allows the modeling of various shapes of distribution. For example, Gamma distribution should be a better choice than Normal distribution in modeling debris removal times where heterogeneous factors are involved.

• Weibull distribution is the most widely used for modeling reliability data. It is best used for modeling useful life time of components, which decays exponentially. It is particularly convenient to model asymmetric distributions.

• Five-point estimation is useful when analysts only have limited data points on performance time from simulator runs or they do not have any simulator data but have access to people who have experience observing operators performance of the action.

Analysts can obtain the estimation by asking questions like: Out of 100 crews, how long it would take the five slowest crews to complete this action? How long it would take the five fastest crews? Within how many minutes would half of the crews complete the action?

3-28

Guidance for Calculating Assuming that the HRA analyst estimates the parameters of the probability distributions (central tendency and dispersion) for and , the calculation of is performed using Equation (3.7) and any general calculation software. The IDHEAS-ECA Software has a function to calculate for the assumed distribution and parameters of and . Table 3-6 shows an example of the implementation of Equation (3.7) using OpenBUGS [9] and it is assumed that and are normally distributed with mean (; central tendency) and standard deviation

(; dispersion) of = 30 , = 3 , = 20 , and =

2 , respectively.

Table 3-6 Example Implementation of Equation (3.7) using OpenBUGS [9]

model {

t.avail ~ dnorm(mu.t.avail, tau.t.avail) # distribution of time available tau.t.avail <- pow(sigma.t.avail, -2) t.reqd ~ dnorm(mu.t.reqd, tau.t.reqd) # distribution of time required tau.t.reqd <- pow(sigma.t.reqd, -2)

Pt <- step(t.reqd - t.avail) # this node gives the probability that time required > time available

}

data list(mu.t.avail=30, sigma.t.avail=3, mu.t.reqd=20, sigma.t.reqd=2) # parameters of the distributions Running the script in Table 3-6 estimates that 2.7 x 103 . The script in Table 3-6 may be modified for other probability distributions supported by OpenBUGS taking into account the appropriate parameterization of the selected probability distributions. In the special case shown in the example (i.e., both and are normally distributed), Equation (3.7) can be calculated as [8]:

= 1 (3.8) 2 +

2 where [] is the standard normal cumulative distribution function 2 for the term inside the brackets. Figure 3-8 shows a graphical representation of for the example in Table 3-6, where is proportional to the area under where the two probability distributions intersect.

2 In Microsoft Excel, the function for the standard normal cumulative distribution function is =norm.s.dist(z,true) where z is the term inside the brackets in Equation (3.8).

3-29

0.25 Time Required Time Available 0.2 probability density 0.15 0.1 0.05 0

10 15 20 25 30 35 40 time (minutes)

Figure 3-8 Graphical Representation of Pt 3.7. Step 7: Calculate the Overall Human Error Probability Using the results from Step 5 ( ) and Step 6 ( ), the overall HEP is calculated using Equation (3.1). This calculation is performed by the IDHEAS-ECA software. Alternatively, it can be performed manually.

3.8. Step 8: Analyze HRA Uncertainties and perform sensitivity analysis PRA is a probabilistic model that characterizes the aleatory uncertainty associated with accidents at NPPs in that the results are given in terms of the likelihoods of accident sequences.

The purpose of the uncertainty analysis that is performed as part of the PRA process is to characterize uncertainties associated with the results of the PRA model. NUREG-1855 [10]

provides guidance for treatment of three types of uncertainty in PRA: parameter uncertainty, model uncertainty, and completeness uncertainty. The assessment of uncertainty on HEPs is a required part of the PRA.

Assessment of the uncertainty in the HEPs should be performed (at least for the significant HEPs) to the extent that these uncertainties need to be understood and addressed in order to make appropriate risk-informed decisions. Step 8 of IDHEAS-ECA is to analyze uncertainties associated with the obtained mean HEPs and perform the sensitivity analysis. This step adapts the guidance in HRA good practices (NUREG-1792 [11]) as follows:

1) Systematically analyze and document uncertainties in Steps 1-5. The uncertainties should include (1) those epistemic uncertainties existing because of lack of knowledge of the true expected performance of the human for a given context and associated set of PIFs, and (2) consideration of the combined effect of the relevant aleatory (i.e., random) factors to the extent they are not specifically modeled in the PRA and to the extent that they could alter the context and PIFs for the HFE.

2) Develop uncertainty distributions for the significant HEPs to capture the center, body, and range of an HEP associated with the uncertainty factors. If different and significant levels of an uncertainty factor, for example Training and Experience, are known to exist, it is random as to which personnel will perform the action at any given time. Thus, the mean for the single HFE/HEP should represent the average Training and Experience level, and the uncertainty should reflect the uncertainty attributable to the variation of the levels and any other relevant factors.

3-30

3) Perform sensitivity analyses that demonstrate the effects on the risk results for extreme estimates in the HEPs based on at least the expected uncertainty range. Analysts may propagate the extreme HEPs of the distributions through the quantitative analysis of the entire PRA, such as by a Monte Carlo technique. For the uncertainty that results from whether a PIF attribute should be included in the HEP calculation, IDHEAS-ECA recommends using sensitivity analysis or bounding analysis for the PIF attributes that are important contributors to the HEPs of the HFEs which have significant impact on the risk (at PRA level). Note that, in some cases, it may be sufficient to address the uncertainties only with qualitative arguments without the need to specifically quantify them (e.g., justifying why a change in the HEP has little relevance to the risk-informed decision to be made). In other cases, the HEP uncertainties may have significant impact on the risk-informed decision to be made. Analysts may choose to explicitly model the HFE as two or more different events, one for each representative situation. For example, one HFE is for the situation when a less experienced crew is on shift and one for the situation when a more experienced crew is on shift.

In IDHEAS-ECA, various sources of information are used to assess the impact of PIF attributes on HEPs. These sources collectively represent a range of the impacts of the PIFs. The IDHEAS-ECA method derives a best-estimate impact (i.e., the base HEPs and PIF weights) from a variety of available human error data and use the best-estimate impact for point estimates. A future development of IDHEAS-ECA uncertainty analysis will be to use an uncertainty distribution to represent PIF impacts. Monte Carlo sampling can be used to calculate the integrated uncertainty distribution of multiple PIF attributes with uncertainty distributions.

3.9. Summary of IDHEAS-ECA Relation between the IDHEAS-ECA steps IDHEAS-ECA consists of eight steps. Performing an HRA with IDHEAS-ECA is to perform all the eight steps and document the results of each step. The results of one step serves as the inputs to subsequent steps. Below is the outline of the relationship of the steps:

1) The process begins with analyzing the event scenario. The results of the analysis include scenario definition, operational narrative, scenario context, and a list of HFEs in the event. The results from Step 1 serve as the inputs to all other steps.

2) Step 2 focuses on each HFE identified in Step 1. The results of the analysis include the HFE definition, the task diagram and/or HFE timeline that graphically illustrates the success and failure paths of an HFE, and the critical tasks that must be accomplished for the success of the HFE.

3) Step 3 focuses on critical tasks. The results include the characterization of every critical task and the identification of the applicable CFMs determined by the macrocognitive functions required to perform the cognitive activities in the task. Task characterization specifies the information in Step 1 and Step 2 (i.e., the operational narrative and context of the scenario, the HFE definition, and the task diagram/timeline) for individual critical tasks.

4) Step 4 focuses on determining PIF attributes applicable to individual CFMs. The results are the applicable PIF attributes for every CFM. The determination of applicable PIF attributes is based on the scenario context, HFE definition, and task characterization.

3-31

5) Step 5 is to calculate HEPs of each applicable CFMs, and these HEPs are probabilistically summed to result in of the HFE. This step takes the CFMs identified in Step 3 and PIF attributes identified in Step 4 as the input to the calculation.

6) Step 6 focuses on analyzing time uncertainty of each HFE and calculating by estimating the distributions of time available and time needed for the HFE. While the estimation can be made with the results of Step 1 and Step 2, the detailed analysis of the critical tasks and relevant PIFs in Step 3 and Step 4 help to refine the estimation of the time needed.

7) Step 7 takes the results from Steps 5 and 6 and calculates the overall HEP using Equation (3.1).

Steps 1, 2, 3, and part of Step 6 all require information collection. These steps are equivalent to the qualitative analysis portion in many HRA methods. They transform the qualitative information that analysts collect for the HRA into structured elements that assist HRA quantification in later Steps. Steps 4 and 5, the calculation of of Step 6, and Step 7 consist of HRA quantification. The quantification is based on the specific formats of IDHEAS-ECA qualitative analysis steps. The IDHEAS-ECA software assists HRA analysts to perform Steps 4, 5, 6, and 7 after analysts complete the first three steps and document the results in the IDHEAS-ECA Worksheets.

Documentation of Analysis Results IDHEAS-ECA requires HRA analysts to document the results of the first six steps in five worksheets. Table 3-7 summarizes the content of the Worksheets. The details of the Worksheets are shown in Appendix A, and the use of the Worksheets is demonstrated in the examples of Appendix C.

Table 3-7 Summary of IDHEAS-ECA Worksheets Worksheet Content of the Worksheet Section A.1 Operational narrative This section includes the initiating event, initial conditions, boundary conditions, and event timeline. It may also include important scenario deviations and past operational experience review.

Section A.2 HFE identification List of all the important human actions, including the HFEs in the PRA model and additional important human actions that should be analyzed.

Worksheet A Section A.3 Scenario context documenting This section analyzes and documents the conditions that challenge or Step 1 results facilitate human performance. The documentation includes four categories of context relating to environment, system, personnel, and tasks.

Section A.4 Initial assessment of PIFs Based on the assumed boundary conditions of the event and context analysis, initially assess the PIFs and make the lists of PIFs that have no impact on HEPs of the HFEs and the PIFs that may impact the HEPs and need further assessment.

3-32

Table 3-7 Summary of IDHEAS-ECA Worksheets (continued)

Worksheet Content of the Worksheet Section B.1 HFE definition This section defines the HFE at a high level with basic information relevant Worksheet B to human performance of the action.

Documenting Section B.2 Task diagram and identification of critical tasks Step 2 results, An important human action may consist of multiple discrete tasks. A task Every HFE has diagram shows the tasks needed to achieve the action, the paths of the its own tasks, and the interteam coordination needed to achieve the tasks. Critical Worksheet B tasks are those required to meet the success criterion of the action; failing any of the critical tasks would fail the action.

Section C.1 Analysis of cognitive activities and identification of Worksheet C applicable CFMs documenting Analysis of cognitive activities are based on the macrocognitive functions.

Step 3 and Section C.2 Task characterization Step 4 results. Several characteristics for individual critical tasks are specified consistent Every critical with the operational narrative and context of the scenario, the HFE task has its definition, and the task diagram/timeline.

own Section C.3 Assessment of PIFs Worksheet C. Assessment of PIFs is to select the applicable PIF attributes from the PIF tables in Appendix B.

Worksheet D Worksheet D documents the HFEs, critical tasks, CFMs, and applicable PIF organizing the attributes for HEP estimation. The HEP of every CFM can be calculated information using equations and the base HEP and PIF weight values in the PIF tables needed for of Appendix B.

Step 5.

Worksheet E documents the estimation of time available and time needed Worksheet E for every HFE. The distribution of the time can be estimated as a single documenting number, the mean and standard deviation by assuming a normal the results of distribution, or a five-point estimation of probability distribution (at 5th, 25th, Step 6. 50th, 75th, and 95th percentile). , the HEP from time uncertainties, then can be calculated based on the estimation.

IDHEAS-ECA Summary With the documented results in Worksheets A-E, HRA analysts can calculate the probability of an HFE using the IDHEAS-ECA software. Alternatively, analysts can manually calculate the HEP using the base HEPs and PIF weights in Appendix B. Table 3-8 summarizes what is needed for every IDHEAS-ECA step, including the object of the analysis, key outputs, the Worksheet to be filled out, and the corresponding function of the IDHEAS-ECA software. Notice that the IDHEAS-ECA software has an additional function of generating an analysis report that summarizes the analysis results.

3-33

Table 3-8 Summary of the IDHEAS-ECA Process Note: IDHEAS-ECA calculates the HEP of an HFE in two parts: attributing to human failures under the condition that time available is adequate, and attributing to uncertainties in time available and time needed for the HFE.

Input Output Documentation IDHEAS-ECA Step (Object of (analysis results) required software analysis)

Step 1. The Operational narrative, Worksheet A N/A Scenario scenario scenario context, and analysis list of HFEs Step 2. An HFE HFE definition, task Worksheet B, N/A HFE analysis diagrams, and list of one for each critical tasks HFE Step 3. A critical Task characterization Worksheet C, N/A Critical task task (CT) and applicable CFMs one for each failure analysis critical task Step 4. A CFM PIF attributes Worksheet C, Software helps go Assessment of applicable for the one for each through the PIF PIF attributes CFM critical task attributes and select for CFMs them Step 5. An HFE HEP for every CFM Worksheet D, Software calculates Calculate with all and by summing one for each based on HEPs of CFMs CTs, HEPs of all the CFMs HFE selected PIF and sum to CFMs, of the critical tasks attributes and PIF attributes Step 6. An HFE Distributions of time Worksheet E, Software allows Time available and time one for each specification of the uncertainty needed, and the HFE time distributions analysis and calculated . and calculates .

Step 7. Software calculates Calculate the overall HEP of overall HEP an HFE by probabilistically summing and .

Step 8.

Uncertainty analysis and documentation Basis or Software generates HRA justification a summary report Documentation needs to be for analysts to edit documented in basis/justifications the worksheets and uncertainties 3-34

4 DISCUSSION AND CONCLUDING REMARKS 4.1. From IDHEAS-G to IDHEAS-ECA IDHEAS-G is a general HRA methodology from which application-specific HRA methods can be developed. IDHEAS-G consists of a Cognitive Basis Structure, an HRA process implementing the Cognitive Basis Structure, supplementary guidance for performing the HRA process, and an interface (Human Error Tables) for generalizing human error data. IDHEAS-G is intended to be general enough so it can be adapted to all nuclear HRA applications. It has the following features [1]:

• IDHEAS-G has a basic set of CFMs at three levels of detail and 20 PIFs each with a comprehensive list of attributes. Those allow the modeling of the variety of human actions and contexts in NPP HRA applications. Yet, using all the detailed CFMs and PIF attributes can be very time consuming for HRA analysts.

• IDHEAS-G provides multiple approaches for estimating HEPs. It is intended that different approaches may be adapted for specific HRA applications, depending on the available resources and data.

• IDHEAS-G establishes a set of Human Error Tables that generalize human error data from various sources to the IDHEAS-G CFMs and PIFs. Yet, using the data in the Tables to inform HEPs requires integrating the data for the specific HEP estimation approach.

Developing an application-specific HRA method from IDHEAS-G is to have a method specific for the application, concise and easy to use, and ideally having a HEP model that allows analysts to calculate HEPs. IDHEAS-G recommends the following approach for developing an application-specific HRA method:

• Define the scope of the application, requirements, and available sources for the intended use

• Keep the qualitative analysis the same as that in IDHEAS-G

• Develop application-specific sets of CFMs, PIFs, and an HEP calculation model.

The NRC defines the development of IDHEAS-ECA method as the following:

• Scope: the method should allow for the performance of event and condition assessments for NPP HRA applications. Specifically, it should be able to model operator actions outside control rooms under severe operating conditions, such as implementation of FLEX strategies.

• Requirements: The method should be easy to use and should not over-burden HRA analysts. It should allow HRA analysts to quickly explore What-If questions in an HRA.

• Data sources: IDHEAS-DATA, the data in NRCs SACADA database, and estimated HEPs in the NRC 2018 FLEX-HRA Expert Elicitation.

With the above definition, the following approach was made to develop IDHEAS-ECA method:

• Adapt the same guidance for the scenario, HFE, and task analysis as well as the guidance for time uncertainty analysis as those in IDHEAS-G.

• Use the five high-level CFMs (failure of D, U, DM, E, and T) to model failure of a critical task.

• Use all the 20 PIFs but with a consolidated subset of the attributes.

4-1

• Use the HEP calculation model in IDHEAS-G for analysts to directly calculate HEPs of CFMs for any selection of PIF attributes

• Integrate the available human error data to obtain the base HEPs and PIF weights needed in the HEP calculation model.

Table 4-1 summarizes the commonality and differences between IDHEAS-ECA and IDHEAS-G.

The left column shows the elements in IDHEAS-G. The right column highlights how the IDHEAS-G element is implemented in the eight-step process of IDHEAS-ECA.

Table 4-1 Summary of IDHEAS-ECA Development from IDHEAS-G IDHEAS-G IDHEAS-ECA Scenario analysis Step 1:

• Same guidance as in IDHEAS-G

• Specifications on guiding questions for identifying context HFE and task analysis Step 2:

• Same as in IDHEAS-G

• Specific guidance and options on estimating time distribution Modeling the failure of the critical Step 3:

tasks in an HFE - three levels of

• Use the five high-level cognitive failure modes, CFMs in progressively details i.e., failure of the macro-cognitive functions

• Specific guidance on assessing applicable CFMs Modeling context with PIFs - 20 Step 4:

PIFs each with a comprehensive

• All 20 PIFs preserved list of attributes

• A compressed set of PIF attributes based on human error data available (combining attributes with similar effects)

• Specific guidance on assessing PIF attributes HEP estimation - Several Steps 5 and 7 approaches to estimate HEPs

• Use the HEP calculation model in IDHEAS-G along with generalized human

• Have all the base HEPs and PIF attribute weights error data by integrating the generalized human error data Time uncertainty analysis Steps 6 and 7

• Same as in IDHEAS-G

• Specific guidance and options on estimating time distribution Uncertainty documentation and Step 8 sensitivity analysis

• Same as in IDHEAS-G with concise guidance 4-2

4.2. Integration of Human Error Data for IDHEAS-ECA IDHEAS-G generalized human error data from various sources into IDHEAS-DATA, which consists of three sets of human error data tables: HEP tables, PIF weight tables, and PIF interaction tables. The tables document human error data that are generalized into the IDHEAS-G taxonomy (the cognitive failure modes and PIF attributes). The generalized data are used to inform HEPs in various approaches to HEP estimation. In developing IDHEAS-ECA, the NRC staff integrated the available data as of July 2019 in the Human Error Tables to develop the base HEPs and PIF weights for every CFM and PIF attribute in IDHEAS-ECA. Because of the limited amount of data, the integration involves interpolation, reasoning, and engineering judgment. Appendix B of this report presents the integrated base HEPs and PIF weights.

There will be a separate NRC Research Information Letter (RIL) report documenting the basis of deriving every base HEP or PIF weight, and that report will include the source references from which the human error data are integrated. Below are some general strategies the NRC staff used in the integration:

1) Multiple data points for a base HEP or PIF weight The human error data are first evaluated for their uncertainties and practicality in the source documents. The NRC staff considered that the NPP operational data that were systematically collected for HRA had the highest practicality while cognitive experiments performed in research laboratories with students had the least practicality. The NRC staff used high practicality data to anchor a base HEP or PIF weight and used other data points to adjust the uncertainties in the high-practicality data points.

For the multiple data points that have about the same level of practicality and certainty, the NRC staff used the median of the data points as the base HEP or PIF weight.

2) Data points on the combined effects of several CFMs and/or PIF attributes When there were multiple data points with combined effects of two or three CFMs or PIF attributes, the NRC staff performed data fitting to get the best-fit base HEP or PIF weight. When there were only a few data points or a variety of CFMs and PIFs involved in the data points, the NRC staff combined the data points to estimate the range then used the middle of the range as the base HEP or PIF weight.

3) No data point for a PIF weight The available data in the IDHEAS-DATA do not have numeric human error information for many attributes in the PIFs such as Work Process or Teamwork and Organizational Factors. Yet, there have been studies demonstrating that those attributes impact human performance in measures other than human error rates, such as increasing personnel workload or reducing situational awareness. The NRC staff assigned the PIF weight as 1.1 or 1.2 for those attributes, pending for future updates as relevant human error data become available.

4) Consistency checking and adjustment with benchmark values After the initial base HEPs and PIF weights are developed, they are checked for internal consistency against the literature that ranks the likelihood of certain types of human errors and the contribution of various PIFs. The NRC staff also used reported rates of human events and estimated HEPs from the NRC 2018 FLEX HRA expert elicitation as benchmarks to check and adjust some base HEPs and PIF weights within their uncertainty ranges.

4-3

4.3. Future Development and Improvement This report, referred to as Rev. 0, presents the first version of the IDHEAS-ECA method. There are many areas for future development and improvement of the method, such as:

1) Continuous effort on use of human error data to inform HEPs The base HEPs and PIF weights in this report are the first version of integrating the data generalized using the IDHEAS-DATA. Because of the limited amount of data available, the NRC staff used interpolation, judgment, and benchmarking to develop the full set of base HEPs and PIF weights. In the long-term, generalizing human error data as new data become available should be a continuous effort, and there should be periodic integration and updates of the base HEPs and PIF weights based on the up-to-date available data in the Human Error Tables.

The NRC has the Scenario Authoring, Characterization, and Debriefing Application (SACADA) program collecting operator simulator training data. The SACADA program continuously generates operator performance data classified as satisfactory, unsatisfactory, or deviated from the training objectives. The frequency of unsatisfactory performance is considered as human errors. There are also operator simulator data collection programs going on in other organizations such as the Korea Atomic Energy Research Institute. Several research organizations such as the Organisation for Economic Co-operation and Development Halden Reactor Project have been conducting human performance experiments with nuclear reactor simulators. The experimental results provide human error data relevant to NPP operations. The NRC also has plans to reach out to other sources of human performance data. The NRC staff intends to continuously generalize accessible data and add it to the IDHEAS-DATA.

Even if there are multiple data points for a base HEP or PIF weight, judgment and reasoning are still needed in generalizing and integrating the human error data because of uncertainties and complications in the data sources. The data sources as well as the process and considerations in generating the base HEPs and PIF weights should be documented. The NRC staff will develop such documentation aside from this method report.

2) Probabilistic Distribution of base HEPs and PIF weights The base HEPs and PIF weights in this version are single point numbers. In reality, those numbers inherit the uncertainties and variability in the source data as well as additional uncertainties in the process of data generalization and integration. In the future, it is desirable to develop the probabilistic distribution of the base HEPs and PIF weights to represent their center, body, and range of the numbers.

3) Supplementary guidance and examples to inform assessment of PIF attributes To model scenario context with the IDHEAS-ECA PIFs, analysts need to assess the applicability of relevant attributes. This requires engineering judgment. Analyst-to-analyst variability arises from the uncertainties from the information available for analysts to make judgments as well as from their interpretation of the attributes. The definitions of the attributes in this version of the method are kept concise and general for broad applications. They should be periodically updated for clarification and precision.

Moreover, additional attributes may be needed to model new HRA applications or unusual events.

4-4

4) Assessment of recovery The HEP quantification model in IDHEAS-ECA allows analysts to assign a recovery factor to credit recovery of HFEs in the HEP. The method provides the criteria for crediting recovery and qualitative guidance on assessing the recovery factor. However, a caveat is that the method does not provide guidance on numeric values of recover factors that should be assigned to a critical task in a given scenario. It is left open to the analysts judgment.

The NRC staff did not provide reference numeric values for crediting recovery because the staff had not thoroughly studied this topic to build a solid technical basis for the likelihood of recovery. Some existing HRA methods provide numeric values for crediting recovery. For example, the Standardized Plant Analysis Risk-HRA (SPAR-H) method

[12]-[14] credits recovery based on time available. It assigns a recovery factor (multiplier) of 0.1 for Extra Time and a factor of 0.01 for Expansive Time. Yet, other recovery mechanisms are not weighted in these recovery factors. In the IDHEAS At-Power method (NUREG-2199 [6]), the failure modes for which recovery is feasible are associated with numeric values of the recovery factor for different combinations of applicable PIF attributes. Those numbers were estimated through formal expert judgment. The estimated recovery factors range from 2 to 20. However, those numbers were estimated specifically for the context that licensed NPP crews perform well trained procedures in control rooms in internal, at-power events. The Cause-Based Decision Tree method [15] also provides numeric values for crediting recovery while the technical basis and application scope of those numbers are unclear. Therefore, we consider that it is premature to provide numeric recovery factors for a method that is intended for a broad range of HRA applications.

The recommendation in the current version of IDHEAS-ECA method is for analysts to assign a number to the recovery factor and document the basis and justification for recovery feasibility and the assigned number. It is desirable for the method to provide quantitative guidance on assessing the likelihood of recovery in the future.

5) Dependency The current version of the IDHEAS-ECA method does not include HFE dependency analysis. This is another area requiring further study. Conventional dependency models in existing HRA methods were originated from the Technique for Human Error Rate Prediction (THERP) method [16]. The dependency model in THERP was intended to compensate for the limited PIFs modeled in the method. Several modifications in other HRA methods such as SPAR-H or the Fire HRA guidelines (NUREG-1921 [5]) have been developed to improve the original THERP dependency model. In IDHEAS-ECA, the PIFs and PIFs attributes are comprehensive and detailed and the effect of dependency can be largely represented by adjusting the applicable PIF attributes.

Moreover, THERP quantifies HEPs of individual procedure steps that inherit similar context, while IDHEAS-ECA quantifies HEPs of critical tasks which are at a higher level than the activity steps and are deemed different in the context. Thus, it is likely that the conventional dependency models are not applicable to IDHEAS-ECA. This is an area to be resolved in near future.

6) Testing and validating the method The authors of this report applied the method to three scenarios to demonstrate how the method works. The analysis results are documented in Appendix C. In 2019, the NRC 4-5

held a workshop in which six HRA analysts used the IDHEAS-ECA software to calculate HEPs of the HFEs in implementing FLEX strategies. The analysts were not required to fill out the IDHEAS-ECA Worksheets and they directly started from the software. Thus, the analysts essentially performed Steps 4, 5, 6, and 7 of the IDHEAS-ECA process without performing Steps 1, 2, and 3.

The NRC staff intends to test the IDHEAS-ECA method by implementing it on a trial basis in regulatory applications. The staff further intends to document the examples and lessons learned from its trial use to further improve the method and its guidance.

4.4. Concluding Remarks Overall, IDHEAS-ECA is developed as a complete, off-the-shelf HRA method. IDHEAS-ECA is used to analyze human events and estimate HEPs in PRA applications. IDHEAS-ECA builds upon existing HRA methods by providing a systematic process and guidelines to analyze and model human actions and the associated scenario context. Further, it uses a human error database to calculate HEPs and includes an extensive set of PIFs to represent the context of scenarios under various operational conditions, such as using FLEX equipment.

IDHEAS-ECA is envisioned to be used by the staff involved in the NRCs PRA applications.

The intent is for the method to be applicable to the same situations that existing HRA methods model (e.g., nuclear power plant internal events while at-power) and beyond (e.g., external events, low power and shutdown events, spent fuel storage and transportation, and events where FLEX equipment are used). Given the wide range of contextual factors included in its model, it is feasible that IDHEAS-ECA could also be used for applications beyond the nuclear domain.

4-6

5 REFERENCES

[1] J. Xing, Y. J. Chang, and J. DeJesus, The General Methodology of an Integrated Human Event Analysis System (IDHEAS-G) Draft Report, U.S. Nuclear Regulatory Commission, NUREG-2198 (ADAMS Accession No. ML19235A161), Aug. 2019.

[2] U.S. Nuclear Regulatory Commission, Staff Requirements - Meeting with Advisory Committee on Reactor Safeguards, 2:30 p.m., Friday, October 20, 2006, Commissioners Conference Room, One White Flint North, Rockville, Maryland (Open to Public Attendance), U.S. Nuclear Regulatory Commission, SRM M061020 (ADAMS Accession No. ML063120582), Nov. 2006.

[3] A. M. Whaley, J. Xing, R. L. Boring, S. M. L. Hendrickson, J. C. Joe, K. L. Le Blanc, and S. L. Morrow, Cognitive Basis for Human Reliability Analysis, U.S. Nuclear Regulatory Commission, NUREG-2114 (ADAMS Accession No. ML16014A045), Jan. 2016.

[4] M. Barriere, D. Bley, S. Cooper, J. Forester, A. Kolaczkowski, W. Luckas, G. Parry, A.

Ramey-Smith, C. Thompson, D. Whitehead, and J. Wreathall, Technical Basis and Implementation Guidelines for A Technique for Human Event Analysis (ATHEANA),

U.S. Nuclear Regulatory Commission, NUREG-1624, Rev. 1 (ADAMS Package No.

ML003736288), May 2000.

[5] S. Lewis, S. Cooper, K. Hill, J. Julius, J. Grobbelaar, K. Kohlhepp, J. Forester, S.

Hendrickson, E. Collins, B. Hannaman, and M. Presley, EPRI/NRC-RES Fire Human Reliability Analysis Guidelines, Electric Power Research Institute and U.S. Nuclear Regulatory Commission, EPRI 1023001/NUREG-1921 (ADAMS Accession No. ML12216A104), Jul. 2012.

[6] J. Xing, G. Parry, M. Presley, J. Forester, S. Hendrickson, and V. Dang, An Integrated Human Event Analysis System (IDHEAS) for Nuclear Power Plant Internal Events At-Power Application, U.S. Nuclear Regulatory Commission and Electric Power Research Institute, NUREG-2199, Vol. 1 (ADAMS Accession No. ML17073A041), Mar. 2017.

[7] D. C. Bley, D. R. Buttemer, and J. W. Stetkar, Light water reactor sequence timing: its significance to probabilistic safety assessment modeling, Reliab. Eng. Syst. Saf., vol.

22, no. 1-4, pp. 27-60, Jan. 1988.

[8] M. Modarres, M. P. Kaminskiy, and V. Krivtsov, Reliability Engineering and Risk Analysis: A Practical Guide, 3rd ed. Boca Raton, FL: CRC Press, 2017.

[9] OpenBUGS, Version 3.2.3 rev 1012 (www.openbugs.net). 2014.

[10] M. Drouin, A. Gilbertson, G. Parry, J. Lehner, G. Martinez-Guridi, J. LaChance, and T.

Wheeler, Guidance on the Treatment of Uncertainties Associated with PRAs in Risk-Informed Decisionmaking, U.S. Nuclear Regulatory Commission, NUREG-1855, Rev. 1 (ADAMS Accession No. ML17062A466), Mar. 2017.

[11] A. Kolaczkowski, J. Forester, E. Lois, and S. Cooper, Good Practices for Implementing Human Reliability Analysis, U.S. Nuclear Regulatory Commission, NUREG-1792 (ADAMS Accession No. ML051160213), Apr. 2005.

[12] D. Gertman, H. Blackman, J. Marble, J. Byers, and C. Smith, The SPAR-H Human Reliability Analysis Method, U.S. Nuclear Regulatory Commission, NUREG/CR-6883, Aug. 2005.

5-1

[13] R. L. Boring and H. S. Blackman, The Origins of the SPAR-H Methods Performance Shaping Factor Multipliers, in 2007 IEEE 8th Human Factors and Power Plants and HPRCT 13th Annual Meeting, Monterey, CA, USA, 2007, pp. 177-184.

[14] A. M. Whaley, D. L. Kelly, R. L. Boring, and W. J. Galyean, SPAR-H Step-by-Step Guidance, Idaho National Laboratory, INL/EXT-10-18533, Rev. 2 (ADAMS Accession No. ML112060305), May 2011.

[15] G. W. Parry, A. J. Spurgin, P. Moieni, and A. Beare, An Approach to the Analysis of Operator Actions in Probabilistic Risk Assessment, Electric Power Research Institute, TR-100259, Jun. 1992.

[16] A. D. Swain and H. E. Guttmann, Handbook of Human Reliability Analysis with Emphasis on Nuclear Power Plant Applications, U.S. Nuclear Regulatory Commission, NUREG/CR-1278 (ADAMS Accession No. ML071210299), Aug. 1983.

[17] J. Forester, H. Liao, V. N. Dang, A. Bye, E. Lois, M. Presley, J. Marble, R. Nowell, H.

Broberg, M. Hildebrandt, B. Hallbert, and T. Morgan, The U.S. HRA Empirical Study -

Assessment of HRA Method Predictions against Operating Crew Performance on a U.S.

Nuclear Power Plant Simulator, U.S. Nuclear Regulatory Commission, NUREG-2156 (ADAMS Accession No. ML16179A124), Jun. 2016.

5-2

Appendix A IDHEAS-ECA WORKSHEETS Worksheet A. Scenario analysis Event title Scenario description This section documents the information collected by HRA analysts or what was documented in the PRA model about the event.

Section A.1 - Operational narrative This section includes the initiating event, initial conditions, boundary conditions, and event timeline. It may also include important scenario deviations and past operational experience review.

Initiating event:

Initiating conditions:

Boundary conditions:

Event timeline - Document important system states and required human actions along the timeline of event progression. The event timeline is estimated based on expected event progression and the time that important system activities (S) and expected human (H) actions occur. The time was estimated in minutes after the event started.

Time System activity (S) or Human action (S) 0:00 Event begins Section A.2. Human Failure Event (HFE) identification List of all the important human actions, including the HFEs in the PRA model and additional important human actions that should be analyzed.

List of HFEs:

Section A.3. Scenario context This section analyzes and documents the conditions that challenge or facilitate human performance. The documentation includes four categories of context relating to environment, system, personnel, and tasks.

Environment and situation context:

System context:

Personnel context:

Task context:

Section A.4 Initial assessment of PIFs Based on the assumed boundary conditions and context analysis, list the PIFs that need or do not need further analysis.

The following PIFs should have no impact on the HFE:

A-1

The following PIFs may impact crew performance of specific tasks, thus they should be further analyzed for individual CFMs of critical tasks:

Worksheet B. HFE Analysis HFE1: (name)

Section B.1 HFE definition This section defines the HFE at the high level with basic information relevant to human performance of the action.

Success criterion:

Starting and ending point:

Procedures / Instructions:

Key indications:

Special equipment / tools:

Time available / urgency:

Specifications:

Section B.2 Task diagram and identification of critical tasks An important human action may consist of multiple discrete tasks. Task diagram is to elucidate the tasks needed to achieve the action, the paths of the tasks, and interteam coordination needed to achieve the tasks. Critical tasks are those required to meet the success criterion of the action; failing any of the critical tasks would fail the action.

Task diagram and/or timeline:

List of critical tasks:

HFE1-T1:

HFE1-T2:

HFE1-T3:

A-2

Worksheet C. Analysis of critical tasks in an HFE Critical task HFE1-T1: Task name - Brief description of task goal Section C.1. Analysis of cognitive activities and identification of applicable CFMs The analysis of cognitive activities is based on the macrocognitive functions.

Cognitive activities Detection (Respond to alarms, Get information, Monitor parameters or status) -

Understanding (Assess situation, Diagnose problems, Make predictions) -

Decisionmaking -

Action Execution -

Interteam coordination -

List of applicable CFMs:

Section C.2 Task characterization Special requirements -

Cue -

Personnel -

Procedure -

Competing goals and alternative strategies -

Multitasking -

Additional task characteristics -

Section C.3 Assessment of PIFs Assessment of PIFs is to select the applicable PIF attributes from the PIF tables in Appendix B. Assessment of PIFs is performed only for the PIFs relevant to the event as determined in Worksheet A Section A.4.

HFE1-T1-CFM1 HFE1-T1-CFM2 HFE1-T1-CFM3 HFE1-T1-CFM4 HFE1-T1-CFM5 A-3

Worksheet D - HEP Calculation for Pc Worksheet D documents the HFEs, critical tasks, CFMs, and applicable PIF attributes for HEP estimation. The HEP of a CFM can be calculated using the base HEPs and PIF weights in Appendix B.

HFE1:

Critical Task Applicable Applicable PIF attribute CFMs HFE1-T1: T1-CFM1:

Failure of Detection T1-CFM2:

T1-CFM3:

T1-CFM4:

T1-CFM5:

HFE1-T2:

A-4

Worksheet E: Time uncertainty analysis of the HFEs Worksheet E documents the estimation of time available ( ) and time needed ( ) for every HFE to calculate . The distribution of the time can be estimated as a single number (for only), the mean and standard deviation (SD) by assuming a normal distribution, or a five-point estimation of probability distribution (at 5th, 25th, 50th, 75th, and 95th percentile) or Gamma distribution or Weibull distribution.

HFE Estimation of Time available and Estimation of Time needed and basis/justification basis/justification HFE1 HFE2 HFE3 A-5

Appendix B BASE HUMAN ERROR PROBABILITIES AND PERFORMANCE-INFLUENCING FACTOR WEIGHTS Appendix B presents the base human error probabilities (HEPs) of the three base performance influencing factors (PIFs) in Tables B-1 through B-3, and it presents the PIF weights for the rest of the PIFs in Table B-5 through Table B-15. Each table is for one PIF except that Table B-4 contains PIF weights for several PIFs in the environment PIF category, and Table B-14 contains PIF weights for two PIFs, Mental Fatigue and Time Pressure / Stress.

Each row in a table is for one attribute, with the first row for the No impact state of a PIF. The first column in a table is an identifier assigned for a PIF attribute. For example, the attributes for PIF Scenario Familiarity have the identifiers SF1, SF2, SF3, while SF0 is the identifier for No impact, the base state of the PIF. The second column is the description of every PIF attribute.

The remaining five columns contain the base HEP of a cognitive failure mode (CFM) or the PIF weight on the CFM imposed by the PIF attribute of the row. These five columns are for failure of Detection (D), Understanding (U), Decisionmaking (DM), Action Execution (E), and Interteam coordination (T). One exception is Table B-3 in which the base HEPs are separately presented for each CFM.

The base HEPs for the No impact states of the base PIFs in Tables B-1, B-2, and B-3 (i.e.,

SF0, Inf0, C0, C10, C20, C30, and C40) are shown as zero. However, in the case that the three base PIFs are in their No impact state, (see Equation (3.5)) is not zero and should be assigned a value of the lowest HEP of a CFM, which is 1x10-4 for failure of Detection or Action Execution, and 1x10-3 for failure of Understanding, Decisionmaking, or Interteam coordination.

Table B-1 Base HEP for Scenario Familiarity PIF Attribute D U DM E T SF0 No-impact 0 0 0 0 0

• frequently performed tasks in well-trained scenarios,

• routine tasks SF1 Unpredictable dynamics in 6.6E-4 6.6E-3 6.6E-3 6.6E-4 NA known scenarios

• Shifting task objectives,

• Dynamic decisionmaking is required SF2 Unfamiliar elements in the 5E-3 5E-2 5E-2 5E-3 NA scenario

• non-routine, infrequently performed tasks,

• unlearn a technique and apply one that requires the application of an opposing philosophy B-1

Table B-1 Base HEP for Scenario Familiarity (continued)

PIF Attribute D U DM E T SF3 Scenarios trained on but E-3 E-2 E-2 E-3 NA infrequently performed Scenario is unfamiliar, 1.2E-2 E-1 E-1 3.3E-2 NA rarely performed

• notice adverse indicators that are not part of the task at hands

• notice incorrect status that is not a part of the routine tasks Extremely rarely performed 3.3E-2 3E-1 3E-1 3.5E-1 NA

• Lack of plans, policies and procedures to address the situation

• No existing mental model for the situation

• Rare events such as the Fukushima accident SF4 Bias or preference for NA 2.6E-2 2.6E-2 NA NA wrong strategies exists, mismatched mental models B-2

Table B-2 Base HEP for Information Availability and Reliability PIF Attribute D U DM E T Inf0 No impact - Key 0 0 0 0 0 information is reliable and complete Inf1 Information is temporarily NA 5E-3 5E-3 NA NA incomplete or not readily available Inadequate updates of information

• Feedback information is not available in time to correct a wrong decision or adjust the strategy implementation

• Different sources of information are not well organized thus personnel cannot readily access all the information needed

• Primary source of information is not available and secondary source of the information is in lower resolution Information is moderately NA 5E-2 5E-2 NA NA incomplete - a small portion of key information is missing Information is largely NA 2E-1 2E-1 NA NA incomplete

• Key information is masked

• Key indication is missing B-3

Table B-2 Base HEP for Information Availability and Reliability (continued)

PIF Attribute D U DM E T Inf2 Low unreliable or uncertain NA E-2 E-2 NA NA

• Personnel is aware that source of information could be temporally unreliable

• Pieces of Information change over time thus they become uncertain by the time personnel use them Moderately unreliable or NA 5E-2 5E-2 NA NA uncertain

• Source of information could be unreliable and personnel likely recognize this

• Conflicts in key information Highly unreliable NA E-1 E-1 NA NA

• Key information is highly uncertain Extremely unreliable NA 3E-1 3E-1 NA NA

• Key information is misleading

• Key information is inaccurate B-4

Table B-3 Base HEPs for Task complexity PIF Attribute Detection C0 No impact on HEP 0 C1 Detection overload with multiple competing signals Few (<7) 3E-3

- track the states of multiple systems, Multiple (7~11)

- monitor many parameters, 1E-2

- memorize many pieces of information detected

- Many types or categories of information to be Many (11~20) detected 1E-1 Excessive amount (>20) 3E-1 C2 Detection is moderately complex E-3

- Criteria are not straightforward,

- Information of interest involves complicated mental computation

- Comparing for abnormality C3 Detection demands for high attention E-3

- Need split attention

- Need sustained attention over a period of time

- Need intermittent attention C4 Detection criteria are highly complex E-2

- multiple criteria to be met in complex logic,

- Information of interest must be determined based on other pieces of information

- Detection criteria are ambiguous and need subjective judgment C5 Cues for detection are not obvious 5E-2

- detection is not directly cued by alarms or instructions and personnel need to actively search for the information C6 No cue or mental model for detection E-1

- no rules / procedures / alarms to cue the detection; Detection of the critical information is entirely based on personnels experience and knowledge B-5

Table B-3 Base HEPs for Task Complexity (continued)

PIF Attribute Understanding C10 No impact - straightforward diagnosis 0 with clear procedures or rules C11 Working memory overload E-2 for <11 messages

- need to decipher numerous messages (indications, alarms, 5E-2 for 11~15 spoken messages)

E-1 for 15-20

- Multiple causes for situation assessment: Multiple independent 3E-1 for > 20 influences affect the system and system behavior cannot be explained by a single influence alone C12 Relational complexity (Number of 2E-2 for 2 relations unchunkable topics or relations in one understanding task) 4.5E-2 for 3 relations

- Relations involved in a human E-1 for 4 relations action are very complicated for 3E-1 for more than 4 understanding relations

- Need to integrate (use together) multiple relations C13 Understanding complexity - Requiring E-2 high level of comprehension C14 Potential outcome of situation E-2 assessment consists of multiple states and contexts (not a simple yes or no)

C15 Ambiguity associated with assessing E-1 the situation

- Key information for understanding is cognitively masked

- Pieces of key information are intermingled or coupled C16 Conflicting information, cues, or E-1 symptoms B-6

Table B-3 Base HEPs for Task Complexity (continued)

PIF Attributes Decisionmaking C20 No impact - simple, straightforward choice 0 C21 Transfer step in procedure -integrating a few cues 4.5E-3 C22 Transfer procedure (Multiple alternative strategies 1.2E-2 to choose) - integrating multiple cues C23 Decision criteria are intermingled, ambiguous, or 1E-2 difficult to assess C24 Multiple goals difficult to prioritize, e.g., advantage 3.3E-2 for incorrect strategies C25 Competing or conflicting goals (e.g., choosing one 1.4E-1 goal will block achieving another goal, Low preference for correct strategy, Reluctance & Viable Alternative)

C26 Decision-making involves developing strategies or 5E-2 action plans C27 Decisionmaking requires diverse expertise 1E-1 distributed among multiple individuals or parties who may not share the same information or have the same understanding of the situation C28 integrating a large variety of types of cues with 1.7E-1 complex logic B-7

Table B-3 Base HEPs for Task Complexity (continued)

Action PIF Attributes Execution C30 No impact - Simple execution with a few steps 0 C31 Straightforward Procedure execution with many steps E-3 C32 Non-straightforward Procedure execution 5E-3

- Very long procedures, voluminous documents with checkoff provision

- Multiple procedures needed C33 Simple continuous control that requires monitoring 3.4E-4 parameters C34 Continuous control that requires manipulating dynamically 2.6E-3 C35 Long-lasting action, repeated discontinuous manual control 2E-2 (need to monitor parameters from time to time)

C36 No immediacy to initiate execution - time span between 5E-3 annunciation (decision for execution made) and operation C37 Complicated or ambiguous execution criteria E-2

- Multiple, coupled criteria

- Restrictive, irreversible order of multiple steps

- Open to misinterpret C38 Action execution requires close coordination of multiple 5E-2 personnel at different locations - transport fuel assemblies with fuel machines C39 Unlearn or break away from automaticity of trained action 1E-1 scripts B-8

Table B-3 Base HEPs for Task Complexity (continued)

Interteam PIF Attributes coordination C40 No impact - Clear, streamlined, crew-like communication and 0 coordination C41 Complexity of information communicated -

simple (e.g., notifying / requesting to ex-MCR) - 1.5E-3 Moderate - E-2 High - 5E-2 Extremely high - E-1 C42 Complex or ambiguous command-and-control E-2 C43 Complex or ambiguous authorization chain E-2 C44 Coordinate activities of multiple diverse teams or E-2 organizations B-9

Table B-4 PIF Weights for Environmental PIFs PIF Attribute D U DM E T ENV0 No impact - nominal weather and 1 1 1 1 1 environmental factors ENV1 Coldness on action execution NA NA NA 1.5 NA Moderate cold (<5°C) - 1.5 2 Extreme coldness on manipulating 5 instrumentation - 2 20 Extreme coldness on physically demanding execution -5 Extreme coldness on high precision manipulations (e.g., connecting lines to pump, remove air from lines and pumps) - 20 ENV2 Moderate coldness (<5°C) for non- 1.1 1.1 1.1 NA 1.1 execution ENV2 Extreme coldness for non-execution 2 2 1.1 NA 2 ENV3 Heat (>33°C) or high humidity 1.1 1.1 1.1 1.5 1.1 ENV4 Poor lighting, low luminance (L=0.15, 2 NA NA 2 NA compared to no impact L=1.5) for reading information or execution ENV5 Strong ambient light, glare, reflection 2 NA NA 1.5 NA ENV6 Very low visibility (e.g., heavy smoke 5 NA NA 5 NA or fog) for detecting targets or execution ENV7 Loud or burst noise 1.7 1.1 1.1 1.1 1.1 ENV8 Wearing heavy protective clothes NA NA NA 1.5 NA and/or gloves ENV9 Slippery surface (e.g., icing) NA NA NA 1.5 NA ENV10 Strong winds, rain, or objects close to NA NA NA 1.5 NA road on physically demanding tasks ENV11 Strong winds, rain, or objects close to NA NA NA 2 NA road impeding vehicle movement ENV12 High or chaotic traffic impeding NA NA NA 1.5 NA vehicle movement ENV13 Unstable or vibrating surface or work NA NA NA 2 NA site B-10

Table B-5 PIF Weights for System and I&C Transparency PIF Attribute D U DM E T SIC0 No impact 1 1 1 1 NA SIC1 System or I&C does not behave as 1.1 1.1 1.1 1.1 NA intended under special conditions SIC2 System or I&C does not reset as intended 1.1 1.1 1.1 10 NA SIC3 System or I&C is complex or non- NA 2 NA NA NA transparent for personnel to predict its behavior SIC4 System or I&C failure modes are not NA 2 NA NA NA transparent to personnel Table B-6 PIF Weights for Human-System Interface PIF Attribute D U DM E T HSI0 No impact - well designed HSI supporting 1 1 1 1 1 the task HSI1 Indicator is similar to other sources of 1.5 NA NA NA NA information nearby HSI2 No sign or indication of technical difference 3 NA NA NA NA from adjacent sources (meters, indicators)

HSI3 Related information for a task is spatially 1.5 2 NA NA NA distributed, not organized, or cannot be accessed at the same time HSI4 Un-intuitive or un-conventionnel indications 2 NA NA NA NA HSI5 Poor salience of the target (indicators, 3 NA NA NA NA alarms, alerts) out of the crowded background HSI6 Inconsistant formats, units, symbols, or 5 NA NA NA NA tables HSI7 Inconsistent interpretation of displays NA 5.7 NA NA NA HSI8 Similarity in elements - Wrong element NA NA NA 1.2 NA selected in operating a control element on a panel within reach and similar in design in control room HSI9 Poor functional localization - 2~5 displays / NA NA NA 2 NA panels needed to execute a task B-11

Table B-6 PIF Weights for Human-System Interface (continued)

PIF Attribute D U DM E T HSI10 Ergonomic deficits NA NA NA 3.38 NA

- Controls are difficult to maneuver

- Labeling and signs of controls are not salient among crowd

- Inadequate indications of states of controls - Small unclear labels, difficult reading scales

- Maneuvers of controls are un-intuitive or unconventional HSI11 Labels of the controls do not agree with NA NA NA 5 NA document nomenclature, confusing labels HSI12 Controls do not have labels or indications NA NA NA 10 NA HSI13 Controls provide inadequate or ambiguous NA NA NA 4.5 NA feedback, i.e., lack of or inadequate confirmation of the action executed (incorrect, no information provided, measurement inaccuracies, delays)

HSI14 Confusion in action maneuver states (e.g., NA NA NA 10 NA automatic resetting without clear indication)

HSI15 Unclear functional allocation (between NA NA NA 9 NA human and automation)

B-12

Table B-7 PIF Weights for Equipment and Tools PIF Attribute D U DM E T TP0 No impact - tools & parts are well 1 1 1 1 1 maintained under proper administrative control TP1 Tools / parts are complex or 1.1 NA NA 1.1 NA difficult to use TP2 Failure modes or operational 1.1 NA NA 1.1 NA conditions of the tools are not clearly presented (e.g., ranges, limitations, and requirements)

TP3 Tool does not work properly due to 1.1 NA NA 1.1 NA aging, lack of power, incompatibility, improper calibration etc.)

TP4 Document nomenclature does not 2 NA NA 2 NA agree with equipment labels TP5 Personnel are unfamiliar or rarely 2 NA NA 2 NA use the tool / parts TP6 Tools or parts lack of proper 2 NA NA 2 NA administrative control (so could be missing or temporally not available)

B-13

Table B-8 PIF Weights for Staffing PIF Attribute D U DM E T STA0 No impact - adequate staffing 1 1 1 1 1 STA1 Shortage of staffing (e.g., key 1.1 1.1 1.1 1.1 1.1 personnel are missing, unavailable or delayed in arrival, staff pulled away to perform other duties)

STA2 Lack of backup/lack of peer check 1.1 1.1 1.1 1.1 1.1 or cross-checking (e.g., an overseer or independent reviewer is not available)

STA3 Ambiguous or incorrect 1.1 1.1 1.1 1.1 1.1 specification of staff roles and responsibilities STA4 Inappropriate staff assignment 1.1 1.1 1.1 1.1 1.1 (e.g., lack of skills)

STA5 Key decision makers knowledge 1.1 1.1 1.1 1.1 1.1 and ability are inadequate to make the decision (e.g., lack of required qualifications or experience)

STA6 Lack of administrative control on 1.1 1.1 1.1 1.1 1.1 fitness-for-duty B-14

Table B-9 PIF Weights for Procedures, Guidance, and Instructions PIF Attribute D U DM E T PG0 No impact - well validated 1 1 1 1 1 procedures like most EOPs PG1 Procedure design is less than 1.2 1.1 1.1 1.2 1.1 adequate (difficult to use)

- Requires calculation (e.g., unit conversion)

- No placeholders

- Graphics or symbols not intuitive

- Inconsistency between procedure and displays PG2 Procedure requires judgment 1.6 1.6 1.6 3 1.1 PG3 Procedure lacks details 2.2 2.2 2.2 2.2 1.1 PG4 Procedure is ambiguous, 1.5 5 5 3 5 confusing PG5 Mismatch - Procedure is available 1.1 17 17 1.1 10 but does not match the situation (e.g., needs deviation or adaptation)

PG6 No verification in procedure for 20 NA NA 20 10 verifying key parameters for detection or execution PG7 No guidance to seek confirmatory NA 30 30 NA 10 data when data may mislead for diagnosis or decisionmaking B-15

Table B-10 PIF Weights for Training PIF Attribute D U DM E T TE0 No impact - professional 1 1 1 1 1 staff have adequate training required TE1 Inadequate training Frequent Frequent Frequent Frequent Frequent frequency / refreshment (<6 (<6 (<6 (<6 (<6 months) - months) - months) - months) - months) -

1 1 1 1 1 Infrequent Infrequent Infrequent Infrequent Infrequent (6-12m) - (6-12m) - (6-12m) - (6-12m) - (6-12m) -

1.2 1.2 1.2 1.2 1.2 Highly Highly Highly Highly Highly infrequent infrequent infrequent infrequent infrequent

(> 4years) (> (> (> (>

-5 4years) - 4years) - 4years) - 4years) -5 10 10 10 TE2 Inadequate training 1.5 1.5 1.5 1.5 1.5 practicality - no hands-on training

• Not drilled together

• Training on parts, not whole scenario together TE3 Inadequate training on 1.1 2 2 2 NA procedure adaptation:

Training focuses on procedure-following without adequately training personnel to seek alternative interpretations, evaluate the pros and cons of alternatives, and adapt the procedure for the situation TE4 Inadequate amount of 1.8 3 3 6.1 NA training - no qualification exam

• Less than adequate training specification /

requirement B-16

Table B-10 PIF Weights for Training (continued)

PIF Attribute D U DM E T TE5 Operator inexperienced 3 3 3 3 NA (e.g., a newly qualified tradesman, but not an expert)

TE6 Poor administrate control 2 2 10 10 NA on training (e.g., not included in the Systematic Approach of Training Program)

TE7 Inadequate training or 14 NA NA NA NA experience with sources of information (such as applicability and limitations of data or the failure modes of the information sources)

TE8 Inadequate specificity on 20 NA NA NA NA urgency and the criticality of key information such as key alarms TE9 Not trained to seek NA 10 10 NA NA confirmatory information when dismissing critical data TE10 Premature Termination of NA 15 NA NA NA Critical Data Collection in diagnosis due to inadequate training on system failure modes TE11 Poor training on NA NA 5 NA NA assessing action margin in deciding implementation delay TE12 Poor training on NA NA 11 NA NA interpreting procedure in the context of the scenario for decisionmaking TE13 Poor training on the NA NA NA 10 NA importance of data in frequently checking data for execution B-17

Table B-11 PIF Weights for Teamwork and Organizational Factors PIF Attribute D U DM E T TF0 No impact - adequate, crew-like 1 1 1 1 1 teams TF1 Inadequate team 2 2 2 2 2

• Inadequate teamwork resources (short of personnel, knowledge gaps)

• Distributed, or dynamic teams

• Poor team cohesion ((e.g.,

newly formed teams, lack of drills / experience together)

TF2 Poor command & control 1.5 1.5 1.5 1.5 1.5

• Unclear allocation of functions and responsibilities

• Inadequate coordination between site personnel and decision-makers (e.g., adapt or modify planned actions based on site situation)

• Inadequately verify the plan with decision-makers

• Inadequate supervision in overseeing action execution and questioning current mission TF3 Poor information management in NA NA NA NA 2 multiple-team tasks TF4 Poor communication capabilities NA NA NA NA 2 between teams TF5 Competing resources available for NA NA NA NA 1.5 multiple teams B-18

Table B-12 PIF Weights for Work Processes PIF Attribute D U DM E T No impact - licensed personnel 1 1 1 1 1 WP0 with good work practices Lack of practice of self- or cross- 10 1.1 1.1 10 1.1 WP1 verification (e.g., 3-way communication)

Lack of or ineffective peer- 10 1.1 1.1 10 1.1 WP2 checking / supervision WP3 Poor work prioritization, scheduling 1.1 1.1 1.1 1.1 1.1 WP4 Lack of or ineffective 1.1 1.1 1.1 1.1 1.1 instrumentation (e.g., pre-job briefing) for personnel to be aware of potential pitfalls in performing the tasks WP5 Lack of or ineffective 1.1 1.1 1.1 1.1 1.1 instrumentation (e.g., supervision) for safety issue monitoring and identification WP6 Lack of or ineffective 1.1 1.1 1.1 1.1 1.1 instrumentation for safety reporting WP7 Hostile work environment 1.1 1.1 1.1 1.1 1.1 B-19

Table B-13 PIF Weights for Multitasking, Interruption, and Distraction PIF Attribute D U DM E T MT0 No impact 1 1 1 1 1 MT1 Distraction by other on- Weak - 1.2 1.1 1.1 Weak - 1.2 Weak -

going activities that 1.2 demand attention Moderate - 2 Moderate -

2 Moderat High - 2.8 e-2 High - 2.8 High -

2.8 MT2 Interruption taking away Weak - 1.1 Weak - Weak Weak - 1.1 Weak -

from the main task 1.1 - 1.1 1.1 Moderate - Moderate -

2.8 Moderate Moder 2.8 Moderat

- 1.5 ate - e - 2.8 Frequent or 1.5 Frequent long - 4 Frequent or long - 4 Frequent or long- Frequ or long -

1.7 ent or 4 long-1.7 MT3 Concurrent visual Low NA NA NA NA detection and other tasks demanding -

2 Moderate demanding -

5 High demanding -

10 MT4 Concurrent auditory Auditory / NA NA NA NA detection and other tasks visual -10 Auditory /

auditory -

20 MT5 Concurrent diagnosis and NA Low NA NA NA other tasks demandin g-3 High demandin g - 30 B-20

Table B-13 PIF Weights for Multitasking, Interruption, and Distraction (continued)

PIF Attribute D U DM E T MT6 Concurrent Go/No-go NA NA 2 NA NA decision-making MT7 Concurrently making NA NA 5 NA NA intermingled complex decisions / plans MT8 Concurrently executing NA NA NA 2.3 NA action sequence and performing another attention/working memory task MT9 Concurrently executing NA NA NA 5 NA intermingled or inter-dependent action plans MT10 Concurrently NA NA NA NA 5 communicating or coordinating multiple distributed individuals or teams B-21

Table B-14 PIF Weights for Mental Fatigue and Time Pressure and Stress PIF Attribute D U DM E T MF0 No impact 1 1 1 1 1 MF1 Sustained (>30mins) high- 2.5 1.1 1.1 2.5 1.1 demanding cognitive activities requiring continuous attention (e.g.,

procedure-situation mismatches demand constant problem-solving and decisionmaking; information changes over time and requires sustained attention to monitor or frequent checking.)

MF2 Time pressure due to perceived 2 2 1.1 3 1.1 time urgency Lack of self-verification due to 10 2 2 10 2 MF3 needs to rush the task completion (speed-accuracy trade-off)

MF4 Reluctance to execute an action NA NA NA 2 NA plan due to potential negative impacts (e.g., adverse economic impact, or personal injury)

MF5 Long working hours (greater than 1.5 1.5 1.1 1.5 1.1 4hrs) with high cognitively demanding tasks Sudden increase in workload from a 1.2 1.2 NA 1.2 1.2 MF6 long period of low to high MF7 Sudden decrease in workload from 1.8 1.1 NA 1.8 1.2 high to normal MF8 Emotional stress (e.g., anxiety, 1.2 1.2 1.2 1.2 1.2 frustration)

Physical stress or fatigue (e.g., long 1.1 1.1 1.1 1.1 1.1 hours1.157407e-5 days <br />2.777778e-4 hours <br />1.653439e-6 weeks <br />3.805e-7 months <br /> exposure to ambient noise, MF9 disturbed dark and light rhythms, air pollution, disruption of normal work-sleep cycles, ill health)

MF10 Sleep deprivation 2 1.2 1.1 2 1.2 B-22

Table B-15 PIF Weights for Physical Demands PIF Attribute D U DM E T PD0 No impact 1 PD1 Physically strenuous - possibly exceeding physical NA NA NA 1.5 NA limits (e.g., lifting heavy objects, moving heavy things, opening / closing rusted or stuck valves)

PD2 High spatial or temporal precision NA NA NA 2 NA PD3 Precise motor coordination of multiple persons NA NA NA 2 NA PD4 Unusual, unevenly balanced loads (e.g., reaching high NA NA NA 5 NA parts)

PD5 Loading or unloading objects using crane/hoist NA NA NA 10 NA B-23

Appendix C EXAMPLES Appendix C provides three full examples to demonstrate performing an HRA with IDHEAS-ECA and documenting the results in the Worksheets. The event scenarios of the first two examples are from the US HRA Empirical Study documented in NUREG-2156 [17]. The study evaluated the performance of different HRA methods by comparing method predictions to actual crew performance in three simulated accident scenarios conducted in a U.S. nuclear power plant (NPP) simulator. The analyst teams were given the information package about the scenarios, observed NPP crews performing similar scenarios with the simulator, interviewed the operators, and then performed HRA with different methods. We used the same information package and the crew performance information documented in the report to perform HRA with IDHEAS-ECA.

The event scenario in the third example was used in the NRC 2018 FLEX HRA Expert Elicitation. The scenario describes a beyond-design-basis event that causes Station Blackout and leads to the implementation of FLEX strategies. An expert panel developed the scenario and specified the context associated with a set of human actions in FLEX strategies. The panel estimated the HEPs of those actions and the uncertainties. The NRC staff applied IDHEAS-ECA process to analyze two of the human actions to demonstrate the use of IDHEAS-ECA to a beyond-design-basis event.

C.1. Example 1: A simple steam generator tube rupture (SGTR) event This section documents the HRA of the scenario in IDHEAS-ECA Worksheets. The information we used for the analysis is from the Information Package in the Empirical Study, the Handbook of Conduct of Operations, the procedures, and the observations of NPP crew performance of the scenario in a simulator documented in NUREG-2156.

Worksheet A. Scenario Analysis Event title: Steam Generator Tube Rupture Event description:

Below presents the original scenario description provided to the HRA analysts in the US Empirical Study. The scenario is a simple SGTR event without any additional complications.

Detailed information about the procedures used, conduct of operations, and crews simulator performance can be found in NUREG-2156.

Situation from start

• All participating crew members in control room (Shift Manager, Unit Supervisor, Shift Technical Advisor and two Reactor Operators)

• The plant is operating at 100%

• Core burnup is 19,000 megawatt days per metric ton uranium (MWD/MTU), end of life (EOL)

• Steam generator tube rupture

• About 1 min after the start of the scenario, a tube rupture occurs in steam generator C.

The leak size is about 500 gallon per minute (gpm) at 100% power.

Procedures

• 0POP05-E0-E000 Reactor Trip or Safety Injection

• 0POP05-EO-EO300POP05-EO-EO30 Steam Generator Tube Rupture C-1

Section A.1 - Operational narrative This section includes the initiating event, initial conditions, boundary conditions, and event timeline. It may also include important scenario deviations and past operational experience reviews.

Initiating event About 1 minute after the start of the scenario, a tube rupture occurs in steam generator C.

The leak size is about 500 GPM at 100% power.

Initiating conditions

• All participating crew members in control room (Shift Manager, Unit Supervisor, Shift Technical Advisor and two Reactor Operators)

• The plant is operating at 100%

• Core burnup is 19,000 MWD/MTU (EOL)

Boundary conditions

• This is a standard scenario that crews are trained on frequently

• No complications to the event

• All related instrumentation, equipment, and components function as designed.

• Adequate staffing

• Procedures are available and are well trained on Event timeline - Document important system states and required human actions along the timeline of event progression.

The event timeline is estimated based on expected event progression and the time that important system activities (S) and expected human (H) actions occur. The time was estimated in minutes after the event started.

0:00 Start of scenario 1:00 (S) Radiation alarms 4:00 (S) Reactor trip 5:00 (H) Place AFW 13 in pull-to-lock (PTL) 5:00 (H) Enter E-30 SGTR response procedure 20:00 -30:00 (H) Isolate the ruptured steam generator (SG) 30:00 beyond (H) Cool down and depressurize reactor cooling system (RCS)

Section A.2. Human Failure Event (HFE) identification List of all the important human actions, including the HFEs in the PRA model and additional important human actions that should be analyzed.

List of HFEs One HFE is identified in the PRA event tree.

HFE1: Fail to isolate the ruptured SG.

C-2

Worksheet A. Scenario Analysis (continued)

Section A.3. Scenario context This section analyzes and documents the conditions that challenge or facilitate human performance. The documentation includes four categories of context relating to environment, system, personnel, and tasks.

Environment and situation context

• All crew activities in this scenario are performed in the main control room

• No additional complications beside the initiating event System context

• Required equipment and/or instrumentation is available

• No complicating or unexpected malfunctions

• Indications are available and reliable Personnel context

• The human-machine interface in the control room is well designed

• The operators have worked as cohesive crews for a long time

• Operator work process follows the requirements in the Handbook of Conduct of Operations

• The Symptom based EOPs are in place and well designed for the scenario under evaluation; operators recognize the event and are familiar with applicable procedures and actions.

Task context

• The scenario is a well-practiced classic event, covered in training, and practiced in simulator requalification exercises, such that crews are expected to know the alarm pattern of an SGTR event.

• Crew responses are clearly specified in EOP steps and frequently practiced in training scenarios

• Execution actions are considered step-by-step following procedures

• Control panel indications needed for diagnosis are simple and easily found.

• Parameters and trends are easily available and no calculations or trend annotation or memorization is needed

• The crew needs to complete the isolation of the ruptured SG and the cooldown and depressurization of RCS within two to three hours after the initiating event. Normally, the operators take about 30-40 minutes to complete both tasks.

• No simultaneous event occurs, thus the crews do not need to perform parallel multiple tasks and distractions and interruptions of the crews are expected to be nominal.

C-3

Worksheet A. Scenario Analysis (continued)

Section A.4 Initial assessment of PIFs Based on the assumed boundary conditions and context analysis, the following PIFs have no impact on the HFE:

All the Environmental factors, Information availability and reliability, System and IC transparency, Equipment and tools, Human-system interface, Staffing, Training, Work process, Team and organization factors, Scenario familiarity, Mental fatigue, and Physical demands.

The following PIFs appears nominal to the overall scenario, but they may impact crew performance of specific tasks, thus they should be further analyzed for individual parts of the HFE:

Task complexity, Procedures, Multitasking / interruption / distraction, Stress and time pressure.

Worksheet B. HFE Analysis HFE1: Fail to isolate the ruptured steam generator Section B.1 HFE definition This section defines the HFE at the high level with basic information relevant to human performance of the action.

Success criterion: Isolate the ruptured SG and control pressure below the SG PORV setpoint to prevent radiation leakage to the environment Starting and ending point: The HFE starts at the secondary radiation alarms immediately for the tube rupture that cues the operator about the occurrence of an abnormal situation, and ends at completing the RCS cooldown and depressurization Procedures / Instructions: Emergency Operating Procedure - Zero (EOP-0) Reactor Trip Or Safety Injection and EOP-3 Steam Generator Tube Rupture Key indications: Radiation alarms Special equipment / tools: None Time available: A time-critical action as the crew needs to isolate the ruptured SG before SG PORV opening. The time available is 2-3 hours.

Specifications: None for the standard scenario and a normal PWR simulator C-4

Worksheet B. HFE Analysis (continued)

Section B.2 Task diagram and identification of critical tasks An important human action may consist of multiple discrete tasks. The task diagram elucidates the tasks needed to achieve the action, the paths of the tasks, and interteam coordination needed to achieve the tasks. Critical tasks are those required to meet the success criterion of the action; failing any of the critical tasks would fail the action.

Task diagram This is a standard EOP action so the success path of the action is straightforward and clearly defined in the EOP. Below is the task diagram based on the two EOPs required for the action.

The key indication for the action is the radiation alarms. In responding to the alarms, the crew first performed the immediate actions to ensure the reactor is tripped, then followed the EOP-0 to diagnose the event. The crew is expected to enter EOP-3 to handle the SGTR event.

The task diagram shows two tasks: Trip the reactor and Perform EOP-3. The first task is not critical to the success of the HFE because the system will automatically trip the reactor if the crew fails to manually trip it. Because the reactor will trip automatically if not manually tripped, the success or failure of manually trip reactor should have no effect on the HFE.

Manually tripping the reactor early would buy the crew more time to handle the event. The second task is critical because failing the task leads to failure of the HFE.

Critical tasks The task diagram indicates one critical task:

HFE1-T1: Enter and perform EOP-3 Worksheet C. Modeling failure of the critical tasks Worksheet C is for every critical task thus there can be multiple Worksheet Cs if an HFE has multiple critical tasks.

C-5

Worksheet C for Critical Task HFE1-T1 HFE1-T1: Enter and perform EOP the task goal is to isolate the SG and control pressure below the SG PORV setpoint.

Section C.1. Analysis of cognitive activities and identification of applicable CFMs Analysis of cognitive activities are based on the macrocognitive functions.

Cognitive activities Detection (Respond to alarms, Get information, Monitor parameters or status) - Crew needs to read parameter information from indicators and monitor parameters.

Understanding (Assess situation, Diagnose problems, Make predictions of system behaviors)

- Entering EOP-3 needs to assess the situation and diagnose the ruptured SG.

Decisionmaking -Once the ruptured SG is identified, the EOP-3 directed the operator to isolate the ruptured SG and to cooldown and depressurize the RCS. The EOP-3 steps are to be performed upon the parameters that meet the specified conditions. The crew does not need to choose from alternatives or make plans. In other words, the procedure made the decision which is clear and there are no alternative decisions in this event.

Action execution - The crew needs to execute EOP-3 step by step to isolate the ruptured SG and to cooldown and depressurize the RCS. EOP-3 has many steps and some steps are control manipulations Interteam coordination - The action is performed by a single crew in the control room. The on-site operator actions (e.g., take SG chemistry samples) does not affect the success of the HFE. The communication between the MCR crew and onsite operator (interteam coordination) has little impact on the HFE success.

Applicable CFMs Based on the analysis of the cognitive activities, the following CFMs are applicable to T1:

T1-CFM1: Failure of Detection T1-CFM2: Failure of Understanding T1-CFM4: Failure of Execution Section C.2 Task characterization Special requirements - The task needs to be performed before reaching the SG PROV setpoints.

Cue - The cue to start the task is the secondary radiation alarm.

Personnel - Adequate well-trained crew Procedure -EOP-0 and EOP-3 have been implemented in simulator training. The procedures have been optimized based on training feedback. It is expected that the crew in the scenario will follow the procedures without any notable reason to deviate from the procedure instructions.

Competing goals and alternative strategies - None.

Multitasking - Parallel tasks are distributed by the two ROs handling the activities related to the primary and secondary systems, thus individual crew members are not impacted by multitasking.

C-6

Worksheet C for Critical Task HFE1-T1 (continued)

Section C.3 Assessment of PIFs Assessment of PIFs is to select the applicable PIF attributes from the PIF tables.

Assessment of PIFs is performed only to the PIFs relevant to the event as determined in Section A.4. Four PIFs were identified as relevant to the event and should be assessed for specific tasks and CFMs: Task complexity, Procedures, Multitasking / interruption /

distraction, Stress and time pressure.

T1-CFM1 Task complexity: C0 - No impact. The detection activities are simple and straightforward; alarms and parameters are easy to recognize.

Procedures: PG0 - No impact. EOP-3 is clear and straightforward Multitasking: MT0 - No impact; no need for multitasking, nominal distraction and interruption by following the Conduct of Operations.

Mental fatigue and Time pressure and stress: MF0 - No impact; a standard, frequently trained EOP scenario; plenty of time T1-CFM2 Task complexity: C30 - No impact. The diagnosis of SGTR is symptom-based and the EOP-0 and EOP-3 provide multiple opportunities for the diagnosis.

Procedures: PG0 - No impact. EOP-3 is clear and straightforward Multitasking: MT0 - No impact; no need for multitasking, nominal distraction and interruption by following the Conduct of Operations.

Mental fatigue and Time pressure and stress: MF0 - No impact; a standard, frequently trained EOP scenario; plenty of time T1-CFM4 Task complexity: C31 - EOP-3 has multiple proceduralized steps and some steps need control manipulation Procedures: PG0 - No impact. EOP-3 is clear and straightforward Multitasking: MT0 - No impact; no need for multitasking, nominal distraction and interruption by following the Conduct of Operations.

Mental fatigue and Time pressure and stress: MF0 - No impact; a standard, frequently trained EOP scenario; plenty of time Worksheet D - HEP estimation Worksheet D documents the HFEs, critical tasks, CFMs, and applicable PIF attributes for HEP estimation.

HFE1: Fail to isolate the SG and control the pressure Critical Task Applicable CFMs Applicable PIF attribute HFE1-T1: Enter T1-CFM1: failure of Detection No impact and perform T1-CFM2: failure of Understanding No impact EOP-3 T1-CFM4: failure of Execution Task complexity C31 -

Straightforward procedure execution with many steps C-7

Worksheet E: Time uncertainty analysis of the HFEs Worksheet E documents the estimation of time available and time needed for every HFE. The distribution of the time can be estimated as a single number, the mean and standard deviation (SD) by assuming a normal distribution, or a five-point estimation of probability distribution (at 5th, 25th, 50th, 75th, and 95th percentile).

HFE Time available Time needed HFE1 2-3 hours in PRA models Mean: 18mins SD: 5mins The estimation is based on operator simulator training data in NUREG-2156.

C.2. Example 2: Loss of Component Cooling Water and Reactor Coolant Pump Sealwater This is Scenario 2 used in the US HRA Empirical Study. The scenario was designed to have multiple concurrent component and control failures to increase the operators workload and to distract the operators attention to prevent a reactor coolant pump (RCP) seal failure. The success criteria for crew responses are to trip the RCPs and to start the positive displacement pump (PDP) to provide RCP seal injection to prevent the seal water inlet temperature or the lower sealwater bearing temperature exceeds 230 degree F (to prevent an RCP seal failure).

The criteria determine that operator actions are time critical.

This section documents our analysis of the scenario in IDHEAS-ECA Worksheets. The information we used for the analysis is from the Information Package in the Empirical Study, the Handbook of Conduct of Operations, the procedures, and the observation of NPP crew performance of the scenario in a simulator documented in NUREG-2156.

Worksheet A. Scenario Analysis Event title: Loss of CCW and RCP sealwater with complication PRA event description:

This section presents the original scenario description provided to the HRA analysts in the US Empirical Study. Detailed information about the procedures used, conduct of operations, and crews simulator performance can be found in NUREG-2156.

Plant Technical Information Component Cooling Water (CCW)

CCW pump 1A, powered by E1A CCW pump 1B, powered by E1B CCW pump 1C, powered by E1C RCP sealwater charging pump 1A, powered by E1C charging pump 1B, powered by E1A positive displacement pump (PDP), powered by 1G8-bus (remains energized), cooled by air (does not use CCW)

C-8

Loss of CCW and RCP Sealwater In the beginning of the scenario the plant is operating at 100 percent power and all five crew members are in the control room. The CCW B train is out of service so that the CCW pump B is unavailable.

Two minutes into the scenario, the distribution panel 1201 fails. As a consequence, the crew has to establish manual control of following controlling channels:

• A and B SGs

• PZR level control

• rod control

• nuclear instrumentation (NIS)

• PZR pressure control For this scenario, it is of particular importance for the crew to establish manual control of feedwater flow to SGs A and B.

The failed distribution panel is unrelated to the loss of CCW and sealwater but increases the complexity of the scenario. It masks the status of CCW and sealwater by keeping the crew busy because of the number of alarms.

The feedwater regulation valve on SG A remains fully open and cannot be operated manually, feeding the SG. If the crew does not trip the reactor, there will be an automatic turbine trip on high SG level (87 percent), which would cause a reactor trip.

When the reactor trips, one AFW pump cannot start because of the loss of the distribution panel 1201. In addition, Bus E1C will have a bus lockout caused by a bus fault (the busbar is deenergized and the DG breaker cannot be closed), and the CCW pump 1A breaker will trip because of a failed and seized shaft. As a result, there are no CCW pumps in service (the pump B is out of service, pump A is tripped, and pump C is de-energized), and no charging pump is running (pump A is de-energized). If charging pump 1B is started, it will trip 2 minutes after the reactor trip.

According to procedure 0POP04-RC-0002 Reactor Coolant Pump Off Normal, any RCP that experiences a simultaneous loss of seal injection flow and loss of CCW flow to thermal barrier shall be stopped within 1 minute after determining that both RCS seal injection flow and thermal barrier cooling were lost. The risk of a seal failure increases after 1 minute.

Procedure 0POP04-RC-0002 Reactor Coolant Pump Off Normal and procedure ES-01 (reactor trip response) both have guidance to start the positive displacement pump (PDP) when CCW and seal injection are lost. However, the PDP can only be started if the RCP seal temperatures are below 230 degrees Fahrenheit (F), and reaching these procedure steps takes some time. In accordance with the Westinghouse RCP vendor manual, if seal inlet or seal inlet bearing reach 230 degrees F, the potential for seal damage is too great to risk placing seal injection in service.

Procedures that may be used

• 0POP04-VA-0001 Loss Of 120 VAC Class Vital Distribution

• 0POP04-RC-0002 Reactor Coolant Pump Off Normal

• 0POP05-E0-E000 Reactor Trip Or Safety Injection

• 0POP05-E0-ES01 Reactor Trip Response C-9

• 0POP09-AN-02M3 Annunciator Lampbox 2M03 Response Instructions (page 10, CCW PUMP 1A(2A) TRIP)

• 0POP09-AN-04M7 Annunciator lampbox 4M07 Response Instructions (page 3, RCP 1A(2A) SEAL WTR INJ FLOW LO)

Section A.1 - Operational narrative This section includes the initiating event, initial conditions, boundary conditions, and event timeline. It may also include important scenario deviations and past operational experience review.

Initiating event About 2min into the scenario, the Distribution Panel 1201, 120V AC Class Vital Distribution, failed. That led the crew to establish manual control of the following controlling channels:

• A and B SGs.

• PRZ level control

• Rod control

• Nuclear Instrumentation (NIS)

• PRZ pressure control The crew needs to take the equipment above in manual control, in particular they need to take manual control of the feedwater flow to SG A and B.

Initiating conditions

• All participating crew members in control room (Shift Manager, Unit Supervisor, Shift Technical Advisor and two Reactor Operators)

• The plant is operating at 100% power

• Core burnup is 19,000 MWD/MTU (End of life)

• CCW pumps 1A and 1C are in service. Charging pump 1A is in service.

• B train is out of service for CCW pump 1B and ECW pump 1B planned maintenance.

The following equipment is unavailable:

o CCWP 1B o ECWP 1B o Diesel Generator 12 o AFWP 12 Boundary conditions

• The event begins with the failure of Distribution Panel 1201 and ends with the crew successfully tripping the RCPs and starting the Positive Displacement Pump (PDP) to prevent RCP seal LOCA.

• Reactor trip on high SG level - The Feedwater regulation valve on SG A cannot be operated manually and remains fully open, feeding the SG. If the crew does not trip the reactor, there will be an automatic turbine trip on high SG level (87%), which causes a reactor trip.

• When the reactor trips, Bus E1C will have bus lockout due to a bus fault. (The busbar is deenergized and the DG breaker cannot be closed.). CCW pump 1A breaker will trip due to failed, seized shaft. There are no CCW pumps in service (B pump out of service, A pump tripped, C pump de-energized), and no charging pump running (A pump de-energized). If charging pump 1B is started, it will trip 2 minutes after reactor trip.

C-10

Worksheet A. Scenario Analysis (continued)

Section A.1 - Operational narrative (continued)

Event timeline - Document important system states and required human actions along the timeline of event progression.

Trip the RCPs after the loss of CCW and start the PDP to provide seal injection before either the seal water inlet or lower seal water bearing temperature is greater than 230 degrees F (per ES01 Step 6 or 0POP04-RC-0002: Reactor coolant pump off-normal) to avoid potential (not necessarily immediate) RCP seal LOCA. The time to reach 230 degrees F is about 7 to 9 minutes from the loss of CCW.

0:00 Start of scenario 2:00 (S) DP-1201 failure 3:00 (S) Reactor trip (S) Loss of CCW and sealwater (H) Start EOP-0 5:00 - 8:00 (H) Start procedure ES-01 Detect no CCW or sealwater Stop all RCPs Start 0POP04-RC-0002 Reactor Coolant Pump Off Normal procedure Start PDP 7:00 - 9:00 (S) RCP temp reaches greater than 230 degrees F 7:00 beyond (H) PDP must not be started after RCP temp is greater than 230 degrees F Section A.2. Human Failure Event (HFE) identification List of all the important human actions, including the HFEs in the PRA model and additional important human actions that should be analyzed.

List of HFEs One HFE is identified in the PRA event tree.

HFE1: Failure of the crew to trip the RCPs and to start the Positive Displacement Pump (PDP) to prevent RCP seal LOCA Note: A possible deviation is that operators may start the PDP after RCP temperature exceeds 230 degrees F. This would be an error of commission. We model this deviation as a separate HFE. This HFE was not modeled in the US HRA Empirical Study.

HFE2: Crew starts the PDP after RCP temperature reaches 230 degrees F.

Section A.3. Scenario context This section analyzes and documents the conditions that challenge or facilitate human performance. The documentation includes four categories of context relating to environment, system, personnel, and tasks.

Environment and situation context

• All key readings and actions to successfully implement the HFE are performed inside the main control room

• No complicating or unusual environmental factors C-11

Worksheet A. Scenario Analysis (continued)

Section A.3. Scenario context (continued)

System context

• The required equipment and instrumentation are available

• The failure of DP-1201 caused failures of the controlling channels for A and B SGs, PRZ level control, Rod control, Nuclear Instrumentation (NIS), and PRZ pressure control. The crew needs to take the equipment above in manual control, in particular they need to take manual control of the feedwater flow to SG A and B.

• The human-machine interface in the control room is well designed, but a lot of Train A indications were not available because of DP-1201 failure.

Personnel context

• The operators have worked as a cohesive crew for a long time

• Operator work process generally follows the requirements in the Handbook of Conduct of Operations, but some conduct of operation, such as monitoring control boards and acknowledging alarms, may not be optimal in the scenario when many system failures occurred.

• The Symptom based EOPs are in place and applicable for the scenario under evaluation

• The loss of CCW and sealwater to RCPs is usually trained in an EC-00 (loss of offsite power procedure) scenario once every 2 years.

Task context

• The crew are trained to respond to loss of CCW (Loss of RCP seal cooling) or loss of PDP charging flow (loss of the seal injection) scenarios caused by loss of offsite power. However, the scenario under the evaluation is different from the training. The crew has not been trained to respond to a loss of CCW and RCP sealwater in concurrence with failure of DP-1201.

• The required responses are clearly specified in EOP steps. The actions are considered step-by-step following procedures. Multiple procedures may be used for the scenario.

• Control panel indications needed for diagnosis are simple and can be easily found.

• Parameters and trends are readily available and no calculations or trend annotation or memorization is needed

• Time is critical for the required actions but the time available is only 7 to 9 minutes, barely enough for the crews to perform all the needed actions. Manual control of inventories due to failure of the distribution panel increases the time needed to perform the actions.

Many things happening at the same time made it difficult to detect the priority items.

Operators may experience multitasking, interruption, and distraction by the alarm cascade and required actions after the DP-1201 failure and the other concurrent failures especially related to SG water level control.

C-12

Worksheet A. Scenario Analysis (continued)

Section A.4 Initial assessment of PIFs Based on the assumed boundary conditions of the event and context analysis, the following PIFs should have no impact on the HFE:

All the Environmental factors, Information availability and reliability, System and IC transparency, Equipment and tools, Staffing, Work process, Team and organization factors, Mental fatigue, and Physical demands.

The following PIFs appear nominal to the overall scenario, but they may impact crew performance of specific tasks, thus they should be further analyzed for individual parts of the HFE:

Scenario familiarity, Task complexity, Human-system interface, Procedures, Training, Multitasking / interruption / distraction, Stress and time pressure.

Worksheet B. HFE Analysis Worksheet B for HFE1: Failure of the crews to trip the RCPs and start the Positive Displacement Pump (PDP) to prevent RCP seal LOCA HFE1: Failure of the crews to trip the RCPs and start the Positive Displacement Pump (PDP) to prevent RCP seal LOCA Section B.1 HFE Definition This section defines the HFE at a high level with basic information relevant to human performance of the action.

Success criteria: Trip the RCPs after the loss of CCW and start the PDP to provide seal injection before either sealwater inlet or lower sealwater bearing temperature exceeds 230 degrees F.

Starting and ending point: Starts at the failure of the DP-1201 and ends at stopping the RCP and starting PDP.

Procedures / Instructions: The following procedures are available and may be used. Among them EOP-0 and ES-01 were the highest priority procedures in the scenario:

• 0POP05-E0-E000 Reactor Trip Or Safety Injection

• 0POP05-E0-ES01 Reactor Trip Response

• 0POP04-VA-0001 Loss Of 120 VAC Class Vital Distribution

• 0POP04-RC-0002 Reactor Coolant Pump Off Normal

• 0POP09-AN-02M3 Annunciator Lampbox 2M03 Response Instructions (page 10, CCW PUMP 1A(2A) TRIP)

• 0POP09-AN-04M7 Annunciator lampbox 4M07 Response Instructions (page 3, RCP 1A(2A) SEAL WTR INJ FLOW LO)

Key indications: Loss of the DP-1201, loss of ESF bus C (The running charging pump was fed by the C-bus and consequently all CCW and sealwater was lost to the RCPs), indications of no CCW, indications of no RCP sealwater, and indication of RCP seal temp Special equipment / tools: None Time available: A time-critical action. Time available for stopping RCP is 1min after the loss of both seal injection flow and loss of CCW flow. Time available for starting PDP is 7~9mins before the RCP temp exceeds 230 degrees F.

C-13

Worksheet B. HFE Analysis (continued)

Section B.1 HFE Definition (continued)

Specifications:

• 1-minute criterion in POP4-RC2 for stopping the RCPs - any RCP that experiences a simultaneous loss of seal injection flow and loss of CCW flow to thermal barrier shall be stopped within 1 minute.

• The PDP pump must not be started if the RCP seal temperatures exceeds 230 degrees F (according to 0POP04-RC-0002 Reactor Coolant Pump Off Normal and ES-01).

Section B.2 Task diagram and identification of critical tasks An important human action may consist of multiple discrete tasks. The task diagram elucidates the tasks needed to achieve the action, the paths of the tasks, and interteam coordination needed to achieve the tasks. Critical tasks are those required to meet the success criterion of the action; failing any of the critical tasks would fail the action.

Task diagram and timeline At 2 minutes into the scenario, the DP-1201 was lost. The crew needs to quickly take control of the affected equipment. In this scenario, because SG A feedwater regulation valve cannot be put in manual, the SG A water will rise and eventually causes an automatic reactor trip. The crew is expected to manually trip the reactor before an automatic setpoint and enter EOP-0.

Within one minute of the DP-1201 failure, there are alarms and indicators to indicate the loss of CCW and RCP sealwater. The crew should recognize the loss of CCW and RCP sealwater and enter ES-01.

In ES-01, there are steps for the crew to perform procedure 0POP04-RC-0002 Reactor Coolant Pump Off Normal to stop RCP. However, if operators only recognize the loss of CCW without recognizing the loss of RCP sealwater, they may perform activities to recover CCW.

Later in ES-01, there are steps for the crew to start PDP to protect the seals. However, the crew must not to start the PDP if the seal temperature exceeds 230 degrees F.

Critical tasks The task diagram indicates two critical tasks:

T1: Stop the RCP within 1min after the loss of CCW and RCP sealwater.

T2: Start the PDP before seal temp reaches 230 degrees F (in 7~9 mins).

C-14

HFE2: Start the PDP after the RCP temp reaches 230 degrees F (Note that this HFE is not modeled in the US Empirical Study. We only performed its HFE analysis for demonstration purpose without going into further detail.)

HFE2: Start the PDP after the RCP temp reaches 230 degrees F Section B.1 HFE definition This section defines the HFE at a high level with basic information relevant to human performance of the action.

Success criterion: NOT starting the PDP after either sealwater inlet or lower sealwater bearing temperatures exceeds 230 degrees F.

Starting and ending point: The action may begin as the crew enters ES-01 and recognizes that the seal flow is low; it ends as the crew starts the PDP after the seal temp exceeds 230 degrees F.

Procedures / Instructions:

• 0POP05-E0-ES01 Reactor Trip Response

• 0POP04-RC-0002 Reactor Coolant Pump Off Normal

• 0POP09-AN-04M7 Annunciator lampbox 4M07 Response Instructions (page 3, RCP 1A(2A) SEAL WTR INJ FLOW LO)

Key indications: indications of no CCW, indications of no RCP sealwater, indication of RCP seal temp Special equipment / tools: None Specifications:

• The PDP pump must not be started if the RCP seal temperatures are 230 degrees F or higher (according to 0POP04-RC-0002 Reactor Coolant Pump Off Normal and ES-01).

Section B.2 Task diagram and identification of critical tasks An important human action may consist of multiple discrete tasks. The task diagram elucidates the tasks needed to achieve the action, the paths of the tasks, and interteam coordination needed to achieve the tasks. Critical tasks are those required to meet the success criterion of the action; failing any of the critical tasks would fail the action (HFE).

Task diagram The assumption for this action is that the crew recognizes the loss of CCW and RCP sealwater and enters E0-ES01. In E0-ES01, there are steps for the crew to start the PDP to protect the seals. However, the crew must not to start the PDP if the seal temperature exceeds 230 degrees F.

Critical tasks One critical task is identified:

HFE2-T1: Not start the PDP after seal temp reaches or exceeds 230 degrees F.

C-15

Worksheet C. Modeling failure of the critical tasks There may be multiple Worksheets C, one for each critical task of an HFE.

Worksheet C for Critical Task HFE1-T1 Critical task HFE1-T1: Trip the RCPs - the task goal is to recognize the simultaneous loss of CCW and RCP sealwater and stop the RCPs Section C.1. Analysis of cognitive activities and identification of applicable CFMs Analysis of cognitive activities are based on the macrocognitive functions.

Cognitive activities Detection - Crew needs to recognize the simultaneous loss of CCW and RCP sealwater.

Understanding - The task does not require Understanding activities because, from training, operators know to stop RCPs upon recognizing the simultaneous loss of CCW and RCP sealwater.

Decisionmaking - The task does not require decisionmaking activities because, from training, operators know to stop RCPs upon recognizing the simultaneous loss of CCW and RCP sealwater.

Action execution - The crew needs to stop RCPs.

Interteam coordination - The action is performed by a single crew in the control room. It does not involve interteam coordination.

Applicable CFMs Based on the analysis of the cognitive activities, the following CFMs are applicable to T1:

HFE1-T1-CFM1: Failure of Detection HFE1-T1-CFM4: Failure of Execution Section C.2 Task characterization Special requirements - The task needs to be performed within minutes after the loss of CCW and RCP sealwater.

Cue - The cues for starting the task include the alarms of CCW pump trip and PDP trip and the indications of no CCW flow and no sealwater. Alternatively, operators may also use the cue of loss of Bus C to recognize the loss of CCW and RCP sealwater based on their knowledge.

Personnel - Adequate well-trained crew Procedure - Well practiced EOP to respond to individual symptoms. However, immediately recognizing the simultaneous loss of CCW and sealwater is guided by knowledge, not procedures.

Competing goals and alternative strategies - None.

Multitasking - The operators need to handle multiple, concurrent system malfunctions.

Section C.3 Assessment of PIFs Assessment of PIFs is to select the applicable PIF attributes from the PIF tables in Appendix B.

Assessment of PIFs is performed only for the PIFs relevant to the event as determined in Section A.4. The following PIFs were identified as relevant to the event and should be assessed for specific tasks and CFMs:

Scenario familiarity, Task complexity, Human-system interface, Procedures, Training, Multitasking / interruption / distraction, Stress and time pressure.

C-16

Worksheet C for Critical Task HFE1-T1 (continued)

Section C.3 Assessment of PIFs (continued)

HFE1-T1-CFM1 Scenario familiarity:

SF3 - Scenario is unfamiliar, rarely performed (notice adverse indicators that are not part of the task at hand)

Task complexity:

C1 - Detection overload with multiple competing signals C6 - No cue or mental model for detection Multitasking / Interruption / Distraction: MT3 - Concurrent visual detection and moderate demands of other tasks HFE1-T1-CFM2 All the PIFs are no impact. As long as the detection succeeds, stopping the RCP is a simple, one step activity and can be performed instantly.

Worksheet C for Critical Task HFE1-T2 Critical task HFE1-T2: Start PDPs - the task goal is to start the PDP before the RCP temperature reaches or exceeds 230 degrees F.

Section C.1. Analysis of cognitive activities and identification of applicable CFMs Analysis of cognitive activities are based on the macrocognitive functions.

Cognitive activities Detection - Crew needs to recognize the simultaneous loss of CCW and RCP sealwater.

Understanding - Crew needs to understand that the PDP pump must be started before the RCP seal temperature reaches 230 degrees F (according to 0POP04-RC-0002 Reactor Coolant Pump Off Normal and ES-01).

Decisionmaking - Procedure directs to start the PDP; The crew does not need to make decisions but only needs to follow the procedure.

Action execution - Press a button to start the PDP.

Interteam coordination - None Applicable CFMs Based on the analysis of the cognitive activities, the following CFMs are applicable to HFE1-T2:

HFE1-T2-CFM1: Failure of Detection HFE1-T2-CFM2: Failure of Understanding HFE1-T2-CFM4: Failure of Execution Section C.2 Task characterization Special requirements - The task needs to be performed before 230 degrees F and CANNOT be performed after 230 degrees F.

Cue - The cues for starting the task include the alarms of CCW pump trip and the indications of no CCW flow and no sealwater. Alternatively, operators may also use the cue of loss of Bus C to recognize the loss of CCW and RCP sealwater based on their knowledge.

Personnel - Adequate well-trained crew Procedure - The procedure 0POP04-RC-0002 Reactor Coolant Pump Off Normal and ES-

01) are available for the situation. However, the criteria for entering the procedures are not clear, and there are several alternative procedures that might be applicable.

Competing goals and alternative strategies - None.

C-17

Multitasking - The operators need to handle multiple, concurrent system malfunctions.

Worksheet C for Critical Task HFE1-T2 (continued)

Section C.3 Assessment of PIFs Assessment of PIFs is to select the applicable PIF attributes from the PIF tables in Appendix B.

Assessment of PIFs is performed only to the PIFs relevant to the event as determined in Section A.4. The following PIFs were identified as relevant to the event and should be assessed for specific tasks and CFMs:

Scenario familiarity, Task complexity, Human-system interface, Procedures, Training, Multitasking / interruption / distraction, Stress and time pressure.

HFE1-T2-CFM1 Scenario familiarity:

SF3 - Scenario is unfamiliar, rarely performed (notice adverse indicators that are not part of the task at hand)

Task complexity:

C1 - Detection overload with multiple competing signals Multitasking / Interruption / Distraction:

MT3 - Concurrent visual detection and moderate demands of other tasks HFE1-T2-CFM2 Scenario familiarity:

SF2 - Unfamiliar elements in the scenario - non-routine, infrequently performed tasks Task complexity:

C14 -Potential outcome of situation assessment consists of multiple or ambiguous states and context.

HFE1-T2-CFM4 None of the PIFs impact this simple action.

Worksheet D - HEP estimation Worksheet D documents the critical tasks, CFMs, and applicable PIF attributes of an HFE for HEP estimation.

HFE1: Failure of the crews to trip the RCPs and start the Positive Displacement Pump (PDP) to prevent RCP seal LOCA Critical Task Selected CFMs Selected PIF attribute HFE1-T1: Stop T1-CFM1: failure SF3 - Scenario is unfamiliar, rarely performed the RCP within of Detection C1 - Detection overload with multiple competing 1min after the signals loss of CCW and C6 - No cue or mental model for detection RCP sealwater MT3 - Concurrent visual detection and moderate demands of other tasks T1-CFM4: failure No impact of execution HFE1-T2: Start T2-CFM1: failure SF3 - Scenario is unfamiliar the PDP before of Detection C-18

seal temp C1 - Detection overload with multiple competing reaches 230 signals degrees F (in 7~9 MT3 - Concurrent visual detection and moderate demands of other tasks mins).

T2-CFM2: failure SF2 - Unfamiliar elements in the scenario of Understanding C14 - Ambiguous states of situation assessment MT3 T2-CFM4: failure No impact.

of Execution Worksheet E: Time uncertainty analysis of the HFEs Worksheet E documents the estimation of time available and time needed for every HFE. The distribution of the time can be estimated as a single number, the mean and standard deviation (SD) by assuming a normal distribution, or a five-point estimation of probability distribution (at 5th, 25th, 50th, 75th, and 95th percentile).

HFE1 is special in time available. It has two critical tasks, each having its own required time available. Therefore, Pt has to be separately quantified for each critical task.

HFE Time available Time needed HFE1- 1 minute after simultaneous loss Mean: 6.5mins T1 of CCW and RCP sealwater. SD: 1.5mins This is specified in the The estimation is based on operator simulator procedure performance data in NUREG-2156.

HFE1- The system time available is 7-9 After operators performed HFE1-T1, they already T2 minutes in the PRA model. detected the loss of CCW and sealwater. Then, However, operators need to the time needed for the task is the time to enter perform HFE1-T1 first, so the the RCP procedure and start the PDP. The mean time available for HFE1-T2 is the time for detecting the loss of CCW and sealwater system time available subtracted is 5.5min with a deviation of 1.5min. The mean by the time needed for HFE1-T1 time taken from detecting the loss of CCW and sealwater to entering the RCP procedure is approximately 3min with a standard deviation 2min, based on operator simulator performance data in NUREG-2156.

C.3. Example 3: Human actions of implementing FLEX strategies in a beyond-design-basis event This is the FLEX-designed scenario in the NRCs 2018 FLEX-HRA Expert Elicitation. This example used the scenario description and the inputs from the expert panel as the source information for IDHEAS-ECA analysis.

C-19

Worksheet A. Scenario analysis Event title: Deploy FLEX generator in an extended loss of AC power (ELAP) event Event documentation:

Blockage of intake by seaweed/jellyfish ingestion, silt, physical damage to screens, frazile ice, unusually low tide or low river water level, dam break downstream dropping water level quickly, or barge collision, etc. Loss of Turbine Lube Oil CoolerMust get Turbine shut down, loss of H2 Seal Oil System CoolingMust vent Main Generator, loss of Condenser Vacuum Pumps, loss of Circulating Watervacuum lossno Steam Generator (PWR [pressurized-water reactor]) or Reactor (BWR [boiling-water reactor]) Feed Pumpsno steam dumps (smaller atmospheric dumpsPWR) or turbine bypass valves (BWR), loss of EDG Cooling water, loss of closed cycle cooling water systems (Service Water, Emergency Service Water or RHR [residual heat removal] Service Water), Instrument Air Systemno interstage cooling and no Air Dryer cooling For PWRs:

Letdown Heat Exchangersmust isolate Letdown (will wipe out Resin Beds if the temperature is over 140Fmust bypass but still put HOT water to VCT [volume control tank]or divert to Liquid Radwaste which will steam out Aux Building. Must commence Cooldown to keep PZR [pressurizer] Level under control WHILE Charging.) Sampling System Coolersno boron verification, eventually could be a bigger concern verifying SDM [shutdown margin]. Control Room HVAC [heating, ventilation, and air conditioning]causes erroneous instrument readings and strange alarms. Switchgear Room Ventilationneed to open doors as in SBO if in summer conditions. ISOPHASE Bus Duct Cooling Systemnot immediate but could cause problems, not a big issue for now. Hot Penetration Room Cooling. Loss of motor cooling to most secondary pumpsneed to close MSIVs [Main Steam Isolation Valves] and Feedwater Isolations (only AFW [auxiliary feedwater] in useeventually only the Motor Driven AFW Pumps when SGs [steam generators] cooldownBUT you cannot cooldowninability to go on Shutdown Cooling. Therefore, must maintain Hot Standby BUT no cooling to the RCP [reactor coolant pump] thermal barrier so must keep Charging running to inject into seal. BUT as stated above, Letdown securedwhere is VCT getting makeup? Shift Suction to RWST [refueling water storage tank]. This is good for a while to keep the RCP Seals cool. Loss of Motor Cooling to RCPsMotor Winding Temp alarmsmust securego to natural circulation, loss of Motor Cooling to Shutdown Cooling Pumps (if installed)

Necessary to go on Shutdown Cooling. So, you cannot go on Shutdown Cooling.

Must simmer away feeding SGs with AFW Tanks and alternate sources of inventory (that needs to be replenished). Must keep RCP Seals cool by charging into Seal Injection and letting down to Radwaste.

Need to get FLEX pump setup to put water either into CST [condensate storage tank]

(if motor-driven AFW pump available) or directly into the steam generators to keep RCS [reactor coolant system] cool.

For BWRs:

Controlling reactor pressure on safety relief valves (SRVs), Reactor Core Isolation Cooling (RCIC) starts on low reactor level and injects water into the reactor pressure vessel (RPV) to maintain water level. Depressurize with SRVs and use RCIC to put water into the RPV from the CST. Either torus (or suppression pool) level and C-20

temperature increases (RPV pressure control and reduction via SRVs)no cooling water to cool torus. RCIC loses barometric condenser. Torus (suppression pool) will heat up and may start steamingmay need to have torus venting, need to get FLEX pump setup to put water either into CST (if RCIC is available) or directly into the RPV.

Section A.1: Event description Initiating event An external hazard caused flooding in a single-unit nuclear power plant and led to damage of plant systems (see initial conditions below).

Initial conditions

• The reactor trips automatically immediately after the external hazard impacts the plant.

• The external hazard caused an ELAP event immediately after the plant is impacted.

• Some plant systems, equipment, and structures that do not have direct impact on plant safety were damaged.

• The indications of the plant parameters key to responding to the event to protect plant safety were available.

• Debris on the FLEX generator transportation route needs to be removed to bring the FLEX equipment to its designated setup location.

• Some of the work areas were flooded but accessible for work.

Boundary conditions (i.e., assumptions) for HEP estimation

• System and environment

- The event occurs at a single unit plant.

- The work area to setup and operate the FLEX equipment is accessible, but there are adverse environmental factors including water and debris in working areas, cold, moderately strong wind, and darkness.

- The FLEX generator is available.

- The key plant information related to the FLEX equipment deployment is presented with uncertainty that could affect the decisionmaking (e.g., the expected time of the tasks would be completed.).

- RCS leakage is nominal (low-leakage from the RCP seal package).

- The FLEX generator is not preinstalled. It needs to be brought from the FLEX storage building outside the plant access control area.

- The FLEX generator is operated outside of buildings unless specified otherwise.

• Personnel

- The plant staffing is at the minimal staffing level (i.e., typical staffing on holiday and night shifts).

- The onsite personnel responding to the initiating event are not going to be relieved for 8-12 hours.

- Decisionmakers are in the TSC emergency response organization after the TSC is in operation.

- It has been 2-3 years since plant personnel were trained on FLEX strategies.

• Human actions

- The FLEX generator needs to be in operation within four hours after the SBO.

- All the human actions are feasible.

C-21

Scenario timeline Document important system states and required human actions along the timeline of event progression.

Time Required responses and cues 0:00 ELAP 1:00 Operators declare the occurrence of an ELAP event within 1 hour1.157407e-5 days <br />2.777778e-4 hours <br />1.653439e-6 weeks <br />3.805e-7 months <br /> (based on procedure).

1:00 - 6:00 Operators perform FLEX actions after the ELAP declaration, including load shed of DC power, debris removal, use of portable generator, use of portable pumps, and refilling of condensate storage tank. The actions are initiated by the main control room crew based on procedure instructions. The corresponding FLEX Support Guidelines (FSGs) are available for the implementation of every FLEX actions.

6:00 and beyond The offsite emergency response personnel become available onsite for event mitigation.

Section A.2: Identification of important human actions List of all the important human actions, including HFEs in the PRA model and additional important human actions that should be analyzed.

Action 1: transportation, placement, connection, and local control of portable pumps Action 2: transportation, placement, connection, and local control of portable generators Action 3: refilling of water storage tanks using alternate water sources Action 4: ELAP declaration Action 5: deep dc load shed (the initial dc load shed was performed after an SBO event was declared, before the declaration of the ELAP event)

This example only analyzes Action 2 and Action 4 to demonstrate IDHEAS-ECA process.

Section A.3. Scenario context This section analyzes and documents the conditions that challenge or facilitate human performance. The documentation includes four categories of context relating to environment, system, personnel, and tasks.

Environment context Action 2: transportation, placement, connection, and local control of portable generators

• VisibilityWorkplace has moderately poor lighting (e.g., because of darkness, fog, smoke, dust).

• Water level is moderately poor; some workplaces or travel paths are in water (e.g.,

1-3 feet deep).

• Worksite is moderately cold.

• Worksite accessibility and habitability are not affected after debris removal.

Action 4: ELAP declaration

• The main control room crew is expected to make the declaration.

• The main control room lighting is by emergency lights.

Situation context - key challenges of the situation to human performance

• The hazard scenario includes damage to some onsite routes so personnel need to assess site damage and identify the deployment pathway.

C-22

• A personnel error or delay in one FLEX action could have ripple effects on the other FLEX actions because, for the first six hours, the plant is in a minimum staffing level to respond to the event.

• Resource management can be challenging:

o Prioritize items, such as actions and resource allocation between damage assessment, system restoration, and FLEX deployment.

o Assess and prioritize resources to perform the action in the given time.

• Command-and-control between parties in different locations can be challenging.

System context

• Human-system-interface (HSI) Indications and controls are well designed and not damaged.

• Parts and tools - All the tools and parts are available or accessible according to FLEX implementation orders.

Personnel context

• Staffing - minimum required staffing on site

• Guidelines/ procedures -FSGs are available to provide guidance to implement all FLEX actions. FSGs instructions may not be specific, lacking details, need judgment or minor adaptation.

• Training -

o Not included in Systematic Approach to Training o In this more complicated scenario, personnel found it hard to put their training and experience into action.

o The training does not explicitly cover all potential situations. The personnel need to be flexible to respond to the uncertainties in situations.

o That tabletops and drills on the scenario are performed infrequently.

o Lack of or less previous experienceNone of the operators have experienced a real event like the event in the analysis.

• Interteam coordination -

o Coordination: three types of teamwork difficulties are anticipated:

interdependence with other stakeholders (e.g., evacuation of the surrounding populations and working with firefighters) difficulty in anticipating events for unexpected action sequences (e.g.,

no one had anticipated that an air compressor would be needed to open the venting valve remotely, and sourcing one significantly delayed the venting) an individual could be assigned to work within a team and transfer to another team after completion of the first teams task. If the first team does not complete the task in time, it will affect the performance of the subsequent team because the same personnel work in both teams.

o Command and control - the MCR crew has the initial ultimate decision-making authority which is transferred to the technical support center after the center is in operation.

o Close coordination of activities is neededThe MCR crew coordinates the event mitigation activities before the TSC is in operation. After the TSC is in operation, the TSC, MCR, and Offsite Support Center coordinate the event mitigation activities and manpower assignments.

C-23

o Coordination between site personnel and decisionmakers is expected to be challenging. That affects modifying the planned actions for event mitigation based on site situation.

o Personnel are likely unable to verify the plan because of inadequate communication (of the goals, negative impacts, deviations) with decisionmakers.

o Supervision in monitoring actions and questioning the current mission is likely inadequate.

Task context

• Information needed for FLEX actions may not be readily available and may be presented with a large uncertainty because of unclear status of equipment damage and problems in remote communications.

• The scenario is not clearly recognized based on procedures or guidance; personnel must rely on knowledge to develop a mental model. The situation may not fully match prior training. Personnel barely used FLEX equipment in real scenarios.

• Personnel may concurrently execute intermingled or interdependent action plans.

• The decisionmaker has multiple issues to address in parallel. (In Fukushima, the tsunami warnings affected the site superintendents planning of accident management because he was concerned that the tsunami might damage seawater pumps.)

• Personnels work may be interrupted or distracted from time to time because the continuous effect of the external hazard and the other ongoing activities.

• Most FLEX actions require on-field team effort with heavy physical effort (e.g.,

establishing power connection with long power cables.)

Section A.4 Initial assessment of PIFs Based on the assumed boundary conditions and context analysis, the following PIFs are assumed to have no impact on the important human actions of the analysis:

System and IC transparency, Equipment and tools, Human-system interface, Staffing, Team and organization factors.

The following PIFs may impact personnel performance of specific tasks thus they should be further analyzed:

Environmental factors, Scenario familiarity, Information availability and reliability, Task complexity, Procedures, Multitasking / interruption / distraction, Stress and time pressure.

Worksheet B. Modeling important human actions HFE1 (Action 4): Fail to declare ELAP within one hour Section B.1 HFE definition This section defines the HFE at a high level with basic information relevant to human performance of the action.

An external hazard caused an ELAP that required the implementation of FLEX strategies. In the event, the MCR crew needs to timely declare an ELAP event because the essential dc power will only last for 4 hours4.62963e-5 days <br />0.00111 hours <br />6.613757e-6 weeks <br />1.522e-6 months <br />. The HFEs success criterion is to declare an ELAP event within one hour after the event start.

C-24

• Success criteria: An ELAP event should be declared within one hour after the event to have enough time to shed the dc load to extend the dc power availability and to use FLEX generator to charge the essential batteries for longer dc power availability.

• The plant SBO procedure requires declaring an ELAP event if the ac power is not and does not expected to be restored within an hour.

• The information about when the ac power (i.e., emergency diesel generators and the normal and emergency offsite power sources) could be available may be uncertain or not timely for various reasons.

• Parties include the MCR crew, field operators, and maintenance personnel. The Control Room Supervisor (CRS) is the key decisionmaker, the Shift Manager (SM) declares the emergency, and the offsite emergency response officer (ERO) interacts with the CRS.

• Information may not be available timely.

• Training (drill) occurs every two years.

• Guidelines and procedures exist but require adaptation of judgment to the scenario.

Section B.2 Task diagram and identification of critical tasks The task diagram elucidates the tasks needed to achieve the action, the paths of the tasks, and interteam coordination needed to achieve the tasks. Critical tasks are those required to meet the success criteria of the action; failing any of the critical task would fail the action.

Task diagram Declaration of ELAP includes these tasks:

• Identify the conditions corresponding to an ELAP event.

• Declare an ELAP event is on-going.

• Plan the various ELAP-related activities.

• Prioritize resources for recovering the EDG or performing load shed.

Critical tasks The key to the success of this action is declaring ELAP within one hour after the event. The critical task is:

HFE1-T1: Declare ELAP within one hour after the SBO HFE2 (Action 2): Fail to implement the FLEX generator Section B.1 HFE definition This section defines the HFE at a high level with basic information relevant to human performance of the action.

The action (deploying FLEX generator to charge essential batteries) is one of the few FLEX actions to be initiated right after ELAP is declared based on the ELAP procedure. The action ends at the 480VAC emergency buses being powered by the FLEX generator. The success criterion of the action is correctly operating FLEX generator to power the 480 VAC emergency buses before the depletion of dc power. The FLEX generator is loaded on a trailer in the C-25

FLEX storage building. It needs to be transported from the FLEX storage building to its operating location, staged, and connected properly. There are FSGs to guide the action.

The debris in the transportation route needs to be removed before the action can be performed. Removing debris is considered as a separate important human action because it is performed by a different group of people and affects the deployment of all FLEX equipment.

Section B.2 Task diagram and identification of critical tasks Task diagram The action begins at the ELAP cuing the operator to decide to deploy the FLEX generator and ends at the 480VAC emergency buses being powered by the FLEX generator. The cue of starting a FLEX generator deployment to power the 480VAC emergency buses is explicitly stated in the ELAP procedure. Possible reasons for not deploying a FLEX generator are not having sufficient manpower or the FLEX generator transportation routes or set up is not accessible. None of these reasons exist in this analysis.

Deploying the FLEX generator to power the 480 VAC emergency buses starts with the MCR giving the order to the OSC manager to deploy a team to implement the order. The field crew needs to communicate with the MCR crew to specify the generator operating locations, the location to connect the FLEX generator to charge the dc power, and to align the emergency buses to be powered by the FLEX generator from the specified location. Prior to deploying the FLEX generator, the SBO procedure instructs the crew to assess the FLEX equipment deployment location, and if needed, remove the debris in the equipment transportation routes.

Performing FLEX location assessment and debris removal is modeled separately.

Critical tasks The action is broken down into these three critical tasks because they are performed at different locations, by different groups of personnel, with different procedures.

The critical tasks for this action are the following:

HFE2-T1: Transporting and staging the generator HFE2-T2: Connecting the generator to the emergency buses (including alignment of the buses)

HFE2-T3: Operating the generator.

Worksheet C. Modeling of the critical tasks of an important human action Worksheet C may have multiple tables, one for each critical task of the HFE Critical Task T1 of HFE1 Critical task HFE1-T1: Declare ELAP - The goal is to make the decision of declaring ELAP within 60 mins of the initiating event.

Section C.1. Analysis of cognitive activities and identification of applicable CFMs Analysis of cognitive activities are based on the macrocognitive functions.

Cognitive activities Decisionmaking - The declaration of an ELAP event is made under the uncertainties of when the ac power (emergency diesel generator, emergency offsite power and the offsite power from the power grid) will be available.

C-26

Applicable CFMs HFE1-T1-CFM3: Failure of Decisionmaking Section C.2 Task characterization Requirement - the decision is based on the judgment of adequate confidence that the ac power is not expected to be restored within one hour after the event.

Cue - The cue to start the task is explicitly stated in the SBO procedure.

Information - Information needed is the status of the ac power recovery. The information may come from multiple sources.

Personnel - The field operators provide the prospect of the recovery of various ac power sources to the MCR crew, specifically the shift supervisor, to declare an ELAP event.

Procedure - The SBO procedure provides instruction to declare an ELAP event.

Competing goals or strategies - wait for any ac power sources to become available.

Section C.3 Assessment of PIFs Assessment of PIFs is to select the applicable PIF attributes from the PIF tables in Appendix B.

The following PIFs are relevant to the tasks and should be assessed for applicable attributes:

Information availability: Inf1 - Information is moderately incomplete - a small portion of key information is missing.

Task complexity: C25 - Competing or conflicting goals (e.g., choosing one goal will block achieving another goal, Low preference for correct strategy, Reluctance & Viable Alternative)

Critical Task T1 of HFE2 Critical task HFE2-T1: Transporting and staging the generator - the task goal is to transport the generator from the FLEX equipment building to the specified location and stage it properly Section C.1. Analysis of cognitive activities and identification of applicable CFMs Analysis of cognitive activities are based on the macrocognitive functions.

Cognitive activities Action execution - Transport and stage the generator to the pre-designated location Applicable CFMs HFE2-T1-CFM4: Failure of Execution Section C.2 Task characterization Specific Requirement: Need to be able to communicate with the MCR.

Cue and Supporting Information: A procedure-instructed task Procedure: FSG Personnel: Non-licensed personnel Task Support: None Location: From the FLEX equipment building to the specified onsite generator location.

Cognitive Activity: Action execution Concurrent Tasks: No concurrent task for the personnel performing the transportation Interteam coordination consideration: Coordination with MCR and TSC on clearing the FLEX generator transportation route and staging the generator.

Section C.3 Assessment of PIFs Assessment of PIFs is to select the applicable PIF attributes from the PIF tables.

C-27

Assessment of PIFs is performed only on the PIFs relevant to the event as determined in Section A.4. The following PIFs were identified as relevant to the action and should be assessed for specific tasks and CFMs:

Scenario familiarity: SF3 - Scenario is unfamiliar, rarely performed Justification: Implementation of the FLEX strategy and equipment is rarely performed. Non-planned situations are likely to occur in a beyond-design-basis event.

Task complexity: C38 - Action execution requires close coordination of multiple personnel at different locations Environmental factors - ENV11 - Strong winds, rain, and partial flooding impeding vehicle movement Teamwork factors: TF1 - Inadequate team, e.g., poor team cohesion. The team members to deploy the FLEX generator is formed based on the available personnel. They have likely not been trained as a team before the event of analysis.

Critical Task T2 of HFE2 Critical task HFE2-T2: Connect the generator - The goal is to power the 480 VAC emergency buses Section C.1. Analysis of cognitive activities and identification of applicable CFMs Analysis of cognitive activities are based on the macrocognitive functions.

Cognitive activities Action execution - Aligning the buses and connecting cables as specified in the procedure Applicable CFMs HFE2-T2-CFM4: Failure of Execution Section C.2 Task characterization Specific Requirement Certain steps of aligning the buses and connecting cables have to be performed in the exact order as specified in the FSG.

Cue and Supporting Information The OSC specifies the individuals to perform the task.

Procedure FSGs; the FSGs instructions are clear.

Personnel Field operators Task Support Coordinate the MCR operators to align the 480 VAC emergency buses to the FLEX generator Location Onsite building or shelter Concurrent Tasks A team dedicated to the task Interteam coordination Consideration Based on the available onsite field operators, the team members may not have previously worked together for this type of task Section C.3 Assessment of PIFs Assessment of PIFs is to select the applicable PIF attributes from the PIF tables.

Scenario familiarity: SF3 - Scenario is unfamiliar, rarely performed. Justification:

Implementation of the FLEX strategy and equipment is rarely performed. Non-planned situations are likely to occur in a beyond-design-basis event.

Task complexity:

C32 - Non-straightforward Procedure execution C37 - Complicated or ambiguous execution criteria; Restrictive, irreversible order of multiple steps Training: TE4 - Inadequate amount of training - no requalification exam on FLEX actions specified in the plants Systematic Approach to Training. Staff are under-trained for the types C-28

of actions. This is a once-in-a-lifetime event. The site does not emphasize training on this type of event as much as responding to more frequent events.

Teamwork and organizational factors: TF2 - Poor command & control - Inadequate coordination between site personnel and decision-makers (e.g., adapt or modify planned actions based on site situation). The FLEX generator team needs to communicate with the MCR to align the emergency buses to be powered by the FLEX generator and the MCR and TSC to know the debris removal status and details in setting up the FLEX generator.

Communication can be difficult because of unfamiliar communication protocol or less than adequate common mental models of the various parties. In addition, the communication could be challenged by the unavailability of AC powered communication equipment.

Procedures, guidance, and instructions: PG3 - Procedure lacks details. The procedure for aligning buses and connecting the generator may not have adequate detail. The specifications on some steps may not match the situation.

Critical Task T3 of HFE2 Critical task HFE2-T3: Operate the generator - The goal is to start and run the generator to power the 480 VAC emergency buses until the recovery of the stationed ac power or the plant reaches to a safe and stable state.

Section C.1. Analysis of cognitive activities and identification of applicable CFMs Analysis of cognitive activities are based on the macrocognitive functions.

Cognitive activities Action execution -Personnel need to start the generator and to open/close breakers as required during the operation.

Applicable CFMs HFE2-T3-CFM4: Failure of Execution Section C.2 Task characterization Specific Requirement Starting and running the generator to power the 480 VAC emergency buses may require opening/closing certain breakers Cue FSG instructed task Procedure FSG Personnel Field operators Location Onsite building or shelter Concurrent Tasks Powering the 480 VAC emergency buses requires continuous monitoring. The personnel performing this task may be assigned to other concurrent tasks.

Section C.3 Assessment of PIFs Assessment of PIFs is to select the applicable PIF attributes from the PIF tables.

• Training: TE2 - Inadequate training practicality. Staff are infrequently trained to perform the actions. This is a once-in-a-lifetime event. The site does not emphasize training on starting and operating the generator as much as responding to more frequent events. Starting and operating a FLEX generator may require manipulations that are different from those for normal diesel generators. (Note: This is based on the assumption in 2018 that FLEX equipment is not included in the plants implementation of the Maintenance Rule. Had the equipment been included in the Maintenance Rule implementation, the generator would have been periodically tested for maintenance C-29

purposes and the equipment operators would start and run the generator in testing. In that case the Training PIF would be considered as No Impact.)

• Multitasking, interruption, and distraction: MT2 - Moderate interruptions are taking away from the main task. Personnel monitoring the status for continuous operation of the generator likely have other main tasks (Note: This assessment is hypothetical and HRA analysts should identify the potential other tasks that may cause MT2).

Worksheet D - HEP estimation Worksheet D documents the HFEs, critical tasks, CFMs, and applicable PIF attributes for HEP estimation. The HEP of every CFM can be calculated using the HEP model in Step 5 of IDHEAS-ECA process and the values in the PIF tables of Appendix B.

HFE1: Fail to declare ELAP Critical Task Applicable CFMs Applicable PIF attribute T1: Declare T1-CFM3: failure of Information availability: Inf1 - Information is ELAP Decisionmaking moderately incomplete.

Task complexity: C25 - Competing or conflicting goals HFE2: Fail to use FLEX generator T1 - Transport T1-CFM4: failure of Scenario familiarity: SF3 - Scenario is Execution unfamiliar, rarely performed Environmental factors - ENV11 - Strong winds, rain, and partial flooding Teamwork factors: TF1 - Inadequate team T2 - Connect T2-CFM4: failure of Scenario familiarity: SF3 - Scenario is Understanding unfamiliar, rarely performed Task complexity: C32 - Non-straightforward Procedure execution Task complexity: C37 - Complicated or ambiguous execution criteria Training: TE4 - Inadequate amount of training Teamwork and organizational factors: TF2 -

Inadequate coordination between site personnel and decision-makers Procedures, guidance, and instructions:

PG3 - Procedure lacks details.

T3 - Operate T3-CFM4: failure of Training: TE2 - Inadequate training Execution practicality.

Multitasking, interruption, and distraction:

MT2 - Moderate interruptions are taking away from the main task.

Worksheet E: Time uncertainty analysis of the HFEs Worksheet E documents the estimation of time available and time needed for every HFE. The distribution of the time can be estimated as a single number, the mean and standard deviation C-30

(SD) by assuming a normal distribution, or a five-point estimation of probability distribution (at 5th, 25th, 50th, 75th, and 95th percentile). Pc, the HEP from time uncertainties, then can be calculated based on the estimation.

HFE Time available Time needed HFE1 60 minutes Making the decision of declaring ELAP can be (Fail to declare done quickly with checking the status of power ELAP within an and the time after SBO. However, postponing hour) the decision to be confident that the power cannot be restored can take any time from one to 59 minutes. Thus, we make a five-point probability distribution estimation 5th -

25th -

50th -

75th -

95th -

HFE2 4 hours4.62963e-5 days <br />0.00111 hours <br />6.613757e-6 weeks <br />1.522e-6 months <br /> Transport and stage: 45 +/- 15 mins (Fail to deploy Connect: 30 +/- 15 mins FLEX generator)

C-31

Appendix D INTRODUCTION TO THE IDHEAS-ECA SOFTWARE Performing a human reliability analysis (HRA) with IDHEAS-ECA (Integrated Human Event Analysis System - Event and Condition Assessment) has eight steps:

Step 1. Analyze the event scenario. This includes defining the scenario being analyzed, developing the operational narrative, determining the scenario context, and identifying the human failure events (HFEs) to be modeled.

Step 2. Analyze the HFE. This includes defining the HFE, analyzing the tasks in the HFE with a task diagram and/or timeline, and identifying critical tasks for human error probability (HEP) quantification.

Step 3. Model the failure of the critical tasks in the HFE. This includes characterizing the critical tasks, identifying cognitive activities required to achieve the tasks and subsequently identifying cognitive failure modes (CFMs) applicable to the tasks.

Step 4. Assess the PIFs applicable to every CFM. This step uses the results of the scenario context, HFE definition, and task characterization to select the applicable PIF attributes for every CFM.

Step 5. Calculate of an HFE. is the probabilistic sum of the HEPs of all the CFMs of the critical tasks.

Step 6. Analyze and quantify of an HFE. is the HEP attributed to uncertainty in time available and time needed for performing the HFE.

Step 7. Calculate the overall HEP of an HFE by probabilistically adding and Step 8. Analyze uncertainties in the HRA results and perform sensitivity analysis as needed.

The purpose of the IDHEAS-ECA software is to facilitate the implementation of Steps 4-8 above with the goal of calculating the human error probabilities (HEPs) of HFEs.

Before using the software, HRA analysts should perform Steps 1-3 and document the results in IDHEAS-ECA Worksheets A-C. The results of Steps 1-3 include:

• Operational narrative, scenario context, and list of HFEs

• HFE definition, task diagrams, and list of critical tasks

• Task characterization and applicable CFMs Once the above three steps are performed, analysts use the software to calculate the HEP of an HFE. Figure D-1 shows the structure implemented in the software to calculate the HEP. On the top of the structure is the HFE of which the HEP is to be calculated. The HEP of an HFE is the probabilistic sum of two parts, (time) and (cognition), as shown in the level below the HFE.

is the HEP attributed to the uncertainty in time available and time needed to complete the action. is the HEP attributed to cognitive failures assuming that the time available for performing the human action of the HFE is adequate. of an HFE is the probabilistic sum of the HEPs of the critical tasks in the HFE. The critical tasks are shown in Figure D-1, one level below . The HEP of a critical task is the probabilistic sum of the cognitive failure modes (CFMs) applicable to the critical task. The five CFMs in IDHEAS-ECA are shown in Figure D-1, one level below the critical tasks. The CFMs are the failures of five macrocognitive functions, namely Detection, Understanding, Decisionmaking, Action Execution, and Interteam coordination. Finally, the HEP of a CFM is determined by the PIF attributes applicable to the CFM.

D-1

HFE Time (Pt) Cognition (Pc)

Critical Critical Critical Task Task Task Action Interteam Detection Understanding Decisionmaking Execution coordination Attributes of Performance Influencing Factors A switch that the users can open and close Figure D-1 The structure of calculating a human failure events human error probability in the IDHEAS-ECA software The software is used to calculate the HEP of an HFE and to document the calculation. An analysis includes three main functions: calculating , calculating , and documentation. The analysis begins with an HFE and generates , , and the overall HEP of the HFE as the outputs. For each HFE analysis, the software provides the following three functions:

1) Calculating  : The analyst specifies the critical tasks and their applicable CFMs, then selects applicable PIFs and PIF attributes for every CFM to calculate .

2) Calculating  : The analyst enters the parameters of the distributions of time available and time required of the HFE for the software to calculate .

3) Documentation: All parameters that the analyst entered to calculate and and the other relevant information are documented in a rich text file to be integrated in the overall analysis document.

The above three functions are discussed in detail below.

Calculate is a function of the PIF attributes, CFMs, and critical tasks applicable to the HFE of analysis.

Twenty PIFs are used to group the PIF attributes. The PIFs and their PIF attributes differ between the CFMs. The analysts responsibilities in calculating include specifying critical tasks, specifying the applicable CFMs for each specified critical task, and specifying the PIF attributes applicable to the analysis for each specified CFM. The software provides a graphical user interface for the analyst to perform these tasks. The PIF (and PIF attributes) have two different types of impacts on HEPs: Base HEP and PIF weights. In the software, the PIF attributes affecting the base HEP are displayed in red text; and the PIF attributes contributing to the PIF weights are displayed in black text.

In IDHEAS-ECA, most PIF attributes statuses are modeled as binary states, i.e., present and not-present. For these PIF attributes, the analyst simply checks or unchecks a PIF attribute to represent the present and not-present state of the PIF attribute, respectively. Some PIF attributes statuses cover a wide spectrum. The impacts on HEP between the two ends of the spectrum is significant. IDHEAS-ECA uses multiple discrete states to represent the possible statuses. In the software, an attribute scale between 1 and 10 is provided for the analysts to D-2

specify the appropriate status. Anchor values with corresponding status descriptions are provided in the software to assist the analyst in determining the appropriate status of the PIF attribute.

To calculate , the analyst first needs to specify the number of critical tasks and the applicable CFMs of each critical task. The software interface provides checkboxes for the analyst to include and exclude CFMs by checking and unchecking the corresponding boxes respectively.

Next, the analyst identifies the applicable PIFs and their attributes for each CFM of the critical task being analyzed. The software provides a set of five radio buttons for the analyst to switch to different sets of PIFs and attributes relevant to each of the five cognitive failure modes (Detection, Understanding, Decisionmaking, Action Execution, and Interteam coordination).

The PIFs and attributes are presented using a tree structure (PIF tree) with two levels. The first level shows all PIFs relevant to a CFM, and the second level shows the PIF attributes relevant to the CFM. Each PIF attribute has a checkbox for the analyst to assign its presence or absence with respect to its impact on the HEP of the CFM. Every time a PIF attribute is checked or unchecked, the Software immediately recalculates the resulting HEPs and updates the displays accordingly. The following are the affected items whose displays are updated by the software:

• The PIF attributes checkbox

• The list of PIF attributes checked (by the analysts)

• The checked PIF attributes shown in the CFM panels

• The HEPs of the cognitive failure mode, the critical task, the Pc (sum of all critical tasks HEPs), and the HFE.

Table D-1 summarizes the analysts operation, software responses to the analysts operation, and software displays (graphical user interfaces) to implement the analysts operation to calculate Pc.

D-3

Table D-1 The Operation, Calculation, and Display for Calculation of Pc Analyst Operation Software Responses Display 1.1 Enter a Critical The software displays the following to Task implement the analysts operation:

1.2 Select an 1.1 Three critical tasks and checkboxes to applicable CFM include and exclude the critical tasks 1.3 Select an 1.4&1.5 Responding 1.2 Five CFMs for each critical task, and applicable PIF to the analysts checkboxes to include and exclude the 1.4 Check all the selection of the CFMs applicable applicable PIF 1.3 Radio buttons to select PIFs applicable attributes of the attributes, the to a CFM and a PIF tree to display the PIF software calculates PIFs 1.5 Assess and select all relevant HEPs and 1.4 The PIF tree (in 1.3 above) shows the the scale of every updates their PIFs and their attributes. Each attribute multi-scale displays. has a checkbox to include and exclude a attribute. PIF attribute in the analysis.

1.5 A pop-up window with a numeric up-down control for the analyst to specify the PIF attributes status in a scale from 1 to 10. The anchor values and corresponding status descriptions are provided.

Repeat 1.3 for all the Every time the applicable PIFs analysts actions Repeat 1.2 for all the affect HEP, all CFMs of a critical task relevant HEPs are Repeat 1.1 for all the recalculated and critical tasks displays are updated.

Calculate Pt An HFE has one and only one Pt, i.e., Pt is performed for the whole HFE. The analyst estimates the probabilistic distributions of time required and time available for performing the HFE. The software calculates Pt by the convolution of the two distributions. The software offers five options for the distribution of time available. Those are: Normal distribution, Gamma distribution, Weibull distribution, Five-point estimation, and Constant (Single-point). The software offers the first four options for both the time available for the time required curves. The software excludes the Constant option for time required, because the time required should never be a single point. With both distributions specified, the software calculates the Pt and displays the time available and time required distributions after the analysts click the Plot and Update Pt button in the software.

To calculate Pt, the analyst specifies the distributions of the time required and time available for the HFE. The IDHEAS-ECA software provides various options for analysts to model the distributions of time required and time available. The options for time required distribution include:

• Normal distribution (specifying the mean and standard deviation parameters)

• Gamma distribution (specifying the alpha and beta parameters)

• Weibull distribution (specifying the shape and scale parameters)

D-4

• Percentile distribution (specifying the 5th, 25th, 50th, 75th, and 95th percentiles)

The option for time-available distribution include:

• Normal distribution (specifying the mean and standard deviation parameters)

• Gamma distribution (specifying the alpha and beta parameters)

• Weibull distribution (specifying the shape and scale parameters)

• Percentile distribution (specifying the 5th, 25th, 50th, 75th, and 95th percentiles)

• Constant (specifying a single value)

Once the time-required and time-available distributions are specified, by clicking the Plot and Update Pt button in the software, the two distributions are plotted on a display area and the Pt is calculated. The HEP of the HFE is also updated along with the update of Pt. The software provides options for specifying the time unit used in the analysis, including seconds, minutes, hours, and days.

The software uses Monte Carlo sampling techniques to calculate Pt. Monte Carlo sampling is used to calculate Pt of all other combinations. The sampling size for time required and time available is one million each. Every time an analyst clicks the Plot and Update Pt button, Monte Carlo sampling is executed. For certain distribution combinations, one million samples may produce slightly different Pt results each time. The differences are considered to have negligible effects on the HEP.

Table D-2 summarizes the analysts operation, software responses to the analysts operation, and relevant software displays to implement the analysts operation to calculate Pt.

Table D-2 The Operation, Calculation, and Display for Calculation of Pt Software Analyst Operation Display Responses 2.1 Select the option for the time-available distribution 2.5 The software Software displays 2.2 Estimate and enter the parameters for the plots the two the distribution distribution distributions, options and fields 2.3 Select an option for the time-required distribution calculates Pt, to enter 2.4 Estimate and enter the parameters for the and updates the parameters for distribution corresponding time-available 2.5 click the Plot and Update Pt button displays. and time-required.

Documentation The Documentation tab supports the document of the HRA analysis and results. After the analyst completes the HEP calculation for an HFE, the software provides two options for documentation. The first option is to generate a document in rich text format that has all parameters specified to calculate the HFEs HEP, the HEPs (i.e., HEPs of CFM, critical tasks, Pc, Pt, and HFE), and the other relevant information (e.g., HEP impact of each PF attribute).

This option provides a convenient way for the analyst to integrate to the information into the final analysis report. The second option is to save the analysis in a file that can be used to reanalyze the event at a later time or be shared with other analysts. The software allows for the analyst to retrieve a saved analysis from a file and to have all the parameters and displays identical to the saved analysis.

D-5