Text

NUREG-2198, the General Methodology of an Integrated Human Event Analysis System (IDHEAS-G)
	ML21127A272
Person / Time
Issue date:	05/31/2021
From:	Segarra J; Office of Nuclear Regulatory Research
To:	;
	Malone Tina
References
	NUREG-2198
	Download: ML21127A272 (372)
	v • d • e

NUREG-2198 The General Methodology of An Integrated Human Event Analysis System (IDHEAS-G)

Office of Nuclear Regulatory Research

AVAILABILITY OF REFERENCE MATERIALS IN NRC PUBLICATIONS NRC Reference Material Non-NRC Reference Material As of November 1999, you may electronically access Documents available from public and special technical NUREG-series publications and other NRC records at the libraries include all open literature items, such as books, NRCs Library at www.nrc.gov/reading-rm.html. Publicly journal articles, transactions, Federal Register notices, released records include, to name a few, NUREG-series Federal and State legislation, and congressional reports.

publications; Federal Register notices; applicant, licensee, Such documents as theses, dissertations, foreign reports and vendor documents and correspondence; NRC and translations, and non-NRC conference proceedings correspondence and internal memoranda; bulletins and may be purchased from their sponsoring organization.

information notices; inspection and investigative reports; licensee event reports; and Commission papers and their Copies of industry codes and standards used in a attachments. substantive manner in the NRC regulatory process are maintained at NRC publications in the NUREG series, NRC regulations, The NRC Technical Library and Title 10, Energy, in the Code of Federal Regulations Two White Flint North may also be purchased from one of these two sources: 11545 Rockville Pike Rockville, MD 20852-2738

1. The Superintendent of Documents U.S. Government Publishing Office These standards are available in the library for reference Washington, DC 20402-0001 use by the public. Codes and standards are usually Internet: www.bookstore.gpo.gov copyrighted and may be purchased from the originating Telephone: (202) 512-1800 organization or, if they are American National Standards, Fax: (202) 512-2104 from American National Standards Institute

2. The National Technical Information Service 11 West 42nd Street 5301 Shawnee Road New York, NY 10036-8002 Alexandria, VA 22312-0002 Internet: www.ansi.org Internet: www.ntis.gov (212) 642-4900 1-800-553-6847 or, locally, (703) 605-6000 Legally binding regulatory requirements are stated only in A single copy of each NRC draft report for comment is laws; NRC regulations; licenses, including technical available free, to the extent of supply, upon written specifications; or orders, not in NUREG-series publications.

The views expressed in contractor prepared publications in request as follows:

this series are not necessarily those of the NRC.

Address: U.S. Nuclear Regulatory Commission The NUREG series comprises (1) technical and Office of Administration administrative reports and books prepared by the staff (NUREG-XXXX) or agency contractors (NUREG/CR-XXXX),

Division of Resource Management & Analysis (2) proceedings of conferences (NUREG/CP-XXXX),

Washington, DC 20555-0001 (3) reports resulting from international agreements E-mail: distribution.resource@nrc.gov (NUREG/IA-XXXX),(4) brochures (NUREG/BR-XXXX), and Facsimile: (301) 415-2289 (5) compilations of legal decisions and orders of the Commission and the Atomic and Safety Licensing Boards and of Directors decisions under Section 2.206 of the NRCs regulations (NUREG-0750).

Some publications in the NUREG series that are posted at the NRCs Web site address www.nrc.gov/reading-rm/ DISCLAIMER: This report was prepared as an account doc-collections/nuregs are updated periodically and may of work sponsored by an agency of the U.S. Government.

differ from the last printed version. Although references to Neither the U.S. Government nor any agency thereof, nor any employee, makes any warranty, expressed or implied, material found on a Web site bear the date the material or assumes any legal liability or responsibility for any third was accessed, the material available on the date cited partys use, or the results of such use, of any information, may subsequently be removed from the site. apparatus, product, or process disclosed in this publication, or represents that its use by such third party would not infringe privately owned rights.

NUREG-2198 The General Methodology of An Integrated Human Event Analysis System (IDHEAS-G)

Manuscript Completed: November 2020 Date Published: May 2021 Prepared by:

J. Xing Y. J. Chang J. DeJesus Segarra Office of Nuclear Regulatory Research

ABSTRACT This report describes a human reliability analysis (HRA) methodology developed by the U.S.

Nuclear Regulatory Commission (NRC) staff, the General Methodology of an Integrated Human Event Analysis System (IDHEAS-G). IDHEAS-G was developed in response to the staff requirements memorandum (SRM) M061020, dated November 8, 2006, to the Advisory Committee on Reactor Safeguards (ACRS). The SRM directed the ACRS to, work with the

[NRC] staff and external stakeholders to evaluate different human reliability models in an effort to propose a single model for agency use or guidance on which model(s) should be used in specific circumstances. IDHEAS-G is intended to be a human-centered, general methodology used to develop application-specific HRA methods by the NRC. It integrates the strengths of existing HRA methods and enhances HRA in: (1) application scope, (2) scientific basis, (3) HRA variability, and (4) data for HRA. An example of the use of the IDHEAS-G framework is the development of Research Information Letter 2020 IDHEAS for Event and Condition Assessment (IDHEAS-ECA) and its associated software tool. IDHEAS-ECA has proven to be a useful HRA method for supporting the NRCs risk-informed decisionmaking processes.

iii

TABLE OF CONTENTS ABSTRACT ................................................................................................................... iii LIST OF FIGURES......................................................................................................... ix LIST OF TABLES .......................................................................................................... xi EXECUTIVE

SUMMARY

.............................................................................................. xv ACKNOWLEDGMENTS .............................................................................................. xxi ABBREVIATIONS AND ACRONYMS ....................................................................... xxiii 1 INTRODUCTION ..................................................................................................... 1-1 1.1 Purpose of the Report .................................................................................................... 1-1 1.2 Background .................................................................................................................... 1-1 1.3 Strategic Approach to Human Reliability Analysis Method Development ...................... 1-3 1.4 Overview of IDHEAS-G .................................................................................................. 1-4 1.5 Organization of the Report ............................................................................................. 1-6 1.6 Perspectives on the Development of the IDHEAS-G Human Reliability Analysis Method ........................................................................................................................... 1-8 2 COGNITION MODELCOGNITIVE BASIS STRUCTURE..................................... 2-1 2.1 Overview of the Cognition Model for Human Performance and Reliability..................... 2-1 2.2 Overview of the Cognitive Basis Structure ..................................................................... 2-2 2.3 Macrocognitive Functions .............................................................................................. 2-6 2.3.1 Detection ............................................................................................................ 2-6 2.3.2 Understanding .................................................................................................. 2-10 2.3.3 Decisionmaking ................................................................................................ 2-13 2.3.4 Action Execution ............................................................................................... 2-18 2.3.5 Interteam Coordination ..................................................................................... 2-24 2.4 An Example of Macrocognitive Functions across Distributed Teams .......................... 2-27 2.5 Failure of Human Actions ............................................................................................. 2-28 2.6 Summary ...................................................................................................................... 2-29 3 COGNITION MODELPERFORMANCE-INFLUENCING FACTOR STRUCTURE ........................................................................................................... 3-1 3.1 Modeling the Context of Important Human Actions ........................................................ 3-1 3.2 Performance-Influencing Factor Structure ..................................................................... 3-3 3.2.1 Overview ............................................................................................................ 3-3 3.3 Details of the Performance-Influencing Factor Structure ............................................... 3-6 3.3.1 Environment- and Situation-Related Performance-Influencing Factors ............. 3-6 3.3.2 System-Related Performance-Influencing Factors ........................................... 3-13 3.3.3 Personnel-Related Performance-Influencing Factors....................................... 3-16 3.3.4 Task-Related Performance-Influencing Factors ............................................... 3-26 v

3.4 Links between Performance-Influencing Factor Attributes and Cognitive Mechanisms ................................................................................................................. 3-35 3.5 Effects of Performance-Influencing Factor Attributes on Macrocognitive Functions .... 3-36 3.5.1 Assessing Performance-Influencing Factor States........................................... 3-36 3.5.2 Quantifying the Effects of Performance-Influencing Factors ............................ 3-37 3.5.3 Quantifying the Effects of Multiple Performance-Influencing Factors ............... 3-37 3.6 Summary ...................................................................................................................... 3-38 4 AN INTEGRATED PROCESS FOR HUMAN RELIABILITY ANALYSIS WITH IDHEAS-G ............................................................................................................... 4-1 4.1 Overview of the IDHEAS-G Human Reliability Analysis Process................................... 4-1 4.1.1 General Human Reliability Analysis Process ..................................................... 4-1 4.1.2 IDHEAS-G Human Reliability Analysis Process ................................................. 4-2 4.2 Stage 1: Scenario Analysis ........................................................................................... 4-4 4.2.1 Overview of Scenario Analysis ........................................................................... 4-4 4.2.2 Development of Operational Narrative ............................................................... 4-5 4.2.3 Identification of Event Context ............................................................................ 4-9 4.2.4 Identification and Definition of Important Human Actions ................................. 4-14 4.2.5 Summary of Stage 1 Analysis .......................................................................... 4-15 4.3 Stage 2: Modeling of Important Human Actions .......................................................... 4-15 4.3.1 Overview of Modeling Important Human Actions ............................................. 4-15 4.3.2 Task Analysis ................................................................................................... 4-16 4.3.3 Representation of Task Failure with Cognitive Failure Modes ......................... 4-20 4.3.4 Representation of Important Human Action Context with Performance-Influencing Factors ........................................................................................... 4-28 4.3.5 Summary of Stage 2 Analysis .......................................................................... 4-30 4.4 Stage 3: Estimation of the Human Error Probability of an Important Human Action ........................................................................................................................... 4-30 4.4.1 Overview of Human Error Probability Estimation in IDHEAS-G ....................... 4-30 4.4.2 Estimation of Time Required for the Important Human Action ......................... 4-31 4.4.3 Estimation of the Error Probability of a Cognitive Failure Mode ....................... 4-32 4.4.4 Documentation and Communication of the Assumptions Made for Estimating Human Error Probabilities or Parameters in a Quantification Model ................................................................................................................ 4-37 4.5 Stage 4: Performing Integrative Analysis .................................................................... 4-37 4.6 Summary of the IDHEAS-G Human Reliability Analysis Process ................................ 4-38 5 TIME UNCERTAINTY ANALYSIS ........................................................................... 5-1 5.1 Time Uncertainty Model ................................................................................................. 5-1 5.2 Guidance on Estimating the Distribution of Time Available ........................................... 5-3 5.3 Guidance on Estimating the Distribution of Time Required ........................................... 5-4 5.3.1 Estimation of Time Required .............................................................................. 5-6 5.4 Summary ...................................................................................................................... 5-10 6 GENERALIZATION OF HUMAN ERROR DATA FOR ESTIMATION OF HUMAN ERROR PROBABILITIES ......................................................................... 6-1 6.1 Human Error Data .......................................................................................................... 6-1 6.2 Data Generalization and Integration .............................................................................. 6-5 6.2.1 HEP Table .......................................................................................................... 6-6 vi

6.2.2 PIF Impact Table ................................................................................................ 6-7 6.2.3 PIF Interaction Table .......................................................................................... 6-8 6.3 Demonstration of Human Error Data Generalization and Integration........................... 6-10 6.3.1 Generalization of Data to the HEP Table to Inform the Base Human Error Probabilities ...................................................................................................... 6-10 6.3.2 Mapping between SACADA Database and IDHEAS-G.................................... 6-12 6.4 Summary ...................................................................................................................... 6-21 7 GENERAL DISCUSSION AND COMMENTS.......................................................... 7-1 7.1 Areas for Human Reliability Analysis Method Enhancement ......................................... 7-1 7.1.1 Application Scope ............................................................................................... 7-1 7.1.2 Scientific Basis ................................................................................................... 7-1 7.1.3 Human Reliability Analysis Variability................................................................. 7-3 7.1.4 Data in Human Reliability Analysis ..................................................................... 7-4 7.1.5 IDHEAS-G Human Error Probability Quantification Approaches........................ 7-4 7.2 Areas that Need Further Research in IDHEAS-G .......................................................... 7-4 7.2.1 Validation of IDHEAS-G HEP Quantification Model ........................................... 7-4 7.2.2 Guidance on Combined Effect of Multiple Performance-Influencing Factors ............................................................................................................... 7-5 7.2.3 Treatment of Errors of Commission.................................................................... 7-5 7.2.4 Dependency between Important Human Actions ............................................... 7-5 7.2.5 Potential Variability in Application-Specific Quantification Models ..................... 7-6 7.2.6 Updating the Basic Quantification StructureRefining Cognitive Failure Modes and Performance-Influencing Factors..................................................... 7-7 7.2.7 Definition of Critical Tasks .................................................................................. 7-7 7.3 Common Human Reliability Analysis Practices Not Included in IDHEAS-G .................. 7-7 8 REFERENCES ........................................................................................................ 8-1 APPENDIX A COGNITIVE MECHANISMS UNDERLYING HUMAN PERFORMANCE AND RELIABILITY ................................................ A-1 APPENDIX B LINKS OF PERFORMANCE-INFLUENCING FACTOR ATTRIBUTES TO COGNITIVE MECHANISMS................................. B-1 APPENDIX C INSIGHTS INTO PERFORMANCE-INFLUENCING FACTORS FROM THE COGNITIVE LITERATURE ............................................ C-1 APPENDIX D COGNITIVE BASIS FOR THE COMBINED EFFECT OF PERFORMANCE-INFLUENCING FACTORS ON HUMAN ERROR PROBABILITIES ................................................................................ D-1 APPENDIX E SCENARIO ANALYSIS ...................................................................... E-1 APPENDIX F IDENTIFICATION AND DEFINITION OF IMPORTANT HUMAN ACTIONS .............................................................................................F-1 APPENDIX G TASK ANALYSIS ............................................................................... G-1 APPENDIX H IDENTIFICATION OF COGNITIVE FAILURE MODES ..................... H-1 vii

APPENDIX I ASSESSMENT OF PERFORMANCE-INFLUENCING FACTORS ..... I-1 APPENDIX J QUANTIFICATION OF HUMAN ERROR PROBABILITY ................. J-1 APPENDIX K IDHEAS-G TREATMENT OF DEPENDENCY BETWEEN HUMAN FAILURE EVENTS ............................................................................ K-1 APPENDIX L UNCERTAINTY ANALYSIS AND DOCUMENTATION ..................... L-1 APPENDIX M DEMONSTRATION OF THE IDHEAS-G HUMAN RELIABILITY ANALYSIS PROCESS ...................................................................... M-1 viii

LIST OF FIGURES Figure ES-1 Overview of the Cognitive Basis Structure and Performance-Influencing Factor Structure ................................................................................................... xv Figure ES-2 Illustration of the IDHEAS-G HRA Process..xviii Figure 1-1 Strategic Approach to HRA Method Development ............................................. 1-4 Figure 1-2 IDHEAS-G Diagram ........................................................................................... 1-5 Figure 2-1 The Cognition Model for Human Performance and Reliability............................ 2-1 Figure 2-2 Cognitive Basis Structure Representation of a Human Task ............................. 2-3 Figure 2-3 The Cognitive Basis Structure ............................................................................ 2-4 Figure 2-4 Components of the Detection Macrocognitive Function ..................................... 2-6 Figure 2-5 Components of the Understanding Macrocognitive Function ........................... 2-10 Figure 2-6 Components of the Decisionmaking Macrocognitive Function ......................... 2-13 Figure 2-7 Components of the Action Execution Macrocognitive Function ....................... 2-19 Figure 2-8 Components of the Interteam Coordination Macrocognitive Function.............. 2-24 Figure 2-9 Teamwork between Teams versus Interaction within a Coherent Team.......... 2-25 Figure 2-10 An Example of Macrocognitive Functions in Emergency Response Management .................................................................................................... 2-27 Figure 3-1 Overview of the PIF Structure ............................................................................ 3-2 Figure 3-2 Illustration of the PIF Structure ........................................................................... 3-4 Figure 3-3 Connection of the Cognitive Basis Structure and PIF Structure ......................... 3-5 Figure 4-1 General HRA Process ........................................................................................ 4-2 Figure 4-2 IDHEAS-G HRA Process ................................................................................... 4-3 Figure 4-3 Illustration of the IDHEAS-G HRA Process ........................................................ 4-3 Figure 4-4 Iterative Process for Scenario Analysis .............................................................. 4-5 Figure 4-5 Composition of an Event Operational Narrative ................................................. 4-6 Figure 4-6 Iterative Process for Modeling of Important Human Actions ............................ 4-16 Figure 4-7 IDHEAS-G Task Structure ................................................................................ 4-17 Figure 4-8 Overview of Process for HEP Estimation in IDHEAS-G ................................... 4-31 Figure 4-9 Illustration of Modeling the Effects of PIFs on HEPs ........................................ 4-34 Figure 4-10 Overview of the IDHEAS-G Process ................................................................ 4-39 Figure 5-1 Illustration of Pt as the Convolution of the Probabilistic Distribution Functions of Time Available and Time Required ................................................ 5-3 Figure 5-2 Timeline Illustration Diagram .............................................................................. 5-4 Figure 6-1 Illustration of IDHEAS-G Data Generalization and Integration ........................... 6-5 ix

Figure 6-2 IDHEAS-G Steps to Generalize Human Error Data ........................................... 6-6 Figure 6-3 Illustration of Human Error Data to Inform PIF Weights ..................................... 6-8 Figure C-1 The Effect of Detection Complexity on Information Detection ........................... C-2 Figure D-1 Illustration of Three Ways of PIF Combination .................................................. D-2 Figure E-1 Composition of an Event Operational Narrative .............................................. E-12 Figure F-1 A Master Logic Diagram Structure .....................................................................F-1 Figure F-2 Example of DC Availability in an ELAP Event Related to Human Error in Shedding the DC Load .......................................................................................F-6 Figure G-1 IDHEAS-G Task Structure ................................................................................. G-2 Figure G-2 Types of Task Relations Illustrated with Task Diagrams ................................... G-5 Figure G-3 An Example Task Diagram for the See-and-Flee Event.................................... G-6 Figure G-4 An Example Timeline for the See-and-Flee Event ............................................ G-7 Figure G-5 An Interteam Coordination Diagram in NPP Crisis Management [118] ............. G-8 Figure G-6 Generic Depiction of Operations in Response to an In-cabinet Aspirating Smoke Detection VEWFD (Very Early Warning Fire Detection Systems)

Alert Followed by Alarm where a De-energization Strategy is Being Used ..... G-12 Figure G-7 Crew Response Diagram for HFE-3 in the Example of NPP Fire Event ......... G-16 Figure G-8 Timeline for the Example HFE in the NPP Fire Event ..................................... G-18 Figure H-1 (a) Macrocognition Taxonomy and (b) Cognition Failure Taxonomy................. H-2 Figure J-1 Diagram of a Formal Expert Elicitation Process ................................................. J-7 Figure K-1 Overview of the IDHEAS-G Dependency Model ............................................... K-4 Figure K-2 Illustration of Dependency Adjustment of HEP Based on IDHEAS-G Context Structure .............................................................................................. K-9 Figure K-3 (a) Diagram for Modeling Hypothetically Independent HFEs. (b) Diagram for Modeling Dependent HFEs. ....................................................................... K-10 Figure M-1 A Simplified Electric Diagram ............................................................................ M-5 Figure M-2 PATH-1 Procedure on the Step of Checking RCP Thermal Barrier Cooling Water Hi/Lo Flow Alarm Illuminated. ................................................................. M-8 Figure M-3 The Critical Tasks and Their Completion Time in the H.B. Robinson Event ... M-15 Figure M-4 A PRA Fault Tree on RCP Seal Failure........................................................... M-17 Figure M-5 Task Diagram to Prevent RCP Seal Failure .................................................... M-19 Figure M-6 Task Diagram to Deploy a FLEX Generator .................................................... M-39 x

LIST OF TABLES Table ES-1 PIFs in IDHEAS-G ...............................................................................................xvi Table ES-2 IDHEAS-G HRA Process Stages and Stepsxviii Table 3-1 PIF Workplace Accessibility and Habitability ...................................................... 3-8 Table 3-2 PIF Workplace Visibility ...................................................................................... 3-9 Table 3-3 PIF Noise in Workplace .................................................................................... 3-10 Table 3-4 PIF Cold/Heat/Humidity .................................................................................... 3-11 Table 3-5 PIF Resistance to Physical Movement ............................................................. 3-12 Table 3-6 PIF System and I&C Transparency to Personnel ............................................. 3-13 Table 3-7 PIF Human-System Interface ........................................................................... 3-14 Table 3-8 PIF Tools and Parts Availability and Usability .................................................. 3-16 Table 3-9 PIF Staffing ....................................................................................................... 3-17 Table 3-10 PIF Procedures, Guidance, and Instructions .................................................... 3-18 Table 3-11 PIF Training ...................................................................................................... 3-20 Table 3-12 PIF Team and Organization Factors ................................................................ 3-21 Table 3-13 PIF Work Processes ......................................................................................... 3-25 Table 3-14 PIF Information Availability and Reliability........................................................ 3-26 Table 3-15 PIF Scenario Familiarity ................................................................................... 3-28 Table 3-16 PIF Multitasking, Interruptions, and Distractions .............................................. 3-29 Table 3-17 PIF Task Complexity ........................................................................................ 3-31 Table 3-18 PIF Mental Fatigue ........................................................................................... 3-33 Table 3-19 PIF Time Pressure and Stress ......................................................................... 3-34 Table 3-20 PIF Physical Demands ..................................................................................... 3-35 Table 4-1 IDHEAS-G HRA Process Stages and Steps ...................................................... 4-4 Table 4-2 Guideline Questions to Collect Information for the Operational Narrative .......... 4-8 Table 4-3 Task Characterization in Task Analysis............................................................ 4-19 Table 4-4 Taxonomy of Cognitive Activities...................................................................... 4-20 Table 4-5 Failure of Macrocognitive Functions as the High-Level CFMs ......................... 4-21 Table 4-6 Middle-Level CFMs .......................................................................................... 4-22 Table 4-7 Detection CFMs ................................................................................................ 4-23 Table 4-8 Understanding CFMs ....................................................................................... 4-24 Table 4-9 Decisionmaking CFMs ..................................................................................... 4-25 Table 4-10 Action Execution CFMs .................................................................................... 4-26 xi

Table 4-11 Interteam Coordination CFMs .......................................................................... 4-27 Table 4-12 Example Probing Questions to Assess CFM Applicability................................ 4-27 Table 4-13 IDHEAS-G PIFs ................................................................................................ 4-28 Table 4-14 Summary of IDHEAS-G .................................................................................... 4-40 Table 5-1 Typical Factors Contributing to Tr ....................................................................... 5-8 Table 5-2 Uncertainty Factors that Modify the Distribution of Tr ......................................... 5-9 Table 6-1 Summary of Example Meta-Analysis on PIF Combination ................................. 6-9 Table 6-2 An Example Datapoint for the HEP Table ........................................................ 6-11 Table 6-3 Example datapoint generalized for PIF Impact Table....................................... 6-12 Table 6-4 Mapping between SACADA Error Modes and IDHEAS-G CFMs..................... 6-13 Table 6-5 Mapping between SACADA Context Factors and IDHEAS-G PIF Structure ........................................................................................................... 6-16 Table 7-1 Example Categorizations of PIFs ....................................................................... 7-3 Table A-1 Cognitive Mechanisms for Detection ................................................................. A-1 Table A-2 Cognitive Mechanisms for Understanding ......................................................... A-1 Table A-3 Cognitive Mechanisms for Decisionmaking ....................................................... A-1 Table A-4 Cognitive Mechanisms for Action Execution ..................................................... A-2 Table A-5 Cognitive Mechanisms for Interteam Coordination............................................ A-2 Table B-1 Cognitive Mechanisms for PIF Workplace Accessibility and Habitability .......... B-1 Table B-2 Cognitive Mechanisms for PIF Workplace Visibility........................................... B-2 Table B-3 Cognitive Mechanisms for PIF Noise in Workplace and Communication Pathways ........................................................................................................... B-2 Table B-4 Cognitive Mechanisms for PIF Cold/Heat/Humidity........................................... B-2 Table B-5 Cognitive Mechanisms for PIF Resistance to Physical Movement.................... B-2 Table B-6 Cognitive Mechanisms for PIF System and I&C Transparency to Personnel .......................................................................................................... B-3 Table B-7 Cognitive Mechanisms for PIF Human-System Interface .................................. B-3 Table B-8 Cognitive Mechanisms for PIF Tools and Parts Availability and Usability ......... B-4 Table B-9 Cognitive Mechanisms for PIF Staffing ............................................................. B-4 Table B-10 Cognitive Mechanisms for PIF Procedures, Guidance, and Instructions .......... B-4 Table B-11 Cognitive Mechanisms for PIF Training............................................................. B-5 Table B-12 Cognitive Mechanisms for PIF Team and Organization Factors ....................... B-6 Table B-13 Cognitive Mechanisms for PIF Work Processes ............................................... B-6 Table B-14 Cognitive Mechanisms for PIF Information Availability and Reliability .............. B-7 Table B-15 Cognitive Mechanisms for PIF Scenario Familiarity .......................................... B-7 xii

Table B-16 Cognitive Mechanisms for PIF Multi-tasking, Interruptions, and Distractions ........................................................................................................ B-8 Table B-17 Cognitive Mechanisms for PIF Task Complexity ............................................... B-8 Table B-18 Cognitive Mechanisms for PIF Mental Fatigue .................................................. B-9 Table B-19 Cognitive Mechanisms for PIF Time Pressure and Stress .............................. B-10 Table B-20 Cognitive Mechanisms for PIF Physical Demands .......................................... B-10 Table C-1 Decisionmaking Error Variation with Information Accuracy ............................... C-1 Table C-2 Assessment Error Rate (%) ............................................................................... C-3 Table E-1 Narrative Information Coverage of a Safety Issue........................................... E-16 Table G-1 Task Characterization for HRA .......................................................................... G-9 Table G-2 Taxonomy of Cognitive Activities..................................................................... G-10 Table H-1 Failure of the Macrocognitive Functions as the High-Level CFMs .................... H-2 Table H-2 Detection CFMs ................................................................................................. H-5 Table H-3 Understanding CFMs ........................................................................................ H-6 Table H-4 Decisionmaking CFMs ...................................................................................... H-7 Table H-5 Action Execution CFMs ..................................................................................... H-8 Table H-6 Interteam Coordination CFMs ........................................................................... H-8 Table H-7 Reference Questions for Identifying Failures of the Macrocognitive Functions Applicable to a Critical Task............................................................ H-12 Table H-8 Example Reference Questions for Identifying Detailed CFMs in NPP At-Power Internal Event Applications (adapted from NUREG-2199, Vol. 1) ........ H-13 Table M-1 Scenario Timeline of the Fire Event at the H. B. Robinson Steam Electric Plant ................................................................................................................ M-10 Table M-2 Task Characteristics of Detecting RCP Abnormality Alarms ........................... M-20 Table M-3 Task Characteristics of Entering AOP-018 ...................................................... M-21 Table M-4 Task Characteristics of Reopening FCV-626 or Tripping the RCPs................ M-21 Table M-5 Timeline of the Baseline Scenario ................................................................... M-30 Table M-6 Task Characteristics of Detect ELAP Procedure Instruction to Deploy a FLEX Generator .............................................................................................. M-40 Table M-7 Task Characteristics of Transporting the FLEX Generator.............................. M-40 Table M-8 Task Characteristics for Connecting the FLEX Generator .............................. M-41 Table M-9 Task Characteristics of Operating the FLEX Generator to Power the 480 VAC Emergency Buses ................................................................................... M-41 Table M-10 Time Available for Successful Deployment of FLEX Generator ...................... M-43 xiii

EXECUTIVE

SUMMARY

Following the issuance of the Commissions policy statement on the use of probabilistic risk assessment (PRA) methods in nuclear regulatory activities (60 Federal Register 42622), the U.S. Nuclear Regulatory Commission (NRC) staff focused on improving human reliability analysis (HRA), which is an essential part of PRA. The Commission, in its staff requirements memorandum (SRM) M061020, directed the Advisory Committee on Reactor Safeguards (ACRS) to, work with the [NRC] staff and external stakeholders to evaluate different human reliability models in an effort to propose a single model for agency use or guidance on which model(s) should be used in specific circumstances. In response to SRM M060120, the NRC staff evaluated several HRA methods by conducting two international collaborative research projects that compared the results obtained from the HRA methods to simulator experiments.

Based on the results of the comparisons, which are known as the International HRA Empirical Study (NUREG-2127) and the U.S. HRA Empirical Study (NUREG-2156), the NRC staff identified areas for HRA improvement and decided to develop an enhanced HRA methodology to integrate the strengths of the existing HRA methods and improve HRA in the areas of application scope, scientific basis, variability, and data. The enhanced HRA methodology is referred to as general methodology of an Integrated Human Event Analysis System (IDHEAS-G). IDHEAS-G is intended to be a human-centered, general methodology used to develop application-specific HRA methods and consists of two parts: a cognition model of human performance and an HRA process that implements the cognition model.

Cognition Model The cognition model integrates current research in cognitive and behavioral science and is based on an extension of the cognitive basis framework documented in NUREG-2114. The cognition model consists of a cognitive basis structure and a performance influencing factor (PIF) structure for which an overview is shown in Figure ES-1 and briefly explained in the subsequent paragraphs.

Human action Tasks and cognitive activities Cognitive Macrocognitive Basis Functions Structure Processors Cognitive Mechanisms Performance-Influencing Factor Structure

Context categories

PIFs

PIF attributes Figure ES-1 Overview of the Cognitive Basis Structure and Performance-Influencing Factor Structure xv

A human action can be divided into several tasks. The cognitive basis structure describes how humans succeed or fail at a task and the underlying cognitive mechanisms for the success or failure. The cognitive basis structure is a way to model the cognitive demands of a task and it is based on the concept of macrocognitive functions. Macrocognitive functions are the high-level brain functions that must be successfully accomplished to achieve the cognitive activities demanded by a task. The representation of a task with the cognitive basis structure is founded on the following macrocognitive functions:

Detection (D) is noticing cues or gathering information in the work environment.

Understanding (U) is the integration of pieces of information with a persons mental model to make sense of the scenario or situation.

Decisionmaking (DM) includes selecting strategies, planning, adapting plans, evaluating options, and making judgments on qualitative information or quantitative parameters.

Action execution (E) is the implementation of the decision or plan to change some physical component or system.

Interteam coordination (T) focuses on how various teams interact and collaborate on an action.

The first four macrocognitive functions (D, U, DM, and E) may be performed by an individual or a team, and interteam coordination is performed by multiple groups or teams. Each macrocognitive function is achieved through a series of microcognitive information processes, referred to as processors. Cognitive mechanisms enable the success and reliability of the processors and are effective within their capacity limits. PIFs are the factors that positively or negatively affect human performance. Consequently, they affect the capacity limits of the cognitive mechanisms. When the PIFs make the cognitive mechanisms ineffective by challenging their capacity limits, they increase the chance of error associated with the processors and macrocognitive functions, which subsequently influence the likelihood of failure of a task, and, therefore, the human action.

The PIF structure models the context of the human action using 20 PIFs in four categories, which are shown in Table ES-1. The set of PIFs in IDHEAS-G is based on an extensive review of the literature, existing HRA methods, human performance databases, and operational experience in various domains (e.g., nuclear, aviation, transportation, and chemical processing).

Table ES-1 PIFs in IDHEAS-G Environment and System Personnel Task situation

Work location

System and

Staffing

Information accessibility and instrumentation

Procedures, availability and habitability and control guidelines, and reliability

Workplace visibility transparency instructions

Scenario familiarity

Noise in workplace to personnel

Training

Multi-tasking, and communication

Human-system

Team and interruption, and pathways interfaces organization distraction

Cold/heat/humidity

Equipment and factors

Task complexity

Resistance to tools

Work

Mental fatigue physical movement processes

Time pressure and stress

Physical demands xvi

Each PIF has a set of PIF attributes, which are the assessable traits of a PIF. A PIF attribute describes a way that the PIF challenges the cognitive mechanisms and increases the likelihood of errors in the processors.

Cognition Model Implementation through an HRA Process The HRA process that implements the cognition model consists of the following four stages:

Stage 1Scenario analysis. The purpose of this stage is to understand the event and collect information about human actions from broad perspectives. This includes developing an operational narrative, analyzing the scenario context, and identifying and defining important human actions (e.g., the ones considered in a PRA or human failure events (HFEs)). IDHEAS-G provides a structured process to query and document the qualitative information used as the foundation of human error probability (HEP) quantification.

Stage 2Modeling of important human actions. The purpose of this stage is to model important human actions for structured analysis and HEP quantification. This includes identifying and characterizing critical tasks in an important human action, representing potential task failure with cognitive failure modes (CFMs), and representing the context of the important human action with PIFs. IDHEAS-G provides guidelines for task analysis, as well as a basic set of CFMs and a comprehensive taxonomy of PIFs from its cognition model.

Stage 3HEP quantification. The purpose of this stage is to estimate the HEP for important human actions, which has two parts: (1) the error probability attributed to the uncertainties and variability in the time available and time required to perform the action

( ) and (2) the error probability attributed to the CFMs ( ). IDHEAS-G provides several approaches to HEP estimation, along with the human error data generalized in the IDHEAS-G framework. IDHEAS-G uses a time uncertainty model, which incorporates (convolves) the probability distributions of time available and time required, and the generalized human error data to estimate and , respectively. The overall HEP is then 1 (1 )(1 ).

Stage 4Integrative analysis. While Stages 2 and 3 analyze individual important human actions, Stage 4 analyzes all the important human actions as a whole. This includes assessing the dependencies between important human actions and documenting uncertainties in the event and its analysis. IDHEAS-G provides a new approach to assess dependency between important human actions and supplementary guidance for uncertainty analysis by consolidating existing guidelines.

The NRC staff developed guidelines to implement each stage in the HRA process. These guidelines can be found in the appendices of this report. Figure ES-2 illustrates the IDHEAS-G HRA process as a group of steps and Table ES-2 provides a crosswalk between the IDHEAS-G stages discussed above and the steps shown in Figure ES-2.

xvii

PIF attributes of every CFM Scenario for every CT PRA context and Step 5: Calculate model list of applicable Step 4: Assess PIFs Step 1: Determine PIFs applicable to every scenario context CFM List of Step 1: HFE and its applicable Step 7:

Develop scenario narrative definition CFM(s) for Calculate Develop scenario timeline the CT(s) overall HEP HFE and its Step 2: List of CT(s) Step 3: Characterize the Step 1: Identify HFE definition Analyze tasks CT(s) and select applicable Step 1: Define HFE and identify CFMs CT(s) in HFE HFE and its definition and Step 6: Estimate parameters Step 6: Analyze HFE timeline of distribution (subset of scenario timeline, if there are Step 6: Calculate multiple HFEs in the scenario)

Step 6: Estimate parameters of distribution and Step 8:

Uncertainty and dependency analysis and documentation CFM = cognitive failure mode = error probability due to CFMs CT = critical task = error probability due to uncertainty in and HEP = human error probability = time available HFE = human failure event = time required PIF = performance-influencing factor and = mean and standard deviation of PRA = probabilistic risk assessment and = mean and standard deviation of Figure ES-2 Illustration of the IDHEAS-G HRA Process Table ES-2 IDHEAS-G HRA Process Stages and Steps IDHEAS-G Stages IDHEAS-G Steps Stage 1 - Scenario analysis Step 1: Develop scenario narrative Step 1: Develop scenario context Step 1: Identify HFE Step 1: Define HFE Stage 2 - Modeling of Step 2: Analyze tasks and identify CT(s) in HFE important human actions Step 3: Characterize the CT(s) and select applicable CFMs Step 4: Assess PIFs applicable to every CFM Stage 3 - HEP quantification Step 5: Calculate Step 6: Analyze HFE timeline Step 6: Estimate parameters of and distributions Step 6: Calculate Step 7: Calculate overall HEP Stage 4 - Integrative analysis Step 8: Uncertainty and dependency analysis and documentation HRA Areas Improved by IDHEAS-G IDHEAS-G improved the following four areas by integrating the inputs from the broad technical community, adopting the strengths of existing HRA methods, and incorporating state-of-the-art cognitive and behavioral science.

xviii

Application scope IDHEAS-G provides an application-independent process for performing HRA along with a comprehensive set of CFMs and PIFs, which allows the expansion of the application scope of HRA into other applications (e.g., use of flexible and coping strategies equipment and external events) and nuclear-related domains.

Scientific basis The application-independent HRA process in IDHEAS-G is based on state-of-the-art cognitive and behavioral science. Human performance is modeled using a cognition-based approach that is more comprehensive than that found in existing HRA methods. The cognitive basis structure provides an explicit picture of how and why personnel succeed or fail in performing expected tasks in a complex work environment and the PIF structure describes the challenges to human performance. IDHEAS-G includes the modeling of cognitive activities in a teamwork and organizational environment.

HRA variability The structure and traceability of the HRA process in IDHEAS-G along with the guidelines for analyzing and documenting an event should reduce the inter-analyst variability. IDHEAS-G also has an improved approach to time uncertainty analysis, which emphasizes the identification of factors that contribute to the uncertainty in time available and time required to perform the action. The traceability of the IDHEAS-G HRA process allows analysts to identify sources of variability in the results of the HRA and attempt to reconcile differences in the results. The scientific basis that underpins the IDHEAS-G HRA process should reduce the variability that results from applying the methodology across different applications.

Data for HRA Perhaps the major contribution that IDHEAS-G can make to improve HRA practice is to open the methodology for incorporating human error data across a number of domains. The structure of the IDHEAS-G HRA process is general enough to incorporate data across domains (e.g., nuclear, aviation, transportation, and chemical processing). Also, the basic quantification structure allows for generalization of data in different domains. Moreover, the NRC staff has been developing an HRA database drawing from nuclear power plant operator simulator data. This database is known as Scenario Authoring, Characterization, and Debriefing Application (SACADA). SACADA is structured using the same cognitive framework as IDHEAS-G so that the data can be used to support HEP estimation. Given that IDHEAS-G emphasizes context, the compilation of data pertaining to the impact of environmental and organizational factors on personnel actions will be needed. The availability of data to support the application of IDHEAS-G will significantly increase the feasibility of its use. Over time, as more applicable data are compiled, incorporating data from different domains should become one of the major strengths of IDHEAS-G.

IDHEAS-G can be used to develop application-specific HRA methods. An example of developing an application-specific HRA method using IDHEAS-G and incorporating data across multiple domains is the publication of the Integrated Human Event Analysis System for Event and Condition Assessment (IDHEAS-ECA). IDHEAS-ECA was published in February 2020 as Research Information Letter 2020-02 and can be found using the Agencywide Documents Access and Management System (ADAMS) accession number ML20016A481. IDHEAS-ECA has proven to be a useful HRA method for supporting the NRCs risk-informed decisionmaking processes.

xix

ACKNOWLEDGMENTS The authors of IDHEAS-G wish to thank the many esteemed scientists and engineers who provided invaluable contributions to the development of the methodology and production of this report.

We wish to thank John Stetkar, Stetkar and Associates, for his performance of an in-depth review of the 2019 draft report. He provided insightful comments and constructive suggestions.

In particular, he recommended that the authors complete the development of the IDHEAS-G-based dependency model to solve long-standing issues in modeling dependency between human failure events. The inclusion of the new dependency model in this report is largely attributed to Mr. Stetkars review work.

We thank Kevin Coyne and Nathan Siu from the NRC for their support and guidance on the IDHEAS-G development. They provided significant feedback on the early versions of IDHEAS-G. Their inputs have become an inseparable part of IDHEAS-G. Specifically, they recommended that IDHEAS-G should provide an interface for using human performance data to inform human error probability estimates. We adopted this recommendation and developed the important piece of IDHEAS-G for generalizing human error data from various sources, as documented in Chapter 6 of the report.

We thank Dr. Emilie Roth, Roth Cognitive Engineering, Inc., for working with the NRC staff to develop the framework of modeling interteam coordination in IDHEAS-G and for developing an illustrative example for interteam coordination. Her developmental work was incorporated into Chapter 2 of this report.

We thank the following individuals (in alphabetical order) who worked on the NRCs efforts to develop a HRA method for agency use. Their insights on HRA method improvement were migrated into the IDHEAS-G development.

Vinh Dang, Paul Scherrer Institute John Forester (retired), Idaho National Laboratory Stacey Hendrickson (deceased), Sandia National Laboratories Erasmia Lois (retired), U.S. Nuclear Regulatory Commission Gareth Parry (retired), Jensen Hughes, Inc.

Mary Presley, Electric Power Research Institute We thank our own fellow NRC staff (in alphabetical order) who reviewed the various drafts of IDHEAS-G and provided valuable comments: Valerie Barnes (retired), Michael Cheok, Susan Cooper, Carmen Franklin, Michelle Kichline, Stephanie Morrow, Jeff Mitman, Sean Peters, Song-Hua Shen, and Mark Thaggard.

We thank the following individuals (in alphabetical order of their affiliations) who participated in the NRCs external review of the draft IDHEAS-G report in 2016. They provided valuable comments from different perspectives. The NRC staff consolidated their comments and thoroughly revised the report based on the inputs.

Mary Presley - Electric Power Research Institute David Gertman (retired) - Idaho National Laboratory Paul Amico, Erin Collins, and Jeff Julius - Jensen Hughes Inc.

xxi

Sun Yeong Choi, Wondea Jung, Jaewhan Kim, Seunghwan Kim, Yochan Kim, and Jinkyun Park - Korea Atomic Energy Research Institute Claire Blackett, Andreas Bye, and Salvatore Massaiu - OECD Halden Reactor Project, Norway Garill Coles - Pacific Northwest National Laboratory Emilie Roth - Roth Cognitive Engineering, Inc.

Ali Mosleh - University of California Los Angles Zahra Mohaghegh - University of Illinois Urbana-Champaign xxii

ABBREVIATIONS AND ACRONYMS 10 CFR Title 10 of the Code of Federal Regulations ACRS Advisory Committee on Reactor Safeguards ADAMS Agencywide Documents Access and Management System AFW auxiliary feedwater ANS American Nuclear Society ASME American Society of Mechanical Engineers ASP accident sequence precursor (program)

ATHEANA A Technique for Human Event Analysis CBDT cause-based decision tree CCW component cooling water CFM cognitive failure mode cm centimeters CRD crew response diagram CVC chemical and volume control D detection (one of the five macrocognitive functions)

DC direct current DI&C digital instrumentation and control DM decisionmaking (one of the five macrocognitive functions)

E action execution (one of the five macrocognitive functions)

ELAP extended loss of alternating current power EOC error of commission EOF emergency operations facility EOO error of omission EOP emergency operating procedure EPP end path procedure EPRI Electric Power Research Institute Ex-CR Outside the Control Room FCV flow control valve FLEX flexible and coping (strategies) gpm gallons per minute HA human action HCR/ORE Human Cognitive Reliability/Operator Reliability Experiments HEART Human Error Assessment and Reduction Technique HEP human error probability HF human factor HFE human failure event HRA human reliability analysis HSI human-system interface xxiii

HuREX Human Reliability Data Extraction I&C instrumentation and control IDHEAS Integrated Human Event Analysis System IDHEAS-G General Methodology of an Integrated Human Event Analysis System IHA important human action IROFS items relied on for safety ISA integrated safety analysis LOCA loss-of-coolant accident LOSC loss-of-seal cooling MCR main control room NDE nondestructive examination NDM naturalistic decisionmaking NPP nuclear power plant NRC U.S. Nuclear Regulatory Commission PGI procedure, guidance, and (or) instruction PIF performance-influencing factor PPE personal protective equipment PRA probabilistic risk assessment [same as PSA]

PSA probabilistic safety assessment [same as PRA]

psig pounds per square inch gauge RCP reactor coolant pump RCS reactor coolant system RNO response not obtained RO reactor operator SACADA Scenario Authoring, Characterization, and Debriefing Application SDP significance determination process SG steam generator SGTR steam generator tube rupture SI safety injection SLIM-MAUD Success Likelihood Index Methodology Multi-Attribute Utility Decomposition SPAR-H Standardized Plant Analysis Risk Human Reliability Analysis SRM staff requirements memorandum T interteam coordination (one of the five macrocognitive functions)

THERP Technique for Human Error Rate Prediction TSC technical support center U understanding (one of the five macrocognitive functions)

UF6 uranium hexafluoride xxiv

1 INTRODUCTION 1.1 Purpose of the Report In 1995, the U.S. Nuclear Regulatory Commission (NRC) adopted a policy stating the following:

[t]he use of probabilistic risk assessment (PRA) technology should be increased in all regulatory matters to the extent supported by the state-of-the-art in PRA methods and data and in a manner that complements the NRCs deterministic approach and supports the NRCs traditional defense-in-depth philosophy. [1]

PRA models the reliability of systems and personnel to mitigate a system abnormality and prevent it from developing undesired consequences. It addresses three key questions: what can go wrong, how likely is it to go wrong, and what are the consequences [2]. Human reliability analysis (HRA) is an essential part of PRA. HRA is an engineering approach that systematically analyzes human performance for events or specified conditions.

This report presents a new HRA methodology, referred to as the General Methodology of an Integrated Human Event Analysis System (IDHEAS-G) and the work performed to develop it.

The development work, described in the appendices to this report, serves as supplementary guidance for using IDHEAS-G and also can be used as a reference resource for HRA in general.

The NRC staff highlights the following intended uses of IDHEAS-G:

IDHEAS-G is a method to perform HRA for all nuclear applications. Based on cognitive science, it analyzes human failures with a cognition model.

IDHEAS-G is a general methodology and can be used as high-level guidance for developing application-specific HRA methods or tools.

IDHEAS-G can serve as a platform to generalize and integrate human error data from various sources for human error probability (HEP) estimation. IDHEAS-G uses the cognition model as the basis to analyze an event scenario, model important human actions, and quantify HEPs. The cognition model has a structured taxonomy that can model human errors from cognitive perspectives at different levels of detail and link the errors to causal factors. The generalized data can also serve as anchors in addressing HRA variability (see the third area for HRA method improvement in Section 1.2).

IDHEAS-G is also a method for systematically analyzing human events, including identifying potential human failures and root causes. The NRC staff has used IDHEAS-G to analyze and document some notable human events and simulator experiments to test the methodology.

1.2 Background To date, about 50 HRA methods have been developed worldwide. In the United States, notable HRA methods used in the nuclear industry include the following:

The Handbook of Human Reliability Analysis with Emphasis on Nuclear Power Plant Applications (i.e., Technique for Human Error Rate Prediction (THERP)) [3]

Accident Sequence Evaluation Program Human Reliability Analysis Procedure [4]

1-1

Success Likelihood Index Methodology Multi-Attribute Utility Decomposition (SLIM-MAUD) [5], [6]

Standardized Plant Analysis Risk Human Reliability Analysis method (SPAR-H) [7]-[9]

A Technique for Human Event Analysis (ATHEANA) [10], [11]

Human Cognitive Reliability (HCR)/Operator Reliability Experiments (ORE) method

[12]

Cause-Based Decision Tree (CBDT) method [12]

Improving HRA has been an NRC research focus since the 1995 publication of the NRC PRA policy statement [1]. In 2005, the NRC published NUREG-1792, Good Practices for Implementing Human Reliability Analysis [13]. After evaluating various HRA methods against these practices, the NRC issued NUREG-1842, Evaluation of Human Reliability Analysis Methods Against Good Practices, in September 2006 [14]. In Staff Requirements Memorandum (SRM) M061020 [15] to the Advisory Committee on Reactor Safeguards (ACRS),

dated November 8, 2006, the Commission directed the ACRS to evaluate the different human reliability methods to propose a single model for the agency to use or guidance on which model(s) should be used in specific circumstances. To evaluate the methods, the NRC staff performed two international collaborative research projects, the International HRA Empirical Study [16]-[19] and the U.S. HRA Empirical Study [20], comparing HRA methods and simulator experiments. These studies provided valuable lessons from use of the HRA methods and identified areas for HRA improvement. Based on the results of the empirical studies, the NRC decided to develop an enhanced HRA method, referred to as the Integrated Human Event Analysis System (IDHEAS). The method was intended to integrate the strengths of existing HRA methods and enhance HRA in the four areas described below:

(1) Application scopeEach existing HRA method was developed for a specific application domain, and most were developed for a procedure-based response to internal events occurring at-power in NPPs. As a result, the methods are not necessarily adequate to model human actions for NPP events that result from external hazards or events in other domains. The use of PRA is expanding to include many diverse applications. As the application of PRA grows and covers broader contexts, HRA must be able to expand with it and support the growth areas. Over the years, some HRA studies have been performed for contexts other than internal, at-power events. The studies either adapted the methods used for internal, at-power events or used the general HRA concepts. A consistent methodology is needed to address HRA across different applications and different domains.

(2) Scientific basisExisting HRA methods were built on behavioral observations of human performance and cognitive science. Without explicitly modeling the intrinsic cognitive mechanisms underlying human errors, an HRA method may result in different interpretations of the same observed phenomena and poor understanding of the causes of human errors. HRA methodology should be enhanced to include the advances made in cognitive and behavioral science in the past decades.

(3) HRA variabilityHRA results, especially the HEP for a human failure event (HFE) can vary significantly, depending on the HRA model or method used and the analyst applying the method. The International HRA Empirical Study [16]-[19] and the U.S. HRA Empirical Study [20] identified three types of HEP variability in a given scenario: method-to-method, analyst-to-analyst, and crew-to-crew. The International 1-2

HRA Empirical Study indicated that key sources of the variability including weak guidance for performing the qualitative analysis and poor understanding of performance-influencing factors (PIFs) could affect the HEP.

(4) Data for HRAThe use of empirical data for HEP estimation has been limited in past studies by the lack of relevant data and discrepancies between the formats of available data and HRA methods. A lack of a strong data basis in the methods may challenge the method validity and introduce additional variabilities in HEP estimation.

Section 7.1 provides further discussion of how IDHEAS-G addresses these four areas of HRA method enhancement.

1.3 Strategic Approach to Human Reliability Analysis Method Development Based on lessons learned from previous studies, the NRC staff took the following strategic approach to develop IDHEAS and improve the state of HRA. Figure 1-1 illustrates the interaction of the activities in the strategic approach.

Developed a cognitive basis for HRA. The purpose of the cognitive basis is to synthesize the fundamentals of human cognition into a structure that supports HRA method development and HRA practices. NUREG-2114, Cognitive Basis for Human Reliability Analysis, issued January 2016, documents the cognitive basis [21].

Developed a generic HRA methodology based on cognitive and behavioral sciences.

The methodology should be independent of specific HRA applications and apply to a wide range of HRA applications in the nuclear domain. The methodology should also integrate the strengths of existing HRA methods and conform to existing PRA standards.

This methodology, documented in this report, is referred to as IDHEAS-G.

Documented and generalized human error data from various sources and generalized the data in the IDHEAS-G framework to inform HEP quantification. Along with the development of IDHEAS-G, the NRC staff used the cognition model to generalize empirical data from the literature and various operational databases to inform HEPs.

This will be a longer-term and ongoing effort as more data become available.

Implemented the methodology in the context of internal NPP events that initiate while the reactor is at-power. The work, a collaboration with the Electric Power Research Institute (EPRI), was documented in NUREG-2199, Volume 1, An Integrated Human Event Analysis System (IDHEAS) for Nuclear Power Plant Internal Events At-Power Application [22], published in March 2017. Experience gained from the application-specific approach informed the approach to the general methodology presented in this report.

Developed quantification tools for other applications using IDHEAS-G.

1-3

Cognitive basis framework (NUREG-2114)

Literature and human factors practices IDHEAS General Human error Methodology data Existing HRA methods (NUREG-2198) generalization Operational experience IDHEAS Internal At-Power IDHEAS for actions outside Other specific HRA Application the control room applications (NUREG-2199, Vol. 1) (e.g., IDHEAS for Event and Condition Assessment)

Plant-specific PRA models Figure 1-1 Strategic Approach to HRA Method Development 1.4 Overview of IDHEAS-G Performing HRA requires both qualitative analysis (analyzing the human event) and quantification (estimating HEPs). Key HRA results include (1) scenarios describing expected and deviating human and system activities, (2) identification of important human actions that may lead to an undesired or an unsafe system state, (3) identification of the ways that a human can fail, (4) identification of the factors that affect human performance, and (5) estimation of HEPs. HRA methods achieving these results include qualitative analysis, which is to understand event scenarios and the context of important human actions in the scenario, and quantitative analysis to estimate HEPs.

IDHEAS-G consists of two parts: a cognition model of human performance and reliability, and an IDHEAS-G-specific HRA process that implements the cognition model. The cognition model integrates current research in cognitive and behavioral science. It consists of a cognitive basis structure and a PIF structure. The cognitive basis structure describes how humans succeed or fail at a task and the underlying cognitive mechanisms for the success or failure. The PIF structure describes which factors affect the likelihood of success or failure and how those factors impact the effectiveness of the cognitive mechanisms and increase the likelihood of human failures. Both structures have cognition-based taxonomies representing an important human action and the associated context of the action.

IDHEAS-G implements its cognition model to the full span of the general HRA process (see Section 4.1.1). The HRA process of IDHEAS-G consists of four stages:

(1) Stage 1Scenario analysis. The purpose of this stage is to understand the event and collect information about human actions from broad perspectives. This includes developing an operational narrative, analyzing the scenario context, and identifying and defining important human actions (i.e., the ones considered in a PRA). IDHEAS-G 1-4

provides a structured process to query and document the qualitative information used as the foundation of HEP quantification.

(2) Stage 2Modeling of important human actions. The purpose of this stage is to model important human actions for structured analysis and HEP quantification. This includes identifying and characterizing critical tasks in an important human action, representing potential task failure with cognitive failure modes (CFMs), and representing the context of the important human action with PIFs. IDHEAS-G provides guidelines for task analysis, as well as a basic set of CFMs and a comprehensive taxonomy of PIFs from its cognition model.

(3) Stage 3HEP quantification. The purpose of this stage is to estimate the HEP for important human actions. IDHEAS-G provides several approaches to HEP estimation, along with the human error data generalized in the IDHEAS-G framework.

(4) Stage 4Integrative analysis. While Stages 2 and 3 analyze individual important human actions, Stage 4 analyzes all the important human actions as a whole. This includes addressing the dependencies between important human actions and documenting uncertainties in the event and its analysis. IDHEAS-G provides supplementary guidance for uncertainty analysis by consolidating existing guidelines.

Figure 1-2 illustrates the composition of IDHEAS-G. The cognition model is incorporated into all four stages of the IDHEAS-G HRA process. In particular, the cognition model is the basis for modeling important human actions. The cognition model allows integration of human error data available from various sources (i.e., experiments, operating experience, and expert judgments),

and the integrated data can inform HEP quantification. The outputs of one stage serve as the inputs to subsequent stages. Moreover, each stage represents the analysis of an important human action from a different perspective, and the outputs of each stage provide valuable insights into the success and failure of an important human action.

Cognitive Basis Structure Cognition Model PIF Structure Human error data Stage 1 Stage 2 Stage 3 Stage 4 Scenario Modeling of HEP Integrative analysis important quantification analysis human actions Figure 1-2 IDHEAS-G Diagram While IDHEAS-G follows the general analysis flow used by most existing HRA methods, it has new features, briefly described below, that advance the state of practice of HRA:

The cognition model explains how and why humans succeed and fail at a task and why PIFs affect the likelihood of failure.

1-5

A basic set of CFMs models human errors independent of application domain, and structured PIF states represent the context in which the important human action occurs.

A time uncertainty model calculates the HEPs attributed to uncertainties in time available and time required to perform important human actions.

Integrated human error data support HEP quantification.

A framework is used for generalizing and integrating human error data to inform HEP estimation.

1.5 Organization of the Report This report is organized into seven chapters and 13 appendices. The seven chapters represent the main body of the text and describe IDHEAS-G with a focus on new developments.

Chapter 1 provides an introduction and overview of IDHEAS-G. Chapters 2 and 3, respectively, describe the two parts of the cognition model, cognitive basis structure and PIF structure.

Chapter 4 explains the IDHEAS-G HRA process (i.e., the four stages: scenario analysis, modeling of important human actions, HEP quantification, and integrative analysis) that implements the cognition model. Chapter 5 describes the time uncertainty model. Chapter 6 explains the IDHEAS-G framework for generalizing human error data. Chapter 7 discusses perspectives on applying IDHEAS-G, future research, and concluding remarks. Chapter 8 lists the references used in this report.

The 13 appendices document the work performed to develop IDHEAS-G and serve as supplemental guidelines for performing the various stages of the HRA process. These supplemental guidelines are intended to help HRA analysts use IDHEAS-G. The supplementary guidelines use many examples to facilitate readers understanding of the guidance. Many side-by-side examples demonstrate different perspectives on a topic or demonstrate the implications of the guidance in different HRA applications, such as NPP operations, radioactive medical equipment operation, or response to chemical material release.

In addition, the appendices demonstrate how to perform the stages of the HRA process in IDHEAS-G and give full examples of using IDHEAS-G for event analysis. Each appendix can be viewed as a standalone document to assist with a specific part of the HRA process. Below is a brief summary of the appendices.

APPENDIX A. Cognitive Mechanisms Underlying Human Performance and ReliabilityThis appendix presents prevalent cognitive mechanisms that have been studied in cognitive and behavioral science and their association with macrocognitive functions.

APPENDIX B. Links of PIF Attributes to Cognitive MechanismsThis appendix generalizes experimental findings and operational experience on how PIF attributes can lead to ineffectiveness of cognitive mechanisms. The information explains why and how PIFs affect human performance and increase the likelihood of human errors.

APPENDIX C. Insights into Performance-Influencing Factors from the Cognitive Literature This appendix explains the insights obtained from the cognitive literature about the PIF attributes and their effects on HEPs.

APPENDIX D. Cognitive Basis for the Combined Effect of PIFs on HEPs.This appendix describes the cognitive basis for the quantitative treatment of PIF combinations.

APPENDIX E. Scenario AnalysisThis appendix provides step-by-step guidance for performing scenario analysis to establish an overall understanding of event context and event 1-6

evolution. The guidance provides structured ways to acquire information on human aspects of an event evolution and organize context information for modeling and quantifying important human actions.

APPENDIX F. Identification and Definition of Important Human ActionsImportant human actions (or HFEs) are the objects of HRA quantification. PRA models prospectively identify human actions for modeling, but actual events may involve actions that are not included in PRA models. This appendix provides guidance on identifying and defining critical human actions with and without the presence of PRA models.

APPENDIX G. Task AnalysisThis appendix provides guidance for identifying and characterizing critical tasks in a human action. Task characterization includes, but is not limited to, cognitive activities demanded by a task, timing, and relation between tasks. The guidance also includes several task analysis methods for HRA. This appendix provides a specific, logical framework and vocabulary for performing task analysis for HRA.

APPENDIX H. Identification of Cognitive Failure ModesThis appendix discusses the basic set of CFMs in IDHEAS-G and provides guidance and examples of adapting the basic set of CFMs for a specific HRA application.

APPENDIX I. Assessment of Performance-Influencing FactorsThis appendix provides guidance and examples of how to structurally represent the context of an important human action with PIFs.

APPENDIX J. Quantification of Human Error ProbabilityThis appendix provides guidance and examples of using different approaches to quantify HEPs based on HRA methods and data available.

APPENDIX K. IDHEAS-G Treatment of Dependency between Human Failure Events This appendix describes the IDHEAS-G dependency model along with some insights for improving dependency analysis in HRA. The model is capable of systematically identifying changes in the context of an HFE that result from the failure of the preceding HFE, modeling the changes at different levels of the IDHEAS-G HRA process, and re-estimating the HEP based on the changes.

APPENDIX L. Uncertainty Analysis and DocumentationThis appendix generalizes existing guidance on uncertainty analysis and documentation. It emphasizes that uncertainty analysis is not an add-on to the HRA process. Instead, analysis and documentation of uncertainties are embedded in every stage of the IDHEAS-G HRA process.

APPENDIX M. Demonstration of the IDHEAS-G HRA ProcessThis appendix provides two examples of implementing the IDHEAS-G process for performing HRA. The first example is an actual event the March 28, 2010, fire event at the H.B. Robinson Steam Electric Plant, Unit 2 and the second example analyzes a hypothetical event.

We emphasize the importance of the using the supplementary guidance in the appendices of this report along with the IDHEAS-G HRA process described in Chapter 4. HRA deals with human events under uncertainties and performing HRA requires analysts subjective judgment, which inevitably introduces analyst-to-analyst variability in the results. Structured guidance describing the detailed process of an HRA helps explain the uncertainties and trace the sources of analyst-to-analyst variability. This ultimately improves HRA quality. The guidance documented in the appendices of this report is developed for this purpose.

1-7

1.6 Perspectives on the Development of the IDHEAS-G Human Reliability Analysis Method IDHEAS-G includes advances in several HRA areas. First, it is an application-independent HRA method. Also, it can be used for any nuclear-related HRA application for human event analysis, including various types of NPP events such as internal and external hazards, important human actions within and outside the main control room, during at-power and shutdown operations, and nuclear material handling events. Second, it is based on cognitive and behavioral science. Its underlying cognition model makes HRA more explainable: it explains how and why a person may fail an action, as well as why and how various contexts of an important human action affect the likelihood of its success or failure. Third, IDHEAS-G delineates a structured HRA process for consistently analyzing an event and documenting the results in a transparent manner. IDHEAS-G also provides step-by-step guidance for qualitative analysis and quantification that clearly specifies the objective, process, inputs, and outputs of each stage. IDHEAS-G makes the HRA process transparent and traceable. Therefore, it improves the consistency of HRA performed by different analysts. If there is analyst-to-analyst variability in the outcomes of HRA, the differences can be easily traced for reconciliation.

Last and probably most important, IDHEAS-G has a built-in interface with human error data, which allows the use of empirical data for HEP estimation. The cognition model makes it possible to generalize human error data from different events that share the same CFMs and PIFs. In particular, in responding to SRM 090204B, dated February 4, 2009 [23], on developing an HRA database, the NRC staff developed an operator simulator training database, referred to as the Scenario Authoring, Characterization, and Debriefing Application (SACADA) [24].

SACADA collects simulator training data on licensed NPP operators. It is structured on the same set of macrocognitive functions as those in IDHEAS-G, but its taxonomy of operator failure modes and PIFs is oriented towards operator simulator training. The NRC staff developed the mapping between SACADA and IDHEAS-G taxonomies. Therefore, the SACADA data directly support HEP estimation in IDHEAS-G. An additional benefit of IDHEAS-G is that it connects to an operational human performance database and makes use of the data.

In the longer term, the NRC staff should continue generalizing human error data from a variety of sources to support HEP estimation.

Throughout the development of IDHEAS-G, the ACRS guided the NRC staff in developing structured guidelines for all the HRA areas to enhance HRA quality. For example, scenario analysis has been recognized as an essential step in HRA. Yet, existing HRA methods generally lack explicit guidelines on how to perform scenario analysis and what information should be collected for HRA through scenario analysis. As a result, HRA analysts may seek various ways to perform the analysis or even skip some steps. The International HRA Empirical Study [16]-[19] found that lack of clear guidance for qualitative analysis led to inconsistent information collection for HRA and that was a major cause leading to HRA variability. The NRC staffs approach was to integrate the strengths of existing HRA methods and develop additional guidelines for the areas where guidance is lacking or weak. The staff documented the appendices as IDHEAS-G supplementary guidance. While the supplemental guidance supports the use of IDHEAS-G, it also supports HRA practices performed with other methods.

In summary, IDHEAS-G provides a link from what is factually known per current science to the HRA/PRA models, i.e., the NRC staff is making better use of the available evidence to support decisionmaking. Evidence includes what is embedded in theories, models, and data. Even though the link is not perfect, establishing the link allows for the systematic enhancement of the link as new research theories, models, and data become available.

1-8

The methodology emphasizes the importance of the thorough qualitative analysis in HRA and provides detailed qualitative guidance. Every step of the qualitative analysis yields insights into risk-important scenarios, which tells people what they might fix to prevent human failure events.

The implication is that HRA is not just about getting an HEP estimate, but more about understanding human performance in a scenario and identifying the potential problems that may be fixed to improve human reliability.

1-9

2 COGNITION MODELCOGNITIVE BASIS STRUCTURE IDHEAS-G is a general methodology. It is based on the cognition model developed from cognitive research for HRA. The cognition model has two parts: a cognitive basis structure described in this chapter and the PIF structure described in Chapter 3. This chapter and Chapter 3 are essential to understand the scientific basis for the human performance modeling in IDHEAS-G. Chapter 4 describes the implementation of the human performance model in the IDHEAS-G HRA process.

This chapter presents the cognitive basis with cognitive mechanisms, which are the fundamental explanations of why personnel may succeed or fail at a task. Chapter 3 presents performance influencing factors with PIF attributes, which are the most basic elements affecting human reliability. APPENDIX A and APPENDIX B present the links of the PIF attributes to the cognitive mechanisms in a series of tables. The links are inferred from the cognitive and psychological literature. In the context of the IDHEAS-G methodology, these tabulations summarize the identified functional relationships. Using links in the tabulations enhances analysts confidence in their assessment of risk contributors.

In addition to serving as the basis of the IDHEAS-G development, the cognition model can be used more generally to identify causes, mechanisms, and PIFs to consider for any situation involving human errors. The cognition model gives HRA analysts a structured tool, based on cognitive research, to identify the factors relevant to a given human failure event. As a result, the cognition model would be useful for other HRA methods or human factors applications.

2.1 Overview of the Cognition Model for Human Performance and Reliability One of the objectives in developing IDHEAS-G is to have an HRA methodology based on cognitive science showing why and how personnel fail actions. Through extensive literature review and analysis, the NRC staff developed the Cognition Model for Human Performance and Reliability as the basis for IDHEAS-G. The cognition model describes the nature of human performance in applied work domains where human tasks are complex and often involve multiple individuals or teams. Figure 2-1 shows a diagram of the cognition model.

Cognition Model for Human Performance and Reliability Important Critical Cognitive Basis Structure Human Tasks Action and and Cognitive Context of Activities PIF Structure the Action Figure 2-1 The Cognition Model for Human Performance and Reliability The inputs to the cognition model are the critical tasks of an important human action (including its cognitive activities) and its context (the conditions under which the action is performed). The 2-1

output is the success or failure of the task. The cognition model explains why and how a task is a success or failure and what influences the success or failure. The cognition model consists of two parts: a cognitive basis structure, which explains how personnel perform a task correctly and reliably and a PIF structure, which describes why the task is a success or failure and what influences the success or failure. The cognition model can be used for HRA to (1) represent how a task may fail through the cognitive basis structure, (2) represent the influence of context on human reliability through the PIF structure, and (3) explain why the context or PIFs affect task failure. These establish the basis for HEP quantification. The cognition model can also guide HRA analysts as they inquire and organize information in the HRA qualitative analysis.

In the first phase of the IDHEAS project, the NRC staff led the development of a cognitive basis framework for HRA, documented in NUREG-2114 [21]. This early cognitive basis framework synthesizes research in cognitive and behavioral models and theories, cognitive mechanisms, and some examples of PIFs. Given the broad scope of IDHEAS-G application, the NRC staff performed an expanded review and synthesis of literature in cognitive and behavioral psychology, human error analysis, and human factors. The scope of the additional work spans to human performance in complex situations or severe operating conditions such as multiple simultaneous events, multiple teams and organizations involved, distributed locations, and dynamic decisionmaking. As a result, the NRC staff extended the cognitive basis framework in NUREG-2114 into the structured cognition model to support HRA in various applications. This chapter presents the cognitive basis structure, and Chapter 3 presents the PIF structure.

Chapter 4 describes the implementation of the cognition model in HRA.

2.2 Overview of the Cognitive Basis Structure A human action can be divided into several tasks. Any human task involves various cognitive activities such as monitoring parameters or operating equipment. Performing cognitive activities demands brain resources. The cognitive basis structure is a way to model the cognitive demands of a task. It is based on the concept of macrocognitive functions (see Section 7.1.2 for an explanation of the use of macrocognition to model human performance). The macrocognitive functions are the high-level brain functions that must be successfully accomplished to achieve the cognitive activities demanded by a task. As illustrated in Figure 2-2, the cognitive basis structure represents a task, and this representation is founded on five macrocognitive functions: detection, understanding, decisionmaking, action execution, and interteam coordination. The macrocognitive functions describe human performance at the individual or team level. The first four macrocognitive functions may be performed by an individual, a group or a team, and the interteam coordination macrocognitive function is performed by multiple groups or teams.

2-2

Human action Task 1 Task 2 Task 3 and cognitive activities and cognitive activities and cognitive activities Under- Decision- Action Interteam Detection standing making execution coordination Figure 2-2 Cognitive Basis Structure Representation of a Human Task Each macrocognitive function is described as follows:

Detection (D) is noticing cues or gathering information in the work environment.

Emphasized in this macrocognitive function are the sensory and perceptual processes that allow humans to perceive copious amounts of information and focus selectively on those pieces of information that are pertinent to the task being performed.

Understanding (U) is the integration of pieces of information in the work environment with a persons mental model to make sense of the scenario or situation. Cognition in this macrocognitive function ranges from automatic, effortless recognition and understanding to more effortful thinking and deliberate attempts to make sense of multiple pieces of information.

Decisionmaking (DM) includes selecting strategies, planning, adapting plans, evaluating options, and making judgments on qualitative information or quantitative parameters.

Action execution (E) is implementation of the decision or plan to make a change in some physical component or system.

Interteam coordination (T) is the macrocognitive function that focuses on how various teams interact and collaborate on a task. In the present effort, IDHEAS-G uses this macrocognitive function primarily to include coordination, collaboration, and communication between teams. This macrocognitive function focuses on the emergent aspects of interteam interaction to avoid duplicating the within-team interaction already included in the four previous macrocognitive functions.

Each macrocognitive function is achieved through a series of microcognitive information processes, referred to as processors. Performing a cognitive activity may demand some or all the processors of a macrocognitive function. Notice that the Interteam coordination macrocognitive function addresses only interactions between teams of personnel (e.g., between the main control room crew and local operators). It does not to address interactions among individuals within a team (e.g., among supervisors and operators of the main control room crew) 2-3

to achieve a consensus decision or plan of action. Within-team interaction is a part of each individual macrocognitive function. This can be seen in the process of the macrocognitive functions described next.

For each macrocognitive function, the cognitive basis structure provides a causal tree, shown in Figure 2-3, describing the process for accomplishing the macrocognitive function. A causal tree connects the macrocognitive functions with a series of processes (referred to as processors 1) 0F that achieve each function, the cognitive mechanisms that enable the processors, and PIFs that affect the mechanisms. The set of processors describes the generic cognitive process of achieving a macrocognitive function. For example, to achieve Detection, personnel start with a mental model of what to detect and the criteria of successful detection, followed by attending to the sources of the information to be detected, and then perceiving and recognizing the information. To achieve the task in a realistic job setting, personnel need to continue the Detection macrocognitive function while verifying the perceived information and making corrections, as needed, and then retaining or communicating the results of the detection. These last two processors involve within team interaction such as peer checking, supervision, and communication. In Figure 2-3, the processors for each of the macrocognitive functions are labeled as D1-D5, U1-U5, DM1-DM6, E1-E5, and T1-T7, respectively, and are explained in Section 2.3.

Macrocognitive Cognitive PIFs Processors functions mechanisms Processor - D1 Cognitive Detection mechanism PIF 1 Processor - D5 Cognitive Processor - U1 PIF 2 mechanism Understanding Processor - U5 Cognitive PIF 3 mechanism Task and Processor - DM1 Decision-cognitive activities making Processor - DM6 Cognitive PIF 17 mechanism Processor - E1 Action Cognitive PIF 18 execution Processor - E5 mechanism PIF 19 Processor - T1 Cognitive Interteam mechanism coordination Processor - T7 Figure 2-3 The Cognitive Basis Structure Cognitive mechanisms, shown to the right of the processors in Figure 2-3, are behavioral or neural processing aspects that enable the success and reliability of the processors. Section 2.3 1 The term processors as used in this report is the same as proximate causes in NUREG-2114 [21].

2-4

explains the components of each macrocognitive function, which include the processors and cognitive mechanisms.

Examples of cognitive mechanisms of the detection macrocognitive function are attention, working memory, and vigilance. Some mechanisms are essential for the macrocognitive function to be accomplished; others ensure the reliability of a macrocognitive function. For example, in the detection macrocognitive function, working memory is essential for retaining perceived information in the brain, while vigilance maintains the brains neural network to actively respond to external stimuli. Cognitive mechanisms work randomly (i.e., it involves a chance or probability) because of the brains random neural activities. Thus, human errors that result from the ineffectiveness of cognitive mechanisms are probabilistic, not deterministic.

Cognitive mechanisms work effectively under certain conditions, referred to as capacity limits.

Outside its capacity limits, a cognitive mechanism becomes less effective or completely ineffective. For example, in the detection macrocognitive function, working memory has a capacity limit of approximately 7 to 11 items, beyond which working memory deteriorates [25].

Cognitive experiments show that the percentage of correctness of remembered items decreases as the number of items to be remembered in the task exceeds seven [26]. Working memory can also be influenced by other capacity limits. For example, working memory needs attention to be consolidated, it decays over time, and other concurrent tasks can interfere with it

[27].

The purpose of including cognitive mechanisms in the cognition model is to understand how the processors and macrocognitive functions can be achieved reliably and how they are affected by various PIFs, shown at the far right of Figure 2-3. Thousands of research papers have reported findings on cognitive mechanisms, many of which are interrelated or intermingled. This report did not intend to make an exhaustive list of all the cognitive mechanisms reported in the literature, nor to reconstruct the mechanisms known to be orthogonal or independent of each other. The cognitive basis structure includes only the cognitive mechanisms that are well studied and demonstrated to be prevalent in the success and reliability of the processors. The cognitive basis structure also includes the well-studied, prevalent capacity limits outside of which a cognitive mechanism becomes less effective.

Ineffectiveness of a cognitive mechanism increases the chances of errors in the associated processors. PIFs are the factors that positively or negatively affect human performance and influence the effectiveness of cognitive mechanisms and subsequently influence the likelihood of the success or failure of a task. The links between the cognitive mechanisms and PIFs are explained in Chapter 3 and APPENDIX B.

The purpose of the causal tree is to explain the possible success or failure of a macrocognitive function to accomplish a task. The causal tree identifies the processors needed in achieving the task activities, the potential cognitive mechanisms involved in those processors, and what contexts (PIFs) may enable or inactivate those mechanisms. Therefore, the causal trees illustrate how and why macrocognition may fail.

The generic structure of a causal tree is shown in Figure 2-3. Starting from the left in the figure, the blocks in the first box column represent the macrocognitive functions that the tree is analyzing. The blocks in the second column represent the processors that achieve the macrocognitive functions. Each processor is then linked to a set of cognitive mechanisms (i.e.,

the blocks in the third column). Each cognitive mechanism is connected to the relevant PIFs (in the fourth column) for that mechanism. On the other hand, the causal tree from right to left can be used to identify why personnel fail a macrocognitive function. The PIFs in the event context challenge the capacity limits of the linked cognitive mechanisms and make them ineffective, 2-5

which leads to errors in the connected processors that, in turn, increases the likelihood of failure of the related macrocognitive functions. Note that different processors can be associated with some common cognitive mechanisms, and the same cognitive mechanism may associate with more than one processor. The same is true for the connections between PIFs and the cognitive mechanisms. Each causal tree corresponds to a set of detailed tables in APPENDIX A and APPENDIX B that provide supporting information for the psychological basis of each node in the tree. Together, APPENDIX A and APPENDIX B serve as the psychological foundation for the qualitative and quantitative analysis methodology of IDHEAS-G. Moreover, Appendices A and B serve as a tool for HRA analysts to better understand how and why personnel may succeed or fail human actions under a given context.

Overall, the cognitive basis structure is a multilevel model, which bridges neural information processing (i.e., cognitive mechanisms), to microcognitive processes (i.e., processors), and then to macrocognitive functions. It also bridges individual and within-team performance to interteam and organizational level performance. It provides a comprehensive and explicit picture of how personnel perform tasks in a work environment. As illustrated in Figure 2-2, macrocognitive functions corresponding to the cognitive activities involved can represent a human task. They can also be viewed as generic tasks that constitute complex human performance. Also, the cognitive activities can be viewed as the observable aspects of the macrocognitive functions. A human task can be broken into these cognitive activities and thereby represented by corresponding macrocognitive functions.

2.3 Macrocognitive Functions This section describes the details of each macrocognitive function, including (1) the types of cognitive activities demanded for the function, (2) the processors, and (3) related cognitive mechanisms.

2.3.1 Detection Figure 2-4 illustrates the components of detection, which are explained in Sections 2.3.1.1, 2.3.1.2, and 2.3.1.3, respectively.

Detection cognitive activities Detection processors Detection cognitive mechanisms D.a. Mental model of the D1. Initiate detection -

cues Establish the mental model for information D.b. Perception of sensory to be detected information D2. Select, identify, and D.c. Attention attend to sources of D.d. Working memory

Detect cues information D.e. Vigilance

Acquire (gather) D3. Perceive, recognize, information and classify information D.f. Information foliage D4. Verify and modify the D.g. Pattern recognition outcomes of detection D.h. Shared cognition within D5. Retain, a team document/record, or D.i. Infrastructure for communicate the exporting the outcomes information detected Figure 2-4 Components of the Detection Macrocognitive Function 2-6

2.3.1.1 Cognitive Activities for Detection The macrocognitive function detection obtains meaningful information in the work environment. It allows personnel to focus on target information pertinent to the task from copious amounts of information. Personnel may perceive information through various senses (e.g., seeing, hearing, touching, smelling) or with the use of instruments. Detection recognizes the meaning of the perception.

The following are general types of cognitive activities that require detection:

Detect cues. Cues are brief indications of system or personnel status important for the task being performed. Cues usually need to be detected as soon as they are present.

Examples of cues include system alarms, alert signs or signals, abnormal parameters, trends, or changes in indications. Some cues such as alarms are salient and can automatically capture a persons attention for detection; other cues may not be salient or even ambiguous, and they can be detected only through careful monitoring, searching, inspecting, or comparing.

Acquire (gather) information. Information to be detected can be as simple as a parameter or as complex as pages of a status report. Sources of information can be in various formats such as physical indicators in the work environment, computer displays, telecommunication devices, or a persons voice or body gesture. Humans acquire information by attending to known sources or by searching for sources.

2.3.1.2 Processors for Detection In a complex scenario where large volumes of information are presented dynamically, humans perceive abundant sensory inputs (images, voices, etc.), while they actively process the information needed for the task. That is the purpose of detection. It is not a snapshot perception of stimuli in a work environment. It is also not just passively responding to the onset of a cue. Personnel need to actively seek cues to perform a detection task. This is true even when responding to alarms. Although alarms are salient to capture human attention and trigger detection, in actual operations, experienced personnel have mental models of alarms, and they use those mental models to guide alarm detection. Detection begins with having the mental model of what is to be detected. Before information processing occurs for perception, detection requires forming a mental model about the target information, filtering out irrelevant information, and locating the target information to be detected. The mental model includes knowledge (templates) and criteria for the target information, where the target information may be located, and how the target information may be acquired.

Humans can attend to only a limited amount of information at a time. Detection filters raw information in the environment; selects, perceives, and processes meaningful information; and finally retains or communicates the meaning of the perceived information. Past experience and training affect the meaning associated with a particular percept. That is, the raw sensory stream is imbued with meaning, and meaning is thus subject to the cognition of the individual beholding it. Therefore, the outcome of detection is determined by the sensory stimuli; the processes of filtering, selecting, and perceiving information; and the knowledge that recognizes the meaning of the information.

Although perception is achieved through human sensory organs, some tasks may need special equipment or devices to detect the presence or measure the parameters of the target information. With or without the aid of special equipment, sensory (e.g., visual or auditory) information is perceived as stimuli through means such as attending to alarms, monitoring 2-7

parameters, searching through an area, reading, etc. The perceived information needs to be recognized for its meaning and classified to match ones mental model of the cues of detection.

Mental model matching involves verification (through self-checking, peer-checking, or supervising) and iteration of the detection processes as needed. Finally, the detected information needs to be exported for its use. The information may be retained mentally as inputs to other macrocognitive functions, communicated with others, or physically recorded.

The processors for detection are summarized as follows:

D1. Initiate detectionEstablish the mental model for information to be detected.

D2. Select, identify, and attend to sources of information.

D3. Perceive, recognize, and classify information.

D4. Verify and modify the outcomes of detection.

D5. Retain, document/record, or communicate the outcomes.

While an individual can achieve these processors, each processor may involve within-team collaboration through information sharing, supervision, and peer-checking. For example, information sharing among members of a team can help individuals form the correct mental model for a cue or information to be detected, especially in unfamiliar scenarios or environments. Peer-checking and supervision are also important for verifying the outcomes of detection so that errors can be noticed and corrected.

2.3.1.3 Cognitive Mechanisms for Detection The following cognitive mechanisms are included in the cognitive basis structure for detection.

The open circle bullets list some prevalent capacity limits of the cognitive mechanism.

D.a. Mental model of the cues: The mental model guides all other cognitive mechanisms. It sets up ones expectation for the cues and criteria for the cues.

o wrong or biased expectation o narrowly focused expectation leads to missing relevant information o failure to adjust the expectation based on situation o ambiguous mental model (such as ambiguous or conflicting criteria for the cues to be detected)

D.b. Perception of sensory information: The neuronal process of perceiving sensory information is based on responses of sensory neurons to stimuli.

o The sensory signal is too weak to be perceived.

o The signal is embedded in noise, and thus the perception may be incorrect.

D.c. Attention: This is the control mechanism for selecting pertinent information.

o failure to focus attention at the expected signal o failure to maintain sustained attention o failure to shift attention D.d. Working memory: This control cognitive mechanism retains the perceived information and the objects to be monitored or searched.

o Working memory overflows (i.e., it exceeds the working memory capacity).

o Working memory is lost or erroneous because of interruption or disruption.

2-8

o Working memory for one task is interfered with by the memory for other concurrent tasks.

o Memory is not consolidated o Memory decays over time.

o Similar information or past experience interferes with memory.

D.e. Vigilance: The alertness of the brain is necessary for perceiving stimuli, continuously monitoring, searching for information, and retaining information in working memory.

o Vigilance is reduced after sustained cognitive activities.

o Vigilance is attenuated after a sustained no signal period.

D.f. Information filtering: This is the neuronal process of filtering out irrelevant information and detecting salient changes in the environment.

o Irrelevant information cannot be filtered out because it is unorganized or the mental model of the target information is unclear.

o Salient signal or changes may not pop out for perception because the signal is not salient enough or there are many competing salient signals.

D.g. Pattern recognition: Perceived information is recognized and classified as meaningful cues. The brains sensory systems compare and match perceived patterns with the mental model templates and criteria stored in the mental model.

o wrong templates o criteria too complex to classify meaning of the perceived patterns D.h. Shared cognition within a team: Shared cognition allows individuals within a team to perform detection processes such as establishing mental models, verifying perceived information, or exporting the outcomes of detection.

o Mental models do not have a common ground.

o Shared cognition is not adequate or updated.

o Shared cognition fades as the result of mental fatigue.

o Communication is broken (not initiated, not perceived, or miscommunicated).

o Peer-checking or supervision is missing.

o Group bias causes other potentially correct models to be self-censured, not considered, or rejected.

D.i. Infrastructure for exporting the information detected: The information detected by an individual or team needs to be retained or communicated with others.

o Hardware for recording or transmitting is not available or does not work.

o Personnel are unable to use the infrastructure.

As stated in Section 2.2, many of the cognitive mechanisms are not orthogonal and may have substantial overlap with each other. For example, vigilance and attention have substantial overlap; mental model selection is also addressed in information foliage; signal not salient falls under both information foliage and perception; and narrowly focused expectation seems akin to 2-9

failure to shift attention. With such overlap among the cognitive mechanisms, their failure or ineffectiveness is not a meaningful indicator of failure modes of human tasks. Rather, the cognitive mechanisms can explain how macrocognition may fail and what may lead to the failure.

2.3.2 Understanding Figure 2-5 illustrates the components of understanding, which are explained in Sections 2.3.2.1, 2.3.2.2, and 2.3.2.3, respectively.

Understanding cognitive activities Understanding processors Understanding cognitive mechanisms U1. Assess/select data

Maintain situational awareness U2. Select/adapt/develop the mental model

Assess status based U.a. Data on indirect U3. Integrate data with the mental model to U.b. Selection of data information generate the outcome U.c. Mental model

Diagnose problems of understanding and resolve (situational awareness, U.d. Integration of data with conflicting diagnosis, resolving mental model information conflicts) U.e. Working memory

Make predictions or U4. Verify and revise the U.f. Shared cognition within form expectations for outcome through a team the upcoming iteration of U1, U2, and situation U3 development U5. Export the outcome Figure 2-5 Components of the Understanding Macrocognitive Function 2.3.2.1 Cognitive Activities for Understanding The macrocognitive function of understanding means to mentally interpret the situation in which a task is performed. The output of understanding is an evaluation of current conditions to determine if they are within acceptable limits to conclude a diagnosis or to identify the underlying causes of any abnormalities.

This macrocognitive function integrates individual pieces of cues or information with a persons mental model of the task being performed to form a coherent interpretation of the situation. This macrocognitive function allows humans to question what is known, evaluate what is conjectured, hypothesize and diagnose, and integrate facts with theories. Understanding can result in situational awareness, diagnosis of symptoms, resolution of conflicting information, or hypothesis or prediction about an event progression. Humans may generate different understandings for a given situation because of different mental models or different integration of the information and mental models. The following types of cognitive activities require understanding:

Maintain situational awareness. In becoming aware of the system status, such as the occurrence of a steam generator tube rupture (SGTR), there is no single indication for the status. An operator determines that an SGTR has occurred based on evaluation of 2-10

multiple pieces of information such as main steamline radiation, steam generator (SG) water levels, and blowdown line radiation. Moreover, information on SG water levels may not be helpful in that the SG level control system (at least when the plant is at-power) will maintain levels within the control band. The operator would then need to rely on information such as main feed/main steam flow mismatches or the increase of the charging flow. These pieces of information must be integrated to determine whether there is an SGTR and to identify the ruptured SGs.

Assess status based on indirect information. Assessing system status typically involves integration, processing, and inference from many pieces of information to interpret the information. For example, assessment of NPP core damage involves many aspects of the plant status, such as whether core debris has relocated, whether the reactor pressure vessel is breached, and whether the containment has uncontrolled leakage.

These plant conditions do not have instruments to provide a direct indication. The crew must integrate multiple pieces of information to determine the status.

Diagnose problems and resolve conflicting information. This involves understanding the causes of abnormal signals or the reasons for conflicting information. Examples of diagnosis are determining the causes of a pump vibration simultaneously with multiple component malfunctions.

Make predictions or form expectations for the upcoming situation development.

2.3.2.2 Processors for Understanding The process of achieving understanding involves establishing the mental model of the situation, interpreting pieces of information using the mental model, integrating the information with the mental model, and generating the output. The central theme of the process is its dynamic aspect (i.e., the process iterates until a satisfactory outcome is achieved).

The understanding function involves three concepts: data, mental model, and mental representation. Data are what one perceives from the external world. In complex scenarios, humans select and process the data for understanding. The data that serve as the input to the understanding function may already represent the integration of both the perceived external world and the persons initial understanding of what was perceived. The mental model, also referred to as a persons knowledge base, consists of a persons internal framework for the physical and functional characteristics of the systems, tasks, and mission. Formal education, training, and experience are the basis for this model. People use their mental model to interpret the data to generate a mental representation of the situation, which is understanding. In this paradigm, the knowledge base or mental model is considered relatively static (at least for the period of interest), but the mental representation can be dynamic based on the persons attempt to explain observed data. It is important to recognize the static nature of what people know versus the dynamic understanding of the situation they create.

Klein et al. [28] explain that sensemaking (i.e., understanding) is a process of fitting data to a mental model that helps to filter and interpret the data. The process generates a mental representation or a frame of the situation. A frame defines the elements of the situation, describes the significance of these elements, describes their relationship to each other, filters out irrelevant data, and highlights relevant messages. Frames can organize relationships that are spatial (maps), causal (stories and scenarios), temporal (stories and scenarios), or functional (scripts). The process iterates through testing and improving the frame.

2-11

The processors for understanding are organized as follows:

U1. Assess/select data.

U2. Select/adapt/develop the mental model.

U3. Integrate data with the mental model to generate the outcome of understanding (situational awareness, diagnosis, resolving conflicts).

U4. Verify and revise the outcome through iteration of U1, U2, and U3.

U5. Export the outcome.

While an individual can achieve these elements, some or all of the elements may involve interaction of team members to carry out a task. For example, all of the processors require within-team interaction to develop a team-level situational awareness; team leadership is important to ensure that adequate iteration is made for a thorough diagnosis or resolution of conflicts. The processors model the macrocognitive function understanding of an individual, if the task is performed by a single person or by a coherent team working together to achieve understanding.

2.3.2.3 Cognitive Mechanisms for Understanding The following cognitive mechanisms are included in the cognitive basis structure for understanding. The open circle bullets list some prevalent capacity limits of the cognitive mechanisms.

U.a. Data: This is the input for understanding a situation; it includes all the evidence available in the work environment (including the cues and information detected).

o Data are not sufficient to support the complete and correct understanding of the situation.

o Data are not current.

U.b. Selection of data: Personnel selectively use data for understanding.

o Data are not properly recognized, classified, or distinguished.

o Attention is given to wrong or inappropriate data.

o Improper data or aspects of the data are selected.

U.c. Mental model: This includes frames of past experience and knowledge of the situation.

o Personnel use an incorrect or inadequate mental model to interpret or integrate information.

o A mental model is inappropriately preserved or confirmed when it should be rejected.

o A mental model is inappropriately rejected when it should be preserved or confirmed.

o No mental model exists to interpret the information or situation.

U.d. Integration of data with mental model: Data are evaluated, manipulated, and compared with the mental model to generate the outcomes of understanding.

2-12

o Improper data or aspects of the data are selected for comparison with or identification of a mental model.

o The integration is incorrect or fails to match data to a mental model.

o Mental manipulation or evaluation of the data is inadequate, inaccurate, or otherwise inappropriate.

U.e. Working memory: Working memory maintains the data online in ones awareness for integration of the data and mental model.

o Memory overload occurs.

o Memory fades over time.

U.f. Shared cognition within a team: Members in a team share their mental models of the situation as the team works toward completion of the goal. Team cognition describes the cognitive processes at the individual level that are dependent on and interact with the processes at the team level.

o Individuals mental models do not have a common ground.

o Shared cognition is not adequate or updated.

o Shared cognition fades as the result of mental fatigue.

o Communication is broken (not initiated, not perceived, or miscommunicated).

2.3.3 Decisionmaking Figure 2-6 illustrates the components of decisionmaking, which are explained in Sections 2.3.3.1, 2.3.3.2, and 2.3.3.3, respectively.

Decisionmaking cognitive activities Decisionmaking processors Decisionmaking cognitive mechanisms DM1. Select and implement decisionmaking model

Make a go/no-go DM2. Manage the goals and DM.a. Decisionmaking decision for a pre-decision criteria model specified action DM3. Acquire and select DM.b. Data for

Select among data for decisionmaking multiple options or decisionmaking strategies DM.c. Selection or judgment DM4. Make decision

Change or add to a DM.d. Cognitive biases (judgment, strategies, pre-existing plan or plans) DM.e. Deliberation or strategy evaluation of decision DM5. Simulate or evaluate

Develop a new the decision or plan DM.f. Team decisiomaking strategy or plan DM6. Communicate and authorize the decision Figure 2-6 Components of the Decisionmaking Macrocognitive Function 2-13

2.3.3.1 Cognitive Activities for Decisionmaking Decisionmaking is to determine the optimal choice among the alternatives or develop a plan or strategy to achieve the task goals. Decisionmaking can be determining whether an existing strategy should be implemented or how and when it should be implemented to respond to the situation. Alternatively, it can involve developing complex strategies or plans for a situation.

Decisionmaking can be a one-time activity (i.e., the decision or plan is to be executed once it is made) or dynamic. Dynamic decisionmaking is interdependent decisionmaking that takes place in an environment that changes over time either due to the previous actions of the decision maker or due to events that are outside of the control of the decision maker [29].

Klein [30] defines naturalistic decisionmaking in operational environments associated with crew systems with features such as dynamic and continually changing conditions, real-time reactions to these changes, ill-defined tasks, time pressure, significant personal or corporate consequences for mistakes, and experience of the decision makers. The outcomes of decisionmaking can be as simple as a single instruction or as complicated as an emergency response plan including personnel or organizational collaboration; multiple interdependent actions; and subgoals, success criteria, and contingencies for the planned activities.

Regardless of decision variety and complexity, the macrocognitive function of decisionmaking is to achieve tasks of which the output is an explicit decision, strategy, or plan to guide personnels actions.

In some operational settings, decisionmaking is characterized as being dictated or largely driven by procedures. For example, control room operation of NPPs involves well-trained licensed operators following normal or emergency procedures. It may appear that operators only need to select proceduralized decision paths. In reality, procedures may not be applicable to every situation. For unusual or unexpected situations, it is likely the procedures cannot be verbatim followed. The operators need to interpret or even revise the existing procedures or guidelines to address the situations. In some situations, the operators may need to develop additional plans to supplement the existing procedures and guidance to handle current situation.

The types of cognitive activities that require decisionmaking include the following:

Make a go/no-go decision for a prespecified action. This is a decision on whether an action should be performed.

Select among multiple options or strategies. This often involves prioritization.

Change or add to a preexisting plan or strategy. Changes to a preexisting plan or strategy may include changes of personnel, success criteria, subgoals, plan or strategy constraints, or other factors.

Develop a new strategy or plan. Decisionmakers need to develop a strategy or plan for unforeseen events or dynamically changing situations, which require real-time reactions to the situations.

2.3.3.2 Processors for Decisionmaking Decisionmaking is the judgment of what should be done and the decision to do it. People make decisions based on different decisionmaking models. Normative decisionmaking models emphasize finding the optimal solution. The prospect theory of decisionmaking focuses on minimizing risks or losses in the outcome of a decision. Naturalistic decisionmaking (NDM) theories consider the decisionmaker in a real-world setting in which decisions are typically 2-14

embedded in a larger task. Researchers in this area focus on studying time pressure, uncertainty, ill-defined goals, high personal stakes, and other complexities that characterize decisionmaking in real-world settings [31]. In NDM, decisions are typically based on finding a sufficient solution instead of finding an optimal solution, because the uncertainties and complexities of the situation often make it impossible to determine what is optimal. Rather than choosing an alternative which is considered sufficient to address the current situation among many alternatives, Kleins models point more toward a recognition-primed approach, in which the solution is selected based on the experience of the individual.

Researchers have studied NDM in different work domains, such as aviation [30], [32],

firefighting [30], military combat [30], [33], and NPPs [34] and developed a generalized model of the decisionmaking process. First, a decisionmaking infrastructure needs to be built or adapted if it is not preexisting. The infrastructure includes personnel and their responsibilities, the ways information for decisionmaking is assessed and used, the strategy or rules of integrating judgments, and the decision approval or authorization chain. Within the infrastructure, decisionmakers manage the goals and the other considerations for decisions. For complex scenarios, often there are many goals and many possible criteria for the success of a goal, so selecting goals and criteria is a necessary part of the decision process.

In deciding to achieve the selected goals and meet the selected criteria, decisionmakers assess the situation based on relevant cues from the situation, understanding of the situation, and expectations about how the situation may evolve, or the stated goals for success in comparing the situation to previously encountered situations. A decisionmaker can respond to a typical situation with typical and known solutions. A novel situation, on the other hand, will require new solutions. Decisionmakers review alternative solutions to the problem. The solutions that are considered most typical would be considered first. The solutions are evaluated one at a time, as the decisionmaker mentally simulates the implementation of the solution and the outcome.

Based on this simulation, the decisionmaker implements the solution as-is, changes it somewhat, or discards it and chooses or develops another solution.

A large body of research in NDM has explained real-world decisionmaking by accounting for the environmental and situational impacts on the decision. Zsambok [35] defines NDM as the way people use their experience to make decisions in field settings. It considers the decisionmaker in a real-world setting in which decisions are typically embedded in a larger task. Researchers in this area focus on studying time pressure, uncertainty, ill-defined goals, high personal stakes, and other complexities that characterize decisionmaking in real-world settings [31].

NDM theories describe the decisionmaking process as recognizing the information or data in the situation and forming initial response plans, evaluating the plans through mental simulation, seeking more information to modify the plan or select alternative plans, and finally selecting an applicable course of action.

NDM provides a good framework for modeling the decisions made by NPP operators in accident situations. Within an NPP, the situation consists of an experienced decisionmaker who is largely directed by procedures. Greitzer, et al. [36] present an integrated NDM model that can be used to represent decisionmaking by operators. The model includes the idea of critiquing by modeling additional loops of mental simulation during the pattern recognition process. In the case of experienced operators trained in many situations and recovery strategies, when several procedures are available, the operator may take three approaches when planning a response

[37]:

(1) In a very familiar setting in which the cues match the procedural guidance almost perfectly, the operator may follow the procedures with little diagnosis needed.

2-15

(2) In a familiar setting that deviates just slightly from either procedural guidance or from previously encountered situations, the operator will have to adapt some and plan a response based on an analogous experience.

(3) In a novel setting, the operator will have to construct a new response plan using his or her knowledge of the plant and system and previous experience.

One of the defining features of decisionmaking in NPPs is the dynamic nature of the event.

Maintaining appropriate situational awareness of the event, updating the mental representation of the situation, and planning the response accordingly are important steps [38]. These steps continue even when procedures exist. During the evolving and dynamic NPP event, operators were found to follow their procedures, but they also actively construct a mental representation of plant state and use this mental representation to identify malfunctions, anticipate future problems, evaluate the appropriateness of procedure steps given the situation, and redirect the procedural path when judged necessary [39]. Overall, NDM fits within what seems to be a very structured, procedurally driven environment. The key (and reason why NDM is a fit for NPP operators) is that while decisions are executed through procedures, operators do much cognitive decisionmaking in selecting and implementing procedures (e.g., deciding what an uncontrolled rise in SG water level means is a difficult task).

Largely based on the concepts of NDM, the processors for decisionmaking are organized into the following:

DM1. Select and implement decisionmaking model.

DM2. Manage the goals and decision criteria.

DM3. Acquire and select data for decisionmaking.

DM4. Make decision (judgment, strategies, plans).

DM5. Simulate or evaluate the decision or plan.

DM6. Communicate and authorize the decision.

Depending on the characteristics of the decision to be made, some or all of the processors may be needed. For example, if a decision is solely made by one person, then DM1 (select and implement decisionmaking model) is not needed. Decisionmaking on a team may adopt various infrastructures. All the team members may be interactively involved in each of the processors.

Alternatively, the infrastructure may allocate the processors within the team. For example, in normal or emergency operations in an NPP control room, the decisionmaking infrastructure is already specified; the shift supervisor performs DM2 (manage the goals and decision criteria) through DM6 (communicate and authorize the decision) according to procedures, while other crew members assist the shift supervisor. However, procedures may not be applicable or detailed enough for a scenario, and then the entire crew may iterate the decisionmaking process to make the decision. Nevertheless, the processors model the cognitive process of decisionmaking for decisionmakers within a team, regardless of whether the processors are carried out by one person or multiple individuals on a team.

2.3.3.3 Cognitive Mechanisms for Decisionmaking The following cognitive mechanisms are included in the cognitive basis structure for decisionmaking. The open circle bullets list some prevalent capacity limits of the cognitive mechanisms.

2-16

DM.a. Decisionmaking model:

o Incorrect goals selected. Errors may arise if the operators select the wrong goal.

In a variant of this cognitive mechanism, the operator selects an implausible goal that cannot be achieved.

o Lack of or incorrect goal prioritization. Goals may be ordered incorrectly in an operators mind or given the wrong priority, such that less important goals are addressed first.

o Incorrect judgment of goal success. The threshold used by the operator to judge goal success may be incorrectly set too low or be incorrectly determined to be met when it was not.

DM.b. Data for decisionmaking:

o incomplete data o incorrect or unreliable data o ambiguous data sources or data characterization DM.c. Selection or judgment:

o failure to retrieve previous experiences o incorrect recall of previous experiences o incorrect comparison of the mental model to previously encountered situations DM.d. Cognitive biases:

o Confirmation bias and availability bias may be particularly pertinent to causing errors in this phase of decisionmaking. The confirmation bias occurs when people seek data or information that are likely to be compatible with the beliefs the currently hold [40]. The availability bias (or heuristic) occurs when people judge an event as likely or frequent if instances of the event are easy to imagine or recall [41].

o Overconfidence and anchoring are two other types of biases. Overconfidence affects the operators confidence in the ability of an action to work. Especially if the operator has had previous success with an action, he or she may be overconfident in its ability to work in the present case. The anchoring effect states that people are biased toward the first option they see or the first judgment they make. Therefore, an operator may take an unsuitable action because of bias toward choosing the first action that occurs to him or her.

o Other types of biases may affect decisionmaking.

DM.e. Deliberation or evaluation of decision:

o Inaccurate portrayal of action. This cognitive mechanism includes incorrectly characterizing the action (i.e., forgetting a step of the action during the mental stimulation) or incorrectly predicting how the action will be implemented.

o Incorrect inclusion of alternatives. The operator may forget to include some alternatives that should be considered.

2-17

o Inaccurate portrayal of the system response to the proposed action. This cognitive mechanism manifests in the operator incorrectly predicting how the system will respond to the proposed action.

o Misinterpretation of procedures. Response planning within the NPP is done by consulting procedures. An error may occur because either incorrect procedure selection or inaccurate interpretation of the procedures has complicated logic, making the procedures difficult to use and understand.

o Inadequate updating of mental representation of the situation for evaluating the decision. This is particularly important for dynamic decisionmaking, which describes interdependent decisionmaking that takes place in an environment that changes over time either due to the outcomes of the previous actions or due to unforeseen events.

DM.f. Team decisionmaking through which a team, rather than an individual, chooses alternatives.

o Groupthink can occur when bringing a team together (group cohesion) o The desire for group conformity and unanimity o Lack of a structured decisionmaking process to prevent the groupthink-induced short cuts Compared to other macrocognitive functions, the cognitive mechanisms for decisionmaking and their capacity limits are less delineated in the literature.

2.3.4 Action Execution Figure 2-7 illustrates the components for action execution, which are explained in Sections 2.3.4.1, 2.3.4.2, and 2.3.4.3, respectively.

2.3.4.1 Cognitive Activities for Action Execution Action execution is implementation of the intended actions to achieve the task goal. This function involves human manipulation of hardware or software that would consequently alter the status of the target objects (e.g., machines, systems). Action execution can be as simple as carrying out a few physical steps, or it can involve performing a complex control action in which multiple physical steps are interdependent and require personnel to monitor the status of the target object. Action execution may involve one person, multiple individuals, or a coherent team. Action execution may be completed in a few minutes, days, or even months. While routine actions are typically executed with step-by-step procedures or trained scripts, nonroutine actions often need personnels skill of craft.

Although action execution appears to be achieved through physical movement, it requires many microcognitive processes, such as reading instructions, assessing the action scripts, evaluating action criteria, attending to the target object, monitoring parameters for execution, or assessing system status for controlling the execution. Moreover, complex action execution by teams requires cooperation, coordination, and communication.

There are many ways to classify human actions. Examples are individual versus team actions, or discrete versus continuous actions. Actions are also classified by the ways that actions are performed (e.g., simple versus complex actions). Because IDHEAS-G models human 2-18

performance with macrocognition, it classifies action execution as executing simple versus complex actions based on the cognitive demands of the actions. The cognitively simple or complex actions can be further characterized as a long-lasting actions, control actions, fine-motor actions, or physically strenuous actions.

Action Execution cognitive activities Action Execution processors Action Execution cognitive mechanisms E.a. Physical movement and motor skills E.b. Mental model of the actions and the systems to be acted on E.c. Working memory E.d. Attention

Execution of a E1. Assess action plan and E.e. Vigilance cognitively simple criteria E.f. Sensory feedback of action motor movement E2. Develop or modify

Execution of a action scripts E.g. Automaticity cognitively complex action E3. Coordinate and E.h. Action programming command action

Long-lasting action implementation E.i. Executive control

Control action E4. Implement action E.j. Error monitoring and scripts correction

Fine motor action E5. Verify and adjust E.k. Initiation of action

Physically strenuous execution action execution outcomes E.l. Spatial precision or accuracy of action execution E.m. Timing precision of action execution E.n. Coordinate motor movement of action execution Figure 2-7 Components of the Action Execution Macrocognitive Function The types of activities that require action execution include the following:

Execution of a cognitively simple action. A simple action can be executed through one or several action steps.

Execution of a cognitively complex action. A complex action is one that has interdependent steps or requires personnel collaboration. Below are some characteristics of complex actions:

2-19

o Intermingled action sequences. The action involves many steps that must be performed in a particular sequence, and there are multiple simultaneous, intermingled sequences.

o Multiple locations. The action is performed at multiple locations.

o Multiple system functions. Multiple systems or system functions need to be addressed in the execution of an action. Those system functions may be interconnected.

Long-lasting action. Executing the entire action takes multiple hours or days.

Control action. The execution of action steps is not continuous and need to wait for system parameters to meet certain criteria.

Fine-motor action. Action execution requires fine-motor skills, such as operating a delicate piece of equipment.

Physically strenuous action. Actions are physically strenuous, such as lifting heavy equipment or traveling with heavy materials.

2.3.4.2 Processors for Action Execution Action execution includes receiving the action commands; confirming, clarifying, and questioning the action commands; adapting or developing action scripts or procedures for implementing the action; executing the motor activities according to scripts or procedures (including obtaining needed access keys, tools, and equipment; traveling to the action locations; verifying the target of action; monitoring system status or parameters). Action execution at the macrocognitive function level also includes verifying the action completion and monitoring the actions effectiveness. Compared to relatively simple motor activities in a control room such as using computer interfaces, pushing a button, or turning a switch, action execution outside a control room is more apt to require motor skills (e.g., turning a manual valve or actions needed to transport and set up equipment in flood or high winds).

The processors for action execution include the following:

E1. Assess action plan and criteria.

E2. Develop or modify action scripts.

E3. Coordinate and command action implementation.

E4. Implement action scripts.

E5. Verify and adjust execution outcomes.

Not all of the elements are needed for every action execution. For individuals performing simple, routine tasks, E4 (implement action scripts) may adequately represent the underlying cognitive process; other elements may not be needed or may be an automatic part of E4.

However, if the manual action is complicated or is performed in an unusual scenario or with a different setting from the routine, or the action is performed collaboratively by a team, then most or all of the elements may be needed. For example, team leadership is important to initiate and ensure E1 (assess action plan and criteria) and E2 (develop or modify action scripts); E5 (verify and adjust execution outcomes) may encompass peer-checking by teammates. Thus, these elements model the cognitive process of individuals and within-team interactions.

2-20

2.3.4.3 Cognitive Mechanisms for Action Execution The following cognitive mechanisms are included in the cognitive basis structure for action execution. The open circle bullets list some prevalent capacity limits of the cognitive mechanisms.

E.a. Physical movement and motor skills:

o Personnel are physically unable to perform the action.

o Learned motor skills fade with time.

o There is a tradeoff between motor speed and accuracy.

o There are limitations in fine-motor skill. The time a user takes to reach or hit a target in a human-system interface (HSI) increases with the decreasing size of the target (Fitts Law of motor movement) [42].

o There are limitations in physical movement. Humans have ergonomic limitations (e.g., the amount and duration of force personnel can exert, body coordination, speed of motor activities).

E.b. Mental model of the actions and the systems to be acted on:

o Population stereotypes. This involves whether mappings are consistent with expectations based on experience and conventions. Mappings that are inconsistent with population stereotypes will be more error prone. For example, a typical population stereotype is that green is used for go and red for stop.

Personnel tend to make errors if it is the other way around. This is a pertinent issue for nuclear plant designsometimes, red is used to indicate energized/with flow and green for deenergized/without flow. At other times, green denotes normal position and red an abnormal condition. Above are population stereotypes typically encountered.

o Indication of system status may be different from the design. For example, manual control can lead to system instability; excessive oscillations (overshooting and undershooting target values and trajectories) may result in inadvertently exceeding critical parameter limits (e.g., the SG level may exceed reactor trip setpoints). Another example is that indication of the actual state of a system is incompatible with the personnels knowledge of the system.

o The expected responses of the systems to be acted on may have changed. For example, the unexpected dynamics of the system response to the forces applied differently from the conventions (i.e., there are response lags or shrinks and swells that complicate the ability to control a parameter).

o Movement incompatibility involves the mapping between the direction of movement of the control and the corresponding value being controlled. When personnel move a position switch, rotary, or sliding control, movement compatibility defines the set of expectations that the personnel will have about how the display will respond to the control action. For example, an operator may expect that moving a switch up would make the corresponding parameter displayed go up. Violations of movement compatibility are more likely to result in errors.

2-21

E.c. Working memory is the same mechanism as described for the detection function (see Section 2.3.1.3). In the context of action execution, only one example of capacity limits is provided.

o Memory decays over time. If an action is planned but is not executed immediately or the action is executed with long time intervals between the steps, the unexecuted action scripts are maintained in working memory. Working memory decays over time when it is not attended. Errors of omission typically occur as the result of excessive demands on working memory.

E.d. Attention is the same cognitive mechanism as described for the detection function (see Section 2.3.1.3).

E.e. Vigilance is the same cognitive mechanism as described for the detection function (see Section 2.3.1.3)

E.f. Sensory feedback of motor movement: To ensure that motor movements are precise and coordinated, the brain neural-motor system must constantly receive sensory information to adjust and correct the movements. According to closed-loop accounts of motor control, movement errors are detected by comparing sensory feedback to an acquired reference state. Also, movement programming has been shown to be optimized when the participant is permitted to see his or her hand resting on the starting base before movement initiation.

o Error monitoring and correction. Differences between the reference state and the movement-produced feedback result in an error signal that serves as a basis for a correction.

o Errors of commission often occur because of failures to detect stimulus deviance.

Precise and continuous sensory inputs adjust to motor functions to enhance action correctness and accuracy.

o Incorrect mental computation.

o Incorrect comparison of parameter.

E.g. Automaticity: Action automaticity is the ability to implement actions without occupying the brain with the low-level details required, allowing the action to become an automatic response pattern. It is usually the result of learning, repetition, and practice. The sequence of actions appropriate to solve a problem often must be discovered by trial and error and recalled in the future when faced with the same problem. Many routine tasks are performed almost automatically.

o Automaticity control. Automaticity is limited to the scope of the learning and training environment or context. Such actions become invalid if the context is changed, at which point personnel need to switch behavior by overcoming actions that are otherwise triggered automatically. Such behavioral switching can occur either retroactively based on error feedback or proactively by detection of a contextual cue.

E.h. Action programming: Execution of an action typically requires multiple steps of motor movement. The brain mechanisms programming the execution can program only one action at a time [43].

2-22

o Interference. If a task requires simultaneous action goals, the action programs for different goals can interfere with each other, resulting in loss of one action, incomplete action programming (e.g., missing a movement step or following the wrong order of movement), or transposing movement steps in two action sequences.

o Cost of switching. Performing concurrent tasks requires switching between them. Switching between tasks has the expense of slower and more error-prone execution of the actions. The aspects of the task set, including task variations, task-set overlap, and task-set structure, and modalities of the actions are related to action error rates caused by task switching.

E.i. Executive control: A cognitive system must be capable of running mental processes that virtually simulate action sequences aimed at achieving the goal. The lateral prefrontal cortex is critically involved in broad aspects of executive behavioral control. Neurons in this area take part in the selections of attention for action and the intended action.

Furthermore, the lateral prefrontal cortex is involved in the implementation of behavioral rules and in setting multiple behavioral goals. This area is responsible for strategic planning of macrostructures of event-action sequences.

o incorrect executive control o attention not focused on the intended action E.j. Error monitoring and correction: When executing goal-directed actions, brain neural systems monitor and correct errors in motor movement, especially for delayed or sequenced actions. This requires attention to be focused on the outcomes of motor movement to meet the success criteria.

E.k. Initiation of action execution: Action execution consists of motor movements necessary for a goal-directed activity. Initiating an action execution includes planning and enacting a series of motor movements. Initiation is controlled by brain areas that collectively interact to exert governance and control over executive function and intentionality of movements that require anticipation and the prediction of movement of others. Both planning and enactment require intention, attention, and working memory; therefore, they are subject to capacity limits of those mechanisms.

E.l. Spatial precision or accuracy of action execution: Spatial accuracy is the type of accuracy required of aiming movements for which spatial position of the movement's endpoint is important to the performance. A cognitive system is dedicated to the temporary maintenance of spatial-motoric representations, which are dynamically updated based on the feedback of the motor movement executed. Spatial accuracy is affected by attention, working memory, and sensory feedback of the motor movement.

Spatial accuracy decreases with the duration and complexity of the motor movements.

E.m. Timing precision of action execution: Timing precision is the type of accuracy required of rapid movements in which accuracy of the movement time is important to the performance. A persons ability to follow instructions depends in part on a limited-capacity working memory dedicated to temporarily retain the motoric, spatial, and temporal features of intended actions. Timing accuracy is subject to the capacity limits of working memory and attention.

E.n. Coordinated motor movement of action execution: Planning of action execution involves transformation of sensory inputs and execution instructions into sequences of movement steps. Feedback from ones and others movement aids in planning and executing 2-23

subsequent motor actions. This feedback allows perceiving change, which in turn feeds planning subsequent action steps or movements. The planned messages of motor movement originate at the motor cortex of the brain and stop at the Basal Ganglia, a complicated system that selects which instructions will be executed and which are inhibited. The selection is based on predicted variation in feedback control with changes in task demands and the correlation structure between different personnel. Feedback control relies on feedback information of ones and others execution of planned movement steps. Coordination of motor movements is subject to the capacity limits of execution planning and feedback control.

2.3.5 Interteam Coordination Figure 2-8 illustrates the components of interteam coordination, which are explained Sections 2.3.5.1, 2.3.5.2, and 2.3.5.3, respectively.

Interteam coordination activities Interteam coordination processors Interteam coordination cognitive mechanisms T1. Establish or adapt interteam coordination infrastructure T.a. Interteam coordination T2. Manage information infrastructure T3. Maintain shared T.b. Command

Communication situational awareness T.c. Control

Cooperation T4. Manage resources T.d. Line of communication

Coordination T5. Plan interteam T.e. Data processing and collaborative activities information T6. Implement decisions and management commands T.f. Shared mental model T7. Verify, modify, and control the implementation Figure 2-8 Components of the Interteam Coordination Macrocognitive Function 2.3.5.1 Interteam Coordination Activities Complex work often involves interactions among personnel from multiple entities, such as distributed individuals, cohesive teams, organizations, and authorities. The interteam coordination macrocognitive function models interteam collaborative activities including cooperation, coordination, and communication. This function focuses on how the various distributed entities collaboratively carry out a mission.

In contrast to interteam collaborative activities, within-team interactions are part of the other four macrocognitive functions. Within-team interactions consist of those activities performed by a coherent team such as an NPP control room crew. Examples of within-team interactions include adaptability, shared situational awareness, mutual performance monitoring, motivating team members/team leadership, mission analysis, sharing information, team decisionmaking, assertiveness, interpersonal relations, and conflict resolution. Such interaction takes place to 2-24

support the other four macrocognitive functions. Ineffective interaction is modeled within the failures of those macrocognitive functions.

The other four macrocognitive functions all share some cognitive processes with the interteam coordination function, such as maintaining shared situational awareness, monitoring task performance of other entities, collaboratively making decisions, and collaboratively executing an action. The interteam coordination function is distinct from those cognitive processes in that its focus is on communication, coordination, and collaboration among distributed entities to achieve the mission. The interteam coordination macrocognitive function is to achieve between-team communication, coordination, and collaboration, while the goal of the within-team personnel interaction in the other four functions is to achieve those functions. An example of within-team interaction is an NPP control room crew. It consists of three or more NPP operators working together to perform control room tasks. In contrast, an emergency operation involves control room crew, field operators outside the control room, and personnel in a technical support center (TSC). The interactions among these distributed entities are achieved through the interteam coordination macrocognitive function. Figure 2-9 illustrates the scope of the interteam coordination function and its distinction from within-team interaction.

High Central authority control Interteam coordination Within-team interaction Figure 2-9 Teamwork between Teams versus Interaction within a Coherent Team The following types of activities are included in the interteam coordination macrocognitive function:

Communication. These activities relate to the transmission of information between the involved entities. Such communication often involves a chain of transmission stages involving different team entities. Interteam communication typically requires communication equipment such as phones and radios.

Cooperation. These activities provide infrastructure and instructions and monitoring of the activities of other entities. Cooperation ensures that personnel activities are properly authorized and implemented and rules and regulations are enforced. Cooperation may occur with clear chains of supervision or authority, or it may occur in entities with no 2-25

specified formal leadership structure. Cooperation may also occur with individuals or teams under the supervision of multiple parallel leadership structures.

Coordination. These are activities that oversee the performance of the entities to ensure that all aspects of the work are properly addressed. Coordination includes prioritizing and coordinating tasks and resources, commanding and controlling task execution, changing requirements of a task or situation flexibly, and helping other entities.

The purpose of the interteam coordination function is to achieve between-team collaborative tasks such as management, authorization, and command and control. Often, achieving tasks that require interteam coordination involves resources such as personnel with different expertise, information displays and decision support systems, and communication systems.

2.3.5.2 Processors for Interteam coordination The processors for interteam coordination include the following:

T1. Establish or adapt interteam coordination infrastructure.

T2. Manage information (e.g., collect, analyze, and distribute information).

T3. Maintain shared situational awareness.

T4. Manage resources (e.g., allocation of personnel; allocation of equipment, water, electricity, etc.; prioritization of shared structure).

T5. Plan interteam collaborative activities (e.g., prioritize goals of the mission, make decisions, and generate commands for cooperation, coordination, or communication).

T6. Implement decisions and commands (i.e., manage and direct activities that can include information gathering; diagnostic activities; resource procurement; planning; action execution; allocating and directing resources; communicating with the entities involved to ensure common understanding of the current state, goals and priorities, pending tasks, and roles and responsibilities up and down the command chain).

T7. Verify, modify, and control the implementation.

2.3.5.3 Cognitive Mechanisms for Interteam Coordination The following cognitive mechanisms are included in the cognitive basis structure for interteam coordination:

T.a. Interteam coordination infrastructure: The roles and responsibilities of the teams involved and authority chains.

T.b. Command: The exercise of authority based on certain knowledge to attain an objective.

T.c. Control: The process of verifying and correcting activity such that the objective or goal of the command is accomplished.

T.d. Line of communication: Ability to exercise the necessary liaison to achieve effective command between tactical or strategic teams or units.

T.e. Data processing and information management: The human or computer systems and compatibility of computer systems for collecting and processing data.

2-26

T.f. Shared mental model: Shared understanding of the task, team members, and objectives.

2.4 An Example of Macrocognitive Functions across Distributed Teams This section uses an example to demonstrate that macrocognitive functions are distributed across different teams that collaborate on a mission. The example is a hypothetical abnormal event response. When the emergency plan is activated, the emergency director assumes controlinitially, this would be the shift manager, but responsibility is transferred to the TSC and then often to the remote emergency operations facility (EOF) after the TSC is in operation. The many distributed teams must be coordinated: TSC, main control room, operations support center, EOF, local emergency response personnel, within-plant field teams, offsite field teams, and others. Each team has its own role to be achieved through interteam coordination.

After the TSC is in operation, each teams tasks are the following:

The emergency director in the TSC directs the main control room to implement plant functions. The main control room coordinates with the operations support center to dispatch field operators to implement field actions.

The field operators check the symptoms and report back (via radio).

The TSC evaluators combine this information with other information available to the team in the TSC and forms an understanding of the situation. The TSC evaluators provide input to the emergency director (in the TSC). This leads the supervisor (and his support staff of experts) to determine that action needs to be taken to restore a safety function. They develop a plan for how to accomplish this and obtain the equipment needed to execute the plan. They communicate their plan to the EOF managers and seek approval.

The EOF gives approval and directs control room operators to execute the plan that was developed in the TSC.

Control room operators take action and report back.

Figure 2-10 illustrates the teams and macrocognitive functions involved.

Sends field operator into field Emergency Director (in TSC) to check on symptom Field Operator (in the field)

1. Determines the need to

2. Detects an issue and check a symptom in the reports back (D3-D5) field (DM1-DM3; D1-D2)

Emergency director (in TSC) supported by technical staff

3. Receives the information (D1-D5)

4. Integrates the information with other sources of Communicates decision Management (at Utility Headquarters) information to form understanding (U1-U5) and seeks approval 8. Decides whether to give approval

5. Prioritizes actions addressing this problem (DM1) and communicates approval (DM1-

6. Decides on action to take (this includes developing an Communicates approval DM6) action plan and procuring necessary tools and equipment) (DM2-DM5)

Sends out operating

7. Communicates and seeks higher level management crew to perform action authority (DM6) Operating Crew (in control room)

9. Communicates action plan to operators who will Operating crew reports 10. Takes actions and reports back (E1-perform action (DM6) back E5)

Figure 2-10 An Example of Macrocognitive Functions in Emergency Response Management 2-27

This example highlights several points:

The macrocognitive functions are distributed across different people in different locations.

Macrocognitive functions have a nested nature (e.g., an action directed from one team to another may trigger the entire set of macrocognitive functions for the team that will carry out the action, while an individual team may perform only a subset of macrocognitive functions).

Because the macrocognitive functions are distributed across people and locations, it is very important that the members of the distributed team establish a shared situational awareness. They need to have a common understanding of the situation and a common understanding of the goals to be achieved. If a shared situational awareness is not established, there is an increased chance of communication errors, as well as errors in perception (e.g., the field operator may look at the wrong thing or misinterpret what he or she saw) or action (e.g., the operating crew may take an incorrect action because it did not fully understand the situation).

2.5 Failure of Human Actions The cognitive basis structure describes how humans succeed at tasks, which helps to explain how humans may fail a task:

Failure of any macrocognitive function demanded by a task leads to the failure of the task.

Failure of a macrocognitive function results from errors of one or more processors.

Errors of a processor may occur if one or more associated cognitive mechanisms do not work properly or reliably.

A cognitive mechanism has capacity limits within which it works properly. When task demands or the context of the task approach or exceed the capacity limits, the mechanism works unreliably and may lead to errors of the associated processors. It may take the failure of several mechanisms for an error to occur. Nevertheless, exceeding capacity limits of cognitive mechanisms increases the likelihood of errors.

Here is a simple example to illustrate the failure of a human task (monitoring the status of an NPP). The task requires the macrocognitive function detection, and it demands monitoring multiple parameters and tracking their changes. This requires personnel to hold the parameters in working memory. The capacity of working memory is limited. If the number of the parameters to be tracked exceeds the capacity limit, personnel may miss changes in some parameters and, therefore, fail to correctly track the trends of the parameters. As a result, the overall likelihood of failing the task increases because of working memory overload. The context of the task also plays a role in working memory. For instance, a well-designed HSI alleviates personnels working memory load by visually presenting the trends of parameter changes. Conversely, a poorly designed HSI may exacerbate personnels working memory load by displaying the parameters in different locations. Also, factors such as mental fatigue or sleep deprivation can affect personnels working memory span.

2-28

2.6 Summary This chapter describes the cognitive basis structure as a part of the Cognition Model for Human Performance and Reliability. The cognitive basis structure includes the following components:

Human tasks are achieved through the five macrocognitive functions.

Cognitive processors accomplish the macrocognitive functions.

Cognitive mechanisms enable processors to be achieved reliably; challenges to the capacity limits of the mechanisms can lead to errors in processors.

The cognitive basis structure explains how and why humans may succeed or fail at a task. It addresses the full scope of cognitive activities in complex scenarios. It serves as a basis for understanding human performance and reliability. The next chapter will describe PIFs that can affect cognitive mechanisms and thus the likelihood of success or failure of human tasks.

2-29

3 COGNITION MODELPERFORMANCE-INFLUENCING FACTOR STRUCTURE The conditions that affect human performance of an action are the context for that action. HRA has been using PIFs to represent context and thereby quantify HEPs. Based on existing HRA methods and a review of the literature, operational experience, and various human performance databases, the authors developed a PIF structure as a part of the Cognition Model to model the context of important human actions and its effects on HEPs. Once the qualitative analysis defines the context, IDHEAS-G models the context using the PIFs, and that allows for quantification.

PIFs, also referred to as performance-shaping factors in some HRA methods, are the factors that positively or negatively affect human performance. The cognitive basis structure described in the previous chapter explains that the success or failure of a task can be traced to failures of macrocognitive functions, processors of the functions, and underlying cognitive mechanisms.

PIFs affect tasks demanding cognitive resources and capacity limits of cognitive mechanisms.

PIFs can challenge cognitive mechanisms to make them less effective and thus increase the likelihood of human errors. PIFs may also decrease the likelihood of human errors by alleviating some challenges to cognitive mechanisms.

This chapter presents the PIF structure. Section 3.1 introduces the context of important human actions and how to model the context for HRA purposes. Sections 3.2 and 3.3 describe the PIF structure, including the definitions of the PIFs and the attributes characterizing the impact of PIFs and their effects on human performance. Section 3.4 describes links between PIF attributes and processors of macrocognitive functions through underlying cognitive mechanisms. Finally, Section 3.5 discusses how to quantitatively assess PIFs and their impacts on HEPs.

3.1 Modeling the Context of Important Human Actions Personnel work with systems to perform required actions and achieve the mission of the work.

The context of an important human action describes all the conditions that can affect human performance. Context can be classified as one of four types:

(1) Environment and situation contextThis consists of conditions in personnels work environment and the situation in which important human actions are performed. It includes weather, radiation or chemical materials in the workplace, and any extreme operating conditions.

(2) System contextSystems are the objects of important human actions, through which the work missions are achieved. Systems include operational systems, supporting systems, instrumentation and control (I&C), physical structures, HSI, and equipment and tools.

(3) Personnel contextPersonnel are the people who perform the action. They include individuals, teams, and organizations. Personnel context describes who the personnel are; their qualifications, skill, knowledge, ability, and fitness to perform the action; how they work together; and the organizational measures that help personnel work effectively.

3-1

(4) Task contextAn important human action may consist of one or more discrete tasks.

Task context describes cognitive and physical tasks demanding for personnel and special conditions in the event scenario that make tasks difficult to perform.

HRA uses PIFs to represent the context that would enhance or adversely impact human performance. The cognitive basis structure models the cognitive process of performing an important human action. Meanwhile, the PIF structure models how the context of an important human action influences the performance by affecting the cognitive process. Figure 3-1 illustrates this approach: given an event, the context (environment and situation, system, personnel, and task) consists of the conditions affecting the performance of an important human action, and the PIFs model the context. Then, the PIFs influence the capacity limits of the cognitive mechanisms, which may cause an error in the cognitive process of performing an action and thus affect the outcomes of the performance.

Environment and situation context Event progress Event, mission, System Human Human Human Personnel goals context action action action context Task context PIFs Cognitive mechanisms Processors Macrocognitve functions Success or failure of human actions Figure 3-1 Overview of the PIF Structure Because the IDHEAS-G cognition model aims to provide a basis for modeling important human actions in any HRA application, the NRC staff established the following criteria to develop a PIF structure to model the context of important human actions:

Pertinence and comprehensiveness in lieu of completenessIt is desirable that PIFs in an HRA method include all of the characteristics of systems, environment, tasks, and personnel that may affect human performance. In reality, modeling everything would lose the practicality of a model. In addition, new systems, or changes in systems, and new concepts of operation introduce new characteristics. Thus, modeling context for HRA should use a set of PIFs that are pertinent to the likelihood of human errors and are 3-2

comprehensive enough to address the current knowledge base of human performance issues.

OrthogonalityPIFs should be orthogonal to each other in meaning and scope coverage, and no PIF is the result of other PIFs. Changes in one PIF should not result in changes in other PIFs. Defining all PIFs orthogonal to each other is difficult.

SpecificityEvery PIF models a distinctive aspect of the context, and the scope of what it models should be unambiguous. The scope should have no overlap. A PIF should not be a subset of others.

ExplainablePIFs should be able to explain why and how they affect the likelihood of task success or failure. Quantification of the effect of a PIF on human errors needs to account for how the PIF leads to certain types of human errors.

AssessableA PIF should have objective criteria for its assessment so that the given context of an important human action can be consistently interpreted as the states of relevant PIFs.

QuantifiableTo quantify the effects of PIFs on HEPs, PIFs need to be behaviorally observable and link to human performance measures or, more desirably, human error data.

The developed PIF structure is intended to comply with these criteria. The development of a comprehensive, but not exhaustive, set of PIFs for IDHEAS-G was based on an extensive review of the literature, existing HRA methods, performance databases, and operational experience in various domains (e.g., nuclear, aviation, transportation, chemical processing).

The cognitive basis structure allows the linking of PIFs, through the cognitive mechanisms and processors, to the macrocognitive functions of task performance.

3.2 Performance-Influencing Factor Structure 3.2.1 Overview As a starting point, the NRC staff first reviewed and consolidated all the PIFs in existing HRA methods. The American Society of Mechanical Engineers/American Nuclear Society (ASME/ANS) PRA standard [44] mentions the following PIFs in the definition of performance-shaping factor: level of training, quality and availability of procedural guidance, and time available to perform an action. The NRC staff organized the PIFs according to the four types of context: environment and situation, systems, personnel, and tasks. The staff also defined some new PIFs to address the comprehensiveness and specificity of a PIF model. For example, cognitive studies have shown that performing concurrent tasks increases the probability of human error compared to performing each task alone. Without a specific PIF for concurrent tasks, HRA analysts have modeled it under the PIFs of mental stress or task complexity. However, the impacts of these two PIFs on human errors are very different.

With the high-level definition of PIFs like those in the PRA standard, interpreting a PIF can be very subjective. This causes the PIFs to be interdependent or to overlap. Also, PIFs defined in a general way are not specific enough to link to underlying processors. The NRC staff used a set of attributes to specify every PIF; each attribute represents one distinctive aspect of the PIF that impacts the underlying processors by challenging one or several cognitive mechanisms.

Characterizing a PIF with a set of such attributes makes the PIF specific, assessable, and explainable. The NRC staff identified the attributes from cognitive and behavioral studies, as 3-3

well as human error data from various sources; thus, the attributes inherently have the capability to link to existing human error data for HEP quantification. Using attributes to specify a PIF also allows the analyst to examine, if not completely eliminate, the interdependency between PIFs.

Environment Context and Situation System Personnel Task

Accessibility/habitability *Staffing *Information availability

System and I&C and reliability of workplace including *Procedures, guidance, transparency to *Scenario familiarity travel paths and instructions personnel *Multitasking,

Workplace visibility *Training

HSI PIF *Noise in workplace and

Equipment and

Team and organization interruptions, and communication factors distractions tools *Task complexity pathways *Work processes

Cold/heat/humidity *Mental fatigue

Resistance to physical *Time pressure and movement stress

Physical demands PIF *Poor lighting in *Tools are difficult to use

Procedure is inadequate *Sustained high-workplace *Tools are unfamiliar to personnel attributes *Glare or reflection *Tool does not work

Procedure is difficult to demanding cognitive use activities Note: The PIF attributes on physical structure *Tool or parts are unavailable shown are examples and *Procedure is available, but *Long working hours

Smoke or fog- *Document nomenclature does correspond to the PIFs does not fit the situation *Sleep deprivation induced low visibility not agree with equipment labels highlighted in red.

Links to cognitive See Section 3.4 and Appendix B mechanisms Figure 3-2 Illustration of the PIF Structure Figure 3-2 illustrates the four-layer PIF structure:

(1) PIF Category/Context: PIFs are classified according to the four types of context:

environment and situation, system, personnel, and task.

(2) PIFs: Each category has several high-level PIFs modeling the corresponding context.

Below are the PIFs in the four categories.

environment- and situation-related PIFs o accessibility/habitability of workplace including travel paths o workplace visibility o noise in workplace and communication pathways o cold/heat/humidity o resistance to physical movement system-related PIFs o system and I&C transparency to personnel o HSI o equipment and tools personnel-related PIFs o staffing o procedures, guidance, and instructions o training o team and organization factors o work processes 3-4

task-related PIFs o information availability and reliability o scenario familiarity o multitasking, interruptions, and distractions o task complexity o mental fatigue o time pressure and stress o physical demands (3) PIF attributes: These are the assessable traits of a high-level PIF. A PIF attribute describes a way that the PIF challenges cognitive mechanisms and increases the likelihood of errors in the processors. In Figure 3-2, the PIF attributes shown correspond to the PIFs highlighted in red. Section 3.3 discusses the attributes for all the PIFs.

(4) PIF attribute links to the cognitive mechanisms: A PIF attribute may affect one or more processors of macrocognitive functions by challenging cognitive mechanisms. For example, distraction challenges working memory and attention, which may lead to multiple processor failures, such as not perceiving critical information or incorrectly executing simple actions. (Note that this has significant impacts on the ability to isolate the influence of these PIFs on a single cognitive mechanism or macrocognitive function.)

Section 3.4 introduces the PIF attribute links to the cognitive mechanisms, and APPENDIX B explains them further.

The PIF structure connects to the cognitive basis structure through cognitive mechanisms, as illustrated in Figure 3-3. The two structures together allow for a systematic analysis of human events. Looking at the flow from left to right in Figure 3-3, an analysis can begin with collecting information on event context, then representing the context with applicable PIFs and attributes, evaluating which cognitive mechanisms are challenged by the PIF attributes, and identifying potential human failure modes as the failure of processors and their associated macrocognitive functions. Or, an analysis can begin with a task; identify applicable macrocognitive functions, processors, and cognitive mechanisms; and then identify PIFs that can potentially affect the processors and macrocognitive functions through the links with cognitive mechanisms. In short, the PIF structure can serve as the bridge linking observable event context to the internal processes of human cognition.

PIF Structure Cognitive Basis Structure Context Processor PIFs PIF Cognitive attributes mechanism Processor Detection Environment Cognitive and situation Processor mechanism Understanding System Processor Event Cognitive Task scenario Decision-mechanism Processor making Personnel Cognitive Processor mechanism Action execution Task Cognitive Processor mechanism Interteam Processor coordination Cognitive mechanism Figure 3-3 Connection of the Cognitive Basis Structure and PIF Structure 3-5

3.3 Details of the Performance-Influencing Factor Structure This section presents the details of the PIF structure. For each PIF, Table 3-1 through Table 3-20 present the high-level PIF definition, a discussion, the associated attributes, and some examples of the attributes. APPENDIX B presents the links between PIF attributes and cognitive mechanisms. The PIF structure in this report has the following features:

The high-level PIFs cover all those in existing HRA methods (not in one-to-one mapping).

A PIF has multiple attributes. Every attribute included has been reported in one or more research papers, event or accident reports, or human event databases.

A PIF attribute challenges one or several cognitive mechanisms, as reported in the literature.

The list should be considered a living document, as new PIF attributes can be introduced by new research, modern technologies, changes in operational concepts, and new HRA applications. For example, upgrades to digital I&C systems in NPP control rooms may introduce new PIF attributes associated with the use of computerized procedures. The PIF structure provides a framework to add new PIF attributes.

3.3.1 Environment- and Situation-Related Performance-Influencing Factors Hazards such as steam, fire, toxic gas, seismic events, or flooding can introduce environmental conditions that impede personnel performance. According to NUREG/CR-5680, The Impact of Environmental Conditions on Human Performance, Volumes 1 and 2, issued September 1994

[45], [46], many environmental conditions can adversely affect human performance. Risk analysis typically considers the following environmental conditions:

temperature and humidity

noise

radiation or chemical contamination

light

smoke and fog

high wind

standing or running water

debris

vibration

seismic aftershocks The following are examples of environmental factors that can adversely impact human performance:

Noise, smoke, and precipitation affect information detection.

Harsh environmental conditions, such as extreme heat or cold, may lead to early termination of situation assessment because personnel are unwilling to seek additional data to reconcile conflicts in the information.

Harsh environmental conditions adversely affect decisionmaking (e.g., reducing decisionmakers ability and effort in evaluating available strategies, thoroughly deliberating decisions, or mentally simulating action plans).

3-6

Environmental conditions on travel paths and at worksites can restrict personnels motor movement, reduce their motor skills, or limit the time that they can steadily perform motor activities. Examples of these conditions are wearing heavy protective clothes, high water on travel paths, high winds, extreme heat or cold, earthquake aftershocks, and chemical or other toxic contamination.

Environmental conditions such as noise or smoke can impede interteam collaboration.

The following PIFs represent environmental and situational context:

accessibility or habitability of workplace, including travel paths

workplace visibility

noise in workplace and communication pathways

cold, heat, and humidity

resistance to personnel physical movement In addition, environmental conditions may affect information and tools available for personnel to perform tasks. Although those two PIFs are classified in the category of task-related and system-related context, assessments of the states of such PIFs should consider different categories of context. The tables in this chapter show the PIF definition, the no-impact state, and the attributes. Each table is for one PIF.

3-7

Table 3-1 PIF Workplace Accessibility and Habitability Definition Workplace is the place where the workers perform actions. It includes hardware facilities, physical structures, and travel paths to support personnel in task performance. Workplace may be in an open, unprotected environment or within a building structure. The travel path to the workplace, accessibility controls to enter the workplace, and physical environment in the work should not impede personnel from entering and performing the required actions.

Discussion Accessibility issues are most likely because of adverse environmental conditions and security system operation. For example, accidents or hazards may cause workplace conditions to become less habitable or accessible for a period of time.

The following are example situations that could affect habitability and accessibility:

adverse environmental conditions such as steam, high water, fire, smoke, toxic gas, radiation, electric shock risk, and roadblocks (e.g., because of extreme external hazards)

doors and components that are normally locked and require keys to unlock (e.g., a fire or flood may cause electric security systems to fail locked.)

external hazard damage to stairways and corridors Attributes

Accessibility (travel paths, security barriers, and sustained habituation of worksite) is limited because of adverse environmental conditions, such as steam, high water, fire, smoke, toxic gas, radiation, electricity shock risk, and blocked roads.

Doors or components require keys to unlock.

Habitability is reduced. Personnel cannot stay long at the worksite because of factors like radiation or earthquake aftershocks.

The surface of systems, structures, or objects cannot be reached or touched (e.g., because they are hot).

The worksite is flooded or underwater.

3-8

Table 3-2 PIF Workplace Visibility Definition Visibility of an object is a measure of the ease, speed, and precision that the object is visually detected and recognized. Visibility of a task is generally determined by visibility of the most difficult element which must be detected or recognized so the task can be performed.

Discussion Personnel need to recognize the object of a task and their surroundings to perform activities accurately and reliably. Visibility of an object is a function of the difficulty experienced to discriminate it visually from the background or surrounding environment. Visibility at work is related to the illumination of the workplace. It requires a minimum level of illumination at which personnel can detect objects and discriminate spaces between objects. Luminance is the most important factor for good visibility, which is needed to reliably perform activities such as reading, writing, inspecting objects for errors, and distinguishing cues. Poor visibility impairs personnels detection of information and execution of physical actions requiring visual-motor coordination. Moreover, it affects comfort and effectiveness of teamwork. In addition to luminance, visibility is also affected by light distribution such as reflections or shadows in the workplace.

The following are example situations that could affect visibility:

Insufficient illumination (e.g., poor ambient light, darkness).

Concealing because of fog, smoke, and rain, etc.

Reflectance, shadow, low brightness contrast for tasks dealing with bright contours of objects.

Flickering or vibration of the object. Display vibration may affect the performance of tracking tasks by reducing perceived visibility.

Glare. Glare refers to the brightness that is greater than that for which human eyes are adapted.

Attributes

Low ambient light or luminance of the object that must be detected or recognized

Glare or strong reflection of the object to be detected or recognized

Low visibility of work environment (e.g., those caused by smoke, rain, fog, etc.)

3-9

Table 3-3 PIF Noise in Workplace Definition Noise is unwanted sound disruptive to hearing. Human perceived noise is a function of the sound intensity (loudness), duration, variation of intensity, frequency of the sound waves, and the meaningfulness of the sound.

Discussion Noise types include continuous sound, intermittent sound, speech, nonspeech, music, and mixtures of sounds. Continuous noise is constant, with no breaks in intensity. Intermittent noise changes in intensity, having gaps of relatively quiet intervals between repeated loud sounds. A major type of practical distractive noise is speech. Speech is a distracter to which humans are especially attuned.

Noise impairs human performance by interfering with cognitive processing or exerting detrimental effects on mental and physical health. It generally does not influence performance speed, but it reduces performance accuracy and short-term/working memory performance. Accuracy in cognitive and communication tasks was most vulnerable to noise effects. Collective research findings have identified the relationship between various types of noise and types of tasks. Below are some examples of noise effects:

Intermittent noise proves to be the more disruptive type of noise. These effects occur more strongly with speech noise and for resource-demanding cognitive tasks.

The effects of noise were stronger when the noise was composed of speech. Intermittent speech noise of relatively short duration is the most disruptive.

Humans adapt to the environment and develop various compensatory strategies to alleviate noise effects. However, intermittent speech of a relatively short duration makes people become unable to effectively recover through compensatory effort because of the limited exposure.

Humans can develop more effective coping strategies for continuous noise of longer duration.

Some low volume continuous sounds such as music can increase personnels alertness.

Attributes

Continuous loud mixture of noise

Intermittent non-speech noise

Relatively continuous speech noise

Intermittent speech noise of relatively short duration 3-10

Table 3-4 PIF Cold/Heat/Humidity Definition Human bodies maintain a core temperature in the vicinity of 98.6°F. Beyond a range of environmental temperature and humidity, the bodys ability to regulate temperature decreases. Cold, heat, and humidity refers to the environmental condition that temperature or humidity can have negative effects on behavior and task performance.

Discussion Cold, heat, and humidity produce thermo stress on humans. While physiological limits of endurance to temperature and humidity may be seldomly reached, personnel are subjected to thermo stress in many work environments, such as in outdoor work under intemperate climatic conditions or loss of ventilation in control rooms. Studies on the relationship between thermo stress and accident occurrence as well as unsafe work behavior have revealed negative effects of thermo stress on task performance. The following are example situations that could affect performance:

Heat begins to impair performance when it exceeds 86°F and exposure exceeds 3 hours3.472222e-5 days 8.333333e-4 hours 4.960317e-6 weeks 1.1415e-6 months . Reaction time, vigilance, and performance of complex tasks are affected by heat.

Performance on tasks requiring manual dexterity declines when temperatures fall below 60°F. Cold exposure of the hands which is critical for manual performance affects the speed and precision of task performance.

The range of temperatures beyond which performance is impaired depends on the kinds of tasks and exposure time. Tasks involving fine movements of the fingers and hands or manipulation of small objects are particularly sensitive to cold effects. Slow cooling is more detrimental to manual performance than rapid cooling to equivalent skin temperatures of the hands.

Comparatively mild levels of cold, heat, and humidity exposure can significant increase the number of errors, speed of incorrect response, and number of false alarms. Complex reaction time slows down in heat, and more errors are made in cold.

Wearing protective clothing can impose thermal stress. The effect of heat on physical work and perceptual/motor task performance may become severe in situations where workers are required to wear heavy protective clothing in restricted or confined areas. Protective clothing worn in radiation zones may not allow for adequate ventilation, which leads to heat and humidity.

Attributes

Cold in workplace

Heat in workplace

High humidity in workplace 3-11

Table 3-5 PIF Resistance to Physical Movement Definition Resistance to physical movement refers to personnels perceived difficulty in making physical movement due to resisting, opposing, or withstanding of external forces such as those imposed by wind, rain, flooding, etc.

Discussion Resistance to physical movement causes physical stress (also referred to as physical fatigue). Physical stress does not lower an individuals knowledge of how to get tasks done, but it causes lowered physical efficiency, reduced attention, and increased susceptibility to loss of balance. Moreover, physical stress can result in unconscious lowering of performance standards. These effects can impact task performance in follow ways:

Errors in timing of movement involving large sequences of movement

Overlooking some important elements in task sequences

Loss in accuracy and smoothness of control movement

Under-control or over-control of movement

Forgetting side tasks The following are example situations that could induce resistance to physical movement:

External forces such as wind, rain, flooding.

Postural instability may be induced by carrying heavy materials on a slippery or unstable surface while not using fall protection; or it can be induced by experiencing unexpected perturbations that cause body acceleration or deceleration. Tasks affected involve standing upright, rapid body movement, or lateral reach during lifting.

Exposure to whole-body vibration interferes with manual tracking and visual acuity. Whole-body vibration may come from operating vehicles, walking or lying on oscillating overhead catwalks, climbing up ladders located on or over machinery, working in ventilation ducts, tending conveyors, and fixing generators, diesels, and turbines.

Protective clothes impose a mechanical burden because body movement is limited by the clothing. That can impact manual dexterity capabilities and psychomotor performance.

Wearing heavy gloves hampers performance of delicate manual tasks.

Attributes

Physical resistance

Postural instability (e.g., slippery surface)

Whole-body vibration

Wearing heavy protective clothes or gloves or both

Resistance to personnel movement with vehicle 3-12

3.3.2 System-Related Performance-Influencing Factors System context consists of conditions in operating and supporting systems, I&C, HSI, and portable equipment and tools. While system availability and reliability are modeled in PRA outside of HRA, system design, maintenance, and administrative control can create conditions impeding human performance.

System context is represented with the following PIFs:

system and I&C transparency to personnel

HSI

equipment and tools These PIFs are presented in Table 3-6, Table 3-7, and Table 3-8, respectively. In addition, systems and I&C can affect information availability and reliability. The PIF information is described in the task-context category. Assessments of the PIF information should also consider system context.

Table 3-6 PIF System and I&C Transparency to Personnel Definition Systems and I&C should be designed for personnel to understand their behaviors and responses in various operating conditions.

Discussion This PIF models the impact of design logic and personnels use of the systems deviating from the design. If the operation of system or I&C is not transparent to personnel, or personnel are unclear about system interdependency, they can make errors because of not understanding the systems in unusual scenarios. Also, some instrumentation, control, electrical, and fluid (water, compressed air, ventilation) systems may be aligned in alternative or unusual configurations when the initiating event occurs. For example, these configurations may apply during testing, maintenance, specific shutdown plant operating states, etc. If a system is not aligned in its normal configuration or the unusual alignment is not apparent, personnel may not correctly confirm that the system is operating properly, easily recognize the effects from equipment damage, or quickly determine how the system should be realigned to cope with the evolving scenario.

Attributes

System or I&C does not behave as intended under special conditions.

System or I&C does not reset as intended.

System or I&C is complex, making it hard for personnel to predict its behavior in unusual scenarios.

System or I&C failure modes are not transparent to personnel.

3-13

Table 3-7 PIF Human-System Interface Definition HSI refers to indications (e.g., displays, indicators, labels) and controls used by personnel to execute actions on systems.

Discussion HSIs are expected to support human performance. For example, advanced alarm displays in NPP control rooms organize alarms according to their urgency to help operators focus on what is most important. HSI design of NPP control rooms generally undergoes rigorous human factors engineering; thus, HSIs should comply with human factors engineering requirements and not impede human performance in normal and typical emergency operation. However, poorly designed HSIs can impede task performance in unusual event scenarios. Even a well-designed HSI may not support human performance in specific scenarios that designers or operational personnel did not anticipate. HSIs may also become unavailable or unreliable in hazardous scenarios.

Attributes HSI attributes depend on the specific interfaces used in an application. New HSI technologies may introduce additional attributes.

The source of indication (e.g., indicators, labels) is similar to other sources nearby.

The source of indication is obscured or masked in many potentially relevant indications.

The indications have low salience; for example o Indications are located outside of personnels direct view (e.g., they are viewed from far away or blocked by constructs, poor lighting).

o Indications appear similar to the surrounding information.

o Indications are difficult for visual perception, such as small fonts, labels that are difficult to read, low legibility, or misuse of colors.

Related information is spatially distributed or unsynchronized.

o Personnel must look around or walk around to get all pieces of information in an information-gathering task.

o Pieces of information needed for a task are not presented at the same time; thus, personnel must remember and relate the various pieces.

3-14

Table 3-7 PIF Human-System Interface (continued)

Attributes (continued):

Indications are confusing or nonintuitive.

o The indication or label can be interpreted differently for reasons such as imprecise axis labeling in an X-Y plot.

o The same information is presented in different formats, which may lead to recognition errors.

o HSI presents information in confusing ways such that interpreting the indications involves complex graphics or complicated logic operations (e.g., AND, OR, NOT, and NOR).

Secondary indications are not promptly available, or personnel are not aware of them.

o Personnel rely on secondary indications when the primary sources of information are not available, but personnel may not know of the existence of secondary indications or may not know how to use them.

Controls are difficult to maneuver.

Personnel do not anticipate the failure modes of controls and their impacts.

Indications of states of controls are inadequate.

There is confusion in action maneuver states.

o Transition in system control states is not acknowledged.

o Controls reset following trips or spurious actions.

Controls provide inadequate feedback (i.e., lack of adequate confirmation of the action executed (incorrect, no information provided, measurement inaccuracies, delays)).

Labels on the controls do not agree with document nomenclature.

Controls are not reliable, and personnel are unaware of the problem.

3-15

Table 3-8 PIF Tools and Parts Availability and Usability Definition The tools, equipment, and parts assessed in an event include all the things needed to support personnel actions. They should be available and readily usable.

Discussion In event scenarios, portable or special tools may be needed. Examples are portable radios, portable generators, torque devices to turn wheels or open flanges, flashlights, ladders to reach high places, and electrical breaker rack-out tools. The tools assessed in an event include all the things needed to support personnel actions. For example, use of a portable diesel pump would include the vehicle to tow the pump to its staging location, the water source, pipes, hoses, junctions and fittings (e.g., to connect to fire hydrants), and other things; ladders or scaffolding may be needed to access equipment that must be operated or local instrumentation that must be checked.

Attributes

Tools are difficult to access or to use (e.g., lack of administrative control of tools).

Tools are unfamiliar to personnel.

o Personnel do not know how to calibrate or use the tools.

o Instructions for use do not state what to do if the equipment or tool is operating outside of the specified range.

Failure modes or operational conditions of the tools are not clearly presented (e.g., ranges, limitations, and requirements).

Critical tool does not work properly because of aging, lack of power, incompatibility, improper calibration, lack of proper administrative control, or other reason.

Tools or parts needed are missing or not available.

Document nomenclature does not agree with equipment labels.

3.3.3 Personnel-Related Performance-Influencing Factors Personnel context includes the conditions related to individuals, teams, and organizations. The following PIFs represent personnel context:

staffing

procedures (including guidance, protocols, and instructions)

training

teamwork factors

work process These PIFs are presented in Table 3-9 through Table 3-13, respectively.

3-16

Table 3-9 PIF Staffing Definition Staffing refers to having adequate, qualified personnel to perform the required tasks.

Staffing includes the number of personnel, their skill sets, job qualifications, staffing structure (individual and team roles and responsibilities). Adequate and qualified staff is normally expected.

Discussion In event scenarios, there may be a shortage of staffing, lack of staff with specific skills, or unclear staff roles and responsibilities. Even in normal operation scenarios, staffing can become a concernfor example, key personnel may be temporally called away for other duties.

Fitness for duty is a part of staff job qualification. It refers to whether an individual is fit to perform the required actions. Factors that may affect fitness for duty include fatigue, illness, drug use (legal or illegal), and personal problems. Personnel may become unfit for duty as the result of excessively long working hours or illness caused by the harsh environment.

Staffing assessment in an event should consider unusual situations like the following:

Some personnel may not be available for a period after an initiating event.

For example, in an NPP external event, the offsite personnel may not be available for a time because of site inaccessibility. Staffing considerations should not be limited only to the HFE being analyzed, but it should be considered within the scope of the entire event. Staffing can be inadequate when many human actions are concurrent. Specifically, analysts need to consider other activities that are not modeled explicitly in the PRA. For example, personnel may be allocated to mitigate failures or damage to non-safety systems that are important for overall plant investment protection or for perceived improvement of overall plant conditions, but are not modeled explicitly in the PRA.

For an extreme event blocking access to the site for a time, onsite personnel must be able to perform the tasks before the offsite personnel are available.

A staffing analysis would be necessary to ensure that sufficient personnel and needed skills are available for all tasks.

If an important human action is performed through teamwork, the team can collectively fulfill the minimum requirement of staffing, knowledge, and abilities. For example, in an external hazard, the plant security force may be required to support the trained plant staff to remove debris on the road and move the portable equipment to the equipment staging location. These activities may need only a trained staff (e.g., to operate a special vehicle) and many helping hands. The other plant staff can supplement the manpower needed if they are supervised by plant staff with the specific skills and knowledge needed.

3-17

Table 3-9 PIF Staffing (continued)

Attributes

shortage of staff (e.g., key personnel are missing, unavailable, or delayed in arrival; staff pulled away to perform other duties)

ambiguous or incorrect specification of staff roles and responsibilities

inappropriate staff assignment (e.g., lack of skills needed)

key decisionmakers knowledge and ability are inadequate to make the decision (e.g., lack of required qualifications or experience)

lack of administrative control of fitness for duty Table 3-10 PIF Procedures, Guidance, and Instructions Definition This PIF refers to availability and usefulness of operating procedures, guidance, instructions (including protocols). Procedures, guidance, and instructions (PGIs) should be validated for their applicability and usefulness. Following PGIs should lead to the success of important human actions.

Discussion Normally, PGIs are expected to be available and facilitate human performance.

However, there are situations in which PGIs give incorrect or inadequate guidance for important human actions. PGIs may not apply to the scenario. Other common problems with PGIs include ambiguity of steps, lack of adequate detail, or conflict with the situation.

Attributes

The PGI is inadequate.

o PGI is not specific about searching for additional information when the primary cues are not available or not reliable.

o PGI does not warn about all the conditions that should be avoided during performance.

o Contingency steps are insufficient.

o Logic is unclear such that the operators are likely to have trouble identifying a way to move forward through the PGI.

o PGI does not warn about the pitfalls of the decision.

3-18

Table 3-10 PIF Procedures, Guidance, and Instructions (continued)

Attributes (continued)

The PGI design is difficult to use.

o The PGI is difficult to use because of factors such as formatting problems, ambiguity, or lack of consistency.

o Multiple guidance documents must be referenced or open at the same time.

o There are no place-holders to maintain ones place in the document.

o The logic to follow PGIs is complex: e.g., sequential presentation of a PGI requires the crew to go through several loops before finding the correct indications to diagnose the plant status.

The PGI lacks details.

o The PGI does not provide sufficient details.

o Engineering judgment is needed to supplement the lack of PGI details.

o The PGI is not specific for the situation so personnel have to fill in the details to make the PGI work for the situation.

The PGI is confusing.

o The PGI requires complex calculations or logic reasoning (e.g., complex logic to follow; a sequentially presented PGI requires personnel to go through several loops before finding the correct indications; PGI logic or layout makes it difficult to follow the PGI step-by-step).

o PGIs that are used for the same important human action are inconsistent (e.g., PGIs use different parameter units such as radius versus diameter, percent versus direct numeric value).

o PGIs conflict with existing policies, requirements, or other documents.

The PGI is available but does not fit the situation (e.g., it requires deviation or adaptation).

The PGI is not available for skill-based tasks.

The PGI is not available; thus, personnel have to find ways to perform the task based on their knowledge.

The PGI is misleading.

3-19

Table 3-11 PIF Training Definition This PIF refers to training that personnel receive to perform their tasks. Included in this consideration are personnels work-related experience and whether they have been trained on the type of the event, the amount of time passed since training, and training on the specific systems involved in the event. It is expected that adequate training is required for professional staff.

Discussion Training may not address all possible event scenarios. For example, NPP operator training focuses on use of normal and emergency operating procedures (EOPs); the training may not adequately emphasize how operators need to develop novel strategies to handle unusual accident or hazard situations.

Attributes

Training frequency is low (greater than 6 months between sessions).

Training duration or the amount of training is not adequate.

Training on procedure adaptation is inadequate. The training focuses on following procedures without adequately training personnel to evaluate all available information, seek alternative interpretations, or evaluate the pros and cons of procedural action plans.

Training is inadequate on collaborative work process as a crew (e.g., inadequate supervision in monitoring actions and questioning current mission; inadequate leadership in initiating assessment of action scripts, facilitating discussion, and avoiding tunnel vision).

Training or experience with sources of information (such as scope and limitations of data and information on the failure modes of the information sources) is inadequate.

Experience in diagnosis (e.g., not being aware of and coping with biases, not seeking additional information, and not avoiding tunnel-vision) is inadequate.

There are gaps in team knowledge and expertise needed to understand the scenario.

There is inadequate specificity on the urgency and criticality of key information such as key alarms, system failure modes, and system design to the level of detail needed for responding to the situation.

The training is inadequate or practice is lacking in the step-by-step completion of action execution.

The training lacks practicality.

Hands-on training on action execution is lacking (e.g., training consists of virtual training, classroom training, or demos only without hands-on practices).

3-20

Table 3-11 PIF Training (continued)

Attributes (continued)

Experience or training is lacking on procedures, guidelines, or instructions for the type of event (e.g., use nonoperators to perform some actions outside the control room).

The action context is infrequently part of training or personnel rarely perform the actions under specific context (greater than 6 months between performance).

Personnel are not trained on the procedures or for the type of actions.

Table 3-12 PIF Team and Organization Factors Definition Team factors refer to everything affecting team communication, coordination, and cooperation.

Discussion Teamwork activities include planning, communicating, and executing important human actions across individuals, teams, and organizations. Examples of teamwork problems seen in event analysis are problems caused by information not being communicated during shift turnover and loss of command and control between the operations center team and field maintenance personnel.

Safety-critical organizations foster safety culture and have mechanisms for identifying, reporting, and correcting human errors or factors that may lead to human failure events. For example, organizations should document and treat any evidence obtained during the review of an operating event indicating intergroup conflict or indecisiveness or an uncoordinated approach to safety. An organization should also maintain an effective corrective action program to address safety issues such as failure to prioritize, failure to implement, failure to respond to industry notices, or failure to perform risk analyses. The attribute of poor safety culture that impedes safety can vary greatly among organizations.

Attributes

Inadequate team information management o Distributed informationThe information needed for understanding the situation is distributed across team entities in distributed locations and needs to be communicated and integrated.

o Unsynchronized informationInformation presented to decisionmakers comes from various sources and represents situations at various times.

o Information overloadInformation allocated to an individual is too overwhelming to be processed promptly.

3-21

Table 3-12 PIF Team and Organization Factors (continued)

Attributes (continued)

Inadequate teamwork resources o lack of sufficient personnel resources to address all issues of concern (shortage of personnel) o lack of sufficient equipment resources to address the issues of concern (shared equipment) o resources not under direct control (need to persuade others to provide the resources) o lack of awareness of resources available (e.g., personnel available) o lack of required expertise of staff

Distributed or dynamic operational teams o Teams have been drilled together.

o Action requires coordination between multiple parties at different locations.

Distributed locations increase the likelihood of breakdowns in communication, increase the work required to maintain shared situational awareness (common ground) and the possibility of divergence in understanding the situation and the goals to be achieved, and make it less possible to catch and correct other errors.

o Teams that involve multiple crafts or multiple organizations (e.g., contractors) may have differences in mental models and disciplinary goals.

o Team cohesion may be inadequate (e.g., lack of understanding of other team members, lack of the required knowledge or experience on the team, lack of a clearly designated decisionmaker on the scene, and not having well-defined roles and responsibilities for team members).

o Leadership or supervision is inadequate to ensure that personnel have a common understanding of evolving situation and goals.

o Roles and responsibilities are not specified or are ambiguous for the situation (i.e., no clear roles and responsibilities). Personnel may be unclear as to their roles and responsibilities or unwilling to take on responsibilities, or there may be no plan for specifying roles and responsibilities for this type of situation.

3-22

Table 3-12 PIF Team and Organization Factors (continued)

Attributes (continued)

Inadequate team decisionmaking infrastructure o Making the decision requires consensus, approval, or both along a chain of command that can lead to delay and possibly increase risk aversion.

o The team does not have an open decisionmaking style, which would encourage everyone to volunteer suggestions and raise concerns. An open style may result in more resilient performance (with team members more likely to catch and correct errors).

o Work process is poor in reconciling different viewpoints.

o The locus of decisionmaking is distributed or shifts from one location to another, with some decisions centralized and others local. Control room operators may act without informing or getting permission from higher-level management. Groups with different situational awareness and expertise and in various locations need to make decisions and develop plans. The transfer of the locus of control from one location to another can contribute to delays or loss of information.

o Making the decision requires varied expertise distributed among multiple individuals or parties who may not share the same information or have the same understanding of the situation.

Team coordination difficulty o Close coordination of activities is necessary. Activities are interdependent, such that the action of one person cannot be achieved until the action of the other is achieved, or the action of one person can complicate or block the action of another.

o There is inadequate coordination between site personnel and decisionmakers to adapt or modify planned actions based on the site situation.

o The team is unable to verify the plan because of inadequate communication (of the goals, negative impacts, deviations) with decisionmakers.

o Supervision is inadequate in monitoring actions and questioning the current mission.

Authorization difficulty o too many levels and roles of authorization entities o no clear lines of authority o approval required from higher-level management chain 3-23

Table 3-12 PIF Team and Organization Factors (continued)

Attributes (continued)

Inadequate communication capabilities between teams o no clear guidance for the content of communication for different purposes (e.g., communication to upper or lower levels, with other parties) o no guidance or protocol for communicating the decisions o unavailable, degraded, or unreliable communication equipment (e.g., signals of wireless devices become unstable in radioactive environment; battery for communication devices is out or not working; unreliable communication channel because of noise or other environmental conditions) o communication difficulties:

using unfamiliar equipment different communication protocols between the parties (e.g., three-way communication requirement is a protocol) partial or full abandonment of routine communication means unfamiliarity of communication parties (e.g., required to communicate with offsite support party with which there is a joint drill less than once per year; involvement of different work groups or organizations) complex content

lack of or ineffective practices (e.g., pre-job briefing) to inform personnel of potential pitfalls in performing the tasks

lack of or ineffective practices (e.g., supervision) for safety issue monitoring and identification

lack of or ineffective practices for safety reporting

lack of or ineffective practices for corrective actions

Poor teamwork practices or drills together o There have been no drills on the command and control structure for the situation.

o The involved parties have not drilled together.

o Good practices are lacking for the following:

identifying and communicating priorities monitoring and coordinating actions tracking pending action maintaining common ground across the distributed team ensuring that time-critical actions are addressed 3-24

Table 3-13 PIF Work Processes Definition Work process refers to aspects of structuring operations and conduct of operations.

Good work processes in safety-critical work domains set high standards of performance and ensure the conduct of control room and field activities in a thorough and professional manner.

Discussion Included in NPP work processes are functions and tasks of plant operations, shift complement and functions, operating practices, pre-job briefings, and work control and authorization. An important aspect of work processes affecting human reliability is verification of personnels task performance. Verification may come in forms of professional self-verification, independent verification, peer-checking, and/or close supervision. In addition, NPP control rooms also have a shift technical advisor performing independent checking and advising. Verification can capture a large portion of errors personnel made in the first place and correct them. Lack of verification greatly reduces human reliability.

Attributes

Lack of professional self-verification or cross-verification (e.g., 3-way communication), peer-checking, independent checking or advising, or close supervision

Poor attention on task goal, individuals roles, or responsibilities, e.g.,

o Poor practice of attention on the task goals (personnel disengages from the goal too early) o Poor practice of keeping personnel in assigned roles and responsibilities o Excessive disturbance of planned work and assigned responsibilities o Bad shift handovers

Poor infrastructure or practice of overviewing operation information or status of event progression

Poor work prioritization, planning, scheduling, e.g, o Poor planning of work orders o Many extra instructions regarding task prioritization and scheduling o The purpose and object of the work permit was not specified o Work permits were not handed in on time and, therefore, delayed other activities o Indistinct information concerning the prioritization of different work activities o Insufficient information in operational order concerning performance of tasks 3-25

3.3.4 Task-Related Performance-Influencing Factors Task-related PIFs include the following:

information availability and reliability

scenario familiarity

multitasking, interruptions, and distractions

task complexity

time pressure, stress, and anxiety

mental fatigue

physical demands These PIFs are presented in Table 3-14 through Table 3-20, respectively.

Table 3-14 PIF Information Availability and Reliability Definition Personnel need information to perform tasks. Information is expected to be complete, reliable, and presented to personnel in a timely and user-friendly way.

Discussion In complex scenarios, large volumes of information are expected to be preprocessed and organized for personnel. Information in event scenarios may be incomplete, unreliable, untimely, or even incorrect or misleading. Personnel could receive information via sensors, instrumentation, alarms, oral communication, local observation, engineering judgment, or other means. Information that is obtained from sensors and instrumentation are usually presented to personnel with the human-system interface (HSI) such as indicators and displays. There are situations that local observations and oral transmittal of information are the only available options to obtain information.

A particular type of information for personnel responding to events is the cues of an event.

The cues are the initial signs or symptoms for personnel to perform required actions. It is assumed that if cues are not available, then the personnel will not respond to the problem, and the required actions will not be performed.

Sensors or indicators may be unreliable or misleading (e.g., damaged or degraded while appearing to be working, false alarms in design, out-of-range, or inherently unreliable sources). Flaws in system state indication - the indications display the demanded position for a component or control function, rather than the actual equipment state. (An example was the pressurizer PORV indications at Three Mile Island, which showed that the valves were supposed to be closed, while one was actually open).

Primary sources of information are not available, and secondary sources of information are not reliable or readily perceived. An example is that secondary sources of information are only available via local observation or oral transmittal.

3-26

Table 3-14 PIF Information Availability and Reliability (continued)

Discussion (continued)

An evaluation of the timeliness and quality of oral information is important to assess information availability and reliability when local observations and oral communications may be the only feasible way to confirm and monitor system status

Information source is obscured because of environmental factors (e.g., labels on the source are located in positions difficult to read).

Attributes

Updates of information are inadequate (e.g., information perceived by one party who fails to inform another party).

Information from different sources is not well organized.

Conflicts in information o Multiple alternative explanations exist for the pattern of symptoms observed.

o The available information contradicts or does not converge to yield a coherent understanding of the situation.

o Information or cues do not match procedures or guidance.

Information updates are inadequate

Different sources of information are not properly organized

Personnel are unfamiliar with the sources or meaning of the information.

Pieces of information change over time at different paces; thus, they may not all be current by the time personnel use them together.

Feedback information is not available in time to correct a wrong decision or adjust the strategy implementation.

Information is unreliable or uncertain.

Primary sources of information are not available, while secondary sources of information are not reliable or readily perceived.

Information is misleading or wrong.

o Sensors or indicators may be unreliable or misleading (e.g., they may be damaged or degraded while appearing to work; false alarms in design, out-of-range, inherently unreliable sources; conflicting data indicate a false situation or a flaw in the system state indication).

o An important cue is masked.

3-27

Table 3-15 PIF Scenario Familiarity Definition The scenario is familiar to personnel, with predictable event progression and system dynamics, and does not bias personnel in their understanding of what is happening.

Discussion Unfamiliar scenarios typically pose challenges to personnel in understanding the situation and making decisions. In addition, responses to unfamiliar scenarios could entail greater uncertainty compared to those for familiar scenarios. In unfamiliar scenarios, personnel are more likely to perform situation-specific actions not identified in the procedures.

Attributes

Scenario is unfamiliar.

o The situation does not match prior training or experience.

o No mental model exists for the situation.

o The scenario is not recognized based on procedures or guidance; personnel have to rely on knowledge to develop a mental model.

A bias or preference for wrong strategies exists.

o An example is anchoring bias. In this case, the mental model that is correct for most situations is not correct for the specific situation. This refers to the stereotype violations or not the usual suspect in psychology. That is, in most situations, there is a stereotypical explanation for a set of data. In this scenario, the usual suspect explanation is not the best explanation.

Personnel are unfamiliar with system failure modes.

Personnel are unfamiliar with worksites for manual actions.

Plans, policies, and procedures to address the situation are lacking.

3-28

Table 3-15 PIF Scenario Familiarity (continued)

Attributes (continued)

Unpredictable dynamics refers to a situation in which systems behave differently from what is expected or external factors make it difficult to predict event progression. Personnel may need to monitor multiple parameters, synchronize information, and constantly update their mental models to understand the situation and make decisions.

o The event evolution and system responses are unpredictable.

o Feedback information is not available in time to correct a wrong decision or adjust the strategy implementation.

o The decision has unintended side effects that are hard to predict.

o Personnel are unable to effectively evaluate the strategies pros and cons.

o The situation involves fast-changing information and cues.

Dynamic decisionmaking is required. Complex system dynamics require constant collection of information to adjust the decision.

Shifting objectives mean that tasks originally given to personnel change over time. This requires a revision in personnels mental models and plan for meeting the original goals.

Table 3-16 PIF Multitasking, Interruptions, and Distractions Definition Multitasking refers to performing concurrent and intermingled tasks. Distraction and disruption refer to things that interfere with personnels performance of their critical tasks.

Discussion Because each task requires multiple cognitive functions, such as detecting cues or parameters, assessing information, and mentally programming sequences of actions, personnel must frequently switch between these tasks during multitasking. Switching between tasks can make errors more likely. An example of multitasking is concurrently implementing multiple procedures; personnel may skip procedure steps when switching between procedures. An example of extreme multitasking is a situation in which decisionmakers must handle several operational systems (e.g., reactor units) that are in different critical states. In this example, related items of information about different systems may be mixed or transposed.

Examples of distractions and interruptions are phone calls, requests for information, and the concurrent activities going on in the work environment. Prolonged interruption refers to situations in which personnel are kept from their critical tasks for a prolonged period or interrupted by cognitively demanding requests.

3-29

Table 3-16 PIF Multitasking, Interruptions, and Distractions (continued)

Discussion (continued)

Experience from actual events has shown that personnel may also be distracted by failures or damage to non-safety systems that are important for overall plant investment protection or for perceived improvement of overall plant conditions, but are not modeled explicitly in the PRA. In some scenarios that involve severe plant damage (e.g., fires, floods, seismic events, etc.), operators may also need to attend to treatment and relocation of personnel who are physically injured. These concerns introduce conflicting strategic and time priorities for decision makers and constraints on the assignment of limited personnel resources. These types of diversions and distractions have occurred in practice, and analysts should account for them. That is why it is essential that the integrated scenario narrative must describe the entire context of the plant damage and not focus only on systems and equipment that are modeled explicitly in the PRA and the distinct human actions that are needed to cope with only those failures.

Attributes

excessively frequent or long interruption during the continuous performance of critical tasks

distraction by other ongoing activities that are relevant to the critical task being performed

Distraction by other ongoing activities that are not directly relevant to the critical task being performed (e.g., damage to systems and equipment that are not modeled explicitly in the PRA, personnel injuries, etc.)

concurrently detecting (monitoring or searching) multiple sets of parameters when the parameters in different sets may be related

concurrently diagnosing more than one complex event that requires continuous seeking of additional data to understand the events

concurrently making decisions or plans that may be intermingled

concurrently executing intermingled or interdependent action plans

Command and control multitasking o The decisionmaker has multiple issues to address in parallel.

o The decisionmaker has multiple individuals (or groups) working independently and in parallel to monitor and control or to manage and supervise.

3-30

Table 3-17 PIF Task Complexity Definition Task complexity, also referred as cognitive complexity, measures task demand for cognitive resources (e.g., working memory, attention, executive control). Nominal complexity refers to the level of complexity that does not overwhelm personnel.

Discussion The cognitive complexity of a task has two parts: the complexity in processing the information to achieve the macrocognitive functions of the task, and the complexity in developing and representing the outcomes to meet the task criteria. For example, a task is to monitor a set of parameters, and the outcome is to identify the parameters outside a certain range or determine the trends of the parameters. The latter imposes higher cognitive demands on personnels working memory; thus, it is more complex. Complexity is characterized by the quantity, variety, and relation of the items to be processed or represented in a task.

Attributes

Detection criteria are complex. For example, there are multiple criteria to be met or complex logic; information of interest must be determined based on other pieces of information and may involve complex computation; or detection criteria are ambiguous.

Detection overloading. For example, personnel may need to concurrently track the states of multiple systems, monitor many parameters, and memorize many pieces of information detected.

Detection requires sustained attention. For example, determining a parameter trend during unstable system status or monitoring a slow-response-system behavior without a clear time window to conclude the monitoring requires attention for a prolonged period.

Cues for detection are not obvious. That is, alarms or instructions do not directly cue detection, so personnel must actively search for information.

Multiple causes for situation assessment: Multiple independent influences affect the system, and system behavior cannot be explained by a single influence.

Relations of systems involved in an action are too complicated to understand

Key information is cognitively masked (e.g., hidden coupling, cascading effects, cognitive masking, and complex logic), and the source of a problem is hard to diagnose because of cascading secondary effects that make it difficult to connect the observed symptoms to the originating source.

The potential outcome of the situation assessment consists of multiple states and context (not a simple yes or no).

Decisionmaking involves developing strategies or action plans.

3-31

Table 3-17 PIF Task Complexity (continued)

Attributes (continued)

Decision criteria are ambiguous and subject to different interpretations.

Multiple, intermingled goals or criteria need to be prioritized.

Goals conflict (e.g., choosing one goal will block achievement of another goal, and multiple competing goals cannot be prioritized).

Decisionmaking requires integration of a variety of types of information with complex logic.

Decisionmaking requires diverse expertise distributed among multiple individuals or parties who may not share the same information or have the same understanding of the situation.

Competing strategies: Multiple strategies can achieve the same goal but with different benefits and drawbacks. These strategies affect each other (e.g., competing resources or delaying critical actions that affect the likelihood of success).

Personnel may need to unlearn or break away from automaticity of trained action scripts.

o Negative transfer between tasks (not used to doing it this way):

Identical or similar tasks performed in different settings, modes, or procedural sequences require different approaches.

Controlled actions may require monitoring of action outcomes and adjusting action accordingly.

o Initiation of the action requires monitoring of certain parameters for a period of time or waiting for a period of time (until the parameters reach a specified threshold).

Action criteria are difficult to use:

o too restrictive to meet o no indication that the criteria are met o not explicit or concrete o too many criteria

Action requires out-of-sequence steps.

3-32

Table 3-17 PIF Task Complexity (continued)

Attributes (continued)

Long-lasting, noncontinuous action sequences, or long-time gap between the cues for execution to initiation of the execution are necessary.

o Actions demand prospective memory (i.e., long lapse time before commencing a follow-up activity).

o An action sequence includes a disconnected activity in the future for which there is no strong memory cue. Performing the action sequence requires personnel to memorize past status over a prolonged period (longer than several hours).

Action sequences are parallel and intermingled.

Action execution requires close coordination of multiple personnel at different locations.

Action execution requires long sustained attention.

Table 3-18 PIF Mental Fatigue Definition In the normal status of mental fatigue, personnel do not experience decrement of vigilance and abilities to perform complex cognitive tasks.

Discussion Mental fatigue can result from performing a task for an extended period of time, nonroutine tasks, and cognitively demanding tasks. Mental fatigue leads to loss of vigilance, difficulty in maintaining attention, reduced working memory capacity, and use of shortcuts in diagnosing problems or making decisions.

Attributes

sustained, high-demanding cognitive activities (e.g., mismatches between procedure and situation demand constant problem-solving and decisionmaking; information changes over time and requires sustained attention to monitor or frequent checking)

long working hours with cognitively demanding tasks

sleep deprivation, exposure to noise, disturbed dark and light rhythms, and air pollution 3-33

Table 3-19 PIF Time Pressure and Stress Definition Time pressure refers to the sense of time urgency to complete a task, as perceived by personnel. This sense of time urgency creates psychological pressure affecting personnel performance.

Discussion Time pressure arises when making a tradeoff between thoroughness in performing the task and completing the task in time. Because time pressure is based on personnels perception and understanding of the situation, it may not reflect the actual situation. Therefore, although time pressure is most likely to occur when marginal or inadequate time is available, it also could occur in scenarios with adequate available time, but personnel have an incorrect understanding of the situation. For example, some training protocols emphasize the importance of making assertive, immediate decisions, and they reward personnel for rapid, correct responses. This type of training can instill an inappropriate sense of urgency, reluctance to question initial impressions, and resistance to deliberative team consultation.

Anxieties, such as concern for families in emergency conditions, fear of potential consequences of the event, and worrying about personal safety, can also increase the level of psychological stress and affect performance. Such concern is prevalent during scenarios that involve extreme hazards such as fires, seismic events, floods, high winds, aircraft crashes, etc..

Attributes

reluctance to execute an action plan because of potential negative impacts (e.g., adverse economic impact)

high time pressure because of perceived lack of adequate time to complete the task or because of training protocols that instill an artificial sense of time pressure and urgency for task performance

mental stress concerning the high workload or task difficulty

emotional stress (e.g., anxiety, frustration)

physical stress (e.g., disturbed dark and light rhythms, air pollution) 3-34

Table 3-20 PIF Physical Demands Definition Physical demands indicate that a task requires extraordinary physical effort, such as twisting, reaching, dexterity, or strong force. Personnel safety refers to that there is the likelihood of personnel injury in performing certain actions for the given scenarios, in particular under extreme operating conditions.

Discussion In practice, personnel safety would most likely apply to scenarios with extreme operating conditions, such as those involving plant damage from internal hazards (fires, floods, etc.), external events (seismic events, floods, high winds, aircraft crashes, etc.), impending or actual core damage, large releases of radiation or toxic chemicals, etc. It accounts for the effect of personnels concerns about their own personal safety and possible harm or known injuries to their co-workers on task performance. The effects from this PIF may be manifested by personal fear, cognitive distractions, an enhanced sense of urgency, additional time delays for cognitive response and action implementation, supervisory reluctance to send personnel into specific plant locations, operator reluctance to perform local actions, etc.

Attributes

Action execution requires highly accurate fine-motor skills, fine-motor coordination, or skills of craft.

Fine or difficult motor actions, such as installing or connecting delicate parts, must be performed.

The task is physically strenuous (e.g., lifting heavy objects, opening or closing rusted or stuck valves, moving heavy things in water or high wind).

There is resistance to motor movement (e.g., wearing heavy clothing; lifting heavy materials; opening or closing rusted or stuck valves; executing actions in water or high wind, in extreme cold or heat, or on unstable ground).

The task is performed in ways or locations that can impact personnel safety.

In summary, the PIF attributes listed in this section are described in terms of their potential adverse impact on macrocognitive functions. The list is independent of work domains or HRA applications. Specific applications may introduce additional PIF attributes. This comprehensive list serves as a basis for developing HRA methods for nuclear-related applications.

3.4 Links between Performance-Influencing Factor Attributes and Cognitive Mechanisms Every PIF attribute in the comprehensive list of PIFs challenges one or several cognitive mechanisms. The challenges to the cognitive mechanisms increase the likelihood of a processor error and subsequent failure of a macrocognitive function (i.e., human error).

However, the presence of one or several PIF attributes does not always result in error of the processorsrather, they increase the likelihood of the failure. To use the PIF list for HRA, potential links between the PIF attributes and cognitive mechanisms, which may lead to 3-35

processor error and subsequent failure of the macrocognitive functions, are inferred. Below are some examples demonstrating that a PIF attribute may affect several processors and vice versa (Appendix A to NUREG-2114 [21] provides more examples).

For every challenge to the cognitive mechanism that may lead to processor error and subsequent failure of the macrocognitive functions, IDHEAS-G generalizes the PIF attributes that can affect the likelihood of macrocognitive failure. The tables in APPENDIX B to this report present the links between the cognitive mechanisms and PIF attributes. The tables are an enhancement of Appendix A to NUREG-2114 [21] in that they provide explicit links between cognitive mechanisms and PIF attributes, while NUREG-2114 links a limited set of processors to high-level PIFs. On the other hand, Appendix A to NUREG-2114 provides a larger set of examples from the research literature to show how a PIF may affect the processors through cognitive mechanisms.

3.5 Effects of Performance-Influencing Factor Attributes on Macrocognitive Functions Abundant evidence in the research literature and operational experience demonstrate that PIF attributes increase the likelihood of errors in macrocognitive functions or their processors. Yet, the quantitative relationship between PIFs and HEPs has been ambiguous. IDHEAS-G needs substantial data to explain the following three aspects:

(1) assessing states of PIFs or attributes (2) quantifying HEPs that change with the state of a PIF or its attributes (3) combining the effects of multiple PIFs on the HEP To gain insights into these aspects, the NRC staff performed a metadata analysis of cognitive and behavioral literature as well as human event databases in various safety-critical work domains (e.g., aviation, transportation, manufacturing, and healthcare). A metadata analysis combines the data from multiple studies to arrive at a conclusion or obtain insights into the answer of an inquiry. The following sections briefly summarize the insights learned from the metadata analysis.

3.5.1 Assessing Performance-Influencing Factor States In quantifying the contribution of PIFs to HEPs, HRA methods may model individual PIFs in a binary fashion. Some methods use several discrete levels to model the state of a PIF.

Examples of labels for different levels include the following:

binary states:

o present versus not present o low versus high o good versus poor o nominal versus poor

multiple discrete states:

o low, medium, high o good, nominal, poor In contrast to HRA methods, most empirical studies in the literature or operational databases assess specific PIF attributes rather than using the entire PIF as a single variable. The attributes are continuous or discrete variables. For example, when assessing the PIF task 3-36

complexity, simply denoting it as low, medium, or high does not link its impact on HEPs. Thus, to quantify HEPs, assessment of PIF states should be based on the states of its attributes.

Each PIF should have a baseline state for reference; that is, all its attributes have no impact on human errors.

3.5.2 Quantifying the Effects of Performance-Influencing Factors Cognitive and behavioral research examines the effects of PIF attributes through experimental measures of human error rates under controlled conditions. The research typically measures human error rates in performing cognitive tasks while systematically varying certain PIF attributes. There have also been metadata studies [47]-[50] that synthesized experimental results for a particular PIF attribute. After performing an extensive literature review, the NRC staff synthesized data and evidence on the effects of PIF attributes on human errors. The identified studies measured human error rates, while varying the states of attributes of one or more PIFs.

The quantitative relationship between human error rates and PIF attributes depends on the definition of the impact of attribute states on human error rates. Experimental studies typically measure human error rates at the no-impact or low-impact state versus higher impact states.

With such data, the NRC staff can calculate the effect of the PIF as a weighting factor, defined as the following:

= (3.1) where is the human error rate at the given PIF state and is the human error rate at the base state of the PIF. APPENDIX D presents several examples of weighting factors derived from the literature.

3.5.3 Quantifying the Effects of Multiple Performance-Influencing Factors HFEs modeled in PRA involve multiple PIFs. HRA methods typically treat the effects of a combination of PIFs in two ways:

(1) holistic estimationExperts estimate the probability of an HFE or a failure mode for a given set of PIFs considering, but not explicitly modeling, the combination of factors.

(2) combination of individual effectsThe HEP is the combination of the impacts of individual PIFs. Existing HRA methods have generally used multiplicative combination, (i.e., the combined HEP is calculated as a baseline probability multiplying the multipliers associated with individual PIFs).

To understand the cognitive basis for the quantitative treatment of combinations of PIFs, the NRC staff studied the experimental literature that examines the individual and combined effects of two or more PIF attributes. APPENDIX D summarizes the NRC staffs study. The NRC staff used the weighting factor defined in Equation (3.1) as a measure of the PIF effect. The staff did not systematically mine the experimental data, nor did it perform a full metadata analysis of the limited sample of the literature. Nevertheless, the initial observation from the data reviewed suggests that the effect of combined PIFs can be roughly estimated by adding the effect of individual PIF weights. Future research, including an extensive metadata analysis of the data in the literature, should be performed to establish the cognitive basis for combining the effects of multiple PIFs.

3-37

3.6 Summary This chapter consolidates knowledge about factors affecting human errors into a PIF structure.

The NRC staff intended to model the context of an important human action with a PIF structure that has the desired traitsit should be comprehensive and pertinent, independent, specific, explainable, assessable, and quantifiable. With respect to comprehensiveness and pertinence, the PIF structure presented in this chapter covers all the PIFs in the reviewed HRA methods and the factors reported in the broad literature and nuclear-specific human event databases, with the intent to make it comprehensive enough for nuclear applications. The PIF categories correspond to different types of context, which supports the independence of the individual PIFs. Moreover, every PIF is characterized by a set of attributes, which make the PIFs specific.

Moreover, attributes were identified from empirical data from experimental studies and operational experience, and they are linked to cognitive mechanisms and, therefore, to the processors of macrocognitive functions. These make the PIFs explainable and quantifiable with human error data. This structure advances the state of practice of using PIFs in HRA.

A specific HRA application may involve only a subset of PIFs from the structure, and various applications may involve different subsets of PIFs. Nevertheless, the subsets of PIFs for various HRA applications share this common structure. This increases consistency between methods and allows comparisons of the HRA quantification results from different methods.

3-38

4 AN INTEGRATED PROCESS FOR HUMAN RELIABILITY ANALYSIS WITH IDHEAS-G This chapter describes the IDHEAS-G HRA process as a method to perform HRA. The method implements the cognition model in the general HRA process. The method has four stages:

scenario analysis, modeling of important human actions, estimation of HEPs, and integrative analysis. Each stage analyzes a human event with different perspectives and levels of detail.

The NRC staff developed step-by-step guidelines for the four stages and detailed supplemental guidance for practical use of the method. The appendices to this report present the step-by-step guidelines for the first three stages and supplemental guidance. The description of each stage in Chapter 4 points to the corresponding guidance appendices. Readers are highly recommended to read and use the materials in Chapter 4 along with the corresponding guidance in the Appendices. In particular, Appendix E provides guidance for performing scenario analysis to establish an overall understanding of event context and event evolution. It provides structured ways to acquire information on human aspects of an event evolution and organize context information for modeling and quantifying important human actions. Appendix F provides guidance on identifying and defining critical human actions with and without the presence of PRA models. Appendix G provides guidance for identifying and characterizing critical tasks in a human action. Together, these three appendices provide a specific, logical framework and vocabulary for performing qualitative analysis for HRA. This qualitative analysis guidance is applicable to any application-specific IDHEAS method derived from IDHEAS-G.

4.1 Overview of the IDHEAS-G Human Reliability Analysis Process 4.1.1 General Human Reliability Analysis Process HRA requires both qualitative analysis and quantification of HEPs, as illustrated in Figure 4-1.

Qualitative analysis involves understanding the event and systematically collecting information for quantification. The full span of performing an HRA generally includes the following activities:

Collect information about the event being analyzed: The event being analyzed is described with a baseline scenario and potential deviating scenarios. Information about scenario context is also collected.

Identify important human actions: Important human actions (IHAs) in a scenario are identified. In the context of a PRA, an IHA is the same as an HFE. In the context of an integrated safety analysis (ISA) 1, an IHA is the same as an item relied on for safety 1F (IROFS). 2 2F 1 An ISA is a systematic analysis developed for facilities subject to the requirements of Subpart H of Title 10 of the Code of Federal Regulations (10 CFR) Part 70, which identifies facility and external hazards and their potential for initiating accident sequences, the potential accident sequences, their likelihood and consequences, and the [IROFS].

2 IROFS are structures, systems, equipment, components, and activities of personnel [emphasis added] that are relied on to prevent potential accidents at a facility that could exceed the performance requirements in 10 CFR 70.61 or to mitigate their potential consequences.

4-1

Perform task analysis of important human actions: An important human action may consist of one or multiple discrete tasks for analysis. Task analysis identifies and characterizes the critical tasks in an important human action.

Perform time analysis and assess feasibility of the important human actions.

Evaluate potential failure modes of an IHA or its critical tasks and assess relevant PIFs.

Estimate failure probability of the IHA (i.e., the HEP).

Analyze dependency between IHAs and adjust the probabilities of the dependent IHAs accordingly.

Identify and document sources of uncertainties throughout the HRA process.

Not every HRA method includes all these steps. Some methods begin with quantification without explicitly requiring qualitative analysis. Many HRA methods do not have a separate step for task analysis. In practice, some HRA analysts may perform task analysis along with collecting qualitative information, while others implicitly identify tasks to be modeled as a part of identifying the failure modes of an IHA.

Collect data/information and interact with PRA Analyze scenarios and develop operational Analyze and quantify HEP narrative of a human failure event:

Identify failure modes

Assess PIFs Identify and define IHAs

Estimate the HEP Identify and analyze tasks Analyze HFE dependency Time and feasibility analysis and adjust HEPs Analyze and document uncertainties Figure 4-1 General HRA Process 4.1.2 IDHEAS-G Human Reliability Analysis Process The IDHEAS-G HRA process includes all the activities described in Section 4.1.1 and organizes the activities into four distinct stages, as described below and shown in Figure 4-2.

Stage 1: Scenario analysisThe purpose of this stage is to understand human performance in the event and collect information for quantification. It includes developing operational narratives, analyzing the scenario/event context that affects human performance, and identifying and defining important human actions in the event.

Stage 2: Modeling of important human actionsThe purpose of this stage is to model the challenges to human performance of an important human action. It includes identifying and characterizing critical tasks in an important human action, identifying potential CFMs of the critical tasks, and assessing the PIFs relevant to the critical tasks.

4-2

Stage 3: HEP quantificationThe purpose of this stage is to estimate the HEP of an important human action. It has two parts: (1) estimating the error probability attributed to the uncertainties and variability in the time available and time required to perform the action and (2) estimating the error probabilities attributed to the CFMs.

Stage 4: Integrative analysisThe purpose of this stage is to assess the dependency between the analyzed important human actions in the scenario/event and document the uncertainties in the event analysis.

Stage 1 Stage 2 Stage 3 Stage 4 Scenario analysis Modeling of HEP Integrative

Develop operational important quantification analysis narrative human actions

Identify event context

Estimate HEPs

Document

Identify and analyze uncertainties

Identify and define critical tasks due to time important human

Identify CFMs uncertainties and

Assess actions

Assess PIFs CFMs dependencies Figure 4-2 IDHEAS-G HRA Process The flow information in the IDHEAS-G HRA process is illustrated in Figure 4-3 as a group of steps and Table 4-1 provides a crosswalk between the IDHEAS-G stages discussed above and the steps shown in Figure 4-3.

PIF attributes of every CFM Scenario for every CT PRA context and Step 5: Calculate model list of applicable Step 4: Assess PIFs Step 1: Determine PIFs applicable to every scenario context CFM List of Step 1: HFE and its applicable Step 7:

Develop scenario narrative definition CFM(s) for Calculate Develop scenario timeline the CT(s) overall HEP HFE and its Step 2: List of CT(s) Step 3: Characterize the Step 1: Identify HFE definition Analyze tasks CT(s) and select applicable Step 1: Define HFE and identify CFMs CT(s) in HFE HFE and its definition and Step 6: Estimate parameters Step 6: Analyze HFE timeline of distribution (subset of scenario timeline, if there are Step 6: Calculate multiple HFEs in the scenario)

Step 6: Estimate parameters of distribution and Step 8:

Uncertainty and dependency analysis and documentation CFM = cognitive failure mode = error probability due to CFMs CT = critical task = error probability due to uncertainty in and HEP = human error probability = time available HFE = human failure event = time required PIF = performance-influencing factor and = mean and standard deviation of PRA = probabilistic risk assessment and = mean and standard deviation of Figure 4-3 Illustration of the IDHEAS-G HRA Process 4-3

Table 4-1 IDHEAS-G HRA Process Stages and Steps IDHEAS-G Stages IDHEAS-G Steps Stage 1 - Scenario analysis Step 1: Develop scenario narrative Step 1: Develop scenario context Step 1: Identify HFE Step 1: Define HFE Stage 2 - Modeling of Step 2: Analyze tasks and identify CT(s) in HFE important human actions Step 3: Characterize the CT(s) and select applicable CFMs Step 4: Assess PIFs applicable to every CFM Stage 3 - HEP quantification Step 5: Calculate Step 6: Analyze HFE timeline Step 6: Estimate parameters of and distributions Step 6: Calculate Step 7: Calculate overall HEP Stage 4 - Integrative analysis Step 8: Uncertainty and dependency analysis and documentation The IDHEAS-G Cognitive Model for Human Performance and Reliability is implemented in every stage. For Stage 1, Scenario Analysis, the cognitive model asks questions to collect context information pertinent to the macrocognitive functions and PIFs and organizes information for input to subsequent stages. Stage 2, Modeling of Important Human Actions, is based on the cognitive model, representing the failure of a human action with the CFMs derived from the cognitive basis structure and representing the event context in the PIF structure. Stage 3, HEP Quantification, is based on the CFMs. The estimation of HEPs relies on human error data generalized using the cognition model. For Stage 4, Integrative Analysis, the NRC staff developed insights on how to use the cognition model to assess the dependency between the important human actions in an event, as documented in APPENDIX K.

The output of each stage serves as the inputs to subsequent stages. Moreover, the outputs of the various stages represent the understanding of the event from different perspectives and at different levels of detail. The outputs of all of the stages together provide an integrated, systematic understanding of the event on what may happen to human performance and how personnel may succeed or fail the action.

Note that Stage 2, Modeling of Important Human Actions, is distinct from HEP quantification in Stage 3. The purpose of modeling an important human action is to construct a failure model for the action. A failure model is independent of the ways of estimating the HEPs. Stage 2 describes what can go wrong with the important human action, how it may fail, and what factors affect the failure. Based on this failure model, the HEP may be estimated in different ways, depending on the availability of human error data relevant to the failure model.

4.2 Stage 1: Scenario Analysis 4.2.1 Overview of Scenario Analysis The objective of this stage is to understand human performance in the event and collect information that may influence important human actions. A PRA event has its safety-related goal or mission. Scenario analysis helps HRA analysts to understand important actions that personnel must perform to achieve the goal and the challenges personnel may face in the event scenario. The analysis allows HRA analysts to gain perspectives on the broad spectrum of 4-4

scenario-specific conditions that may require personnel attention. Such perspectives are essential for an analyst to perform an integrated assessment of the factors that may influence personnel performance in the context of the evolving scenario. This stage includes the following: development of operational narrative, identification of event context, and identification of important human actions.

Development of operational narrative. The operational narrative provides a detailed account of the event scenario, including a storytelling-style representation and timeline. The objective of an operational narrative is to develop an in-depth understanding of the event evolution. The operational narrative specifies the initial conditions, initiating event, and boundary conditions of the event, as well as the scenario progression and consequence. A baseline scenario describes the expected event evolution. Then, the baseline scenario is used as a reference to identify the alternative scenarios that could affect the goal of the event. Several representative scenarios (including the baseline scenario) may be identified and together represent different potential evolutions of the event.

Identification of event context. The event context provides a broad view of the conditions that affect human performance, including those imposed by the environment, systems, personnel, and the actions to be performed. The context influences event progression, human actions to be performed, and human failure modes.

Identification of important human actions. Important human actions are those that personnel must perform to achieve the event goal. A PRA model typically indicates the important human actions required to achieve the success of the event in the event diagram. Identification of important human actions involves working from a PRA model to define those human actions from HRA perspectives, identify additional important human actions that are not included in the PRA model but may affect the goal of the event, and identify important human actions when a PRA model does not yet exist.

The three parts of the analysis support one another to provide a holistic representation of the event. They should be performed iteratively to obtain an integrated understanding of the event.

Figure 4-4 illustrates the iterative process of scenario analysis.

Development of operational narrative Identification of Identification of important human event context actions Figure 4-4 Iterative Process for Scenario Analysis 4.2.2 Development of Operational Narrative The initial conditions of an event define the beginning status of an event. An event scenario begins with an initiating event that disturbs the systems and initial conditions. The baseline scenario of an event is either the typical operational progression following the initiating event or a specific PRA sequence with explicit component failures and human failure events identified.

The baseline scenario is used as the reference for PRA/HRA analysts to gain an understanding 4-5

of the event progression and to identify additional scenarios deviating from the baseline scenario that affect risk assessment.

The operational narrative is a means for HRA analysts to develop an in-depth understanding of the scenario progression. The operational narrative includes two parts: scenario narrative and scenario timeline. The scenario narrative is a storytelling-style documentation of the scenario progression. The scenario timeline documents important human actions and system responses in chronological order. Figure 4-5 illustrates the content of an operational narrative.

Scenario narrative

Overview of the event

Beginning status

Initiating event

Initial conditions

Boundary conditions

Progression and end state Baseline scenario Timeline

Date/time Operational

System response, human narrative of a response, data for human event situation awareness, and notes Deviations from the baseline Additional scenarios scenario Figure 4-5 Composition of an Event Operational Narrative A PRA model has the baseline scenario including the narrative and event progression diagram.

The objective of an HRA operational narrative is to identify and document information specific to human performance along with the PRA model.

4.2.2.1 Baseline Scenarios The baseline scenario should describe the expected event evolution and information about human performance at the appropriate level of detail along a timeline of the scenario progression. It should also include operating experience related to the scenario.

For NPP events, baseline scenarios can be derived in the following situations:

For basic PRA analysis or the condition analysis of the Significance Determination Process (SDP) and Accident Sequence Precursor (ASP) program: These address hypothetical events. The baseline scenario is the expected event progression path based on the given initial conditions, initiating event, and boundary conditions. The baseline scenario describes the expected system responses and personnel actions to the event.

For the event analysis in the SDP and ASP program: These address actual events. The baseline scenario describes what actually occurred in an event, including the actual system and personnel responses.

4-6

Initiating Event An initiating event originates from an internal or external hazard. It causes abnormalities, which may require system automatic interventions, human interventions, or both, to protect safety.

The ASME/ANS PRA Standard [44] defines an initiating event for nuclear reactor safety as follows:

an event either internal or external to that which perturbs the steady state operation of the plant by challenging plant control and safety systems whose failure could potentially lead to core damage or release of airborne fission products. These events include human-caused perturbations and failure of equipment from either internal plant causes (such as hardware faults, floods, or fires) or external plant causes (such as earthquakes or high winds).

Initial Conditions The initial conditions describe the beginning status of systems and personnel that have implications for the scenario progression. The PRA model generally defines the initial conditions of an event. The HRA activities of describing initial conditions should consider the conditions that can affect human performance.

Boundary Conditions The boundary conditions describe the expected systems, site, and personnel status immediately after the initiating event. The boundary conditions specify the consequences of the initiating event. The boundary conditions also specify the scope and the assumptions applied to the HRA. They limit the analysis scope to focus on the primary issues and to make simplified assumptions about the status of systems (e.g., damage associated with the initiating event) and personnel.

Scenario Progression and End State The scenario progression documents the scenario development following the given initiating event, initial conditions, and boundary conditions. The purpose is to describe the scenario progression with the emphasis on important human actions involved. The description should include the safety issues and the expected responses of systems and personnel. At a high level, those responses can be summarized using an analogy to the following macrocognitive functions:

cues for detection

diagnostic information for understanding and decisionmaking

action execution that manipulates systems to achieve the event goal The cues are the information that raises attention for detection and triggers personnels cognitive processes to address the safety issue. The diagnostic information is the information required to make a diagnosis and gain situational awareness for understanding.

Decisionmaking refers to making a response decision based on the situational awareness and diagnosis. Action execution refers to implementing the response decision. Table 4-2 provides guiding questions to collect the narrative information.

4-7

Table 4-2 Guideline Questions to Collect Information for the Operational Narrative Hypothetical Events Safety issue:

- What is the safety issue?

- How does it occur?

- What is the safety significance?

Cues:

- What are the cues?

- How are the cues generated?

- What are the means to detect the cues?

Diagnosis and making decision:

- What is the information for diagnosis?

- How are the diagnosis and decisionmaking performed?

- What are the basis of and constraints on diagnosis and decisionmaking?

- What may mislead personnel to make a wrong diagnosis or decision?

Actions:

- What are the automatic system responses to the safety issue?

- What are the manual actions needed to mitigate the safety issue? How are the actions performed? What are the constraints on performing the actions?

Overarching considerations:

- Specify who does what for each macrocognitive function.

- Discuss the considerations that could have significant effects on operator responses.

- Teamwork and communication should be discussed when applicable.

Timeline The scenario timeline describes the event progression in chronological order. Important timing of system status changes and cues for important human actions should be included. Along the timeline are different types of information for understanding human-system interactions. Each information type is described below.

System automatic responses: A system automatic response is a system status change based on the setpoints or logic of the automatic component actuations or that a system failed to perform its designed function. An example is safety injection (SI) injected coolant into the reactor coolant system (RCS) at 1,600 pounds per square inch gauge (psig).

Human responses: These are important human actions that include detecting the cue, making a diagnosis, entering and exiting procedures, making decisions important to the scenario, and performing actions. The actions could be either physical interference with a system to change the scenario course or actions that should be performed but are not performed, which allows safety degradation in the scenario. Each human response should identify the task and the crew or individual who performs the task.

4-8

Critical data for situational awareness: This refers to the information generated from a system or other source for personnel to diagnose the situation or make decisions.

Examples are the alarms that notify the operator of a system abnormality.

Notes: Included in the notes is information about background, explanation, context, or supplemental information to the system responses, human response, and key data. For example, a required human response is depressurize the reactor pressure vessel to a certain pressure range at a rate less than 100 °F/hr; included in the notes is the task takes about 2 hours2.314815e-5 days 5.555556e-4 hours 3.306878e-6 weeks 7.61e-7 months by periodically manually opening and closing a safety relief valve.

4.2.2.2 Additional Scenarios To perform a risk assessment, the PRA identifies possible event scenarios leading to undesired consequences. Failure of the system or required important human actions may generate new scenarios deviating from the baseline scenario. Additional scenarios are identified by asking what if questions on the failures and consequences of the systems responses and important human actions. The focus on identifying additional scenarios is to develop a high-level risk perspective of the system responses and important human actions that, if failed, would change the scenario progression. In addition, one important purpose of identifying additional scenarios is to determine whether characteristics of system responses or human performance in those scenarios merit distinct and explicit evaluation in the PRA models. In some cases, the PRA team may decide that the differences should be represented by distinct scenarios in the PRA event trees and fault trees. In those cases, human performance should be evaluated in the context of those scenarios according to the same methods and guidelines that are applied for analyses of the baseline scenario. For example, distinct human failure events (HFEs) are defined, evaluated, and quantified, accounting for the scenario context. In other cases, the PRA team may decide that it is not necessary to explicitly account for differences from the baseline scenario by defining distinct new scenarios in the PRA event trees and fault trees. In those cases, the identified possible deviations from baseline scenario conditions introduce a source of uncertainty in the evaluations of the performance-influencing factors (PIFs) and human error probabilities (HEPs) for the defined baseline scenario HFEs. For example, an analyst might need to consider additional PIFs or a broader range of PIF attributes to account for the range of possible deviations within the baseline scenario.

4.2.3 Identification of Event Context Identification of event context refers to the search for the conditions that challenge or facilitate human performance in the event. Event context serves as the high-level guidance for defining and analyzing important human actions. It provides a basis for estimating the HEPs of the important human actions in the event. In HEP estimation, the context is represented by the states of the PIFs. Event context is documented in the following categories:

environment and situation context system context personnel context task context Identification of event context is an exploration of the conditions that are likely to lead to cognitive challenges, as well as the conditions that may positively affect human performance.

Context affects personnel performance by directly impacting systems and personnel or mitigating the adverse effects of other conditions. Event context should describe all of the conditions that may affect human performance in the entire event. Practically, the process of 4-9

searching for event context should focus on the conditions that can affect the macrocognitive functions and lead to undesirable consequences of the event.

Environment and Situation Context Environmental conditions can impact system responses and human performance. Typical causes of environmental conditions include fire, smoke, flood, earthquake, tornado, temperature extremes, and radiation. NUREG/CR-5680 [45], [46] describes the following environmental conditions that should be considered in risk analysis:

temperature and humidity

noise

radiation or chemical contamination

light and glare

smoke and fog

high wind

standing or running water

debris

vibration

seismic aftershocks Environmental conditions may change during the evolution of an event, and they may vary in the different locations where important human actions are performed. Thus, documenting environmental context should annotate the scope to which the context applies.

System Context IDHEAS-G uses the term systems to broadly refer to structures, systems, and components, as well as sensors, equipment, I&C, and HSIs. Systems typically include the following:

Physical structures (and their locations) for personnel and systems to do the work.

Frontline systems that perform accident mitigation functions.

Supporting systems that help primary systems or personnel to achieve their functions.

NUREG-2122, Glossary of Risk-Related Terms in Support of Risk-Informed Decisionmaking, issued November 2013 [51], interprets supporting systems as follows:

In a PRA, support system failures are evaluated to determine the effect of these failures on the operability of other plant systems and components.

Often one support system, such as component cooling water, provides functionality to multiple systems or components, and therefore, needs to be considered in PRA modeling to assess what happens if that capability is lost to multiple systems. Examples of support systems include electrical power, cooling water, instrument air, and heating, ventilation, and air conditioning. Support systems (e.g., cooling water) can require other support systems for operation (e.g., electric power may be needed to operate the cooling water pumps). Front-line systems typically require one or more support systems. In some instances, a failed support system can lead to an undesired plant condition that requires successful mitigation by plant equipment and personnel to prevent core damage from occurring.

4-10

Event-related systems that do not support the mission and expected outcomes of the event but are related to the event by sharing common resources, personnel, or physical structures.

I&C, sensors, equipment, HSIs, and any subjects needed by the systems or personnel.

To identify system context, it is important to understand the operational concepts of the systems (i.e., how the systems are intended to work and how they are intended to interact with personnel). Examples of different types of human-system interactions include the following:

Personnel operate systemsSystems perform their functions as directed by personnel; for example, a radiologist operates a remote after-loading brachytherapy system to perform radiotherapy for patients.

Personnel supervise process control systemsSuch systems, under normal operating conditions, require only occasional fine tuning of system parameters to maintain satisfactory performance, and personnel have overall responsibility for control of the system. Examples are NPP control systems and medical patient-monitoring systems.

Personnel support autonomous systemsSystems automatically perform all of the mission-critical tasks, and the major tasks for personnel are to program changes in inputs or control routines and to serve as a backup in case of a failure or malfunction in a system component. Examples of autonomous systems are small modular reactors and unmanned vehicles.

Passive systemThe system operates on gravity and does not need personnel for operation (it would still need personnel for installation and maintenance, and personnel may decide to intervene in its operation).

The system context includes the conditions that could negatively or positively affect human performance. Identification of system context should focus on conditions that create conflicting priorities, confusion, and distractions to human performance. Those conditions often involve nonsafety systems and equipment that are not the focus of an event analysis (so they may not be modeled in the PRA). For example, NPP operators may be concerned about possible damage to major plant equipment that is not directly relevant to the PRA event scenario, failures or interruptions of nonsafety power supplies that are not explicitly modeled in the PRA, disruptions of low-voltage I&C power supplies that are not modeled, investigation of false fire alarms, and other conditions. System context should include these elements of the event scenario.

Below are some general considerations for identifying system context:

Systems may become unavailable or behave abnormally because of accidents, incidents, hazards, maintenance, repairs, aging, or concurrent activities to protect workers or major equipment. For example, computer systems may become temporarily unavailable because of network congestion; some sensors of NPP reactor systems may become unreliable as the result of an electrical fault; operational system components or equipment may be disabled because of problems in related systems (such as other reactor units in multiunit NPPs).

Electrical faults may reset systems or components to an undesirable status.

4-11

The designed operational range of the system, structure, or component could be exceeded and functions needed to support the component or instrument operation may be inadequate.

Structures may have degraded environmental conditions or be inaccessible because of hazards or construction activities.

Automated systems could be intentionally turned off because of personnels well-intentioned but incorrect belief.

Personnel Context Personnel include all of the people who perform the required actions in an event. Personnel may work in various structures:

IndividualsEvery person has his or her own roles, responsibilities, and assigned tasks.

Team or crewA group of individuals work collaboratively for common goals.

OrganizationThis is a framework outlining authority and communication processes for individuals and teams. The framework usually includes policies, rules, and responsibilities for each individual in the organization.

Explicit consideration of personnel structures and team processes is important in analyzing human performance. This allows HRA analysts to systematically identify the performance challenges and opportunities for errors that arise when the event involves a complex organization distributed across multiple locations with complex communication and command and control structures. Communication, cooperation, and coordination across multiple individuals distributed in time and space are essential in emergency responses. The personnel structure, lines of communication, and chain of command play a critical role in successful performance.

Personnel context includes the conditions that challenge or facilitate personnel to perform the tasks. Personnel context specifies the conditions affecting individuals, teams, or organizations.

The context affects personnels task performance in detecting information, understanding the situation, making decisions, executing planned actions, and interteam coordination. Below are some considerations for the personnel context:

Availability of personnelConsider the amount and types of personnel available to respond to the event relative to the personnel needed. Personnel may become unavailable for reasons such as multiple simultaneous events, environmental effects, or duties unrelated to the event.

Operational limitations of personnelPersonnel may not perform work as expected for reasons such as physical limitations, not being prepared or trained for the type of events, or conformance to special safety or regulatory requirements.

Organizations may not have adequate infrastructure to support teamwork for reasons such as safety culture, authorization restrictions, conflict of interest or goals, or lines of communication.

Availability of personnel supportPersonnel may lack necessary support such as training, tools, procedures or protocols, and expertise for reasons such as hazards, surprise of the event, beyond-design-basis accidents, lack of experience using the supporting items, and need to share the limited supporting items.

4-12

Personnel not in their normal locationsFor example, in an NPP fire event, the shift manager and the shift technical advisor were in another building away from the main control room for shift turnover.

Operating team not in normal configuration: For example, one individual is temporarily performing dual responsibilities for a missing team member.

Personnel substitution: Temporary substitution of an individual familiar with the tasks by another individual who does not normally perform the tasks is likely to affect human performance.

Considerations for personnel context may also address safety culture. Safety culture is the attitude, beliefs, perceptions, and values that employees share about safety. Different organizations define various safety culture metrics. The NRC defines nuclear safety culture as the core values and behaviors resulting from a collective commitment by leaders and individuals to emphasize safety over competing goals to ensure protection of people and the environment.

The NRC defines safety culture traits such as leadership safety value and actions, problem identification and resolution, and effective safety communication. Event analyses and research indicate that the extent to which an organization prioritizes safety over competing goals (e.g., cost, production, schedule) primarily has indirect effects on human performance by affecting the state or condition of other PIFs. For example, in response to perceived financial pressure on the organization, leaders may decide to defer maintenance of equipment, reduce staffing, delay or cut training, choose not to purchase enough tools and field equipment, or otherwise limit the amount and condition of resources available to support human performance.

Task Context Task context includes special conditions for tasks that need to be performed and how these tasks are expected to be performed. The characterization of human-system interaction and conduct of operations specify how tasks are performed. Some aspects such as burden and pace of the tasks may be better understood from the perspective of operational experience.

The conditions affect the task requirements, task difficulty, and demands for resources. In identifying task context, these conditions should be evaluated against the five macrocognitive functions to determine if the situation challenges these functions.

Below are some general considerations for task context:

Use of computerized HSIs and supporting systems increase work for personnel.

Multiple simultaneous events may lead to multitasking, interruptions, and distraction.

Failure or unavailability of operational system components may make event progression unpredictable.

Unusual event evolution may reduce the time available for required important human actions.

Complex events often require personnel to perform tasks in distributed locations.

Personnel may need to perform additional tasks upon failures of automated systems.

Personnel may make nonrequired changes to system status or interfere with system automation with good intentions, yet the changes may lead to undesirable consequences.

4-13

The four categories of context are not intended to represent an exclusive classification system.

Rather, they are intended to guide the search for important context that can significantly affect human performance. HRA analysts may develop questions to probe the possible conditions that can lead to impacts on the macrocognitive functions.

4.2.4 Identification and Definition of Important Human Actions This part of Stage 1 is to identify important human actions as the analysis units of an HRA and define the actions at a high level. Important human actions are those required in the event progression to achieve the mission or goal of the event. PRA models represent some important human actions as HFEs, including pre-initiator, initiator, and post-initiator actions. The definition of important human actions includes success criteria of the action, consequence of the HFE, relevant procedural guidance, cues and indications, and available time (whether or not the action is time critical). The definition establishes the scope for further analysis of the important human action.

Identification of Important Human Actions Identification of important human actions involves searching for important human actions in baseline and deviation scenarios. The search process is outlined as follows:

Important human actions from the baseline scenario: This is the identification of the important human actions from the baseline scenario that affect mission-critical systems.

In a PRA model, these are the HFEs that are the top events in event trees or the basic events in fault trees.

New important human actions may be identified in deviation scenarios.

The search process identifies important human actions interacting with mission-critical systems as well as noncritical systems; manipulations of noncritical systems may impact mission-critical system functions and personnel performing key actions with mission-critical systems.

The search process also identifies errors of commission that impact mission-critical system functions.

Definition of Important Human Actions An identified important human action is defined at the level describing the human failure of the action and linking it to the affected systems. The definition should describe the success criteria of the important human action and the consequences of the failure. The definition may include the following:

success criteria that define the desired end states or outcomes of the systems with the success of the important human action

consequence of failing the action

beginning and ending points of the action

procedures available for the action

the cues and other indications for initiating the action

accident sequence-specific timing of cues and the time available for the action 4-14

4.2.5 Summary of Stage 1 Analysis The outcomes of Stage 1, Scenario Analysis, consist of event context, operational narrative, and important human actions.

Event context provides a holistic view of conditions that influence system and human performance.

Operational narrative, including the baseline scenario and deviation scenarios, provides the evolutionary perspective of the event progression.

Important human actions are the analysis units of an HRA.

These establish a holistic understanding of the event and constitute the foundation for performing an HRA. Scenario analysis provides a framework to systematically document the basic information about an event and enhances HRA analysts understanding of the scenario.

The information and understanding serve as the basis for the rest of the HRA process. The outcomes of scenario analysis also serve as a means to communicate with PRA/HRA analysis team members with different technical disciplines to ensure a cohesive understanding of the assumptions that are applied to the analysis.

4.3 Stage 2: Modeling of Important Human Actions 4.3.1 Overview of Modeling Important Human Actions Stage 2 is to model an important human action for reliability analysis based on the information identified in Stage 1. This stage includes three parts: identification and characterization of critical tasks in an important human action, identification of CFMs of the critical tasks, and representation of the important human action context with PIFs. The outcomes of these three parts generalize the information obtained in Stage 1, Scenario Analysis, and establish the basis for Stage 3, HEP Quantification.

Task analysis. A task is a set of related human activities to achieve a common goal. In IDHEAS-G, the term task refers to a clearly defined piece of an important human action, assigned to expected personnel and required to achieve the success criteria of the action. In short, an important human action can be divided into a set of discrete tasks. Different people or teams, at various locations, and in different time intervals may perform an important human action. Breaking an important human action into discrete tasks can facilitate the assessment of PIFs and HEP estimation. The objective of task analysis is to identify and characterize critical tasks in an important human action. HEP quantification of an important human action is performed on its critical tasks (i.e., the HEP of an important human action is the combined HEPs of all its critical tasks).

Identification of CFMs. HRA methods use human error failure modes as a taxonomy to describe the ways that important human actions may fail and to quantify the HEP of an action.

IDHEAS-G uses macrocognitive functions and their processors to model the cognitive process of performing human tasks. Therefore, the failures of the macrocognitive functions or their processors are used as CFMs to represent various types of task failures. The failure of a task can be represented with one or several CFMs. Identification of CFMs applicable to a task means identifying all the CFMs that can potentially occur for that task.

Assessment of PIFs. The scenario analysis in Stage 1 identifies the event context that challenges or facilitates human performance. The context is descriptive. HRA methods use a set of PIFs to generalize and represent the context. The IDHEAS-G PIF structure (details in 4-15

Chapter 3) consists of a comprehensive set of PIFs and their attributes. The objective of this part of the analysis is to assess the states of the PIFs relevant to the critical tasks. PIF states describe the impact of a PIF on HEPs. PIF states can be discrete or continuous variables.

The Stage 2 analysis begins with organizing the outcomes of Stage 1 as the input to Stage 2, including the following:

important human actions as the units of analysis

scenario boundary conditions and event context for assessing the states of PIFs

operational narrative for performing task analysis and identifying CFMs applicable to the critical tasks of important human actions

definition of the important human action for developing task diagrams and characterizing tasks With the information organized, Stage 2 starts with task analysis to identify and characterize critical tasks of the important human action being analyzed. Identification of applicable CFMs is then performed for every critical task. Assessment of PIF states should be first performed for the entire important human action. Yet, because PIF states may vary for different critical tasks of the important human action, the PIF states assessed for the whole action need to be verified and may change for every critical task. On the other hand, the assessment of PIF states may modify task analysis and CFM identification. If two critical tasks have identical PIF states, they can be merged into the same task. However, if the PIF states are different for different portions of a critical task, then the task should be further broken down. Figure 4-6 illustrates the iterative process of these activities.

Task Analysis Identification of Assessment of CFMs PIFs Figure 4-6 Iterative Process for Modeling of Important Human Actions 4.3.2 Task Analysis The objective of task analysis is to identify critical tasks in an important human action and characterize the tasks. The critical tasks are the ones that are essential to the success criteria of the important human action, and failure of any of the critical tasks will cause the failure of the important human action. Thus, each critical task represents an opportunity for failure of the important human action. Not all of the tasks in an important human action are essential to its success. For example, some required tasks are confirmatory, and incorrectly performing them would not necessarily lead to failure of the important human action. Personnel may also perform secondary tasks that do not necessarily relate to the success criteria of the important human action. However, those noncritical tasks may compete for resources with critical tasks, 4-16

and they may also interfere with the performance of critical tasks. Thus, they should be identified as a part of the characterization of the critical tasks.

Figure 4-7 illustrates how the critical tasks in an important human action relate to the various stages of IDHEAS-G analysis. A human event has one or multiple important human actions, and an important human action is divided into one or multiple critical tasks, which are the basic units of HEP quantification. A critical task consists of cognitive activities, which are achieved through macrocognitive functions; the failure of a critical task is represented with applicable CFMs.

Human Event IHA 1 IHA 2 IHA 3 Critical Critical Critical Critical Critical Task Task 1 Task 2 Task 3 Task Macrocognitive functions required for the task Figure 4-7 IDHEAS-G Task Structure 4.3.2.1 Identification and Graphic Representations of Critical Tasks In performing an HRA, it is important to capture all of the critical tasks along with their relations, cues for the tasks, and timing information. Graphic representations of the tasks and their relations help identify critical tasks and organize the outcomes of task analysis. Graphic representation of the task is developed for the purpose of communication, illustration, and documentation of the task analysis. A thorough task analysis should use the combination of three graphic representations to identify and represent the tasks: a task diagram, a teamwork diagram, and a timeline.

A task diagram represents the success paths of critical tasks required to achieve the important human action; the paths indicate that failure of a critical task can lead to the failure of the important human action. A task diagram also illustrates the orders and relations of the critical tasks to be performed.

For an important human action that involves collaborative teamwork among multiple entities, a teamwork diagram represents the task sequences of the teams and the required interteam interactions such as communication, coordination, command and control, distribution of decisionmaking, and authorization chains.

4-17

The timeline of an important human action represents the occurrence of cues, critical tasks, and important transitions of system states in a chronological order. It also includes necessary noncritical tasks along the timeline to assess their interference with critical tasks.

One issue in developing task diagrams is the level of breaking down an action into tasks. The outcome of task diagrams for the same action may represent various levels of task breakdown or diverse ways of grouping simple activities into a single task. Breaking an action into too many detailed tasks tends to hide the action context and results in the tedious work of quantifying HEPs for all the tasks. Because the critical tasks identified for an important human action are just one way to model the action, there are no universally applicable rules on the level at which an action should break down into tasks. After all, the purpose of representing an important human action with critical tasks is to facilitate PIF assessment and HEP estimation.

The NRC staff offers the following guidelines for developing task diagrams:

Start from the highest level of breaking down an action (i.e., use as few tasks as possible to represent the action.

Further break down the important human action or a high-level task only when the PIF states vary for different portions of the action or task.

Stop breaking down the tasks at the level where there are performance indications or empirical data available to inform HEPs. For example, expert judgment has been a prevalent way to estimate HEPs; if expert judgment is used, the important human action should be broken down to critical tasks at the level with which experts are familiar.

Thus, they are able to make proper judgment.

4.3.2.2 Characterization of Critical Tasks The objective of task characterization is to define the context of the critical tasks that can impact task performance. The characterization of a task determines the states of many PIFs relevant to the task. The characterization also includes identifying cognitive activities involved in the task. The cognitive activities determine the macrocognitive functions and processors required for the task. These are the basis for identifying CFMs applicable to the critical task.

Characterization of a critical task should include, but is not limited to, the characteristics listed in Table 4-3.

4-18

Table 4-3 Task Characterization in Task Analysis Task characteristics Description Task goal The expected outcome of the task with respect to the success criteria of the action.

Specific requirements Specifications on the task goal such as timing requirements, criteria of task outcomes, and how the task goal should be achieved (e.g., monitoring parameters at a certain time interval, using secondary cues when the primary cues are not available, cooling down the RCS within a certain rate).

Cues and supporting The cues to initiate the task and key information needed to perform the information task. A cue could be an alarm, an indication, a procedure instruction, or others (e.g., an onsite report). The supporting information is in addition to the cue required to perform the task.

Procedures Available procedures, guidance, or instructions designed for the task.

Personnel Types of workers needed for the task, minimum staffing required, special skillset required.

Task support Job aids, tools, and equipment needed.

Location Places where the task is performed, special environmental factors about the locations.

Cognitive activities Cognitive activities that are involved in the task and that place demands on their corresponding macrocognitive functions.

Concurrent tasks Concurrent tasks (critical or noncritical) that compete for personnels cognition and resources.

Teamwork Interteam collaborative activities required for the task and considerations requirements for communication facilities (e.g., equipment, tools, devices).

The information in the task characterization can be mapped to various PIF attributes, and thus, along with the action context, it determines the PIF states. For example, assessment of concurrent tasks determines the applicable attributes for the PIF Multitasking, Interruptions, and Distractions (see Table 3-16); specific requirements of a task can be mapped to some attributes of task complexity, such as the attribute complexity and uncertainties in task criteria.

4.3.2.3 Assessment of Cognitive Activities Involved in a Task Cognitive activities in a task are assessed to determine the macrocognitive functions and processors needed to achieve the task goal. Performing a critical task involves the successful performance of one or more specific cognitive and behavioral activities, such as collecting information for decisionmaking and authorizing the decision. In operational documents and domain expert interviews, tasks are generally described in terms of human behaviors with respect to systems. Such descriptions usually provide information about what cognitive activities are involved in a task. The IDHEAS-G cognition model (described in Chapter 2) provides a taxonomy of cognitive activities for each macrocognitive function. This taxonomy 4-19

can be used to assess cognitive activities involved in a task and subsequently identify the macrocognitive functions and processors needed. Table 4-4 summarizes the taxonomy.

Table 4-4 Taxonomy of Cognitive Activities Macrocognitive Types of cognitive activities function Detection

Detect cues (through carefully monitoring, searching, inspecting, or comparing, etc.).

Acquire information (checking, reading, communicating or chatting, computing, etc.).

Understanding

Maintain situational awareness.

Assess status based on indirect information.

Diagnose problems and resolve conflicts in information

Make predictions or form expectations for the upcoming situation development.

Decisionmaking

Make a GO/NO-GO decision for a prespecified action.

Select among multiple options or strategies.

Make changes or additions to a preexisting plan or strategy (e.g., changes of personnel, criteria, subgoals).

Develop a new strategy or plan.

Action Execution

Execute complex actions.

Execute simple actions.

Execute fine-motor actions.

Execute strenuous dexterous actions.

Teamwork (within-team

Communicate.

and between-team

Coordinate (including command and control).

interaction)

Cooperate.

In summary, the outcomes of task analysis consist of a list of critical tasks, the relations of the tasks, and task characterization including the cognitive activities involved in the tasks. In this way, task analysis provides the input for identifying CFMs and PIF states. With this information, HRA analysts can screen which of the CFMs are applicable for a critical task and assess the states of relevant PIFs.

4.3.3 Representation of Task Failure with Cognitive Failure Modes 4.3.3.1 Criteria for Cognitive Failure Modes in Human Reliability Analysis CFMs are the classifications of the various ways that a task may fail. A complete set of CFMs should adequately represent failure of any human task within the application scope of an HRA method. Ideally, CFMs in an HRA method should have the following characteristics:

Completeness: CFMs should adequately represent the ways in which tasks might fail.

Non-overlapping: The scope of individual CFMs should not overlap (i.e., the human failure represented by one CFM is not represented by other CFMs).

4-20

Specificity and sensitivity: The CFMs should be specific enough to differentiate failures caused by different contexts. That is, a CFM should specifically link to a limited set of PIF attributes and be sensitive to changes in the attributes.

Observability: To estimate the HEP of the CFMs using available data or evidence, CFMs should be behaviorally observable and related to data or evidence from existing human performance operating experience.

In addition, since IDHEAS-G is a general methodology, the CFMs in IDHEAS-G should be independent of the HRA application.

4.3.3.2 The Basic Set of Cognitive Failure Modes in IDHEAS-G The NRC staff used the cognitive basis structure of the cognition model (described in Chapter 2) to develop a basic set of CFMs. In the cognitive basis structure, any human task can be achieved through the macrocognitive functions, and each function is achieved through a set of processors. Thus, the failure of a task can be represented by the failure of macrocognitive functions required for the task or the failure of applicable processors. This classification scheme, failure of macrocognitive functions, results in five high-level CFMs, as shown in Table 4-5.

Table 4-5 Failure of Macrocognitive Functions as the High-Level CFMs Macrocognitive function Cognitive Failure Mode Detection Failure of detecting cues/information Understanding Failure of understanding/assessing situation Decisionmaking Failure of making decisions/planning actions Action execution Failure of executing planned actions Interteam coordination Failure of interteam coordination This set of high-level CFMs constitutes a complete representation of cognition failure of a task.

They are, theoretically, non-overlapping because each macrocognitive function is defined with its own scope. However, these CFMs are not specific enough. For example, two tasks that demand the same macrocognitive function may involve different sets of processors, which are affected by different sets of PIFs and may result in different HEPs. However, if the two tasks are modeled with the failure of the same macrocognitive function, then their HEPs would be the same. Thus, the failure of the processors is used as a classification scheme to develop a set of middle-level CFMs. These CFMs are more specific than the high-level CFMs. Table 4-6 shows the middle-level CFMs for failure of detection.

4-21

Table 4-6 Middle-Level CFMs Processors for Detection Middle-Level CFMs for Detection D1. Initiate detectionEstablish mental D1Fail to establish the correct mental model and criteria for information to be model or to initiate detection detected D2. Select, identify, and attend to sources of D2Fail to select, identify, or attend to information sources of information D3. Perceive, recognize, and classify D3Incorrectly perceive or classify information information D4. Verify and modify the perceived D4Fail to verify perceived information information D5. Retain, document/record, or D5Fail to retain, record, or communicate communicate the acquired information the acquired information Theoretically, the middle-level CFMs are more specific than the high-level CFMs. However, they do not necessarily warrant adequate observability for HEP estimation. The processors represent the internal cognitive process of achieving the macrocognitive functions. Most of those internal processes are not behaviorally observable, and they are not directly related to data or evidence of existing human performance measures. Thus, it is difficult to assess the effects of PIFs on these CFMs in order to estimate the HEPs.

To make the middle-level CFMs assessable, the NRC staff recommends developing application-specific, behaviorally observable CFMs by adapting the middle-level CFMs for specific HRA applications. There is no universal rule for developing detailed CFMs because they should be adapted for specific applications. The four criteria for CFMs (see Section 4.3.3.1) should be preserved in developing detailed CFMs.

The NRC staff has developed a reference set of detailed CFMs from the middle-level CFMs. A specific HRA application may either develop its own set of CFMs from the middle-level CFMs or adapt this reference set of detailed CFMs. Shown in Table 4-7 through Table 4-10, these detailed CFMs represent the behaviorally observable failures of the processors, along with the middle- and high-level CFMs for detection, understanding, decisionmaking, and action execution. The staff did not develop specific, detailed CFMs for failure of interteam coordination because middle-level CFMs for failure of interteam coordination are already behaviorally observable. Also, compared to other macrocognitive functions, the interteam coordination function is less studied and, thus, there is limited empirical data to inform any detailed failure modes. Therefore, the middle-level and detailed CFMs for failure of interteam coordination, presented in Table 4-11, are the same. The CFMs for failure of interteam coordination should be updated as more empirical data or evidence on specific modes of interteam coordination failure becomes available.

4-22

Table 4-7 Detection CFMs High-Level CFM: Failure of Detection Middle-Level Detailed CFMs for Detection CFMs Fail to initiate D1-1 Detection is not initiated (e.g., skip steps of procedures for detection detection, forget to check information, fail to realize the need to check information, no mental model for detection)

D1-2 Wrong mental model for detection (e.g., incorrect planning on when, how, or what to detect)

D1-3 Failure to prioritize information to be detected Fail to select, D2-1 Fail to access the source of information identify, or attend to D2-2 Attend to wrong source of information sources of information Fail to perceive, D3-1 Unable to perceive information recognize, or D3-2 Key alarm not perceived classify information D3-3 Key alarm incorrectly perceived D3-4 Fail to recognize that primary cue (other than alarms) is not available or misleading D3-5 Cues (other than alarms) not perceived D3-6 Cues (other than alarms) misperceived (e.g., information incorrectly perceived; failure to perceive weak signals; reading errors; incorrectly interpret, organize, or classify information)

D3-7 Fail to monitor status (e.g., information or parameters not monitored at proper frequency or for adequate period of time, failure to monitor all of the key parameters, and incorrectly perceiving the trend of a parameter)

Fail to D4-1 Fail to self-verify the perceived information against the detection verify the criteria perceived information D4-2 Fail to peer-check the perceived information Fail to D5-1 The detected information not retained or incorrectly retained communicate (e.g., wrong items marked, wrong recording, and wrong data entry) the acquired information D5-2 The detected information not communicated or miscommunicated 4-23

Table 4-8 Understanding CFMs High-Level CFM: Failure of Understanding Middle-Level Detailed CFMs for Understanding CFMs Fail to assess U1-1 Incomplete data selected (e.g., critical data dismissed, critical data or select data omitted)

U1-2 Incorrect or inappropriate data selected (e.g., failure to recognize the applicable data range or recognize that information is outdated)

Incorrect U2-1 No mental model exists for understanding the situation mental model U2-2 Incorrect mental model selected U2-3 Failure to adapt the mental model (e.g., failure to recognize and adapt mismatched procedures)

Incorrect U3-1 Incorrectly assess situation (e.g., situational awareness not integration of maintained, and incorrect prediction of the system evolution or data and upcoming events) mental model U3-2 Incorrectly diagnose problems (e.g., conflicts in data not resolved, underdiagnosis, failure to use guidance outside main procedure steps for diagnosis)

Fail to iterate U4-1 Premature termination of data collection (e.g., not seeking additional the data to reconcile gaps, discrepancies, or conflicts, or failing to revise understanding the outcomes based on new data, mental models, or viewpoints U4-2 Failure to generate coherent team understanding (e.g., assessment or diagnosis not verified or confirmed by the team, and lack of confirmation and verification of the results)

Fail to U5-1 Outcomes of understanding miscommunicated or inadequately communicate communicated the outcome 4-24

Table 4-9 Decisionmaking CFMs High-Level CFM: Failure of Decisionmaking Middle-Level Detailed CFMs for Decisionmaking CFMs Inappropriate DM1-1 Incorrect decision model or decisionmaking process (e.g., incorrect decision model about who, how, or when to make decision, the decision model or process does not support the decision goal DM1-2 Incorrect decision criteria Incorrect goals DM2-1 Incorrect goal selected or priorities DM2-2 Unable to prioritize multiple conflicting goals Data are DM3-1 Critical information not selected or only partially selected (e.g., bias, under- undersampling of information) represented DM3-2 Selected information not appropriate or not applicable to the situation DM3-3 Misinterpretation or misuse of selected information Incorrect DM4-1 Misinterpret procedure judgment or planning DM4-2 Choose inappropriate strategy or options DM4-3 Incorrect or inadequate planning or developing solutions (e.g., plan wrong or infeasible responses, plan the right response actions at wrong times, fail to plan configuration changes when needed, plan wrong or infeasible configuration changes)

DM4-4 Decide to interfere or override automatic or passive safety-critical systems that would lead to undesirable consequences Failure to DM5-1 Unable to simulate or evaluate the decisions effects (e.g., fail to simulate or assess negative impacts or unable to evaluate the pros and cons) evaluate the decision/ DM5-2 Incorrectly simulate or evaluate the decision (e.g., fail to evaluate strategy/plan the side effects or components, or fail to consider all key factors)

DM5-3 Incorrect dynamic decisionmaking Failure to DM6-1 Decision incorrectly communicated communicate or authorize DM6-2 Decision not authorized the decision DM6-3 Decision delayed in authorization 4-25

Table 4-10 Action Execution CFMs High-Level CFM: Failure of Action Execution Middle-Level Detailed CFMs for Action Execution CFMs Fail to assess E1-1 Action is not initiated action plan and criteria E1-2 Incorrect interpretation of the action plan (e.g., wrong equipment/tool preparation or coordination)

E1-3 Wrong action criteria E1-4 Delayed implementation of planned action E1-5 Incorrect addition of actions or action steps to manipulate safety systems outside action plans (e.g., error of commission)

Fail to develop E2-1 Fail to modify, adapt, or develop action scripts for a high-level action or modify action plan scripts E2-2 Incorrectly modify or develop action scripts for the action plan Fail to E3-1 Fail to coordinate the action implementation (e.g., fail to coordinate coordinate team members, errors in personnel allocation) action implementation E3-2 Fail to coordinate activities that must be performed in a sequential or integrated manner.

E3-3 Fail to check the entry conditions to initiate the action execution Fail to perform E4-1 Fail to follow procedures (e.g., skip steps in procedures) the planned action E4-2 Fail to execute simple action E4-3 Fail to execute complex action (e.g., execute a complex action with incorrect timing or sequence, execute actions that do not meet the entry conditions)

E4-3A Fail to execute control actions E4-3B Fail to execute long-lasting actions E4-4 Fail to execute physically demanding actions E4-5 Fail to execute fine-motor actions E4-6 Fail to check the status required for executing critical steps of a task Fail to verify or E5-1 Fail to adjust action by monitoring, measuring, and assessing adjust action outcomes E5-2 Fail to complete entire action scripts or procedures (e.g., omit steps after the action criteria are met)

E5-3 Fail to record, report or communicate action status or outcomes 4-26

Table 4-11 Interteam Coordination CFMs High-Level CFM: Failure of Interteam coordination T1 Fail to establish or adapt the interteam coordination infrastructure T2 Fail to manage information Middle-Level T3 Fail to maintain shared situational awareness CFMs T4 Inappropriately manage resources T5 Fail to plan or make interteam decisions or generate commands T6 Fail to implement decisions or commands T7 Fail to control the implementation 4.3.3.3 Identification of Cognitive Failure Modes Applicable to a Critical Task For each critical task of an important human action, the applicable CFMs are identified. A prerequisite for identification of CFMs applicable to a critical task is the characterization of the critical tasks in terms of the specific activities identified as essential for the task goal, since this information will be used to identify the relevant CFMs. The outcomes of task analysis, along with other outputs of scenario analysis, provide the structured context for the critical tasks of an important human action.

The rationale for identifying potentially relevant CFMs is based on task characterization. The identification is first performed at the high-level CFMs (i.e., the failure of macrocognitive functions). Only those functions involved in the cognitive activities of a critical task need be addressed. For example, if the critical task being evaluated does not involve action execution, then none of the action execution CFMs would apply. Then for each macrocognitive function, every middle-level or detailed CFM is examined with questions probing the relevance of the CFM to cognitive activities. If the answer to the probing question of a CFM is yes for the task being evaluated, the CFM applies to the task. Table 4-12 provides examples of probing questions.

Table 4-12 Example Probing Questions to Assess CFM Applicability Middle-Level or Detailed CFM Example Probing Question D3-2 Key alarm not perceived Does the critical task require responding to an alarm as the or incorrectly perceived primary cue for success?

U1 Fail to assess or select Does the success of the task require data collection to data assess system status?

DM4-1 Misinterpret Does success require making a decision based on procedures procedures (e.g., transfer to another procedure, or initiate action)?

DM4-2 Choose inappropriate Does the procedure allow a choice of strategies?

strategy E1-1 Fail to initiate action Does the task require the manipulation of plant systems?

4.3.3.4 Summary of Representing Task Failure with Cognitive Failure Modes Based on the IDHEAS-G cognition model, the NRC staff developed a basic set of CFMs to represent human failure. The set of the CFMs is structured in three hierarchical levels. The first 4-27

level is failure of macrocognitive functions, the second (middle) level is failure of the processors for every macrocognitive function, and followed by the third level in which each middle-level CFM is divided into several detailed, behaviorally observable failure modes. This hierarchical structure allows flexibility in applying the CFMs to specific HRA uses. This flexibility allows adaptation for different levels of detail in the empirical data available for HEP estimation. A specific HRA application may choose the level of CFMs based on the purpose of the application and the empirical data available, or it may choose to use a mixture of different levels for different macrocognitive functions. For example, an HRA application modeling a large amount of detailed manual actions may use the detailed CFMs for failure of action execution and the high-level CFMs for failure of other macrocognitive functions. A specific HRA application may include only a subset of these CFMs. In short, failure of any task can be represented with one or several applicable CFMs.

4.3.4 Representation of Important Human Action Context with Performance-Influencing Factors 4.3.4.1 Performance-Influencing Factor Structure The PIF structure (described in Chapter 3) in the IDHEAS-G cognition model provides a comprehensive set of PIFs to represent the context of important human actions. The influence of a PIF on task performance is described with its attributes, representing the ways that the PIF challenges cognitive mechanisms underlying macrocognitive functions and processors. Table 4-13 recaptures all the PIFs described in Chapter 3.

Table 4-13 IDHEAS-G PIFs

Environment-and System-related Personnel-related Task-related PIFs situation-related PIFs PIFs PIFs o Accessibility/ o System and o Staffing o Information habitability of I&C o Procedures, availability and workplace, transparency guidance, and reliability including travel to personnel instructions o Scenario paths o HSI o Training familiarity o Workplace o Equipment and o Team and o Multitasking, visibility tools organization interruptions, and o Noise in factors distractions workplace and o Work processes o Task complexity communication o Mental fatigue pathways o Time pressure o Cold/heat/ and stress humidity o Physical o Resistance to demands physical movement 4-28

4.3.4.2 Mapping Event Context and Task Characterization to Performance-Influencing Factor States A base state is defined for every PIF in IDHEAS-G. The base state means that the PIF has no observable impact on HEPs. The other PIF states are poor states that increase HEPs. Every attribute of a PIF represents different levels of poor PIF states because some attributes increase HEPs more than others. The purpose of the assessment of PIF states is to determine applicable attributes based on the context identified in scenario analysis and task characterization. When the context challenges task performance, it maps to applicable PIF attributes. When the context facilitates task performance, it moves the corresponding PIFs to their base state.

4.3.4.3 Modeling Performance-Influencing Factor States To assess the contribution of PIFs to HEPs, the PIFs need to be quantified. Existing HRA methods typically use binary or several discrete levels to model PIF states. Examples of labels for different levels include the following:

binary states:

o present versus not present o low versus high o good versus poor o nominal versus poor

multiple discrete states:

o low, medium, high o good, nominal, poor Contrary to most HRA methods, empirical human error data in the literature and operational databases assess specific PIF attributes rather than using the entire PIF as a single variable.

For example, when assessing the PIF task complexity, simply denoting it as low, medium, or high does not link to its impact on HEPs. Some attributes themselves are continuous or discrete variables. For example, task interruption is an attribute for the PIF multitasking, interruption, and distraction, while cognitive studies show that human error rates continuously increase with interruption time. Thus, to quantify HEPs, defining PIF states should be based on available empirical data and knowledge. Consequently, different PIFs may be defined differently (binary states or multiple discrete states).

Modeling PIFs with binary states or multiple discrete states requires grouping the attributes into arbitrarily defined states based on their effects on HEPs. In modeling PIFs, the analyst needs to clearly define the meaning of each state. Because the effects of PIF attributes on HEPs generally vary continuously, the poor state can represent any place between no impact and maximal impact. As a result, the HEP for the poor states can vary greatly. Thus, modeling of PIF states should be consistent with the following guidance: When modeling PIFs with binary states or a scale (i.e., consisting of multiple discrete states), their definitions must be specified, used consistently in HEP estimation, and documented as a contingency for explaining and using the estimated HEPs.

A good practice of implementing the above guidance is to provide scales to represent the full span of a PIFs possible states. Providing examples for each discrete state would help the scale system to be applied consistently. Like the detailed CFMs, the scales and discrete states of the PIFs should be specified based on human cognition research literature. The PIF scales 4-29

would not solve the problem of translating the scenario context into an HEP. However, they would improve consistency in assessing the PIFs impact on the HEPs, which should also support more consistent derivation of the HEPs, regardless of how that quantification process is accomplished.

4.3.5 Summary of Stage 2 Analysis In summary, Stage 2 analysis establishes a model for analyzing the reliability of an important human action. The model consists of critical tasks and their characterization, applicable CFMs of the tasks, and PIF states associated with the tasks. The critical tasks identified indicate what may go wrong with human performance of the action, the applicable CFMs of the critical tasks represent how the important human action can go wrong, and the PIF states explain what can lead to the failure of the important human action. Ultimately, the model serves as the basis of HEP quantification.

4.4 Stage 3: Estimation of the Human Error Probability of an Important Human Action 4.4.1 Overview of Human Error Probability Estimation in IDHEAS-G Estimation of the HEP of an important human action is based on the modeling of the important human action in Stage 2. Figure 4-8 illustrates the IDHEAS-G process for estimating HEPs.

IDHEAS-G models the HEP of an important human action in two parts: the error probability attributed to uncertainties in time available and time required for the action ( ) and the error probability attributed to the CFMs of all the critical tasks of the important human action ( ). The estimation of the HEP of an important human action is the probabilistic sum of and :

= 1 (1 )(1 ) (4.1) where is the HEP of the important human action being analyzed, and and have already been defined. Note that can also be viewed as the probability that the time required to perform an action exceeds the time available for that action, as determined by the success criteria.

The cognitive failure part of the overall HEP ( ) is the probabilistic sum of the error probabilities of every critical task. It is estimated as follows:

= 1 1 = 1 1 1 1 2 1 (4.2)

=1 where is the total number of critical tasks and is the error probability of the th critical task.

The error probability of every critical task ( ) is the probabilistic sum of the error probabilities of all its applicable CFMs and it is estimated as follows:

= 1 1 = 1 1 1 1 2 1 (4.3)

=1 where is the total number of CFMs applicable to the critical task, and is the error probability of the th CFM applicable to the critical task. The error probability of a CFM applicable to the critical task is a function of the PIF states associated with the critical task.

4-30

= (1, 2, 3, ) (4.4)

Finally, Figure 4-8 shows that different PIFs (there are 20 PIFs in IDHEAS-G; see Section 3.2.1) can be associated with the error probabilities of the CFMs applicable to the critical task.

Time available Time required CFMs PIF states HEP of = (, , )

CFM 1 an IHA Critical

= (, , )

task 1 CFM 2 Critical = (, , )

task 2 CFM 3 Critical = (, , )

CFM 6 task 3 Figure 4-8 Overview of Process for HEP Estimation in IDHEAS-G 4.4.2 Estimation of Time Required for the Important Human Action For time-critical actions, can be calculated using the convolution method described below.

In the convolution method, denotes the time available for an important human action, which is the time from the onset of the cues indicating that the action is needed to the time beyond which the action is no longer useful in mitigating the event consequence, and denotes the time required for personnel to accomplish the action. The time required consists of the time to recognize the needed action, diagnose the problems, make the decision or plan to perform the action, and execute the action.

To calculate , is represented by its cumulative distribution function (), and is represented by its probability density function (). HRA analysts need to estimate the distribution (central tendency and range) of time required and time available. is the convolution of the two distributions, that is

= ( ) = 1 () () (4.5) 0 Chapter 5 presents details on performing time uncertainty analysis to estimate .

4-31

4.4.3 Estimation of the Error Probability of a Cognitive Failure Mode 4.4.3.1 Approaches for Estimating Human Error Probability HEP can be interpreted as the number of errors in performing an important human action divided by the number of times that the action is performed. The error probability of a CFM,

, is interpreted as the number of times the failure mode occurs divided by the number of times that actions or tasks having the CFM are performed. in IDHEAS-G can be estimated in one, or a combination, of the following three ways:

Data-based computation. When adequate human error data are available in the form of number of errors and number of times the task involving the CFM is performed for the given set of PIF states, for the set of PIF states can be computed from the data. The IDHEAS-G framework can generalize and integrate various sources of human error data. Chapter 6 describes using human error data to inform HEP estimation.

Expert judgment. When available numerical data are sparse, expert judgment is used to estimate the distribution of HEPs. This approach relies on the knowledge of the experts who arrive at best estimates of the distribution of the probability of a parameter or a basic event.

This approach is typically used when detailed analyses or evidence concerning the event are very limited. Ideally, this approach provides a mathematical probability distribution that represents the experts best available knowledge about the probability of the parameter or basic event. The estimated HEPs are also referred to as subjective probabilities because they may be informed by, but are not derived from, objective data. The IDHEAS At-Power Application [22] used expert judgment to estimate the HEPs of the failure modes, using the NRCs guidance document for eliciting expert judgment [52], [53]. APPENDIX J summarizes the guidance. Note that obtaining formal expert judgment of HEPs is very resource demanding.

HEP quantification model. In reality, the available human error data are far from adequate to derive HEPs for any combination of PIF states. Thus, many HRA methods rely on a quantification model to calculate HEPs from a set of ad hoc parameters about base HEPs and PIF effects. Employing a quantification model is a tradeoff between the good enough estimation and resource availability. Based on the extensive study of cognitive literature and operational databases on human errors, the NRC staff developed the IDHEAS-G quantification model. Obtaining the parameters in the quantification model requires data-based computation and expert judgment. Once all the parameters in the model are estimated, the quantification model can be used to calculate HEPs for IDHEAS-G CFMs (i.e., ) at any given combination of PIF states. This is described in the next section.

4.4.3.2 IDHEAS-G Human Error Probability Quantification Model The quantification model is based on two assumptions that are derived from cognitive experimental literature.

Assumption 1: Base PIFs and Base HEPs The assumption that the PIFs information availability and reliability, task complexity, and scenario familiarity are the base PIFs is based on the signal detection theory [54] and an extensive literature study that found that the base PIFs can significantly influence the HEPs.

These two bases are further explained below.

4-32

Base PIFs - Signal Detection Theory Nearly all information processing activities underlying the macrocognitve functions take place in the presence of some uncertainty. Thus, brain information processing is to make a correct response among alternatives, that is, detecting the true signal out of the noise. For example, detecting a visual stimulus is to decide whether the stimulus is present or not present. Signal detection theory [54] provides a notation for analyzing human responses in the presence of uncertainty. The theory states that three main components determine human responses as the outcome of brain information processing: (1) the nature of the information, (2) the sensitivity of the brain acquiring and processing the information, and (3) the criterion for making the response.

The probability of the brain making correct responses is a function of information processing reliability. If the information or the intensity of the signal is weak, the information processing is less reliable in discriminating between alternative responses. The PIF information availability and reliability describes different aspects of information strength.

The sensitivity of the brain acquiring and processing the information is another component that determines human responses. The brain has cognitive capacity limits in acquiring and processing information. When a task demands information processing that approaches to or exceeds one or more capacity limits, the brain becomes less sensitive to the information and the human responses become less reliable. The PIF task complexity describes various aspects of task demands on cognitive resources.

Finally, the brain uses a criterion or a set of criteria to discriminate between the correct response and alternatives. The persons mental model of a situation or scenario is developed through learning or training. When a scenario does not match a persons mental model, the individual will be quite uncertain as to what should be the adequate response and will make the response based on judgment. The PIF scenario familiarity describes how well the criteria to make a response are established and how they match the scenario.

The three base PIFs model these three components of information processing. There are many factors modifying these components. For example, using procedures helps a person to have the criteria to make the correct response, while a poorly designed procedure can cause confusion in the criteria, and mental fatigue can make a person less sensitive in acquiring and processing information.

Base PIFs - Influence on HEPs Some PIFs may affect HEPs significantly more than other factors do. Through an extensive literature study, the NRC staff found that three PIFs, information availability and reliability, task complexity, and scenario familiarity, can result in a HEP that varies from a minimal value to 1

[55]. The NRC staff refers to these three PIFs as the base PIFs. The HEPs at various states of these base PIFs are referred to as base HEPs, which can vary from 0 to 1. Moreover, the cognitive literature suggests that the effect of the base PIFs on HEPs generally follows a logarithmic function, as shown with the blue curve in Figure 4-9. The horizontal axis represents the measure of a base PIF state, varying from no impact on the left of the axis to high impact on the right of the axis. For example, if the base PIF is information availability and reliability, the leftmost part of the axis represents 100-percent complete and accurate information, the middle part of the axis may be for 50-percent information available, and the rightmost part of the axis represents no information or wrong information. The vertical axis represents the HEP resulting from the PIF state.

4-33

In a normal work environment, such as a licensed crew of NPP operators performing EOPs in main control rooms without complications, the three base PIFs would fall in the leftmost part of the blue curve in Figure 4-9 and result in very low HEPs. In extreme operating conditions such as those in beyond-design-basis accidents, information may not be complete or reliable, personnel are unfamiliar with the scenarios, and the tasks can be very complex. Any of these base PIFs or the combination of them would result in very high base HEP values that may approach to 1.

The rest of IDHEAS-G PIFs are referred to as modification PIFs. The data in the literature show that they typically modify base HEP values with a weight factor. Figure 4-9 illustrates the impacts of such PIFs: the modification PIFs modify the base HEPs by moving the blue curve up by a factor, as illustrated with the orange and green curves. Modification PIFs vary the base HEPs according to the sum of the weights of the applicable PIF attributes. The weight sum can vary from 1 (no impact) to one or two orders of magnitude.

PIF2 PIF1 1

E-1 HEP E-2 E-3 E-4 States of Base PIFs Figure 4-9 Illustration of Modeling the Effects of PIFs on HEPs Assumption 2: Linear Combination of PIF Effects At present, data are not adequate to allow calculation of the HEPs of all CFMs for any given combination of PIF states, nor has cognitive research clearly explained the mathematical relationship between PIFs and HEPs. The NRC staff performed a limited metadata analysis on experimental studies in which human error rates (i.e., the percentage of human errors) were measured with several PIFs varying independently and jointly. From the metadata analysis, the staff found that the human error data fit well to the simplest linear combination of individual PIF effects, that is, (1 + 2 + 3 + ), where is the ratio of error rates when a PIF varies from its base state to a poor state (see Equation (4.6)); in other words, is the PIF impact weight.

APPENDIX D documents the analysis for this assumption.

4-34

With the above assumptions, the IDHEAS-G quantification model is described as follows:

(1) Modeling PIF states. Every PIF has a base state and multiple poor states. The base state of a PIF has no observable or negligible impact on HEPs. PIF poor states can be defined by their attributes. For instance, every PIF attribute can be one PIF state.

Alternatively, PIF states can be simplified into several discrete states by grouping the attributes according to their impacts on HEPs. For example, the PIF attributes can be grouped into three PIF states (low, moderate, and high impact).

(2) Modeling the impact of PIF states on HEPs. The impact of a PIF state is modeled as follows:

= (4.6) where is the human error rate at the given PIF state and is the human error rate at the base state of the PIF. The NRC staff generalized human error data in a database referred to as IDHEAS-DATA for many IDHEAS-G PIF attributes [55], [56].

The generalized data can be used as an initial estimation of PIF impacts.

(3) Estimating base HEPs for every CFM. The quantification model requires base HEPs for the poor states of the three base PIFs (information availability and reliability, scenario familiarity, and task complexity). The human error data generalized in IDHEAS-DATA are used to estimate the base HEPs for most attributes of these PIFs.

(4) Calculating the HEP of a CFM for a given set of PIF states. The following equation is used to calculate the HEP of a CFM for any given set of PIF states, provided that all the PIF impact weights and base HEPs are obtained:

1

= 1 + ( 1)

=1 (4.7) 1 + (1 1) + (2 1) + + ( 1)

=

where is the base HEP of a CFM for the given states of the three base PIFs; is the PIF impact weight for the given state of modification PIFs, which is calculated using Equation (4.6); is a factor that accounts for the interaction between PIFs, and it is set to 1 for linear combination of PIF impacts unless there are data suggesting otherwise; and is a factor that accounts for the potential recovery from failure of a task, and it is set to 1 unless there are empirical data suggesting otherwise.

Note that the current version of this report does not provide the numeric values of the base HEPs and PIF weights. Although the NRC staff generalized much human error data to obtain the parameters in Equation (4.7), the staff performed only limited integration of the data to derive the parameters. Therefore, some parameters are not yet based on data and, consequently, need expert judgment. More importantly, given the current state of knowledge, those parameters should not be fixed values. Rather, they should be updated periodically as more human error data become available.

4-35

4.4.3.3 Crediting Recovery in the IDHEAS-G HEP Model In PRA, recovery actions focus on the restoration of a lost function that resulted from an equipment failure and, on an as-needed basis, they provide a more realistic evaluation of significant accident sequences. From the perspective of IDHEAS-G, there are two approaches to model recovery actions (or error recovery). The first approach is to define a separate important human action (or HFE) to model the recovery of a preceding important human action.

The modeling of the separate HFE should be consistent with the high-level requirement HR-H of the ASME/ANS PRA standard [44]. The second approach models a recovery action as a parallel task to a critical task within the important human action. In PRA terms, the second approach models the recovery of an HFE for which Equation (4.7) allows HRA analysts to apply a numerical recovery factor to the overall probability of each CFM. This is a simplistic treatment in the formula. In reality, the recovery factor is a function of the states (and corresponding weights) of applicable PIFs. For example, during a particular scenario, specific PIFs may have different influences on the feasibility or reliability of potential recovery from a cognitive error. Therefore, applying a single numerical recovery factor to the composite effects from all PIFs is over-simplistic and not appropriate.

If HRA analysts choose to credit recovery by assigning a numeric recovery factor to the HEP of a CFM, they should focus on the fundamental process for evaluating human performance and estimate based on the full context of the human action. Specifically, crediting recovery should first assess its feasibility for which the following criteria should be used:

(1) A recovery path exists. It should be demonstrated that the event progression allows personnel to go back to the failure point to correctly perform the failed critical task.

Some critical tasks may be irreversible and thus cannot be credited for recovery.

(2) There are cues or indications available to personnel for them to recognize the failure and need for recovery.

(3) There are sufficient staff resources responsible for monitoring plant status and detecting the cues of the failure.

(4) The time of the cue or the time taken to reach a procedural step that indicates the need for recovery is early enough to allow adequate time for recovery.

Recovery is feasible if all the criteria are met. If a critical task is recoverable, IDHEAS-G allows analysts to assign a recovery factor specific to each CFM of the critical task because the potential for recovery is dependent on the failure mode. For example, the error correction opportunities of manipulation tasks will primarily arise from a monitoring activity that is capable of detecting that the plant is not responding as would be expected if the intended action had been completed correctly. These opportunities focus on the crews assessment of the plant feedback.

Then, the recovery factor, , in HEP calculation varies from 1 to any positive number, with 1 being no potential for recovery. IDHEAS-G does not provide reference values of the recovery factor mainly because recovery potential is situation specific. The potential for recovery can be quite different for well-practiced procedural tasks performed in a control room than for rarely performed tasks outside the control room. Below are some recovery mechanisms that can influence recovery potential:

Procedural design - subsequent procedure steps require operators to check or verify the correct performance of important earlier steps.

4-36

Training, work process, and conduct of operations (e.g., plant status check performed for shift turnover).

Unexpected instrument responses to an action.

New alarms that provide cues to indicate potential errors.

Multiple, diverse cues for recognition of the need for recovery.

Finally, analysts should consider the dependency between the error made and recovery. If the recovery relies on the same context as that for the early failure of the critical task, then the recovery potential is reduced because of the dependence. In reality, there are no truly independent opportunities to correct the errors. To actually credit recovery and especially the recovery in multiple CFMs and critical tasks, analysts should carefully review the timeline of the specific recovery paths and identify opportunities for recovery that are sufficiently independent.

4.4.4 Documentation and Communication of the Assumptions Made for Estimating Human Error Probabilities or Parameters in a Quantification Model Stage 3 recommends three approaches to HEP estimation, computation of HEPs from data, expert judgment, and calculation of HEPs from the quantification model. Each approach has its limitations, uncertainties, and assumptions. These should be documented and communicated along with the estimated HEPs of the important human actions of an event. Communicating the assumptions made for the HEPs ensures that HRA analysts properly use the estimated HEPs in HRA applications. It also helps to resolve analyst-to-analyst and method-to-method HEP variation.

4.5 Stage 4: Performing Integrative Analysis Stage 4 is the integrative analysis of the entire event that may include multiple scenarios and important human actions. The analysis includes treating dependency between important human actions and documenting uncertainties in the event analysis. For uncertainty analysis, the staff adapted and synchronized the guidelines from existing HRA methods and regulatory guidance documents. APPENDIX L presents this synchronized guidance. This section only describes analyzing dependency using the IDHEAS-G dependency model.

The state-of-practice approaches to modeling dependency have limitations when attempting to identify and quantify dependent HFEs that result from failures of higher-level cognitive and collaborative processes. APPENDIX K of this report presents a summary of modeling dependency between HFEs in PRA models. The community of HRA practitioners has stressed the need for a new methodology of modeling dependency that is better informed by cognitive and behavioral science. To address this need, the NRC staff developed a new dependency model based on the IDHEAS-G framework. The detailed description of the IDHEAS-G dependency model can be found in APPENDIX K. This section describes the process of performing dependency analysis using the IDHEAS-G dependency model.

The IDHEAS-G dependency model evaluates dependency at the cognitive process level and its effect on the dependent HFE. The effect of dependency is modeled in a manner consistent with how individual HFEs are modeled in IDHEAS-G. The IDHEAS-G dependency model consists of three parts (or steps): identification of dependency context, modeling of dependency context, and calculation of the dependent HEP.

4-37

Step 1 - Identification of dependency context Step 1 involves identifying how the occurrence of an HFE changes the context of subsequent HFEs. The NRC staff defined three types of dependency to facilitate the identification of dependency context and these are: consequential dependency, resource-sharing dependency, and cognitive dependency.

An analyst would identify a consequential dependency when the outcome of a preceding HFE directly affects the performance of a subsequent HFE. Resource-sharing dependency occurs when tasks in an HFE share the same resources with a subsequent HFE. The shared resources may be equipment (e.g., using the same water sources for multiple pumps), or personnel (e.g., there is limited personnel to perform multiple tasks). Cognitive dependency refers to the dependency in the cognitive flow of multiple HFEs. The cognitive flow is composed of the five macrocognitive functions and to successfully accomplish a task, all macrocognitive functions need to be performed successfully. A CFM in a preceding HFE may affect a macrocognitive function of a subsequent HFE.

The main take away from the identification of the dependency context using the three types of dependency is to determine which elements of the subsequent HFEs are affected by the dependency. The affected elements of subsequent HFEs may include HFE feasibility, HFE definition, critical tasks to be performed, applicable CFMs, time available, and applicable PIFs.

Step 2 - Modeling of dependency context Step 2 involves determining how the elements of the subsequent HFEs are affected by the identified dependency context. To model dependency context of a subsequent HFE (HFE2) caused by occurrence of a preceding HFE (HFE1), analysts may systematically examine the changes in HFE2 context. For example,

Are there changes to the HFE definition (e.g., beginning or ending states, personnel, location, etc.)?

Does the occurrence of HFE1 make HFE2 infeasible?

Does the occurrence of HFE1 change the time availability for HFE2?

Are the critical tasks of HFE2 different?

Are there new CFMs to the critical tasks?

Are there changes in PIF attributes applicable to the CFMs?

If the answers to all the questions above are no, then HFE2 is deemed to be independent of HFE1. If the answer to any of the questions is yes, then HFE2 is dependent of HFE1. The changes are then documented for HEP adjustment.

Step 3 - Calculation of the Dependent HEP Finally, in Step 3, the probability of the subsequent/dependent HFE is calculated based on the changes to the dependency context (Step 2) and by applying the same method of estimating the probability of individual HFEs as discussed in Section 4.4.

4.6 Summary of the IDHEAS-G Human Reliability Analysis Process Figure 4-10 illustrates the flow of the IDHEAS-G HRA process and the elements of analysis in the four stages. While the analysis progresses with increasing level of detail from Stage 1 to 4-38

Stage 3, the process maintains the relations between the elements of different stages. Stage 4 goes back to analyze and document the dependencies between high-level important human actions and the uncertainties in the entire analysis. Table 4-14 summarizes the activities to be performed and the outcomes of each stage.

Event Stage 1: Scenario analysis Event scenarios

Develop operational narrative scenarios

Identify event context

Identify important human IHAs IHA 1 IHA 2 IHA 3 actions Critical Critical task 1 Critical task 2 Critical task 3 Stage 2: Modeling of tasks important human actions

Identify and analyze critical Macro-tasks

Identify applicable CFMs cognitive Action Interteam functions Detection Understanding Decisionmaking execution coordination

Assess PIFs Uncertainty and analysis CFMs CFMs CFMs CFMs CFMs CFMs CFM1 PIF1 Stage 3 - HEP quantification HEPs due to CFMs

Estimation of CFM2 PIF2

Estimation of HEP due to time HEP HEP CFM3 PIF3 uncertainty Stage 4 - Integrative analysis Dependency CFM4 PIF4

Document uncertainties

Assess dependencies adjustment Figure 4-10 Overview of the IDHEAS-G Process 4-39

Table 4-14 Summary of IDHEAS-G Stage Activities Outputs Development of

Event contextEnvironment and situation, system, operational personnel, and task context narrative

Operational narrative o Initial conditions, initiating events, boundary Identification of conditions, and consequences of interest Stage event context o Description of scenario: Event progression described 1

Identification and in timeline and narrative stories, including the baseline definition of event sequence and deviation event sequences important human

Important human actions as the basic unit of an HRA and actions the definitions of the actions, including success criteria, beginning and end points, cues, and key information, etc.

Task analysis

Task diagram representing the expected personnel tasks and their relationships Identification of

Timeline representing when cues and important applicable CFMs of information are expected to become available and timing a task of the tasks Stage Representation of

Teamwork diagram representing required collaborative 2 action context with activities between teams PIF states

Task characterization such as task goal and requirements, cognitive activities involved, tools needed, etc.

Macrocognitive functions required for achieving a critical task and applicable CFMs for every critical task

States of PIFs relevant to the task Time uncertainty

Uncertainties in time available and time required to analysis perform an important human action Stage * , the HEP attributed to time uncertainties 3 Estimation of HEPs

, the HEPs attributed to CFMs of the CFMs of

Limitations, uncertainties, and assumptions made in HEP every critical task estimation Dependency

Types of dependency between the important human analysis actions

Adjustment of the HEPs of the important human actions Stage Uncertainty based on the assessment of dependency 4 analysis and

Identification and documentation of the three types of documentation uncertainty in the analysis (model, parameter, and incompleteness uncertainty)

The IDHEAS-G HRA process described in this chapter is generalized into four stages with Stage 1 and Stage 2 focusing on qualitative analysis, Stage 3 on HEP quantification, and Stage 4 for integrative analysis of all the important human actions in the event being analyzed. The methodology emphasizes the importance of the thorough qualitative analysis in HRA and provides detailed qualitative guidance. Every step of the analysis yields insights about analyzed scenarios, which provides information about what can be fixed to improve human reliability.

This is shown in the Outputs column of Table 4-14. The implication is that HRA is not only just calculating the HEP but also understanding the conditions affecting human reliability.

4-40

5 TIME UNCERTAINTY ANALYSIS Cognitive studies show that human errors increase in time-constrained conditions (i.e., when the time available for task performance is less than the time required to complete the tasks). It is important to allocate adequate time for personnel to complete time-critical important human actions. However, real-world events have inherent uncertainties in both the time available for performing an action and the time that personnel need to complete the action. For example, the symptoms of a chemical release may take an uncertain amount of time to become significant enough to be noticed, while the time required for workers to flee from the building would also vary in different scenarios (e.g., it is uncertain how long it takes to go through unfamiliar corridors and stairs). This chapter introduces the IDHEAS-G time uncertainty model to account for HEPs attributable to time uncertainties. The model also includes guidance for estimating uncertainties in time available and time required.

5.1 Time Uncertainty Model IDHEAS-G quantifies the HEP of an HFE in two parts: the error probability attributed to time uncertainty and the error probability attributed to failure of the macrocognitive functions. The HEP equation is the following:

= 1 (1 )(1 ) (5.1)

The terms in Equation (5.1) are defined as follows:

is the error probability resulting from time uncertainties in the time available and time required to perform an action.

is the error probability resulting from failures of macrocognitive functions for all the critical tasks, assuming that the time for performing the tasks is sufficient. Sections 4.4.1 and 4.4.3 and APPENDIX J discuss the estimation of .

accounts for human errors caused by insufficient time available to perform the action at a normal work pace. When there is not enough time to perform an action, personnel either do not complete the action or they rush the action; either one increases the likelihood of errors.

does not account for human errors caused by time pressure, which means that personnel have adequate time, but they are under time pressure so they may try to complete the action as fast as possible. Time pressure is treated as a PIF and contributes to . IDHEAS-G treats and independently.

When personnel have adequate time to perform tasks, the likelihood of human errors is not significantly affected by having more time available, except that more time may yield more opportunities for recovering from human errors. If the time available for an action is only somewhat longer than the time required, then the possibility arises that some individuals might fail to complete the actions. In contrast, even if the time available is less than the time required for a task, some individuals may still complete the task correctly. In real-world events, time required cannot be represented by a single number because it is associated with many sources of uncertainty (e.g., the time one person needs to perform a task may be different for another person under identical conditions because of individual differences, and the time one person needs can vary in different trials). Thus, time required by personnel to perform the actions in an HFE should be represented with a probability distribution function to capture the uncertainties.

Time required represents the time taken for the actions in the HFE to be completed, including information detection, diagnosis, decisionmaking, executing the action, and interteam 5-1

coordination if multiple teams or distributed individuals are involved. Because individuals have their own pace for performing the actions in the HFE, time required is expected to be a distribution for many individuals performing the same actions in the same scenario. This is consistent with the general HRA assumption that actions within an HFE are performed by a nominal crew.

Similarly, because of various uncertainties associated with environment and situation, systems, personnel, and tasks, the time available for the action to be completed (i.e., achievement of the end state of an HFE) should also be described with a probability distribution function. For example, a medium loss-of-coolant accident (LOCA) in an NPP is a class of events covering a range of pipe break sizes (e.g., from 5.08 centimeters (cm) (2 inches) to 15.24 cm (6 inches) in diameter). The simulation for a medium LOCA scenario uses only a representative break size (e.g., 10.16 cm (4 inches) in diameter to represent a range from 5.08 to 15.24 cm (2 to 6 inches)). The system time window for recovering the system to desired states can vary greatly for specific applications; the time window for recovering from a LOCA for a 5.08-cm (2-inch),

10.16-cm (4-inch), and 15.24-cm (6-inch) pipe break could be significantly different.

denotes the time required to perform an important human action, and is the time available for personnel to complete the action. The basic assumption in calculating is that personnel fail the important human action if is greater than . Thus, is the convolution of the probability density functions of and , that is [57]

= ( > ) = ( > = ) = ( > l = ) ( = )

(5.2)

= 1 () () = () ()

0 0 In Equation (5.2), () is the cumulative distribution function of , and it is equal to the probability that is less than or equal to a value of time (i.e., ( )); () is the probability density function of ; () is the cumulative distribution function of , and it is equal to the probability that is less than or equal to a value of time (i.e., ( )); and

() is the probability density function of . The relationship between () and () is given by the following:

() = ()

0 and

()

() =

The relationship between () and () is similar to the two equations provided above.

Figure 5-1 illustrates that corresponds to the shaded area in the convolution of

() (). In the illustration, although the mean value of is significantly greater than that of , there is still an error probability as indicated in the shaded area because of the overlapping tails of the two probability density functions.

5-2

Note: The area in red corresponds to the value of .

sum over all possible values >

Figure 5-1 Illustration of Pt as the Convolution of the Probabilistic Distribution Functions of Time Available and Time Required The time required for personnel to perform the action assumes a normal work pace. It includes the time to detect the cue, understand the situation including diagnosing problems, make decisions or response plans, and execute the planned action (including the time to obtain the needed equipment, travel to the workplace, and perform the action steps). The NRC staff used the cognition model described in Chapters 2 and 3 to develop guidance for estimating the time distribution.

Notice that the time uncertainty model does not credit situations where the time available is excessively greater than the time required. Experimental studies [58]-[61] show that having excessive time has no impact on human error rates in task performance. Some PRA models credit recovery actions (i.e., personnel realize the failure of an important human action and perform the same action again or different actions to recover from the error). Having excessive time makes recovery possible but does not guarantee recovery. The guideline for the IDHEAS At-Power Application [22] has a set of criteria for crediting recovery, and one of the criteria is having excessive time available.

5.2 Guidance on Estimating the Distribution of Time Available Sufficient time means that an important human action can be successfully performed within the time window that the system allows, denoted as . In the definition of an important human action described in scenario analysis, a system time window determines the starting and ending time of the action. Within the system time window, the time available for personnel to complete an important human action should account for the time delay before the cues are present or available to personnel for detection.

Estimating may require reference to engineering calculations [57]. For NPPs, is typically generated by thermal-hydraulic studies or computer simulations. It represents the time lapse from time zero to the time that a selected key parameter would exceed its safety threshold without human intervention. The nuclear industry has been developing computer codes to simulate plant behaviors in various conditions and scenarios. Performing many simulations that include various combinations of plant and equipment conditions can be very resource 5-3

demanding and thus is not practical. On the other hand, many questions concerning event sequence timing are thermal-hydraulic problems. Often low-cost, relatively simple calculations would have adequately answered the question at hand (e.g., the time taken to boil dry the steam generators in a loss of feedwater event). Analysts may use a variety of methods to evaluate the parameter uncertainty without performing numerous resource-intensive thermal-hydraulic simulations. The analytic approach starts by reviewing the preliminary risk results to identify areas where uncertainty analysis is needed and where more sophisticated analyses should be performed to better define the success criteria. This phased approach makes uncertainty analysis affordable. Traditional engineering analyses tend to use point estimates (e.g., the best estimate) and deterministic analysis, but there are physical and analytical uncertainties and operational variability for . Sensitivity studies allow analysts to evaluate the effects of the uncertainties and the variability associated with plant operation. Estimation of the distribution should also consider the effect of human performance, which is the time dependency between important human actions in a PRA scenario. Studies show that there is significant crew-to-crew variability in performance time [16]-[20]. Some crews moved through the response efficiently, resulting in more time available for subsequent actions. Other crews responded less efficiently than expected, resulting in less time available for subsequent actions.

Therefore, any time dependency between the actions in an event may substantially affect the distribution of .

5.3 Guidance on Estimating the Distribution of Time Required NUREG-1852 [62] and NUREG-1921 [63] present a structured timeline to estimate time for an individual HFE, which is illustrated in Figure 5-2. The IDHEAS At-Power method [22] adopted the same timeline analysis. This timeline comprises several elements to capture the various aspects of time during the progression from the initiating event until the time at which the action will no longer be beneficial.

0 Cue Crew Start Action Action received diagnosis complete no longer complete beneficial Figure 5-2 Timeline Illustration Diagram Note that the diagram in Figure 5-2 is the same as that in Figure 3-1 of the IDHEAS At-Power report (NUREG-2199), thus the diagram uses the same labels as in NUREG-2199. The terms 5-4

associated with each timing element are defined next and then further described in the subsequent text:

T0 = start time = start of the event Tdelay = time delay = duration of time until the relevant cue for the action is received by the system and displayed to operators Tsw = system time window Tavail ( ) = time available = time available for response = (Tsw - Tdelay)

Tcog = cognition time consisting of detection, diagnosis, and decisionmaking Texe = execution time including travel, collection of tools, donning of personal protective equipment (PPE), and manipulation of relevant equipment Treqd ( ) = time required = response time to accomplish the action = (Tcog + Texe)

Structuring the timeline in this way allows the analyst to demonstrate, among other things, the feasibility of the action from the perspective of timing. The operator action is feasible when the time available is greater than the time required. The time available (Tavail) is the system time window (Tsw) minus any time delays (Tdelay), for example, time delay until the relevant cue for the action is received by the system and displayed to operators. The time required (Treqd) consists of the time to recognize the needed action (Tcog) and the time to execute the action (Texe); this is also called the crew response time. Each of the timing elements, including the start time, is defined next.

Start time. In Figure 5-2, T0 is modeled as the start of the event, i.e., the occurrence of the initiating event, or the time of the demand for a function or piece of equipment which is unavailable/not responding.

System time window. Tsw is defined as the system time window and is the time from the start of the event until the action is no longer beneficial (typically when irreversible damage occurs, such as core or component damage). It is typically derived from thermal-hydraulic data for the representative PRA scenario and, for HRA quantification, is considered to be a fixed input. The system time window represents the maximum amount of time available for the action.

Delay time. Tdelay represents the time from the start (typically the initiating event) until the time at which the system presents the cue to operators. It is also determined by the system and HSI design given the event. Yet, estimating Tdelay should also consider unique event-specific uncertainties such as the nature of the initiator (fast or slow) or the sensor or detector response times. In some scenarios, the salient cue may be provided only by communications from local operators. Potential delays that might be caused by operator actions or inaction because of the nature of the scenario should also be evaluated. Thus, when assessing delay time, HRA analysts should account for delays that are associated with local confirmation of the plant status during scenarios when that is the primary source of the cue.

Cognition (diagnosis and decisionmaking) time. Tcog is defined as the time for cognition and includes detection of the relevant cues, understanding/diagnosis, and decisionmaking. It is best obtained by simulator observations or talk-throughs or walk-throughs. Yet, Tcog obtained through these methods may not be representative enough because of various uncertainties and individual differences associated with Tcog. Therefore, we propose the following guidance on 5-5

estimating Tcog when adequate observations are not available to verify or modify the observed Tcog, (i.e., when the observation sample is small or no observational results are available).

Execution time. Texe is the time required for the execution of the action. Execution time is defined as the time it takes for the operators to execute the needed action(s) after successful diagnosis and decisionmaking. The execution time includes transit time to various areas in the main control room (MCR) or to the local components, time to collect tools and don PPE if needed, and time to manipulate the MCR or local components. Useful inputs to develop Texe can be obtained from observations of simulator data and walk-throughs or talk-throughs with the operators.

5.3.1 Estimation of Time Required Estimating the distribution of should consider three key aspects: nominal contributors, uncertainty factors, and bias factors. IDHEAS-G recommends the following process for estimating the probability distribution of time required:

Obtain an initial distribution of time required including the central tendency and range.

This information can be obtained by reviewing operational and simulator data and interviewing operators. HRA analysts should collect a range of times using multiple independent estimates to the extent possible. An average crew response time should be obtained, as well as estimates of the times by which the fastest and slowest operating crews would be expected to complete the actions.

Calibrate the initial estimation by reviewing the factors contributing to . For example, factors such as retrieving the tools needed or traveling to the location need to be included when estimating time to line up a pump. Table 5-1 provides some typical contributing factors for .

Modify the distribution by identifying and reviewing uncertainty factors that may change

. For example, operators familiarity with the scenario can significantly change the time required for diagnosing problems. Table 5-2 provides some typical uncertainty factors for .

Verify the estimate by reviewing the bias factors that may occur in the estimation process. Research shows that estimation of time required for human actions tends to be heuristic, and various biases often result in underestimation [64]-[66]. Several common bias factors in time estimation [40], [67], [68] are presented below.

Estimation of may start from the baseline scenario and its context identified in the scenario analysis of an HRA, followed by consideration and evaluation of possible scenario variations.

Experiments on NPP control room operation showed that there was not a consistent set of operational scenarios and context for all crews, especially for emergency operation [16]-[20]. In other words, the EOPs had many branching points for crews to choose, and different crews started from the same point but followed different paths. Once a crew followed a different path, it essentially was working on a different scenario. In some cases, different scenarios may yield a different set of important human actions for HRA to analyze. In other cases, crews perform the same important human action in a different context; therefore, the time required to perform the action can vary. Such context variability should be considered in the estimation of . Note that context variability, although caused by crew differences, is not the same as crew-to-crew variability, which accounts for crew difference in performance time when performing the same action in the same scenario in the same context. In practice, if the variability of conditions within a set of scenarios that use the same HFE result in very large uncertainties in , with a 5-6

corresponding risk-significant contribution from Pt, analysts may then decide to subdivide the scenarios and define variants of the original HFE to account explicitly for those influences.

When evaluating the factors contributing to , HRA analysts should account for sources of uncertainty that affect the total amount of time that is needed to achieve the HFE success criteria, especially for continuous actions that may have scenario-specific constraints. The success criteria for an HFE typically requires that all actions must be completed before plant conditions reach a threshold which alters the event progression, while uncertainties in the time that is needed to fully complete the required action can be important in some scenarios. For example, the success criteria for the manipulation of a cooldown and depressurization of the reactor pressure vessel of a boiling-water reactor with Mark I containment define that the entire action must be completed before the water level in the torus (to suppress containment pressure) reaches a certain setpoint. The criteria also specify that operators need to reduce the temperature by 250 °F at a rate that does not exceed 100 °F per hour. Thus, a minimum of 2-1/2 hours would be needed to complete a cooldown at the maximum allowed rate. However, plant-specific and scenario-specific constraints may not facilitate a continuous cooldown at the maximum allowed rate. For example, degraded flow rates to remove the heat and higher cooling water temperatures would reduce the effectiveness of depressurizing and cooling down the reactor pressure vessel. They can be sources of uncertainty in the estimates for .

Time analysis should consider the interteam coordination macrocognitive function for integrated human actions that require interteam coordination. The timelines and inter-relationships among various teams may be rather complex. Methods are available to display and account for combinations of series, parallel, and functionally dependent activities between the involved teams. In fact, the time required for effective interteam coordination may be the most important source of uncertainty in complex scenarios.

Cognitive biases are tendencies to think in certain ways that can lead to systematic deviations from a standard of rationality or good judgment. Some cognitive biases are referred to as heuristics. Many kinds of bias are identified in psychology. The following are examples:

Frequency bias: This bias pertains to habit intrusion and means that an individuals performance can often be captured by familiar behavioral patterns that occur frequently in the individuals experiences.

Similarity bias: This bias results from relying on a few key features for similarity matching. An example is the confusion arising from being presented with a set of initiating events with overlapping similarity in plant symptoms.

Confirmation bias: The human tendency is to seek and accept information that confirms hypotheses and beliefs.

Salience bias: Human attention is often captured by more salient and prominent indicators and diverted from subtle and modest indicators or information displayed.

Cognitive trap (bounded rationality): In emergency situations, all of a persons resources may be occupied by a primary concern that is psychologically prominent (e.g., a fire),

and the implications of other information are likely to be dismissed or discounted. This could delay diagnosis or impede situational awareness, particularly if cues related to the nonprimary concerns occur simultaneously with the psychologically prominent event.

Studies show that those biases can change, or mostly underestimate, the time required to perform tasks. The following paragraphs summarize the three bias effects on the estimation of time required.

5-7

Table 5-1 Typical Factors Contributing to Tr Macrocognitive Factors Contributing to Time Required Function Detection Travel to source location of information.

Prepare and calibrate equipment needed for detection.

Detect and attend to an indication.

Confirm and verify the indicators.

Record and communicate the detected information.

Understanding Assess the information needed for diagnosis, such as knowledge and status of a valve, pump, heater, battery, etc.

Integrate low-level information to create and/or determine high-level information.

Identify plant status and/or conditions based on several parameters, symptoms and the associated knowledge; collect information and delineate complex information such as a mass and/or energy flow with which two or more systems function.

Delineate conflicting information and unstable trends of parameters (e.g., interpret SG pressure trends when one train has failed).

Wait for continuous or dynamic information from the system to complete diagnosis.

Verify the diagnosis results or reach a team consensus.

Decisionmaking Prioritize goals; establish decision criteria; collect, interpret, and integrate data to reach a satisfying decision.

Make decision based on parameters, choose strategies, or develop a plan.

Coordinate the decisionmakers (especially with hierarchy of decisionmaking or distributed decisionmaking team), achieve consensus needed for the decision, or wait for certain information to make a decision.

Simulate or evaluate the outcome of the decision.

Action Execution Evaluate the action plan and coordinate staff.

Travel and gain access to the action site.

Acquire (deploy, install, calibrate) the tools and equipment (e.g., put on gloves) to perform the actions.

Implement the action steps or continuous action and required timing of steps.

Confirm completion of the actions and wait for system feedback.

Interteam Allocate resources needed for individual teams to perform actions.

coordination Authorize decisions through the required authorization chains.

Communicate key information between teams.

5-8

Table 5-2 Uncertainty Factors that Modify the Distribution of Tr Uncertainty Considerations Factors Environmental Environmental factors affecting allowable time for work (e.g., radiation factors dose limit to work in a high-radiation environment)

Delay in personnel and equipment movement because of external hazards Continuous habitation Plant condition Multiunit events Other ongoing activities that compete for resources Plant-wide conditions introducing scenario-specific sources of distractions, interruptions, possibly conflicting priorities, stress, etc. that may distract personnels attention or cause competing demands resulting in delayed or prolonged actions.

Work site Different paths to worksite accessibility Hurdles to access the worksite (e.g., security system denies access)

Information Visibility of information availability Familiarity with sources of information Procedures/ Applicability of procedures or instructions guidance/ Recency of training instructions applicability and training Decisionmakers Variability of decisionmakers Variability in decision infrastructure Communication in distributed decisionmaking Staff Staff adequacy (e.g., whether concurrent activities would reduce the staff available for the action or whether tasks can be performed concurrently with more than adequate staff)

Command and control Staff experience (e.g., whether less trained, nonregular staff is used)

Equipment, Familiarity with the use of equipment tools, parts, and Potential failure modes of equipment and recovery or backup keys Scenario Familiarity with scenario familiarity Fatigue (mental Time of day and physical) Duration of being on shift Crew-to-crew Crew-to crew-variability in time required to perform the same actions; variability different crews may take different procedure paths, which leads to variability in time required 5-9

Underrepresentation/incomplete representation of the range of times: Estimating relies on subject matter experts judgment or their calibration to simulator data. Given that individuals vary greatly in the time they need to complete tasks, HRA analysts should ensure that the time estimates are representative of a normal operator population. In fact, when estimating for assessing feasibility, HRA analysts should strive to collect a range of crew response times, using multiple independent estimates to the extent possible. Although an estimate of the average crew time for should be obtained, it is also critical to obtain an estimate of the time by which the slowest and fastest operating crews would be expected to complete the action. In other words, the time range by which all crews could be expected to complete the action under the conditions in the scenario should be estimated. Although the availability of training and operations staff may be limited, it is important to interview several trainers or operators for cases in which a small change in the time estimated could render a feasible operator action infeasible or significantly affect the resulting HEP. For actions that occur well after the initiating event or for actions with a long time window, a bounding estimate can often be useful.

Underestimation for complex scenarios: When estimating task completion time, people tend to focus on optimistic aspects of the scenarios and disregard pessimistic aspects, resulting in underestimation of time for complex scenarios. In particular, interteam coordination required for integrated human actions is an important source for underestimating the time required to complete the action. Therefore, analysts, in discussing the time required with trainers and operators, should thoroughly analyze the nominal contributors and modifying factors involved in complex scenarios. The time required to work through the relevant procedures (including verification steps that may not be critical to achieve the necessary actions but nevertheless can require time) should be carefully evaluated (especially when operators are working with multiple procedures). The potential for operating crews to get stuck in a procedure while waiting for particular conditions or to have trouble transitioning to the correct procedure because of misleading or confusing indications should be evaluated.

Underestimation of the effects of interruption: Cognitive studies demonstrate that the effect of interruption on task completion time is typically more severe than expected. Depending on types of tasks, interruption can result in a 30-100 percent increase in task completion time (without counting the interruption time). Analysts will need to discuss with the operators and trainers the types and likely occurrence of any potential interruptions given the scenario conditions and decide how much time should be added in estimating the time required.

Activities that can slow personnel response time (e.g., peer-checking, routine monitoring, communication and coordination, responding to alarms) or extend response time (e.g., simultaneous or parallel activities) should be included in estimates of the time required. In other words, it should not be assumed that personnel are only processing cues, stepping through the procedures, and taking actions.

5.4 Summary This chapter presents an improved approach to time uncertainty analysis for HRA. The analysis emphasizes identifying time uncertainties and accounting for the effects of the uncertainties in the overall HEP for the HFE. The time uncertainty model in IDHEAS-G accounts for time uncertainties based on whether there is sufficient time for personnel to complete the required time-critical actions. The model is consistent with findings on the effects of time on human performance in many cognitive experiments reported in the research literature. The model also includes guidance on identifying sources of uncertainties and estimating time available and time required to perform an important human action. IDHEAS-G expands the current HRA practice of treating time-critical actions in the structure of the model of human performance.

5-10

6 GENERALIZATION OF HUMAN ERROR DATA FOR ESTIMATION OF HUMAN ERROR PROBABILITIES Without explicitly modeling the intrinsic cognitive mechanisms underlying human errors, an HRA method may result in different interpretations of the same observed phenomena and poor understanding of the causes of human errors. Furthermore, the use of empirical data for HEP estimation has been limited by the lack of abundant human reliability data. Lack of a strong data basis in the methods challenges method validity and introduces additional variability in HEP estimation. On the other hand, there are various sources of human error data that have not been used by HRA, mainly because there are significant discrepancies between the formats of available data and HRA methods. Human error data are available from performance of tasks in various domains, in different formats, and at a range of levels of detail. Most of the human error data either cannot be directly used for HRA or they were formatted to support only one application-specific HRA method.

IDHEAS-G enhances HRA methodology by incorporating the advances made in cognitive and behavioral science in the past decades. IDHEAS-G uses its cognition model to represent human failures with a basic set of CFMs and represents the human event context with a PIF structure. The basic set of CFMs represents human failures at three levels of granularity (i.e., failures of macrocognitive functions, failures of the processors in each macrocognitive function, and behaviorally observable failure modes of the processors). Similarly, the PIF structure represents event context at two levels of granularity: PIFs and their attributes.

Underlying cognitive mechanisms can link CFMs and PIFs at any level of granularity. Thus, IDHEAS-G is inherently capable of generalizing human error data of different types of tasks to inform HEP quantification. The CFMs and PIF structure together form a framework for generalizing human error data of various sources and integrating them to support the IDHEAS-G quantification model.

6.1 Human Error Data For a given context, the HEP of an event can be calculated as the number of failure events divided by the total number of times the events has been attempted. The event can be a human action, a task, or a defined CFM. To date, human error data to support calculation of HEPs of all kinds of tasks or CFMs for all possible combinations of PIFs are not abundant. Most HRA methods use a quantification model to estimate HEPs or direct HEPs to expert judgment; the quantification models typically consist of base HEPs for a set of human failure modes and PIF multipliers to adjust base HEPs. Data were not adequate to support the base HEPs and PIF multipliers in the quantification models.

Over the last two decades, much human error data have become available in various fields such as nuclear, aviation, manufacture, and health care. Many cognitive behavioral studies produced experimental data on human error rates in various contexts. Moreover, several human performance databases have been developed to systematically collect operator performance data in NPPs for HRA. Such efforts include the SACADA database [24],

developed by the NRC staff, and the Human Reliability Data Extraction (HuREX) database [69]

developed by the Korea Atomic Energy Research Institute. In addition, there have been many HRA expert elicitation studies to obtain expert judgment of HEPs for specific applications. While individual sources of human error data may not be sufficient to yield HEPs for all kinds of tasks and contexts, consolidating the available data and using the data together would yield more robust and valid HEPs.

6-1

Ideally, the data to inform HEPs would have the following features:

The known numerator and denominator of human error rates are collected within the same context.

Human error rates are measured repetitively to minimize uncertainties in the data.

Human error rates are collected for a variety of personnel so that the data can represent average personnel or operators.

Human error data are collected for a range of task types or failure modes and combinations of PIFs.

Such ideal data do not exist. However, these features can be used as criteria to evaluate real data for their applicability to HRA.

Along with the development of IDHEAS-G, the NRC staff documented human error data in the literature and human performance databases. The NRC staff examined the data for their ability to inform HEPs. The following list contains several example types of human error data to demonstrate if and how the data can be used to inform HEP estimation.

Human error rates with known states of PIFs This type of data provides the numerator and denominator of human error rates for types of tasks performed in the same context or in a known range of contexts. Such data can inform the base HEPs for the CFMs (i.e., ) relevant to the tasks. Below are two examples:

(1) Quantification of unsatisfied task performance in NPP operator simulator training, as collected in the SACADA database by the NRC staff. The SACADA database was built with the same macrocognitive model as that in IDHEAS-G and collects operator unsatisfied task performance for different types of failures in various contexts. The different types of failures can be mapped to the detailed level CFMs in IDHEAS-G, and the various contexts can be mapped to IDHEAS-G PIF attributes. Thus, the SACADA database can inform baseline HEPs of IDHEAS-G CFMs and the quantitative effects of some PIF attributes.

(2) The analysis of human errors in maintenance operations of German NPPs.

Preischl and Hellmich [70], [71] studied human error rates for various basic tasks in maintenance operations. Below are some example human error rates they reported:

o 1/490 for operating a circuit breaker in a switchgear cabinet under normal conditions o 1/33 for connecting a cable between an external test facility and a control cabinet o 1/36 for reassembly of component elements o 1/7 for transporting fuel assemblies This type of data from operational databases inherits uncertainties in the data collection process. For example, the definitions of human failure vary from one database to another, so caution is needed when aggregating human error rates from different sources.

6-2

Human error rates with unknown or mixed context This type of data reports statistically calculated human error rates for specific tasks across a mixture of contexts. Such data cannot inform HEPs of the failure modes because neither the failure modes nor the context was specified. The data could represent the best or worst possible scenarios or the average scenario. This type of data can be used to validate the distribution of HEPs obtained by other means.

HEPs estimated through expert judgment This type of data is not true human error data. They are generated through a formal expert elicitation process, representing the beliefs of the representative technical community on the likelihood of human failure for a given HRA application. Nevertheless, expert judgment has been widely used in risk-informed applications. The resulting estimates of HEPs bear validity and regulatory assurance if the judgment was obtained through a formal, scientifically founded expert elicitation process. This type of data can be used to inform the central tendency and range of HEPs for the context in which the expert judgment was made.

An example of an expert elicitation process used to estimate HEPs is the judgment of HEPs of the crew failure modes in the IDHEAS At-Power Application [22]. The method has 14 crew failure modes, which are a subset of IDHEAS-G behaviorally observable failure modes. A very limited set of PIF attributes is considered for each failure mode.

An expert panel estimated the HEP distributions of the failure modes for the combinations of the PIF attributes.

This type of data has a limitation in that the full context in which the HEPs were estimated is often not well documented. Because expert judgment is typically elicited for a very specific domain of application and the expert panel consists of experienced domain experts, the expert panel makes its own assumptions about the context. For example, in the expert elicitation of HEPs for the IDHEAS At-Power Application [22], the expert panel assumed that NPP operators perform control room tasks by following procedures, and they would make a correct diagnosis with procedures as long as they have the right information. This assumption may not be true for tasks performed outside control rooms. Thus, caution is needed when generalizing expert judgment HEPs to other applications.

Quantification of PIF effects Many sources present the changes in human error rates when varying the states of one or more PIFs. Such data can inform the quantification of PIF effects in the IDHEAS-G quantification model. Below are several examples:

o NUREG/CR-5572, An Evaluation of the Effects of Local Control Station Design Configurations on Human Performance and Nuclear Power Plant Risk, issued September 1990 [72], estimated the effects of local control station design configurations on human performance and NPPs. It estimated that HEP = 2 x 10-2 for ideal conditions and HEP = 0.57 for challenging conditions with poor HSIs and distributed work locations.

o Prinzo et al. [73] analyzed aircraft pilot communication errors and found that the error rate increased nonlinearly with the complexity of the message communicated. The error rate was around 4 percent for an information complexity index of 4 (i.e., the number of messages transmitted per 6-3

communication), 30 percent for an index of 12, and greater than 50 percent for indices greater than 20.

o Patten et al. [74] studied the effect of task complexity and experience on driver performance. The PIF states of the tasks manipulated in the experiment were low experience versus high experience, and low complexity versus high complexity. The mean error rates were 12, 21, 25, and 32 percent respectively for the four combinations of PIF states: low complexity and high experience, low complexity and low experience, high complexity and high experience, high complexity and low experience.

When documenting this type of data, the objective description of PIF states needs to be carefully considered. For example, the PIF state of high complexity in one data source can be referred to as low complexity in another data source. The NRC staff found that PIF attributes more accurately represent the actual context than the subjective assessment of high or low PIF states. In fact, using PIF attributes can make the definition for PIF states more objective.

PIF interaction Most HRA methods treat the combined effects of PIFs on HEPs as the multiplication of the effects of the individual PIFs. Xing et al. [75] reviewed a limited set of cognitive literature in which human error rates were measured, as two or more PIFs varied independently and jointly. They observed that the combined effect of PIFs fits better to the addition than the multiplication of the individual PIF effects. In fact, the broad cognitive literature indicates that the combined effect is not simply the addition or multiplication of individual PIF effects. Instead, the interaction between PIFs may not fit to a single rule and can vary greatly for different combinations of PIFs. The interaction effect can be inferred from human error rates that are collected in a single study or database and with more than one PIF varying independently and jointly.

The significance or ranking of PIFs or types of errors Studies in human error analysis and root causal analysis typically classify and rank the frequencies of various PIFs in reported human events. Some studies correlate PIFs with various types of human errors. Those studies only analyze the relative human error data without reporting how many times personnel performed the kind of tasks. The data from such studies cannot directly inform HEPs, but they can inform which PIFs or attributes are more relevant to the CFMs of the reported human errors. Below are several examples:

o Virovac et al. [76] analyzed human errors in airplane maintenance and found that the prevalent factors with frequent occurrence in human errors are communication (16 percent), equipment and tools (12 percent), work environment (12 percent), and complexity (6.5 percent).

o Kyriakidis et al. [77] analyzed U.K. railway accidents caused by human errors and calculated proportions of PIFs in the accidents. They reported that the most frequent PIFs in the accidents were safety culture (19 percent), familiarity (15 percent), and distraction (13 percent).

The above examples are just a few in the large body of human error data the NRC staff has documented so far. The staff performed a meta-analysis of a subset of the documented data and noticed that the error rate data were generally convergent across 6-4

different sources. For example, most studies of dual tasks showed that the error rate in dual tasks was 1 to 2 times higher than that in a single task. The staff also observed the consistency between the results obtained in controlled cognitive experiments and those from complex scenario simulation. The observation suggests that human error rates measured from cognitive experiments could serve as a baseline reference for estimating HEPs in more complex, real-life scenarios.

6.2 Data Generalization and Integration Sources of human error data measure different types of human actions, tasks, or failure modes and in different contexts. They also describe human errors at different levels of detail. To use different sources together to inform HEPs, the NRC staff generalized them into a common format. IDHEAS-G is based on the cognition model, which is inherently capable of generalizing human error because (1) IDHEAS-G can model any human task with its basic set of CFMs, (2) the CFMs are structured in different levels of detail, and (3) the PIF structure models the context of an important human action with high-level PIFs and detailed PIF attributes. Thus, the NRC staff used IDHEAS-G to generalize various sources of human error data and then integrated the data to inform HEP estimation. Figure 6-1 illustrates this approach.

Sources of human error Data source 1 Data source 2 data Tasks Context Tasks Context Failure PIFs Failure PIFs Generalization modes modes Human error Human error Human error Human error rates of the rates at the rates of the rates at the failure modes PIF states failure modes PIF states Integrate data for the failure modes and PIFs Integration

= ( )

Figure 6-1 Illustration of IDHEAS-G Data Generalization and Integration The NRC staff used IDHEAS-G to generalize human error data in the following steps, as shown in Figure 6-2:

(1) Analyze the tasks and the context of a data source to identify cognitive activities involved in the tasks and whether the tasks are performed with time constraints.

(2) Map the human errors of the source data to IDHEAS-G CFMs.

6-5

(3) Represent the task context with IDHEAS-G PIF structure.

(4) Load the human error data into three sets of IDHEAS-G human error tables.

(5) Evaluate and document uncertainties in the data and mapping.

Through this process, human error data can be generalized to IDHEAS-G CFMs and PIFs and represented in one of the three tables: the HEP Table, the PIF Impact Table, and PIF Interaction Table. Each of these tables provides sets of human error data , as shown in Figure 6-2, that can be used to estimate the error probability of a CFM, . Sections 6.2.1, 6.2.2, and 6.2.3, respectively, describe these tables. The collection of all these tables are referred to as IDHEAS-DATA in the rest of this report.

Analyze data Interpret and Consolidate and source represent data document data Human action IDHEAS-G CFMs HEP Table -

/ tasks IDHEAS-G PIF PIF Impact Table -

Context Structure PIF Interaction Table -

Figure 6-2 IDHEAS-G Steps to Generalize Human Error Data As of 2019, the NRC staff has performed the process delineated in Figure 6-2 on a substantial amount of data from the literature and the NRCs NPP operator simulator training databases (referred to as SACADA). The NRC staff developed the IDHEAS method for Event and Condition Assessment (IDHEAS-ECA) [78]. The method integrated the data documented in IDHEAS-DATA (i.e., the collection of tables described in Sections 6.2.1, 6.2.2, and 6.2.3) to generate the base HEP values and PIF attribute weights. Those base HEPs and PIF weights were the first version of integrating the data in IDHEAS-DATA. Because of the limited amount of data available, the NRC staff used interpolation, judgment, and benchmarking to develop the full set of base HEPs and PIF weights. In the long-term, there should be a continuous effort to generalize human error data as new data become available, and there should be periodic updates of data integration. This chapter focuses on introducing the concepts and methodology of generalizing human error data rather than reporting the integrated HEPs and PIF weights. All the numeric values in this chapter are only to demonstrate the concepts. It is not recommended to use these values in HRA.

6.2.1 HEP Table The HEP Table consolidates data on human error rates or HEPs for every CFM. A data source may contain human error rates for certain tasks or estimated HEPs. The task analysis identifies cognitive activities involved in the tasks. The cognitive activities are then mapped to corresponding IDHEAS-G CFMs. The mapping could be made to one or all three levels of CFMs: failure of macrocognitive functions, failure of processors, and behaviorally observable failure modes. Along with the human error rates or HEPs, the PIF states under which the 6-6

human error rates or probabilities were obtained also are documented. The HEP Table documents the following dimensions of information for every data point:

CFMs human error rate or HEP PIF states or PIF attributes time information, i.e., whether the human error rate or HEP is for tasks performed without time constraints, with time constrains but adequate time, or with inadequate time.

brief narrative of the task or types of failure in the data source, including the work domain (e.g., nuclear, aviation) and type of data source (e.g., experiment, training simulation, event database) uncertainties in the data source and in the mapping to IDHEAS-G CFMs and PIFs As more sources of data are consolidated into the HEP table, a CFM could have multiple data points under the same or different set of PIF states. These data points together can inform the HEP distribution of the CFM.

Integrate the data to inform base HEPs As more sources of data are consolidated into the HEP and PIF Tables, there are multiple data points of various sources for a CFM or a PIF. Before using the data to inform HEP estimation, the context and uncertainties of the data should be evaluated for their reliability and relevance to the HRA application of interest. For example, if the HRA application is for a well-trained crew implementing EOPs in an NPP control room, the analyst may choose to use only the data collected from NPP operator training simulation and not use the data from cognitive experiments in which tasks were performed by college students. However, if there is no NPP operation data available, then using data from other domains is better than not using any data to inform the HEPs of NPP operation.

Multiple data points for a CFM or PIF need to be integrated to inform the HEP or PIF weights.

Integration of multiple data points depends on the intended use. The major purpose of developing the Human Error Data Tables is to support the IDHEAS-G quantification model. The data can be used to generate base HEPs. The base HEPs are the error probabilities of every CFM at various states of the three base PIFs (information availability and reliability, scenario familiarity, and task complexity). The base HEPs vary with CFMs and the states of the base PIFs. To infer the base HEPs, expert judgment or statistical analysis is needed to address the following issues in data:

Some error rates for a base PIF were complicated by other PIFs.

There could be multiple data points from different sources for the HEP of a base PIF; the multiple data points should be carefully integrated to form the HEP distribution.

The HEP distribution should consider the uncertainties annotated for the data point.

Some base HEPs have no data available or generalized.

6.2.2 PIF Impact Table The PIF Impact Table documents the data points at which the human error rates or HEPs of a task are measured for two or more states of a PIF. Such data points would allow calculation of 6-7

the weight of the PIF between the states. The PIF Impact Table has many subtables, one for each PIF. Within a subtable, data points for the same attribute are grouped in the same section of the rows. If a data point has a PIF attribute that is not in the IDHEAS-G PIF structure, the attribute is added to the end of the subtable and annotated. New subtables will likely be added if data sources reveal new PIFs. A PIF subtable contains the following types of information:

PIF states or attributes the PIF description in the original data error rates or HEPs for the PIF states PIF weight, calculated using Equation (4.6) macrocognitive functions of the task or failure mode of the data point brief description of the task and context uncertainties in the data and the mapping of task context to IDHEAS-G PIF structure Integrate human error data to inform PIF weights The first step is to define PIF states. A PIF may be best represented with binary states or multiple states. This can be done by ranking the weights of a PIF in its subtable against the PIF attributes, then aggregating the PIF attributes into different states. Figure 6-3 illustrates this concept. Once the PIF states are defined, the PIF weight can be inferred from the data in the PIF Impact Table.

W5 N: No impact PIF weight W4 L: Low impact W3 M: Moderate impact H: High impact W2 E: Extreme High impact W1 N L M H E States of PIFs Figure 6-3 Illustration of Human Error Data to Inform PIF Weights 6.2.3 PIF Interaction Table The PIF Interaction Table documents data sources in which the human error rates or HEPs of a task are reported as two or more PIFs varying independently and together. The PIF Interaction Table has many subtables, one for each data point, containing the information about human error rates of different states of individual PIFs as well as the error rates under the combination of multiple PIFs. The weights of individual PIFs and the combined weight of multiple PIFs can thus be calculated from the human error rates. The relationships between these weights provide 6-8

insights into the interaction of the PIFs. For example, if the two PIFs examined have no interaction in their impacts on human error rates, then the combined weight is simply the addition of the individual weights. On the other hand, if there is interaction, the combined weight would not be the linear combination of the individual weights.

Integrate human error data to inform interaction of PIFs The data in the PIF Interaction Table constitute the basis for PIF combination in the IDHEAS-G quantification model. Multiple meta-analysis studies in the literature have analyzed the effect of PIF combinations. For example, the NRC staff performed a preliminary analysis of 23 data points in which two PIFs were varied independently and jointly (APPENDIX D). The results showed that the combined weights of two PIFs are roughly equal to the sum of the weights of the two PIFs. This preliminary analysis suggests a weak interaction effect of PIFs on human error rates. Table 6-1 shows the main findings from several such meta-analysis studies. These studies focused on explaining whether additive or multiplicative PIF combination effects better explain existing data. Overall, the findings from the meta-analysis studies were consistent in that 1) PIF combination effects can generally be predicted with linear addition, and 2) there are cases where the combined PIF effect is multiplicative or more than the sum of the individual PIF effects. The NRC staff recommends setting the interaction factor C equal to 1 unless the data suggest otherwise. In the long run, the interaction effect for different PIF combinations can be individually inferred from the PIF Interaction Table as more data become available and analyzed. Appendix D discusses the cognitive and data basis underlying PIF interaction for future work.

Table 6-1 Summary of Example Meta-Analysis on PIF Combination

of studies PIFs analyzed included Main Findings Reference in meta-analysis Noise, 51 Combined effect is no more than the added [79]

temperature, reports single effects and can be predicted from single sleep loss effects.

Noise and 20~30 The majority of evidence indicates that noise [80]

heat reports and heat do not interact significantly within the ranges experienced commonly in the industrial setting.

Distraction, 23 data- Additive fits better than Multiplicative; [75]

experience, points Additive over-estimates for large PIF weights HSI, others Cognitive 40-57 Additive accounted for ~ 91% of job [81]

ability and reports performance data; Multiplicative accounted for motivation on only about 9% of the explained variance.

performance 6-9

6.3 Demonstration of Human Error Data Generalization and Integration This section shows examples of generalizing human error data to the HEP Table and generalizing data to the PIF Impact Table.

6.3.1 Generalization of Data to the HEP Table to Inform the Base Human Error Probabilities The first example demonstrates generalization of human error data into the HEP Table to inform the base HEPs in the IDHEAS-G quantification model. The base HEPs are the error probabilities of every CFM at various states of the three base PIFs: information availability and reliability, scenario familiarity, and task complexity. The base HEPs of a CFM vary with the states of base PIFs. Depending on the information available from the study, the failure of the task may be modeled at different CFM levels (i.e., macrocognitive functions, processors, and detailed CFMs). The example here models the failure of the task at the macrocognitive function level.

Demonstrated next is the process of generalizing human error data to the HEP Table. The data source is a report, The Outcome of Air Traffic Control] Message Complexity on Pilot Readback Performance, by Prinzo et al. [73]. The study analyzed aircraft pilot communication errors and reported that the error rate increased nonlinearly with the complexity of the message communicated. The following is the process of generalizing the data to IDHEAS-DATA Base HEP IDTABLE-IDTABLE-3 for Task complexity.

Analyze the data source: Prinzo et al. [73] The task is that pilots listen to and read back messages from air traffic controllers. The pilots hold the information in their memory and read back at the end of the transmission. The cognitive activities involved are perceiving information and communicating it. The pilots perform the task individually without peer-checking, and the tasks are performed without time constraints.

Readback errors are defined as misreading or missing key messages. Message complexity is defined as the number of key messages in one transmission. The study calculates percent of readback errors at different levels of message complexity from thousands of transmissions.

Identified human error data for generalization: The readback error rates at different message complexity levels are identified as the data for this entry.

Applicable CFMs: The CFM for readback errors is failure of Understanding. While the task is listen to and readback messages, the cognitive activities required are identifying, comprehending, and relating all the key messages in one transmission. Those are the elements in the macrocognitive function Understanding.

Relevant PIF attributes: The primary PIF is Task complexity. The attribute is C11, the number of key messages to be kept. Another PIF present is the Work Process attribute, Lack of verification or peer-checking.

Other PIF attributes present: Some transmissions may be performed with the presence of other PIF attributes such as distraction, stress, or mental fatigue. Those PIFs were not prevalent in the transmissions analyzed but could increase the overall error rates. Pilots flying experience was not correlated with the error rates.

6-10

Uncertainties in the data and mapping: The source audio transmissions are mixture of normal and emergent operation.

The analysis results are documented in the HEP Table as one datapoint. Table 6-2 shows the information documented for this datapoint. All the information items are in one row.

Table 6-2 An Example Datapoint for the HEP Table Other PIFs PIF CFM Error rates Task PIF measure (and REF Uncertainty)

Task Failure of Number of Error Pilots listen to Message (Mixture of [73]

compl Understanding messages rate and read back complexity - # normal and exity key messages of key emergent 5 0.036 messages in operation so 8 0.05 one other PIF transmission attributes may 11 0.11 exist) 15 0.23 17 0.32

>20 >0.5 The next example demonstrates generalizing data to the PIF Impact Table. The data source is the research paper, Effects of Interruption Length on Procedural Errors, by Altmann et al. [82].

The study investigated effects of task interruption on procedural performance, focusing on the effect of interruption length on the rates of different categories of error at the point of task resumption. The following is the process of generalizing the data for PIF Multitasking, Interruption, and Distraction.

Analyze the data source: The task [82] was that individual participants performed procedural sequences of computerized execution steps. The task required individuals memorizing the sequences. The study examined effects of interruption length on procedural performance parametrically across a range of practically relevant interruption durationsfrom about 3 seconds to about 30 seconds. The cognitive activities involved were executing sequential steps. The participants were well trained for the task. They performed the task individually without peer-checking and without time constraint. Performance errors were defined as loss of place in the procedure (sequence errors) and errors involving incorrect execution of a correct step after interruption (non-sequence errors).

Identify human error data for generalization: Both sequence and non-sequence error rates at different lengths of interruption were identified as the data for this entry.

Applicable CFMs: The CFM was failure of action execution.

PIF attributes: The PIF being examined was Multitasking, Interruption, and Distraction. The attribute was Interruption. The PIF Work Process attribute Lack of verification or peer-checking was present for all the human error data measured in the study.

6-11

Evaluate uncertainties in the data and mapping: This study was a well-controlled experimental study and there was no prevalent uncertainty involved.

The analysis results are documented in PIF Impact Table as one datapoint. The sequence-error rates at different lengths of interruption are identified as the human error data for this datapoint. The post-interruption non-sequence errors, although not affected by interruption, is also documented for reference. The reported human error rates for the corresponding CFMs and PIF attributes are then documented along with other items of context information. Table 6-3 shows the information documented for this datapoint. All the information items are in one row.

The top row has column numbers for referencing.

Table 6-3 Example datapoint generalized for PIF Impact Table 1 2 3 4 5 6 7 Other PIFs Task (and error PIF PIF CFM Error rates (%) (and REF measure) measures Uncertainty)

MT2 E Interruption Sequence Non- Individuals executed Interruption - [82]

Length (s) error sequence procedural steps of a Different error computerized task. interruption Performance errors are length Baseline 2 2 loss of place in the (seconds).

procedure (sequence Baseline is 3 4 2 errors) and errors no involving incorrect interruption.

13 10 2 execution of a correct step after interruption 22 14 2 (nonsequence errors).

6.3.2 Mapping between SACADA Database and IDHEAS-G This section shows the mapping between the SACADA database and IDHEAS-G. The NRC developed the SACADA database with the purpose of making it suitable for collecting operator performance information for use in the NPPs operator simulator training program. The collected data would support plant operator training and be shared with the NRC to improve HRA quality, in particular, HEP estimation. Each SACADA data point consists of two information segments: context and performance results. Context is a characterization of the performance challenges to task success. The performance results are the outcomes of a crew performing the task. The data taxonomy uses a macrocognitive function model for the framework. At a high level, the collected information is categorized according to the macrocognitive functions of detecting the plant abnormality, understanding the abnormality, deciding the response plan, executing the response plan, and team-related aspects (i.e., communication, teamwork, and supervision). The structured data allow analysis of the relations between context and error modes in human performance.

Although SACADA and IDHEAS-G are based on the same set of macrocognitive functions, the scope of the macrocognitive functions in IDHEAS-G is broader than those in SACADA. The scope of the functions in SACADA are specific for NPP control room actions performed by licensed crew members. As a result, SACADA has fewer but more specific error modes than IDHEAS-G CFMs. Also, the team-related aspects in SACADA are tuned to NPP control room crew structure, while the interteam coordination function in IDHEAS-G focuses on interaction 6-12

between teams. The within-team communication and supervision are modeled as processors of individual macrocognitive functions. Nevertheless, IDHEAS-G and SACADA taxonomies share the same framework; therefore, the elements can be mapped to each other, but not necessarily in a one-to-one mapping. With the mapping, the error mode statistics of NPP operator simulator training in the SACADA database can be generalized to IDHEAS-G HEP tables, and data on the SACADA context factors can be generalized into the PIF Impact Table and PIF Interaction Table.

Table 6-4 shows the mapping between IDHEAS-G CFMs and SACADA error modes. For brevity, Table 6-4 shows only the IDHEAS-G CFMs that have corresponding SACADA error modes. Thus, the CFMs in this table are a subset of the full list of IDHEAS-G CFMs presented in Chapter 4.

Table 6-4 Mapping between SACADA Error Modes and IDHEAS-G CFMs IDHEAS-G IDHEAS-G SACADA error mode failure of processors detailed failure modes D3 Fail to perceive D3-2 Key alarm not attended to Alarm issues: key alarms not information detected or not responded to D3-3 Critical information not perceived or misperceived Indicator issues: key parameter value not detected or incorrectly read U1 Fail to assess or U1-2 Incomplete data selected Misinterpreted: critical data select data misinterpreted U1-3 Incorrect or inappropriate data selected Discredited: critical data dismissed, discredited, or discounted U3 Incorrect integration U3-2 Incorrectly assess situation Incorrect/incomplete: failure of data and mental to form a correct model U3-3 Incorrectly diagnose understanding or revise initial problems false concept Awareness: lack of awareness of plant conditions 6-13

Table 6-4 Mapping between SACADA Error Modes and IDHEAS-G CFMs (continued)

IDHEAS-G IDHEAS-G SACADA error mode failure of processors detailed failure modes DM4 Incorrect DM4-1 Misinterpret procedure Failed to consult available integration of data and procedure mental model DM4-2 Choose inappropriate strategy or options Following problem: trouble following or using procedure (e.g., misinterpret procedures)

Choice: made incorrect choice.

DM5 Fail to DM5-1 Unable to simulate or Comprehensive: failed to simulate/evaluate the evaluate the decisions effects consider all options decision DM5-2 Incorrectly or incompletely simulate or evaluate the decision against other options DM5-3 Incorrect dynamic decisionmaking DM6 Fail to DM6-1 Decision incorrectly Delayed: delayed making communicate or communicated decision authorize the decision DM6-2 Decision not authorized or delayed in authorization E2 Fail to develop/ E2-1 Fail to modify, adapt, or Action not adapted modify action scripts develop action scripts for a high-level action plan E3 Fail to coordinate E3-1 Fail to coordinate the Action not taken: forget to action implementation action implementation take required actions E3-2 Fail to initiate the action E4 Fail to perform the E4-1 Fail to follow procedures Executed discrete action(s) planned action (e.g., skip steps) incorrectly E4-2 Fail to execute Dynamic manual control:

nonprocedural simple action dynamic manual control problem E4-3 Fail to execute complex action (e.g., control actions, long-lasting actions)

D4 Fail to verify information Team aspects: errors in supervision, teamwork, and D5 Fail to communicate detected information communication*

6-14

Table 6-4 Mapping between SACADA Error Modes and IDHEAS-G CFMs (continued)

IDHEAS-G IDHEAS-G SACADA error mode failure of processors detailed failure modes U4 Fail to iterate the understanding Team aspects: errors in supervision, teamwork, and U5 Fail to communicate the outcome of understanding communication*

DM6 Fail to communicate or authorize the decision E3 Fail to coordinate action implementation E5 Fail to verify or adjust action

Note: The SACADA error modes for team aspects do not have specific correspondence in IDHEAS-G. They are mapped to multiple IDHEAS-G CFMs related to within-team (crew) interaction.

The above mapping shows that most SACADA error modes can be mapped to IDHEAS-G CFMs of different levels. Therefore, the error probabilities collected in SACADA data can inform HEP estimation in IDHEAS-G.

Next, Table 6-5 shows the mapping between the SACADA context factors and IDHEAS-G PIF structure. Because SACADA is designed to be suitable for operator simulator training in NPPs, SACADA context factors constitute only a subset of IDHEAS-G PIFs. Table 6-5 lists all the IDHEAS-G PIFs but only the PIF attributes that have corresponding items in SACADA.

SACADA context factors are categorized by their effects on macrocognitive function while IDHEAS-G are not.

6-15

Table 6-5 Mapping between SACADA Context Factors and IDHEAS-G PIF Structure IDHEAS-G PIF structure SACADA context factors Context factor/ Affected PIF Attributes error cause function Accessibility/

habitability Workplace visibility Noise Loud noise in Noisy background: loud All workplace impeding background noise makes communication communication challenging Change of indicator Degree of change: D status is not intuitive

slight change (i.e., requires and needs working some effort to detect the memory for mental change) comparison

distinct change (i.e., prominent and readily detected)

No mimics: requires operator to D rely on memory Low salience Small indications: can be read D only from a close distance The source of Similar displays: multiple D indication is similar to identical displays in the same other sources nearby bank of control panel Location of controls is Location: E distributed or not in

main or auxiliary control front of work panel board

back panel Confusion in action Unintuitive controls: the control E maneuver states requires counter-intuitive action Controls are difficult to Additional mental effort: E maneuver performing the action requires (e.g., confusing labels, performing activities such as unit unit translation, or translation or mental calculation.

mental calculation)

Controls have Inadequate feedback: E inadequate feedback system/control state feedback is missing or slow.

Appearance of controls Similar controls: similar controls E is not salient (e.g., in the same bank of control many similar controls panel.

nearby and the labels are not visually distinctive) 6-16

Table 6-5 Mapping between SACADA Context Factors and IDHEAS-G PIF Structure (continued)

IDHEAS-G PIF structure SACADA context factors Context factor/ Affected PIF Attributes error cause function Training Inadequate training on Familiarity: D, U procedure adaptation

standard: crew has previously trained on this challenge.

novel: this involves a change in the way the challenge is addressed, such as a new procedure, scenario, or role.

anomaly: standard training must be adapted to "t an anomalous situation (e.g., the procedures do not cover the circumstances).

Procedures, Procedure lacks Detecting mode: D guidance, and detailsProcedure instruction does not direct

procedure-directed check:

personnel to perform procedure directs crew to specific tasks check a speci"c indicator or (e.g., monitoring parameter parameters, looking

procedure-directed monitoring for changes of status)

knowledge-driven monitoring:

knowledge of the situation or expectation of change in the parameter prompts crew to monitor.

awareness/inspection:

nonprocedurally directed monitoring or awareness of plant parameters Procedure lacks Diagnosis basis: U detailsengineering judgments are needed

skill

procedure

knowledge based Procedure lacks Decision basis: D details

procedure: the decision is procedure is not driven by procedures or other available; thus, guidance personnel must find

skill: skill-driven decision; ways to perform the without procedure, operator task based on their can make decision from knowledge memory knowledge: no procedure applicable; crew relies on engineering or technical 6-17

Table 6-5 Mapping between SACADA Context Factors and IDHEAS-G PIF Structure (continued)

IDHEAS-G PIF structure SACADA context factors Context factor/ Affected PIF Attributes error cause function knowledge and operating experience Procedure lacks Guidance: E details

procedure: action guided by procedures Procedure is not skill of craft (nonfaulted available for hardware): in situations without skill-based tasks faulted indications or hardware, the action is guided by skill of the craft, not a written procedure Team and Inadequate teamwork Communicator unavailable:

organization factors resources designated communicator is lack of sufficient needed but is not available personnel resources Information Inadequate updates of Information integration U availability and information

timing of information:

reliability (e.g., information includes slow information feed perceived by a party or delayed information who fails to inform another party) ambiguous information:

information provided by system is Information is vague, unclear, or does not point confusing or uncertain to the nature of the problem Information is Information quality: U incomplete or logically

missing information: includes masked; masked information information is

misleading information:

unreliablehigh information points to an chance it is misleading incorrect diagnosis or wrong; con"icting information:

conflicts in information information points to more than one possible diagnosis or con"icts with other alarms or indications.

Scenario familiarity Scenario is unfamiliar Nonstandard: anomalous All conditions forcing the operator to account for previous discoveries/

incidents/failures Unpredictable Expectation of alarm/indication dynamics change

expected: given the understanding of current plant status (including systems out 6-18

Table 6-5 Mapping between SACADA Context Factors and IDHEAS-G PIF Structure (continued)

IDHEAS-G PIF structure SACADA context factors Context factor/ Affected PIF Attributes error cause function for maintenance or testing),

the alarm or indication is expected to change not expected: operators do not anticipate this alarm or change in indications Unpredictable Unintuitive plant response: plant E dynamics behavior contradicts intuition.

Multitasking, Multitasking and Work Workload All interruptions, and process

normal: all crew members distractions (Note: Two IDHEAS-G have peer-check and backup PIFs are involved in

concurrent demand: one SACADA Workload crew member has own task factor) with no backup; all others have normal peer-check and backup multiple concurrent demands:

overloaded, no peer-check, everyone has own tasks with no backup Multitasking on Multiple demands: multiple parallel competing demands on attention nonintermingled tasks; and distractions multitasking on intermingled tasks Task complexity Detection overloading Status of alarm board Detect alarm

dark: individual alarm or a group of alarms points to the system problem

busy: the alarm boards show some (but not many) other alarms in addition to the critical alarm overloaded: the alarm boards show many other alarms in addition to the critical alarm Cues for detection are Detection mode: D not obvious, (i.e.,

self-revealing: the detection detection is not is based on one or more self-directly cued by revealing cues.

alarms or instructions);

procedure-directed check:

personnel need to procedure directs operators to check the alarm 6-19

Table 6-5 Mapping between SACADA Context Factors and IDHEAS-G PIF Structure (continued)

IDHEAS-G PIF structure SACADA context factors Context factor/ Affected PIF Attributes error cause function actively search for

procedure-directed information. monitoring: the alarm is in the procedure-speci"ed monitoring list aware/inspection: driven by information obtained earlier Task complexity Multiple causes for Information speci"city: U situation assessment:

speci"c: alarm/alarm Multiple independent pattern/indication(s) point to influences affect the the speci"c system problem system and system behavior cannot be not speci"c: the alarm(s) and/or explained by a single indication(s) do not directly point influence; key to the speci"c system problem, information is which requires operator cognitive cognitively masked. effort to integrate the information and identify the speci"c system problem Multiple, intermingled Uncertainty: DM goals or criteria need

clear: no uncertainty or to be prioritized. competing goalsclear Conflicting goals. decision criteria

uncertain: lack of information or ambiguous decision criteria Decision criteria are ambiguous and competing priorities: multiple subject to different competing goals, foreseeable interpretations. severe consequences Con"icting guidance in Con"icting guidance in policies, DM policies, practices, and practices, and procedures procedures involved in decisionmaking.

Controlled actions that Type of action: E require monitoring of

simple and distinct action outcomes and

order: a sequence of discrete adjusting action actions needs to follow a accordingly certain order monitoring: dynamic control actions that require constant monitoring and manipulation to control and maintain a parameter within a certain boundary Teamwork factors Communication Extent of communication: All required

normal: standard level of three-way communication 6-20

Table 6-5 Mapping between SACADA Context Factors and IDHEAS-G PIF Structure (continued)

IDHEAS-G PIF structure SACADA context factors Context factor/ Affected PIF Attributes error cause function within control room, with occasional onsite communication

extensive onsite: high level of close communication with onsite operators extensive within control room:

high level of close communication within control room Time pressure and High time pressure Time criticality: All stress because of perceived

extensive time lack of adequate time

normal time to complete the task barely adequate time Most SACADA context factors can be mapped to the IDHEAS-G PIF structure. The NRC staff could not map the SACADA factors workload, coordination, and memory demands, because IDHEAS-G does not have one specific PIF for these factors. For example, workload has many traits that affect HEPs differently. Thus, IDHEAS-G models workload in multiple PIFs, such as multitasking, time pressure, staffing, and task complexity. IDHEAS-G also does not model communication level, recoverability, and outcomes of diagnosis because there is no empirical evidence showing the relation between these factors and HEPs. One aspect of the SACADA communication level is complexity of communication contents, and IDHEAS-G models this in the PIF task complexity.

The SACADA taxonomy has definitions of several discrete states of a context factor. Thus, human error rates aggregated for different states of a PIF can inform PIF weight. This will be valuable information to support the IDHEAS-G quantification model. Yet, the SACADA context factors have neutral factors (e.g., skill-rule-knowledge bases) and negative factors (e.g., noisy background). The language used in SACADA for a neutral state (e.g., procedure-directed check) does not mean that the procedure is good. Care is needed when using SACADA data with the context factors in neutral states.

6.4 Summary The CFMs and PIFs of IDHEAS-G can be used as a framework to generalize human error data.

Specifically, the CFMs are in the same framework as the SACADA database; thus, it is relatively straightforward to use SACADA data for the HEP estimation in IDHEAS-G. The human error data are generalized into three tables: the HEP Table, PIF Impact Table, and PIF Interaction Table. The data in these tables inform base HEPs, PIF weights, and PIF interaction factors in the IDHEAS-G quantification model. The data in the PIF Impact Tables can be used to define PIF states and relate these states to the weights of HEPs. Nevertheless, the available human error data are as yet incomplete to inform all the parameters in the IDHEAS-G HEP quantification model (i.e., the base HEPs for all the CFMs at any combination of PIF states and the weights of all PIF states). Expert judgment is still needed to bridge the data gaps. Also, 6-21

more data and analysis are needed to improve the validity of the data-informed HEP quantification model.

6-22

7 GENERAL DISCUSSION AND COMMENTS During the IDHEAS-G development, the NRC staff released two early versions of the IDHEAS-G report to over 20 experts in the HRA technical community for review and comment.

This chapter generalizes the NRC staffs insights and the reviewers comments on enhancements made in IDHEAS-G and questions for future research.

7.1 Areas for Human Reliability Analysis Method Enhancement IDHEAS-G includes the Cognition Model for Human Performance and Reliability and its implementation in HRA. In addition, the NRC staff developed supplemental guidance on using IDHEAS-G. This methodology is intended to enhance HRA methods by addressing the four areas discussed in Section 1.2:

Expand application scope.

Enhance the scientific basis.

Reduce HRA variability.

Enable the use of data for HRA.

IDHEAS-G addressed the basic questions in these areas by integrating the inputs from the broad technical community, adopting the strengths of existing HRA methods, and incorporating state-of-art cognitive and behavioral science. This chapter consolidates the NRC staffs insights and reviewers comments on improvements IDHEAS-G made in these areas. Also, this chapter documents the areas where IDHEAS-G needs further improvement.

As a general methodology, IDHEAS-G focuses on providing the basic principles and process for HRA. Thus, this report does not include techniques, tips, or strategies for tradeoffs between the thoroughness and the required resources for performing an HRA. Some commonly used HRA practices are not included in the general methodology. Included at the end of this chapter is a list of some common HRA practices omitted from IDHEAS-G.

7.1.1 Application Scope IDHEAS-G is broad enough to apply to all nuclear-related HRA applications, while still generating a meaningful HRA. The IDHEAS-G approach as presented in this report is an application-independent process for performing HRA, from which application-specific quantification models can be developed. This approach provides a scientific theoretical basis to guide the HRA process. The approach should reduce the variability that could result in applying the methodology across different applications. The structure of the process is general enough to incorporate data across domains, and the guidance for using the structure (such as the identification of standard CFMs and PIFs) should also contribute to reducing the variability that could result in applying the methodology across different applications. Yet, the kinds of important human actions associated with various applications can be much different from those commonly modeled in internal events PRAs, which primarily model actions directed by EOPs.

The differences may lead to the need for more specific CFMs and PIF attributes in new HRA applications.

7.1.2 Scientific Basis An HRA approach based on cognitive and behavioral science should enable analysts to better model human performance. IDHEAS-G offers a detailed cognition-based approach that is more 7-1

comprehensive than that found in todays HRA methods. A potentially larger set of PIFs expands on what is presented in the HRA state of practice.

Cognitive modeling of human performance has involved concepts of human cognition at various levels. Early cognitive modeling focuses on microcognitive or neural information processing in the human brain (e.g., perception, working memory, attention, learning, psychomotor skill).

More recent cognitive engineering research models human performance with macrocognitive functions. The term macrocognition was created by Cacciabue and Hollnagel [83] to distinguish the systemic approach to modeling cognitive systems from the traditional microcognitive approach. West et al. [84] proposed that a macrocognitive architecture exists in the brain of individuals and enables humans to apply information processing abilities (microcognition) to complex, dynamic, multiagent, real-world tasks (macro-cognition). There is also cognitive modeling for team cognition, shared cognition, or shared mental models, which describes cognitive processes of teams in a sociotechnical context. The cognitive basis structure for IDHEAS-G is a multilevel model, bridging neural information processing (i.e., the cognitive mechanisms) to microcognitive processes. Then, bridging the microcognitive processes to microcognitive functions, and bridging individual-level performance into team-level and interteam or organizational level performance. The cognitive basis structure provides a large yet explicit picture of how personnel perform expected tasks through interaction with systems to achieve the mission of human performance in a complex work environment. Thus, IDHEAS-G is built on a state-of-the-art scientific foundation.

The five macrocognitive functions (i.e., detection, understanding, decisionmaking, action execution, and interteam coordination) provide the cognitive foundation for IDHEAS-G.

The taxonomy in the cognitive basis structure (e.g., the processors and cognitive mechanisms) is similar to those modeling or measuring human performance capabilities in the literature. For example, NUREG/CR-5680, Volume 2 [46], lists the following as human capabilities needed to perform actions: (1) attention, (2) vision, (3) perception, (4) psychomotor skill, (5) manual dexterity, (6) cognitive function (reading, arithmetic, and reasoning), and (7) mood and comfort; the U.S. Army Research Institute has developed operation and mission evaluation software around the following task taxonomy: (1) perception, (2) cognitionnumerical analyses, (3) cognitioninformation processing and problem solving, (4) motor skillfine motor discrete, (5) motor skillfine motor continuous, (5) motor skillgross motor light, (6) motor skillgross motor heavy, (7) communicationsoral, and (8) communicationsreading and writing. The Cognition Model in IDHEAS-G organizes these low-level cognitive processes into the macrocognition structure.

Though it was impossible to be exhaustive, the list of PIFs in IDHEAS-G is comprehensive and represents the state of the art of cognitive and behavioral science.

The structure of organizing the factors is compatible with those in existing HRA methods and the literature. Table 7-1 shows the categorization of PIFs in IDHEAS-G and the literature.

7-2

Table 7-1 Example Categorizations of PIFs Groth & Mosleh Organization- Team-based Person- Situation/Stressor-based Machine-

[85] based based based Moray [86] Organization Team and Individual Physical Ergonomics Physical and Group Behavior Devices Management Behavior Behavior IDHEAS-G Personnel Task System Environment and Situation 7.1.3 Human Reliability Analysis Variability IDHEAS-G improves on HRA theory because it delineates a structured and comprehensive HRA process. The guidance should be capable of producing good transparency and traceability of an HRA process. This reduces analyst subjectivity and enhances analyst-to-analyst consistency. The traceability of every IDHEAS-G step allows analysts to identify sources of variability in the outcomes of the HRA and attempt to reconcile differences in the outcomes.

IDHEAS-G improves on the qualitative analysis process in HRA. The analysis includes scenario analysis, important human action identification and definition, and task analysis with graphic representations and task characterization. These provide an adequate modeling structure for presenting the HRA. These steps are often overlooked in HRA method descriptions and yet have a considerable influence on the overall thoroughness and quality of the final HRA. IDHEAS-G includes these in the guidance and emphasizes their importance and the need for analysts to spend some time on these more qualitative aspects of the HRA before performing any quantification. The very detailed description of the macrocognitive task analysis approach should be a significant step toward reducing inter-analyst variability and subjectivity.

This guidance provides clear instructions on how to identify and describe personnel actions, associated PIFs, and the influence of these PIFs on the overall HEP.

The approach of providing a scientific theoretical basis to guide the process should contribute to reducing the variability that could result in applying the methodology across different applications. Once standard sets of CFMs and PIFs are identified and sufficient data are compiled for these failure modes and factors, the consistency between analysts applying the methodology is likely to increase. The guidance provided on the basic quantification structure (such as the identification of standard failure modes and influencing factors to support the structure) should also contribute to reducing the variability that could result when applying the methodology across different applications.

The breakdown of the macrocognitive functions, CFMs, and associated cognitive mechanisms has the potential to improve how HRA analysts define, describe, and justify HEPs. This approach provides a systematic method for analysts, forces analysts to be more transparent when describing PIFs and their influence on the HEP, and should help to reduce inter-analyst variability. At first glance, this seems to be a complex, and perhaps overly burdensome, approach as the analyst systematically works through the CFMs and PIF structures to select the appropriate information. However, the perceived time burden should be significantly reduced as the analyst becomes more familiar with the approach.

The methodology also has an improved approach to time uncertainty analysis. This is an expansion of the current HRA practice of treating time-critical actions in the structure of the 7-3

model of human performance. The analysis emphasizes identifying time uncertainties and counting the effects of the uncertainties on the overall HEP of the HFE. The approach appears novel but reasonable. It would be more useful to demonstrate how the various time uncertainties can significantly change the HEPs.

Nevertheless, some reviewers are skeptical about reducing variability. They consider IDHEAS-G to be more complex than some other HRA methods, which could introduce more variability. Also, the steps in the IDHEAS-G HRA process still require subjective judgment, and thus subjectivity and variability may exist as in other methods. The methodology needs validation and testing to demonstrate whether IDHEAS-G really reduces variability.

7.1.4 Data in Human Reliability Analysis Perhaps the major contribution that the IDHEAS-G approach can make to improve HRA practice is to open the methodology for incorporating human error data across a number of domains.

The basic quantification structure allows for generalization of data in different domains and at different levels of detail. The structure of the process is general enough to incorporate data across domains. Moreover, the NRC staff has been developing an HRA database drawing from NPP operator simulator data. The database is structured using the same cognitive framework as IDHEAS-G so that the data can be used to support HEP estimation. The availability of data to support the application of IDHEAS-G will significantly increase the feasibility of its use. Given that IDHEAS-G emphasizes context, the compilation of data pertaining to the impact of environmental and organizational factors on personnel actions will be needed. An important asset of the IDHEAS-G methodology is that its structure allows the use of data from other domains besides nuclear power. Over time, as more applicable data are compiled, this asset should become one of the major strengths of the approach.

7.1.5 IDHEAS-G Human Error Probability Quantification Approaches IDHEAS-G proposed the basic concepts for quantifying HEPs. Those are included in the discussions of "Modeling PIF States", "Modeling the Impact of PIF States on HEPs", and "Calculating the HEP of a CFM for a Given Set of PIF States" (discussed in Section 4.4.3.2).

The basic concepts can be applied to any HEP quantification scheme. Stage 3 of the IDHEAS-G HRA process described in Chapter 4 proposed multiple approaches to HEP quantification using these concepts. Furthermore, in a more general interpretation of Equation (4.7), the value for could be the HEP that applies with all PIF states at their "best" (i.e., no impact) values, i.e., it needs not necessarily be the HEP that is derived from only the three "base" PIFs, as in the quantification model. The authors propose the quantification model in Equation (4.7) simply as a conceptual way to numerically combine PIFs and relate them to a set of empirically-derived HEPs. Therefore, careful consideration and testing are needed to develop confidence that it provides results that are qualitatively and quantitatively reasonable, and it facilitates consistent estimates of the risk significance of various human actions and their contributors.

The NRC staff implemented the quantification model in Equation (4.7) in IDHEAS-ECA [78].

7.2 Areas that Need Further Research in IDHEAS-G 7.2.1 Validation of IDHEAS-G HEP Quantification Model The IDHEAS-G Human Error Probability Quantification Model described in Section 4.4.3.2 is based on the assumption that the combined effects of the three "base" PIFs, including "Information Availability and Reliability", "Task Complexity", and "Scenario Familiarity," are the primary determinants for the base HEP. The base HEP is then modified further by the effects 7-4

from a linear combination of numerical weights from the other 17 PIFs. Chapter 6 contains an extensive discussion of a process to develop the data and functional relationships. The functional relations are represented by a scale, which is a group of multiple, discrete states of a PIF attribute. By selecting the appropriate PIF attributes state in the scale, its effect on the HEP is determined. A PIF attribute could have different effects on HEPs for different CFMs.

At a very basic level, this quantification model is founded on the concepts of the IDHEAS cognition model for human performance. The functional construct of that model (i.e., the assumptions that three specific PIFs fundamentally determine a "base" HEP, and that the composite effects from all the other PIFs can be evaluated as a modifier to the "base" HEP) are inferred from the human error data the NRC staff have generalized. This has not been formally tested in practical applications or validated to demonstrate that it provides an appropriate way to account for these influences. In 2019, the NRC performed a pilot study in which a group of HRA analysts applied IDHEAS-ECA method, which employed the quantification model to estimate HEPs of several human actions in Flexible Coping Strategies (FLEX). The analysts commented that the results obtained with the quantification model were reasonable. Next, future research needs to validate the basic assumptions and relational structure of the quantification model through pilot applications that can be benchmarked with empirical human performance data.

7.2.2 Guidance on Combined Effect of Multiple Performance-Influencing Factors The IDHEAS-G documentation acknowledges that there are challenges associated with consideration of multiple PIFs. For most important human actions, more than one PIF could apply. Other than a preliminary exploration in APPENDIX D, IDHEAS-G does not address how the impact of multiple PIFs should be combined (e.g., is the total impact a sum of the impacts of individual factors or does a more complex relationship needs to be considered?). It is possible that combinations of PIFs could have negative synergistic effects. The number of variations of possible PIF combinations could be large. It is not clear whether data are sufficient to compile all important combinations of PIFs effects on HEPs. If the impact of PIFs cannot be derived from data, then the ability to consider multiple factors may not differ much from the way they are considered in current HRA methods. Another problem with having too many PIFs is the insensitivity of individual factors (i.e., the effect of important PIFs may be averaged out by a large number of less important factors). Future research should demonstrate how the effects of multiple PIFs are combined and how to differentiate the effects of different combinations within a large number of PIFs.

7.2.3 Treatment of Errors of Commission A systematic way to identify errors of commission is not provided. The supplemental guidance in APPENDIX F, Identification and Definition of Important Human Actions, describes a range of errors of commission, yet the guidance on errors of commission is geared towards event response actions. Other applications (e.g., spent fuel handling, medical applications) are in a context where personnel are doing more routine functions. Errors of commission become very relevant in these applications. Some traditional hazard analysis methods such as Hazard and Operability Analysis and Failure Modes and Effects Analysis may be useful in identifying errors of commission for those applications. In addition, the ATHEANA method [10], [11] provides guidance to find errors of commission.

7.2.4 Dependency between Important Human Actions The HRA technical community has stressed the need for a new methodology of modeling dependency that is better informed by cognitive and behavioral science. APPENDIX K of this 7-5

report presents this new approach and provides examples of assessing dependency between human action.

The state-of-practice approaches to modeling dependency have limitations when attempting to identify and quantify dependent human performance failures that result from failures of higher-level cognitive and collaborative processes. In particular, the approaches assume that dependencies would not be present (or would be extremely low) if actions involved different people or different locations, or if the tasks were not immediately adjacent to each other. These approaches to identifying dependency fail to cover dependencies that might result from higher-level cognitive failures such as misunderstanding the situation, deciding on the wrong goal, or misprioritizing goals. In those cases, there could be causal dependencies across actions that are performed by different people (e.g., because the supervisor gave incorrect guidance to operators); in different locations (e.g., because operators were trying to achieve the wrong goal in both locations); and also when the tasks are not immediately adjacent to each other (e.g., again, because operators misunderstood the situation and were trying to achieve the wrong goals).

The NRC staff developed a new dependency model to perform dependency analysis based on the IDHEAS-G framework. The new approach is presented in Chapter 4 and elaborated in APPENDIX K. The central concept of the IDHEAS-G dependency model is that the failure of an important human action may modify the context of other important human actions; therefore, change the outcomes of the IDHEAS-G analysis of the affected actions such as time availability, applicable CFMs, and PIF assessment. Subsequently, the HEP of the affected important human action is determined by the changed outcomes. The dependency model requires searching for dependency context between the modeled important actions and applicable dependency type, then calculating the affected HEPs.

The dependency model in IDHEAS-G presents a new perspective of analyzing dependency rather than only relying on the similarity between the human actions in a few high-level factors.

Future research needs to examine the validity of this dependency model and test it in various HRA applications, especially those involved in severe context events in which dependency manifests.

7.2.5 Potential Variability in Application-Specific Quantification Models IDHEAS-G has flexibility in PIF selection to the extent that experts (i.e., HRA method developers) can use the available list to select a subset of PIFs for specific HRA applications.

While guidance on selecting PIFs is provided, it is still possible that different expert groups may select different subsets of factors for the same application. Further research needs to validate this and improve the guidance, as needed. Perhaps having several examples of application-specific PIF subsets is the best way to improve consistency in factor selection. The same concern applies to magnitudes of effects for PIFs. This is left up to expert judgment.

Individual expert sessions may lead to different magnitudes of HEP estimation.

The multiple approaches to quantification can be a concern as different quantification approaches could potentially yield significant differences in estimated HEPs. Thus, future research should perform validation to determine whether decision trees, Bayesian updating, simulation, and expert sessions yield comparable findings.

7-6

7.2.6 Updating the Basic Quantification StructureRefining Cognitive Failure Modes and Performance-Influencing Factors While the IDHEAS-G approach seems general enough to be used in an array of nuclear-related HRA applications, its cognition model is based on current literature and experience in cognitive science, human factors, HRA, and human error analysis. It is possible that there are gaps and biases in the state of knowledge supporting the model. For example, application of the approach to actions outside of the control room, such as level-2 or level-3 PRA actions, may need additional CFMs and PIF attributes that are beyond the current state of knowledge. The context and actions associated with response to external hazards, such as external flooding, can be very different from most of the human event analysis that has been conducted.

Personnel response to external hazards typically includes actions in various harsh work environments, which are not well addressed by current HRA methodologies. Examples of such environmental conditions include accidental aircraft impact, extreme winds, tornado-generated missiles, turbine-generated missiles, external fires, accidents from nearby facilities, release of chemicals, transportation accidents, pipeline accidents, and seismic events.

The IDHEAS-G approach does not adequately model detailed cognitive or neural processes specifically related to psychomotor or neuromotor skills. The concept of motor skills suggests the need for physical ability as much as higher-level cognition. The neuromotor processes required for an action execution may be affected differently from the cognitive part of the same action. For example, an action like driving a forklift would include the physical movements associated with driving, as well as other cognitive functions such as detection and understanding that could be affected differently by an environmental condition such as adverse weather. An application of IDHEAS-G involving modeling fine or strenuous motor actions may require expansion of the cognitive basis related to neuromotor skills.

7.2.7 Definition of Critical Tasks HEP quantification in IDHEAS-G is based on the identification of critical tasks, which are treated as binarya task in an HFE is either critical or noncritical. However, even when the task is critical, the importance of the task for the HFE may vary depending on the scenario. The same is true for the importance of the CFMs to the failure of a macrocognitive functionthe CFMs of a macrocognitive function are treated as equally important to the failure of the function. It is possible that a PIF might have a very important impact on a macrocognitive function or CFM but have only minor significance to the important human action. While IDHEAS-G, like any methodology, had to make simplifications in modeling human performance, future research should explore ways to determine and allocate importance among macrocognitive functions or CFMs, or develop a justification for why such an allocation approach is not necessary.

7.3 Common Human Reliability Analysis Practices Not Included in IDHEAS-G IDHEAS-G is a methodology for what should ideally be performed in HRA, rather than a practical manual for how to efficiently perform HRA and make tradeoffs between the thoroughness and resource demands. Therefore, IDHEAS-G does not include some commonly used HRA practices for those purposes. Below is a list of some common practices left out of the general methodology in this report:

preparation for PRA/HRA [13]

HFE feasibility assessment [63]

HFE screening analysis [7]-[11], [63]

7-7

minimum joint HEPThe NRC does not have official guidance on this.

creditable recovery opportunitiesIDHEAS-G does not provide guidance for quantifying the effects of the preinitiator HFE and sequence-based recovery actions (at the cutset or scenario level), which is allowed by the PRA standard [44] (see High Level Requirement HR-H).

Baseline HEP valuesMany HRA methods provide some base failure rate or guidance on magnitude of effects for PIFs [3], [7]-[9], [12].

The guidelines for these HRA practices can be found in the associated references above.

7-8

8 REFERENCES

[1] U.S. Nuclear Regulatory Commission, Use of Probabilistic Risk Assessment Methods in Nuclear Regulatory Activities; Final Policy Statement, U.S. Nuclear Regulatory Commission, Federal Register, Vol. 60, p. 42622 (60 FR 42622), Aug. 1995.

[2] S. Kaplan and B. J. Garrick, On The Quantitative Definition of Risk, Risk Anal., vol. 1, no. 1, pp. 11-27, Mar. 1981.

[3] A. D. Swain and H. E. Guttmann, Handbook of Human Reliability Analysis with Emphasis on Nuclear Power Plant Applications, U.S. Nuclear Regulatory Commission, NUREG/CR-1278 (ADAMS Accession No. ML071210299), Aug. 1983.

[4] A. D. Swain, Accident Sequence Evaluation Program Human Reliability Analysis Procedure, U.S. Nuclear Regulatory Commission, NUREG/CR-4772, Feb. 1987.

[5] D. E. Embrey, P. Humphreys, E. A. Rosa, B. Kirwan, and K. Rea, SLIM-MAUD: An Approach to Assessing Human Error Probabilities Using Structured Expert Judgment.

Volume I: Overview of SLIM-MAUD, U.S. Nuclear Regulatory Commission, NUREG/CR-3518, Vol. I, Mar. 1984.

[6] D. E. Embrey, P. Humphreys, E. A. Rosa, B. Kirwan, and K. Rea, SLIM-MAUD: An Approach to Assessing Human Error Probabilities Using Structured Expert Judgment.

Volume II: Detailed Analysis of the Technical Issues, U.S. Nuclear Regulatory Commission, NUREG/CR-3518, Vol. II, Jul. 1984.

[7] D. Gertman, H. Blackman, J. Marble, J. Byers, and C. Smith, The SPAR-H Human Reliability Analysis Method, U.S. Nuclear Regulatory Commission, NUREG/CR-6883 (ADAMS Accession No. ML051950061), Aug. 2005.

[8] R. L. Boring and H. S. Blackman, The Origins of the SPAR-H Methods Performance Shaping Factor Multipliers, in 2007 IEEE 8th Human Factors and Power Plants and HPRCT 13th Annual Meeting, Monterey, CA, USA, 2007, pp. 177-184.

[9] A. M. Whaley, D. L. Kelly, R. L. Boring, and W. J. Galyean, SPAR-H Step-by-Step Guidance, Idaho National Laboratory, INL/EXT-10-18533, Rev. 2 (ADAMS Accession No. ML112060305), May 2011.

[10] M. Barriere, D. Bley, S. Cooper, J. Forester, A. Kolaczkowski, W. Luckas, G. Parry, A.

Ramey-Smith, C. Thompson, D. Whitehead, and J. Wreathall, Technical Basis and Implementation Guidelines for A Technique for Human Event Analysis (ATHEANA),

U.S. Nuclear Regulatory Commission, NUREG-1624, Rev. 1 (ADAMS Package No.

ML003736288), May 2000.

[11] J. Forester, A. Kolaczkowski, S. Cooper, D. Bley, and E. Lois, ATHEANA Users Guide, U.S. Nuclear Regulatory Commission, NUREG-1880 (ADAMS Accession No. ML072130359), Jun. 2007.

[12] G. W. Parry, A. J. Spurgin, P. Moieni, and A. Beare, An Approach to the Analysis of Operator Actions in Probabilistic Risk Assessment, Electric Power Research Institute, TR-100259, Jun. 1992.

8-1

[13] A. Kolaczkowski, J. Forester, E. Lois, and S. Cooper, Good Practices for Implementing Human Reliability Analysis, U.S. Nuclear Regulatory Commission, NUREG-1792 (ADAMS Accession No. ML051160213), Apr. 2005.

[14] J. Forester, A. Kolaczkowski, E. Lois, and D. Kelly, Evaluation of Human Reliability Analysis Methods Against Good Practices, U.S. Nuclear Regulatory Commission, NUREG-1842 (ADAMS Accession No. ML063200058), Sep. 2006.

[15] U.S. Nuclear Regulatory Commission, Staff Requirements - Meeting with Advisory Committee on Reactor Safeguards, 2:30 p.m., Friday, October 20, 2006, Commissioners Conference Room, One White Flint North, Rockville, Maryland (Open to Public Attendance), U.S. Nuclear Regulatory Commission, SRM M061020 (ADAMS Accession No. ML063120582), Nov. 2006.

[16] E. Lois, V. N. Dang, J. Forester, H. Broberg, S. Massaiu, M. Hildebrandt, P. Ø. Braarud, G. Parry, J. Julius, R. Boring, I. Mnnist, and A. Bye, International HRA Empirical Study - Phase 1 Report: Description of Overall Approach and Pilot Phase Results from Comparing HRA Methods to Simulator Performance Data, U.S. Nuclear Regulatory Commission, NUREG/IA-0216, Vol. 1 (ADAMS Accession No. ML093380283), Nov.

2009.

[17] A. Bye, E. Lois, V. N. Dang, G. Parry, J. Forester, S. Massaiu, R. Boring, P. Ø. Braarud, H. Broberg, J. Julius, I. Mnnist, and P. Nelson, International HRA Empirical Study -

Phase 2 Report: Results from Comparing HRA Method Predictions to Simulator Data from SGTR Scenarios, U.S. Nuclear Regulatory Commission, NUREG/IA-0216, Vol. 2 (ADAMS Accession No. ML11250A010), Aug. 2011.

[18] V. N. Dang, J. Forester, R. Boring, H. Broberg, S. Massaiu, J. Julius, I. Mnnist, H.

Liao, P. Nelson, E. Lois, and A. Bye, International HRA Empirical Study - Phase 3 Report: Results from Comparing HRA Methods Predictions to HAMMLAB Simulator Data on LOFW Scenarios, U.S. Nuclear Regulatory Commission, NUREG/IA-0216, Vol.

3 (ADAMS Accession No. ML14358A254), Dec. 2014.

[19] J. Forester, V. N. Dang, A. Bye, E. Lois, S. Massaiu, H. Broberg, P. Ø. Braarud, R.

Boring, I. Mnnist, H. Liao, J. Julius, G. Parry, and P. Nelson, The International HRA Empirical Study: Lessons Learned from Comparing HRA Methods Predictions to HAMMLAB Simulator Data, U.S. Nuclear Regulatory Commission, NUREG-2127 (ADAMS Accession No. ML14227A197), Aug. 2014.

[20] J. Forester, H. Liao, V. N. Dang, A. Bye, E. Lois, M. Presley, J. Marble, R. Nowell, H.

Broberg, M. Hildebrandt, B. Hallbert, and T. Morgan, The U.S. HRA Empirical Study -

Assessment of HRA Method Predictions against Operating Crew Performance on a U.S.

Nuclear Power Plant Simulator, U.S. Nuclear Regulatory Commission, NUREG-2156 (ADAMS Accession No. ML16179A124), Jun. 2016.

[21] A. M. Whaley, J. Xing, R. L. Boring, S. M. L. Hendrickson, J. C. Joe, K. L. Le Blanc, and S. L. Morrow, Cognitive Basis for Human Reliability Analysis, U.S. Nuclear Regulatory Commission, NUREG-2114 (ADAMS Accession No. ML16014A045), Jan. 2016.

[22] J. Xing, G. Parry, M. Presley, J. Forester, S. Hendrickson, and V. Dang, An Integrated Human Event Analysis System (IDHEAS) for Nuclear Power Plant Internal Events At-8-2

Power Application, U.S. Nuclear Regulatory Commission and Electric Power Research Institute, NUREG-2199, Vol. 1 (ADAMS Accession No. ML17073A041), Mar. 2017.

[23] U.S. Nuclear Regulatory Commission, Staff Requirements - Briefing on Risk-Informed, Performance-Based Regulation, 1:30 p.m., Wednesday, February 4, 2009, Commissioners Conference Room, One White Flint North, Rockville, Maryland (Open to Public Attendance), U.S. Nuclear Regulatory Commission, SRM M090204B (ADAMS Accession No. ML090490812), Feb. 2009.

[24] Y. J. Chang, D. Bley, L. Criscione, B. Kirwan, A. Mosleh, T. Madary, R. Nowell, R.

Richards, E. M. Roth, S. Sieben, and A. Zoulis, The SACADA database for human reliability and human performance, Reliab. Eng. Syst. Saf., vol. 125, pp. 117-133, May 2014.

[25] N. Cowan, Working Memory Capacity, 1st ed. New York: Routledge, 2016.

[26] N. Unsworth, T. S. Redick, R. P. Heitz, J. M. Broadway, and R. W. Engle, Complex working memory span tasks and higher-order cognition: A latent-variable analysis of the relationship between processing and storage, Memory, vol. 17, no. 6, pp. 635-654, Aug. 2009.

[27] M. J. Kane and R. W. Engle, Working-memory capacity and the control of attention: the contributions of goal neglect, response competition, and task set to Stroop interference, J. Exp. Psychol. Gen., vol. 132, no. 1, pp. 47-70, Mar. 2003.

[28] G. Klein, J. K. Phillips, E. L. Rall, and D. A. Peluso, A data-frame theory of sensemaking., in Expertise out of context: Proceedings of the Sixth International Conference on Naturalistic Decision Making., Mahwah, NJ, USA: Lawrence Erlbaum Associates Publishers, 2007, pp. 113-155.

[29] G. A. Klein, J. Orasanu, R. Calderwood, and C. E. Zsambok, Eds., Decision making in action: Models and methods. Westport, CT, US: Ablex Publishing, 1993.

[30] G. Klein, Naturalistic Decision Making, Hum. Factors J. Hum. Factors Ergon. Soc., vol.

50, no. 3, pp. 456-460, Jun. 2008.

[31] R. Lipshitz, G. Klein, J. Orasanu, and E. Salas, Taking stock of naturalistic decision making, J. Behav. Decis. Mak., vol. 14, no. 5, pp. 331-352, Dec. 2001.

[32] P. A. Simpson, Naturalistic Decision Making in Aviation Environments, DSTO Aeronautical and Maritime Research Laboratory, Fishermand Bend, Victoria, Australia, DSTO-GD-0279, Jan. 2001.

[33] T. Elliott, Expert Decision-Making in Naturalistic Environments: A Summary of Research, DSTO Systems Sciences Laboratory, Edinburgh, South Australia, Australia, DSTO-GD-0429, Mar. 2005.

[34] J. S. Carroll, S. Hatakenaka, and J. W. Rudolph, Naturalistic Decision Making and Organizational Learning in Nuclear Power Plants: Negotiating Meaning Between Managers and Problem Investigation Teams, Organ. Stud., vol. 27, no. 7, pp. 1037-1057, Jul. 2006.

8-3

[35] C. E. Zsambok, Naturalistic Decision Making: Where Are We Now?, in Naturalistic Decision Making, C. E. Zsambok and G. Klein, Eds. Mahwah, NJ, US: Lawrence Erlbaum Associates, Inc., 1997, pp. 3-16.

[36] F. L. Greitzer, R. Podmore, M. Robinson, and P. Ey, Naturalistic Decision Making for Power System Operators, Int. J. Hum.-Comput. Interact., vol. 26, no. 2-3, pp. 278-291, Mar. 2010.

[37] P. C. Cacciabue, G. Mancini, and U. Bersini, A model of operator behaviour for man-machine system simulation, Automatica, vol. 26, no. 6, pp. 1025-1034, Nov. 1990.

[38] E. D. Murphy and C. M. Mitchell, Cognitive attributes: implications for display design in supervisory control systems, Int. J. Man-Mach. Stud., vol. 25, no. 4, pp. 411-438, Oct.

1986.

[39] E. M. Roth, Analysis of Decision Making in Nuclear Power Plant Emergencies: An Investigation of Aided Decision Making, in Naturalistic Decision Making, C. E. Zsambok and G. Klein, Eds. Mahwah, NJ, USA: Lawrence Erlbaum Associates, Inc., 1997, pp.

175-182.

[40] D. Kahneman, Thinking, Fast and Slow. New York, NY: Farrar, Straus, and Giroux, 2011.

[41] P. Slovic, B. Fischhoff, and S. Lichtenstein, Facts versus fears: Understanding perceived risks, in Judgment under uncertainty: Heuristics and biases, D. Kahneman, P.

Slovic, and A. Tversky, Eds. Cambridge: Cambridge University Press, 1982, pp. 463-492.

[42] P. M. Fitts, The information capacity of the human motor system in controlling the amplitude of movement., J. Exp. Psychol., vol. 47, no. 6, pp. 381-391, 1954.

[43] R. A. Andersen and C. A. Buneo, Intentional Maps in Posterior Parietal Cortex, Annu.

Rev. Neurosci., vol. 25, no. 1, pp. 189-220, Mar. 2002.

[44] American Society of Mechanical Engineers and American Nuclear Society, Addenda to ASME/ANS RA-S-2008 Standard for Level 1/Large Early Release Frequency Probabilistic Risk Assessment for Nuclear Power Plant Applications, ASME/ANS RA-Sa-2009, Feb. 2009.

[45] D. Echeverria, V. Barnes, A. Bittner, N. Durbin, J. Fawcett-Long, C. Moore, A. Slavich, B. Terrill, C. Westra, D. Wieringa, R. Wilson, D. Draper, D. Morisseau, and J. Persensky, The Impact of Environmental Conditions on Human Performance: A Handbook of Environmental Exposures, U.S. Nuclear Regulatory Commission, NUREG/CR-5680, Vol. 1 (ADAMS Accession No. ML070460030), Sep. 1994.

[46] D. Echeverria, V. Barnes, A. Bittner, N. Durbin, J. Fawcett-Long, C. Moore, A. Slavich, B. Terrill, C. Westra, D. Wieringa, R. Wilson, D. Draper, D. Morisseau, and J. Persensky, The Impact of Environmental Conditions on Human Performance: A Critical Review of the Literature, U.S. Nuclear Regulatory Commission, NUREG/CR-5680, Vol. 2 (ADAMS Accession No. ML071210164), Sep. 1994.

8-4

[47] C. K. W. De Dreu and L. R. Weingart, Task versus relationship conflict, team performance, and team member satisfaction: A meta-analysis., J. Appl. Psychol., vol.

88, no. 4, pp. 741-749, 2003.

[48] W. J. Horrey and C. D. Wickens, Examining the Impact of Cell Phone Conversations on Driving Using Meta-Analytic Techniques, Hum. Factors J. Hum. Factors Ergon. Soc.,

vol. 48, no. 1, pp. 196-205, Mar. 2006.

[49] P. A. Hancock, J. M. Ross, and J. L. Szalma, A Meta-Analysis of Performance Response Under Thermal Stressors, Hum. Factors J. Hum. Factors Ergon. Soc., vol.

49, no. 5, pp. 851-877, Oct. 2007.

[50] E. Salas, D. DiazGranados, C. Klein, C. S. Burke, K. C. Stagl, G. F. Goodwin, and S. M.

Halpin, Does Team Training Improve Team Performance? A Meta-Analysis, Hum.

Factors J. Hum. Factors Ergon. Soc., vol. 50, no. 6, pp. 903-933, Dec. 2008.

[51] M. Drouin, M. Gonzalez, S. Herrick, J. S. Hyslop, D. Stroup, J. Lehner, T. Pratt, M.

Dennis, J. LaChance, and T. Wheeler, Glossary of Risk-Related Terms in Support of Risk-Informed Decisionmaking, U.S. Nuclear Regulatory Commission, NUREG-2122 (ADAMS Accession No. ML13311A353), Nov. 2013.

[52] R. J. Budnitz, G. Apostolakis, D. M. Boore, L. S. Cluff, K. J. Coppersmith, C. A. Cornell, and P. A. Morris, Recommendations for Probabilistic Seismic Hazard Analysis:

Guidance on Uncertainty and Use of Experts, Main Report, U.S. Nuclear Regulatory Commission, NUREG/CR-6372, Vol. 1 (ADAMS Accession No. ML080090003), Apr.

1997.

[53] R. J. Budnitz, G. Apostolakis, D. M. Boore, L. S. Cluff, K. J. Coppersmith, C. A. Cornell, and P. A. Morris, Recommendations for Probabilistic Seismic Hazard Analysis:

Guidance on Uncertainty and Use of Experts, Appendices, U.S. Nuclear Regulatory Commission, NUREG/CR-6372, Vol. 2 (ADAMS Accession No. ML080090004), Apr.

1997.

[54] D. M. Green and J. A. Swets, Signal detection theory and psychophysics, Reprint Edition. Los Altos, California: Peninsula Publishing, 1988.

[55] J. Xing, Y. J. Chang, and J. DeJesus Segarra, Integrated Human Event Analysis System for Human Reliability Data (IDHEAS-DATA) Draft Report, U.S. Nuclear Regulatory Commission, ADAMS Accession No. ML20238B982, Aug. 2020.

[56] J. Xing and Y. J. Chang, Use of IDHEAS General Methodology to Incorporate Human Performance Data for Estimation of Human Error Probabilities, presented at the 14th International Conference on Probabilistic Safety Assessment and Management (PSAM 14), Los Angeles, CA, USA, 2018.

[57] D. C. Bley, D. R. Buttemer, and J. W. Stetkar, Light water reactor sequence timing: its significance to probabilistic safety assessment modeling, Reliab. Eng. Syst. Saf., vol.

22, no. 1-4, pp. 27-60, Jan. 1988.

[58] O. Svenson and A. J. Maule, Eds., Time Pressure and Stress in Human Judgment and Decision Making. Boston, MA: Springer US, 1993.

8-5

[59] T. M. Brown and C. E. Miller, Communication Networks in Task-Performing Groups:

Effects of Task Complexity, Time Pressure, and Interpersonal Dominance, Small Group Res., vol. 31, no. 2, pp. 131-157, Apr. 2000.

[60] H. Topi, J. S. Valacich, and J. A. Hoffer, The effects of task complexity and time availability limitations on human performance in database query tasks, Int. J. Hum.-

Comput. Stud., vol. 62, no. 3, pp. 349-379, Mar. 2005.

[61] M. A. DeDonno, Time Pressure and Decision Making, Doctoral dissertation, Case Western Reserve University, 2009.

[62] A. Kolaczkowski, J. Forester, R. Gallucci, A. Klein, J. Bongarra, P. Qualls, and P.

Barbadora, Demonstrating the Feasibility and Reliability of Operator Manual Actions in Response to Fire, U.S. Nuclear Regulatory Commission, NUREG-1852 (ADAMS Accession No. ML073020676), Oct. 2007.

[63] S. Lewis, S. Cooper, K. Hill, J. Julius, J. Grobbelaar, K. Kohlhepp, J. Forester, S.

Hendrickson, E. Collins, B. Hannaman, and M. Presley, EPRI/NRC-RES Fire Human Reliability Analysis Guidelines, Electric Power Research Institute and U.S. Nuclear Regulatory Commission, EPRI 1023001/NUREG-1921 (ADAMS Accession No. ML12216A104), Jul. 2012.

[64] I. R. Newby-Clark, M. Ross, R. Buehler, D. J. Koehler, and D. Griffin, People focus on optimistic scenarios and disregard pessimistic scenarios while predicting task completion times., J. Exp. Psychol. Appl., vol. 6, no. 3, pp. 171-182, 2000.

[65] M. M. Roy and N. J. S. Christenfeld, Bias in memory predicts bias in estimation of future task duration, Mem. Cognit., vol. 35, no. 3, pp. 557-564, Apr. 2007.

[66] G. Zauberman, B. K. Kim, S. A. Malkoc, and J. R. Bettman, Discounting Time and Time Discounting: Subjective Time Perception and Intertemporal Preferences, J. Mark. Res.,

vol. 46, no. 4, pp. 543-556, Aug. 2009.

[67] D. Kahneman, P. Slovic, and A. Tversky, Eds., Judgment under uncertainty: Heuristics and biases, 24. printing. Cambridge: Cambridge Univ. Press, 1982.

[68] R. A. Josephs and E. D. Hahn, Bias and Accuracy in Estimates of Task Duration, Organ. Behav. Hum. Decis. Process., vol. 61, no. 2, pp. 202-213, Feb. 1995.

[69] W. Jung, J. Park, Y. Kim, S. Y. Choi, and S. Kim, HuREX - A Framework of HRA Data Collection from Simulators in Nuclear Power Plants, Reliab. Eng. Syst. Saf., Jul. 2018.

[70] W. Preischl and M. Hellmich, Human error probabilities from operational experience of German nuclear power plants, Reliab. Eng. Syst. Saf., vol. 109, pp. 150-159, Jan.

2013.

[71] W. Preischl and M. Hellmich, Human error probabilities from operational experience of German nuclear power plants, Part II, Reliab. Eng. Syst. Saf., vol. 148, pp. 44-56, Apr.

2016.

8-6

[72] J. OHara, C. Ruger, J. Higgins, W. Luckas, and D. Crouch, An Evaluation of the Effects of Local Control Station Design Configurations on Human Performance and Nuclear Power Plant Risk, U.S. Nuclear Regulatory Commission, NUREG/CR-5572, Sep. 1990.

[73] O. V. Prinzo, A. M. Hendrix, and R. Hendrix, The Outcome of [Air Traffic Control]

Message Complexity on Pilot Readback Performance, Federal Aviation Administration, DOT/FAA/AM-06-25, Nov. 2006.

[74] C. J. D. Patten, A. Kircher, J. stlund, L. Nilsson, and O. Svenson, Driver experience and cognitive workload in different traffic environments, Accid. Anal. Prev., vol. 38, no.

5, pp. 887-894, Sep. 2006.

[75] J. Xing, Y. J. Chang, and N. Siu, Insights on human error probability from cognitive experiment literature, presented at the 2015 International Topical Meeting on Probabilistic Safety Assessment and Analysis (PSA 2015), Sun Valley, ID, USA, 2015.

[76] D. Virovac, A. Domitrovi, and E. Bazijanac, The Influence of Human Factor in Aircraft Maintenance, PROMET - Traffic Transp., vol. 29, no. 3, pp. 257-266, Jun. 2017.

[77] M. Kyriakidis, K. T. Pak, and A. Majumdar, Railway Accidents Caused by Human Error:

Historic Analysis of UK Railways, 1945 to 2012, Transp. Res. Rec. J. Transp. Res.

Board, vol. 2476, no. 1, pp. 126-136, Jan. 2015.

[78] J. Xing, J. Chang, and J. DeJesus, Integrated Human Event Analysis System for Event and Condition Assessment (IDHEAS-ECA), U.S. Nuclear Regulatory Commission, RIL-2020-02 (ADAMS Accession No. ML20016A481), Feb. 2020.

[79] W. F. Grether, Effects on Human Performance of Combined Environmental Stress, Aerospace Medical Research Laboratory, Wright-Patterson Air Force Base, Ohio, AMRL-TR-70-68, Sep. 1970.

[80] P. A. Hancock and J. O. Pierce, Combined Effects of Heat and Noise on Human Performance: A Review, Am. Ind. Hyg. Assoc. J., vol. 46, no. 10, pp. 555-566, Oct.

1985.

[81] C. H. Van Iddekinge, H. Aguinis, J. D. Mackey, and P. S. DeOrtentiis, A Meta-Analysis of the Interactive, Additive, and Relative Effects of Cognitive Ability and Motivation on Performance, J. Manag., vol. 44, no. 1, pp. 249-279, Jan. 2018.

[82] E. M. Altmann, J. G. Trafton, and D. Z. Hambrick, Effects of interruption length on procedural errors., J. Exp. Psychol. Appl., vol. 23, no. 2, pp. 216-229, Jun. 2017.

[83] P. C. Cacciabue and E. Hollnagel, Simulation of Cognition: Applications, in Expertise and Technology: Cognition & Human-Computer Cooperation, J.-M. Hoc, P. C.

Cacciabue, and E. Hollnagel, Eds. Hillsdale, New Jersey: Lawrence Erlbaum Associates, Inc., 1995, pp. 55-73.

[84] R. West, E. Hancock, S. Somers, K. MacDougall, and F. Jeanson, The Macro Architecture Hypothesis: Applications to Modeling Teamwork, Conflict Resolution, and Literary Analysis, in Proceedings of the 12th International Conference on Cognitive Modeling, Ottawa, Canada, 2013, pp. 427-432.

8-7

[85] K. M. Groth and A. Mosleh, A data-informed PIF hierarchy for model-based Human Reliability Analysis, Reliab. Eng. Syst. Saf., vol. 108, pp. 154-174, Dec. 2012.

[86] N. Moray, Human Factors in Process Control, in Handbook of Human Factors and Ergonimics, 2nd ed., G. Salvendy, Ed. New York, NY: John Wiley & Sons, Inc., 1997, pp. 1944-1971.

[87] P. Meyer, P. Le Bot, and H. Pesme, MERMOS: An extended second generation HRA method, in 2007 IEEE 8th Human Factors and Power Plants and HPRCT 13th Annual Meeting, Monterey, CA, USA, 2007, pp. 276-283.

[88] N. B. Sarter and B. Schroeder, Supporting Decision Making and Action Selection under Time Pressure and Uncertainty: The Case of In-Flight Icing, Hum. Factors J. Hum.

Factors Ergon. Soc., vol. 43, no. 4, pp. 573-583, Dec. 2001.

[89] M. L. Cummings and C. Tsonis, Partitioning Complexity in Air Traffic Management Tasks, Int. J. Aviat. Psychol., vol. 16, no. 3, pp. 277-295, Jul. 2006.

[90] J. V. Baranski, M. M. Thompson, F. M. J. Lichacz, C. McCann, V. Gil, L. Past, and R.

A. Pigeau, Effects of Sleep Loss on Team Decision Making: Motivational Loss or Motivational Gain?, Hum. Factors J. Hum. Factors Ergon. Soc., vol. 49, no. 4, pp. 646-660, Aug. 2007.

[91] E. H. McKinney and K. J. Davis, Effects of Deliberate Practice on Crisis Decision Performance, Hum. Factors J. Hum. Factors Ergon. Soc., vol. 45, no. 3, pp. 436-444, Sep. 2003.

[92] F. A. Drews, M. Pasupathi, and D. L. Strayer, Passenger and Cell Phone Conversations in Simulated Driving, J. Exp. Psychol. Appl., vol. 14, no. 4, pp. 392-400, 2008.

[93] J. D. Lee, D. V. McGehee, T. L. Brown, and M. L. Reyes, Collision Warning Timing, Driver Distraction, and Driver Response to Imminent Rear-End Collisions in a High-Fidelity Driving Simulator, Hum. Factors J. Hum. Factors Ergon. Soc., vol. 44, no. 2, pp.

314-334, Jun. 2002.

[94] B. P. Bailey and J. A. Konstan, On the need for attention-aware systems: Measuring effects of interruption on task performance, error rate, and affective state, Comput.

Hum. Behav., vol. 22, no. 4, pp. 685-708, Jul. 2006.

[95] M. L. Cummings, R. M. Kilgore, E. Wang, L. Tijerina, and D. S. Kochhar, Effects of Single Versus Multiple Warnings on Driver Performance, Hum. Factors J. Hum. Factors Ergon. Soc., vol. 49, no. 6, pp. 1097-1106, Dec. 2007.

[96] S. Hameed, T. Ferris, S. Jayaraman, and N. Sarter, Using Informative Peripheral Visual and Tactile Cues to Support Task and Interruption Management, Hum. Factors J. Hum.

Factors Ergon. Soc., vol. 51, no. 2, pp. 126-135, Apr. 2009.

[97] C. Speier, I. Vessey, and J. S. Valacich, The Effects of Interruptions, Task Complexity, and Information Presentation on Computer-Supported Decision-Making Performance, Decis. Sci., vol. 34, no. 4, pp. 771-797, Nov. 2003.

8-8

[98] G. Keinan, Decision making under stress: scanning of alternatives under controllable and uncontrollable threats, J. Pers. Soc. Psychol., vol. 52, no. 3, pp. 639-644, Mar.

1987.

[99] P. Liu and J. Liu, Combined Effect of Multiple Performance Shaping Factors on Human Reliability: Multiplicative or Additive?, Int. J. Human-Computer Interact., vol. 36, no. 9, pp. 828-838, May 2020.

[100] M. K. Mount, M. R. Barrick, and J. P. Strauss, The Joint Relationship of Conscientiousness and Ability with Performance: Test of the Interaction Hypothesis, J.

Manag., vol. 25, no. 5, pp. 707-721, Oct. 1999.

[101] R. Murray and M. McCally, Combined environmental stresses, Feb. 1973.

[102] U.S. Nuclear Regulatory Commission, Final Safety Culture Policy Statement, U.S.

Nuclear Regulatory Commission, Federal Register, Vol. 76, p. 34773 (76 FR 34773),

Jun. 2011.

[103] American Nuclear Society and Institute of Electrical and Electronics Engineers, PRA Procedures Guide: A Guide to the Performance of Probabilistic Risk Assessment for Nuclear Power Plants, U.S. Nuclear Regulatory Commission, NUREG/CR-2300 (ADAMS Accession Nos. ML063560439 and ML063560440), Jan. 1983.

[104] U.S. Nuclear Regulatory Commission, H. B. Robinson Steam Electric Plant -

Augmented Inspection Team Report 05000261/2010009, U.S. Nuclear Regulatory Commission, IR 05000261/2010009 (ADAMS Accession No. ML101830101), Jul. 2010.

[105] U.S. Nuclear Regulatory Commission, Standard Review Plan for Fuel Cycle Facilities License Applications, U.S. Nuclear Regulatory Commission, NUREG-1520, Rev. 2 (ADAMS Accession No. ML15176A258), Jun. 2015.

[106] U.S. Nuclear Regulatory Commission, H. B. Robinson Steam Electric Plant, Unit 2 -

Final Accident Sequence Precursor Analyses, U.S. Nuclear Regulatory Commission, (ADAMS Accession No. ML11271A139), Sep. 2011.

[107] S. Swaminathan and C. Smidts, The Event Sequence Diagram framework for dynamic Probabilistic Risk Assessment, Reliab. Eng. Syst. Saf., vol. 63, no. 1, pp. 73-90, Jan.

1999.

[108] U.S. Nuclear Regulatory Commission, Reactor Safety Study: An Assessment of Accident Risks in U.S. Commercial Nuclear Power Plants, U.S. Nuclear Regulatory Commission, NUREG-75/014 (WASH-1400), Oct. 1975.

[109] D. J. Wakefield, G. W. Parry, G. W. Hannaman, and A. J. Spurgin, SHARP1-A Revised Systematic Human Action Reliability Procedure, Electric Power Research Institute, TR-101711, Tier 1, Dec. 1992.

[110] D. J. Wakefield, G. W. Parry, G. W. Hannaman, A. J. Spurgin, and P. Moieni, SHARP1-A Revised Systematic Human Action Reliability Procedure, Electric Power Research Institute, TR-101711, Tier 2, Dec. 1992.

8-9

[111] C. Taylor, Improving Scenario Analysis for HRA: Handbook of Good Practices, Organisation for Economic Co-operation and Development, Halden Reactor Project, HWR-1145, 2015.

[112] C. Taylor and P. Le Darz, Improving HRA Practices: Results from the Pilot Case Study, Organisation for Economic Co-operation and Development, Halden Reactor Project, HWR-1178, 2016.

[113] R. I. Milstein, Integrated Safety Analysis Guidance Document, U.S. Nuclear Regulatory Commission, NUREG-1513 (ADAMS Accession No. ML011440260), May 2001.

[114] M. Stamatelatos, H. Dezfuli, G. Apostolakis, C. Everline, S. Guarro, D. Mathias, A.

Mosleh, T. Paulos, D. Riha, C. Smith, W. Vesely, and R. Youngblood, Probabilistic Risk Assessment Procedures Guide for NASA Managers and Practitioners, National Aeronautics and Space Administration, NASA/SP-2011-3421, Dec. 2011.

[115] J. Grobbelaar and K. Gunter, Data and Modeling of Pre-Initiator Human Failure Events in Probabilistic Risk Assessment, Electric Power Research Institute, 3002008094, Mar.

2017.

[116] J. Julius and J. Schroeder, Support System Initiating Events: Identification and Quantification Guideline, Electric Power Research Institute and U.S. Nuclear Regulatory Commission, EPRI 1016741, Dec. 2008.

[117] B. Kirwan and L. K. Ainsworth, Eds., A Guide To Task Analysis: The Task Analysis Working Group. London: Taylor & Francis, 1992.

[118] P. Le Bot, J. Alengry, and C. De La Garza, Organising the Operation of Nuclear Reactors in Extreme Situations: Simulator Based-Test Methodology, in Proceedings of the 39th Enlarged Halden Programme Group (EHPG) Meeting, Fornebu, Norway, 2016.

[119] G. Taylor, S. Cooper, A. DAgostino, N. Melly, and T. Cleary, Determining the Effectiveness, Limitations, and Operator Response for Very Early Warning Fire Detection Systems in Nuclear Facilities (DELORES-VEWFIRE), U.S. Nuclear Regulatory Commission, NUREG-2180 (ADAMS Accession No. ML16343A058), Dec.

2016.

[120] C. E. Rossi, J. T. Beard, T. L. Bell, and W. D. Lanning, Loss of Main and Auxiliary Feedwater Event at the Davis-Besse Plant on June 9, 1985, U.S. Nuclear Regulatory Commission, NUREG-1154, Jul. 1985.

[121] D. Kelly and C. Smith, Bayesian Inference for Probabilistic Risk Assessment: A Practitioners Guidebook. London: Springer, 2011.

[122] M. A. Meyer and J. M. Booker, Eliciting and Analyzing Expert Judgment: A Practical Guide, U.S. Nuclear Regulatory Commission, NUREG/CR-5424, Jan. 1990.

[123] U.S. Nuclear Regulatory Commission, Staff Requirements - COMGEA-11-0001 -

Utilization of Expert Judgment in Regulatory Decision Making, U.S. Nuclear Regulatory Commission, SRM-COMGEA-11-0001 (ADAMS Accession No. ML110740304), Mar.

2011.

8-10

[124] U.S. Nuclear Regulatory Commission, Response to Staff Requirements Memorandum COMGEA-11-0001, Utilization of Expert Judgment in Regulatory Decision Making, U.S. Nuclear Regulatory Commission, SECY-11-0172 (ADAMS Accession No. ML112510091), Dec. 2011.

[125] U.S. Nuclear Regulatory Commission, Staff Requirements - SECY-11-0172 -

Response to Staff Requirements Memorandum COMGEA-11-0001, Utilization of Expert Judgment in Regulatory Decision Making, U.S. Nuclear Regulatory Commission, SRM-SECY-11-0172 (ADAMS Accession No. ML120380251), Feb. 2012.

[126] International Atomic Energy Agency, The radiological accident in the reprocessing plant at Tomsk. Vienna: International Atomic Energy Agency, 1998.

[127] U.S. Nuclear Regulatory Commission, Evaluation of the Containment Protection and Release Reduction for Mark I and Mark II Boiling Water Reactors Rulemaking Activities (10 CFR Part 50) (RIN-3150-AJ26), U.S. Nuclear Regulatory Commission, SECY 0085 (ADAMS Accession No. ML15022A218), Jun. 2015.

[128] M. epin, Comparison of Methods for Dependency Determination between Human Failure Events within Human Reliability Analysis, Sci. Technol. Nucl. Install., vol. 2008, pp. 1-7, 2008.

[129] C. Alós-Ferrer, S. Hügelschfer, and J. Li, Inertia and Decision Making, Front.

Psychol., vol. 7, Feb. 2016.

[130] G. Parry, Establishing Minimum Acceptable Values for Probabilities of Human Failure Events: Practical Guidance for Probabilistic Risk Assessment, Electric Power Research Institute, 1021081, Oct. 2010.

[131] U.S. Nuclear Regulatory Commission, Millstone Power Station Unit 3 - NRC Special Inspection Report 05000423/2005012, U.S. Nuclear Regulatory Commission, IR 05000423/2005012 (ADAMS Accession No. ML051860338), Jul. 2005.

[132] U.S. Nuclear Regulatory Commission, NRC Bulletin 2001-01: Circumferential Cracking of Reactor Pressure Vessel Head Penetration Nozzles, U.S. Nuclear Regulatory Commission, BL-01-01 (ADAMS Accession No. ML012080284), Aug. 2001.

[133] U.S. Nuclear Regulatory Commission, Davis-Besse Nuclear Power Station NRC Augmented Inspection Team - Degradation of the Reactor Pressure Vessel Head -

Report No. 50-346/02-03(DRS), U.S. Nuclear Regulatory Commission, IR 05000346/2002003 (ADAMS Accession No. ML021260141), May 2002.

[134] U.S. Nuclear Regulatory Commission, Palo Verde Nuclear Generating Station - NRC Integrated Inspection Report 05000528/2004003, 05000529/2004003, 05000530/2004003, U.S. Nuclear Regulatory Commission, IR 05000528/2004003, 05000529/2004003, and 05000530/2004003 (ADAMS Accession No. ML042220267),

Aug. 2004.

[135] U.S. Nuclear Regulatory Commission, An Approach for Determining the Technical Adequacy of Probabilistic Risk Assessment Results for Risk-Informed Activities, U.S.

8-11

Nuclear Regulatory Commission, Regulatory Guide 1.200, Rev. 2 (ADAMS Accession No. ML090410014), Mar. 2009.

[136] M. Drouin, A. Gilbertson, G. Parry, J. Lehner, G. Martinez-Guridi, J. LaChance, and T.

Wheeler, Guidance on the Treatment of Uncertainties Associated with PRAs in Risk-Informed Decisionmaking, U.S. Nuclear Regulatory Commission, NUREG-1855, Rev. 1 (ADAMS Accession No. ML17062A466), Mar. 2017.

[137] D. Helton, S. Campbell, K. Compton, J. Xing, J. Corson, and N. Siu, Important Considerations in Selecting a Simulation End-Time in Level 2 PSA Deterministic Analyses, presented at the 13th International Conference on Probabilistic Safety Assessment and Management (PSAM 13), Seoul, Korea, 2016.

[138] U.S. Nuclear Regulatory Commission, Final Precursor AnalysisH. B. Robinson Electrical Fault Causes Fire and Subsequent Reactor Trip with a Loss of Reactor Coolant Pump Seal Injection and Cooling, U.S. Nuclear Regulatory Commission, ADAMS Accession No. ML112411359, Sep. 2011.

[139] S. P. Nowlen, M. Kazarians, and F. Wyant, Risk Methods Insights Gained from Fire Incidents, U.S. Nuclear Regulatory Commission, NUREG/CR-6738 (ADAMS Accession No. ML012600429), Sep. 2001.

[140] S. Sancaktar, WOG 2000 Reactor Coolant Pump Seal Leakage Model for Westinghouse PWRs, Westinghouse Electric Company, LLC, WCAP-15603, Revision 1 (ADAMS Accession No. ML021500485), May 2002.

[141] D. Harrison and C.-Y. Liang, Safety Evaluation by the Office of Nuclear Reactor RegulationWCAP-15603, Revision 1, WOG-2000 Reactor Coolant Pump Seal Leakage Model for Westinghouse PWRs, U.S. Nuclear Regulatory Commission, ADAMS Accession No. ML031400376, May 2003.

[142] Exelon Generation Company, LLC, Peach Bottom Atomic Power Station, Units 2 and 3Fifth Six-Month Status Report for the Implementation of Order EA-12-049, Order Modifying Licenses with Regard to Requirements for Mitigation Strategies for Beyond-Design-Basis External Events, Exelon Generation Company, LLC, RS-15-214 (ADAMS Accession No. ML15245A364), Aug. 2015.

[143] K. Kurokawa, K. Ishibashi, K. Oshima, H. Sakiyama, M. Sakurai, K. Tanaka, M. Tanaka, S. Nomura, R. Hachisuka, and Y. Yokoyama, The official report of The Fukushima Nuclear Accident Independent Investigation CommissionExecutive summary, The National Diet of Japan, 2012.

8-12

APPENDIX A COGNITIVE MECHANISMS UNDERLYING HUMAN PERFORMANCE AND RELIABILITY This appendix presents the prevalent cognitive mechanisms and their associated macrocognitive functions. This information is presented in Section 2.3 for each macrocognitive function and it is reproduced here in a different format for the readers convenience. Table A-1 through Table A-5 present the cognitive mechanisms for the detection, understanding, decisionmaking, action execution, and interteam coordination macrocognitive functions, respectively.

Table A-1 Cognitive Mechanisms for Detection Macrocognitive Function: Detection Identifier Cognitive Mechanism D.a Mental model of the cues D.b Perception of sensory information D.c Attention D.d Working memory D.e Vigilance D.f Information foliage D.g Pattern recognition D.h Shared cognition within a team D.i Infrastructure for exporting the information detected Table A-2 Cognitive Mechanisms for Understanding Macrocognitive Function: Understanding Identifier Cognitive Mechanism U.a Data U.b Selection of data U.c Mental model U.d Integration of data with mental model U.e Working memory U.f Shared cognition within a team Table A-3 Cognitive Mechanisms for Decisionmaking Macrocognitive Function: Decisionmaking Identifier Cognitive Mechanism DM.a Decisionmaking model DM.b Data for decisionmaking DM.c Select, judge, or develop plans, strategies, or working instructions DM.d Cognitive biases DM.e Deliberation or evaluation of decision DM.f Team decisionmaking A-1

Table A-4 Cognitive Mechanisms for Action Execution Macrocognitive Function: Action Execution Identifier Cognitive Mechanism E.a Physical movement and motor skills E.b Mental model of the actions and the systems to be acted on E.c Working memory E.d Attention E.e Vigilance E.f Sensory feedback of motor movement E.g Automaticity E.h Programming sequences or order of execution steps E.i Executive control E.j Error monitoring and correction E.k Initiation of action execution E.l Spatial precision or accuracy of action execution E.m Timing precision of action execution E.n Coordinated motor movement of action execution Table A-5 Cognitive Mechanisms for Interteam Coordination Macrocognitive Function: Interteam coordination Identifier Cognitive Mechanism T.a Interteam coordination infrastructure T.b Command T.c Control T.d Line of communication T.e Data processing and information management T.f Shared mental model A-2

APPENDIX B LINKS OF PERFORMANCE-INFLUENCING FACTOR ATTRIBUTES TO COGNITIVE MECHANISMS This appendix presents the links of the PIF attributes to the cognitive mechanisms as a series of tables. Each table corresponds to each PIF presented in Chapter 3. The links are limited to those inferred from the cognitive and psychological literature on cognitive mechanisms at the time they were reviewed by the NRC staff. Some links are based on the NRC staffs judgment.

In the context of the IDHEAS methodology, these tabulations summarize the identified functional relationships. They might be useful as a tool for analysts to understand how an assessment of a specific PIF attribute affects the respective macrocognitive functions of the HFE, i.e., they serve as a forensic tool for understanding contributors to HFEs. For example, in Table B-1, environmental conditions that adversely affect accessibility or habitability are manifested by reduced perception (D.b.) for Detection and by limited movement (E.a.), spatial precision (E.l.), and timing precision (E.m.) for Action Execution. Using links in the tabulations enhances analysts confidence in their assessment of risk contributors. Each row of every table in this appendix is for one PIF attribute. The cognitive mechanisms that the PIF attribute can impact are shown in the right columns of the tables, indicated with the cognitive mechanism identifiers provided in APPENDIX A.

Several PIF attributes do not have explicit links to specific cognitive mechanisms because those attributes are not specific enough. For example, the attribute for the PIF Training and Experience Training is infrequent can impact many or most cognitive mechanisms of all macrocognitive functions. The word Overarching is used in the cognitive mechanism identifier column for those attributes.

Table B-1 Cognitive Mechanisms for PIF Workplace Accessibility and Habitability Cognitive PIF Attribute Mechanism Identifier Accessibility (travel paths, security barriers, and sustained habituation D.b, E.a, E.l, E.m of worksite) is limited because of adverse environmental conditions, such as steam, high water, fire, smoke, toxic gas, radiation, electricity shock risk, and blocked roads.

Doors or components require keys to unlock. D.b, E.a Habitability is reduced. Personnel cannot stay long at the worksite D.b, E.a because of factors like radiation or earthquake aftershocks.

The surface of systems, structures, or objects cannot be reached or D.b, E.a, E.l, E.m touched (e.g., because they are hot).

The worksite is flooded or underwater. D.b, E.a, E.l, E.m B-1

Table B-2 Cognitive Mechanisms for PIF Workplace Visibility Cognitive PIF Attribute Mechanism Identifier Low ambient light or luminance of the object that must be detected or D.b, E.a, E.f, E.i, recognized E.j Glare or strong reflection of the object to be detected or recognized D.b, E.a Low visibility of work environment (e.g., those caused by smoke, rain, D.i, E.i, E.j fog, etc.)

Table B-3 Cognitive Mechanisms for PIF Noise in Workplace and Communication Pathways Cognitive PIF Attribute Mechanism Identifier Continuous loud mixture of noise D.b, D.c, D.i, E.f, E.i, E.j Intermittent non-speech noise D.b, D.c, D.i, E.f, E.i, E.j Relatively continuous speech noise D.b, D.c, D.i, E.f, E.i, E.j Intermittent speech noise of relatively short D.b, D.c, D.i, E.f, E.i, E.j duration Table B-4 Cognitive Mechanisms for PIF Cold/Heat/Humidity Cognitive PIF Attribute Mechanism Identifier Cold in workplace Overarching Heat in workplace Overarching High humidity in workplace Overarching Table B-5 Cognitive Mechanisms for PIF Resistance to Physical Movement Cognitive PIF Attribute Mechanism Identifier Physical resistance E.a, E.l, E.m Postural instability (e.g., slippery surface) E.a, E.l, E.m Whole-body vibration E.a, E.l, E.m Wearing heavy protective clothes or gloves or both E.a, E.l, E.m Resistance to personnel movement with vehicle E.a B-2

Table B-6 Cognitive Mechanisms for PIF System and I&C Transparency to Personnel Cognitive PIF Attribute Mechanism Identifier System or I&C does not behave as intended under special conditions. D.g, U.a, U.b, U.c, E.f, E.j System or I&C does not reset as intended. D.g, U.c, E.f, E.j System or I&C is complex, making it hard for personnel to predict its D.g, U.c, U.d, E.f, behavior in unusual scenarios. E.j System or I&C failure modes are not transparent to personnel. D.g, U.c, U.d, E.f, E.j Table B-7 Cognitive Mechanisms for PIF Human-System Interface Cognitive PIF Attribute Mechanism Identifier The source of indication (e.g., indicators, labels) is similar to other D.b sources nearby.

The source of indication is obscured or masked in many potentially D.f relevant indications.

The indications have low salience. D.b Related information is spatially distributed or unsynchronized. D.b Indications are confusing or nonintuitive. D.b, D.g Secondary indications are not promptly available, or personnel are not D.a aware of them.

Controls are difficult to maneuver E.a Personnel do not anticipate the failure modes of controls and their E.b impacts.

Indications of states of controls are inadequate. E.b There is confusion in action maneuver states. E.b Controls provide inadequate feedback (i.e., lack of adequate E.f, E.j confirmation of the action executed (incorrect, no information provided, measurement inaccuracies, delays)).

Labels of the controls do not agree with document nomenclature. E.b Controls are not reliable, and personnel are unaware of the problem. E.b B-3

Table B-8 Cognitive Mechanisms for PIF Tools and Parts Availability and Usability Cognitive PIF Attribute Mechanism Identifier Tools are difficult to access or to use (e.g., lack of administrative control E.a of tools).

Tools are unfamiliar to personnel. E.a, E.b Failure modes or operational conditions of critical tools are not clearly E.a, E.b presented (e.g., ranges, limitations, and requirements).

Critical tool does not work properly because of aging, lack of power, E.a incompatibility, improper calibration, lack of proper administrative control, or other reason.

Tools or parts needed are missing or not available. E.a Document nomenclature does not agree with equipment labels. E.b Table B-9 Cognitive Mechanisms for PIF Staffing Cognitive PIF Attribute Mechanism Identifier Shortage of staffing (e.g., key personnel are missing, unavailable or Overarching delayed in arrival, staff pulled away to perform other duties)

Ambiguous or incorrect specification of staff roles and responsibilities D.h, U.f, DM.e, E.j Inappropriate staff assignment (e.g., lack of the skills needed) D.h, U.f, DM.e, E.j Key decisionmakers knowledge and ability are inadequate to make the DM.a, DM.b, DM.c, decision (e.g., lack of required qualifications or experience) DM.d, DM.e Lack of administrative control on fitness for duty Overarching Table B-10 Cognitive Mechanisms for PIF Procedures, Guidance, and Instructions Cognitive PIF Attribute Mechanism Identifier Procedure, guidance, or instruction (PGI) is inadequate. Overarching PGI design is difficult to use. Overarching PGI lacks details. Overarching PGI is confusing. Overarching PGI is available but does not fit to the situation (e.g., it requires D.a, D.b, U.b, U.c, deviation or adaptation). U.d, DM.c, E.b, E.h PGI is not available for skill-based tasks. D.a, D.b, U.b, U.c, U.d, DM.c, E.b, E.h PGI is not available; thus, personnel have to find ways to perform the E.b, E.h task based on their knowledge.

PGI is misleading. D.b, U.b, U.c, DM.c, E.b, E.h B-4

Table B-11 Cognitive Mechanisms for PIF Training Cognitive PIF Attribute Mechanism Identifier Training frequency is low (greater than 6 months between sessions). Overarching Training duration or the amount of training is not adequate. Overarching Training on procedure adaptation is inadequate. The training focuses U.a, U.b, U.c, on following procedures without adequately training personnel to DM.a, DM.b, DM.c, evaluate all available information, seek alternative interpretations, or DM.d, DM.e evaluate the pros and cons of procedural action plans.

Training is inadequate on collaborative work process as a crew D.h. U.f, DM.e, E.j, (e.g., inadequate supervision in monitoring actions and questioning E.n current mission; inadequate leadership in initiating assessment of action scripts, facilitating discussion, and avoiding tunnel vision).

Training or experience with sources of information (such as scope and U.a, U.b, DM.b limitations of data and information on the failure modes of the information sources) is inadequate.

Experience in diagnosis (e.g., not being aware of and coping with U.a, U.b, U.c biases, not seeking additional information, and not avoiding tunnel-vision) is inadequate.

There are gaps in team knowledge and expertise needed to understand U.c the scenario.

There is inadequate specificity on the urgency and criticality of key U.c information such as key alarms, system failure modes, and system design to the level of detail needed for responding to the situation.

The training is inadequate or practice is lacking in the step-by-step E.b, E.g, E.h, E.i, completion of action execution. E.j The training lacks practicality. Overarching Hands-on training on action execution is lacking (e.g., training consists E.g, E.h, E.i, E.j, of virtual training, classroom training, or demos only without hands-on E.l, E.m practice).

Experience or training is lacking on procedures, guidelines, or U.b, U.c, E.b, E.h, instructions for the type of event (e.g., use non-operators to perform E.j some actions outside the control room).

The action context is infrequently part of training or personnel rarely E.b, E.h perform the actions under specific context (greater than 6 months between performance).

Personnel are not trained on the procedures or the type of actions. Overarching B-5

Table B-12 Cognitive Mechanisms for PIF Team and Organization Factors Cognitive PIF Attribute Mechanism Identifier Inadequate team information management D.a, U.a, U.f, DM.b, E.b, T.e Inadequate teamwork resources D.h, U.f, DM.e, E.n, T.a, T.b, T.c, T.d, T.e, T.f Distributed or dynamic operational teams D.h, U.f, DM.e, E.n, T.a, T.b, T.c, T.d, T.e, T.f Inadequate team decisionmaking infrastructure DM.a, DM.d, DM.e Team coordination difficulty E.n Authorization difficulty T.b, T.c Inadequate communication capabilities between teams T.d Lack of or ineffective practices (e.g., pre-job briefing) to inform D.h, U.f, DM.e, E.j personnel of potential pitfalls in performing the tasks Lack of or ineffective practices (e.g., supervision) for safety issue Overarching monitoring and identification Lack of or ineffective practices for safety reporting Overarching Lack of or ineffective practices for corrective actions Overarching Poor teamwork practices or drills together D.h, U.f, DM.e, E.n, T.a, T.b, T.c, T.d, T.e, T.f Table B-13 Cognitive Mechanisms for PIF Work Processes Cognitive PIF Attribute Mechanism Identifier Lack of professional self-verification or cross-verification (e.g., 3-way Overarching communication), peer-checking, independent checking or advising, or close supervision Poor attention on task goal, individuals roles, or responsibilities Overarching Poor infrastructure or practice of overviewing operation information or Overarching status of event progression Poor work prioritization, planning, scheduling Overarching B-6

Table B-14 Cognitive Mechanisms for PIF Information Availability and Reliability Cognitive PIF Attribute Mechanism Identifier Updates of information are inadequate (e.g., information perceived by D.g, U.a, U.b, one party who fails to inform another party). DM.b, E.b, E.f, E.h, T.d, T.e, T.f Information of different sources is not well organized. U.a, U.b, U.d, D.b, D.c Conflicts in information U.a, U.b, U.d, DM.b, DM.c, T.d, T.e, T.f Information updates are inadequate. U.a, U.b, DM.b, E.b, E.f, E.h, T.d, T.e, T.f Different sources of information are not properly organized. U.b, DM.b, T.d, T.e, T.f Personnel are unfamiliar with the sources or meaning of the D.g, U.a, U.b, information. DM.c, T.d, T.e, T.f Pieces of information change over time at different paces; thus, they U.a, U.b, DM.b, may not all be current by the time personnel use them together. T.d, T.e, T.f Feedback information is not available in time to correct a wrong DM.e, E.j, T.b, T.c decision or adjust the strategy implementation.

Information is unreliable or uncertain. U.a, DM.b, T.d, T.e, T.f Primary sources of information are not available, while secondary U.a, DM.b, T.d, T.e, sources of information are not reliable or readily perceived. T.f Information is misleading or wrong. U.a, DM.b, T.d, T.e, T.f Table B-15 Cognitive Mechanisms for PIF Scenario Familiarity Cognitive PIF Attribute Mechanism Identifier Scenario is unfamiliar. U.c, DM.c A bias or preference for wrong strategies exists. DM.d Personnel are unfamiliar with system failure modes. U.c Personnel are unfamiliar with worksites for manual actions. E.b, E.h Plans, policies, and procedures to address the situation are lacking. U.c, DM.c Unpredictable dynamics U.c, U.d, DM.c, DM.e Dynamic decisionmaking is required. DM.b, DM.c Shifting objectives DM.a, DM.b, DM.c, DM.e B-7

Table B-16 Cognitive Mechanisms for PIF Multi-tasking, Interruptions, and Distractions Cognitive PIF Attribute Mechanism Identifier Excessively frequent or long interruption during the continuous D.c, D.d, U.d, U.e, performance of critical tasks E.c, E.d, E.h, E.i, E.j Distraction by other ongoing activities that are relevant to the critical D.c, D.d, U.d, U.e, task being performed E.c, E.d, E.h, E.i, E.j Distraction by other ongoing activities that are not directly relevant to D.c, D.d, U.d, U.e, the critical task being performed E.c, E.d, E.h, E.i, E.j Concurrently detecting (monitoring or searching) multiple sets of D.b, D.c, D.d, D.f parameters where the parameters in different sets may be related Concurrently diagnosing more than one complex event that requires U.b, U.c, U.d, U.e continuously seeking additional data to understand the events Concurrently making decisions or plans that may be intermingled DM.b, DM.c Concurrently executing intermingled or inter-dependent action plans E.d, E.j, E.j Command and control multitasking T.b, T.c Table B-17 Cognitive Mechanisms for PIF Task Complexity Cognitive PIF Attribute Mechanism Identifier Detection criteria are complex. For example, there are multiple criteria D.g to be met or complex logic; information of interest must be determined based on other pieces of information and may involve complex computation; or detection criteria are ambiguous.

Detection overloading. For example, personnel may need to D.d, D.f concurrently track the states of multiple systems, monitor many parameters, and memorize many pieces of information detected.

Detection requires sustained attention. For example, determining a D.c parameter trend during unstable system status or monitoring a slow-response-system behavior without a clear time window to conclude the monitoring requires attention for a prolonged period.

Cues for detection are not obvious. That is, alarms or instructions do D.a not directly cue detection, so personnel must actively search for information.

Multiple causes for situation assessment: Multiple independent U.b, U.d, U.e influences affect the system, and system behavior cannot be explained by a single influence.

Relations of systems involved in an action are too complicated to U.e understand B-8

Table B 17 Cognitive Mechanisms for PIF Task Complexity (continued)

Cognitive PIF Attribute Mechanism Identifier Key information is cognitively masked (e.g., hidden coupling, cascading U.b effects, cognitive masking, and complex logic) and the source of a problem is difficult to diagnose because of cascading secondary effects that makes it difficult to connect the observed symptoms to the originating source.

The potential outcome of situation assessment consists of multiple U.d states and contexts (not a simple yes or no).

Decisionmaking involves developing strategies or action plans. DM.c Decision criteria are ambiguous and subject to different interpretations. DM.a Multiple, intermingled goals or criteria need to be prioritized. DM.a, DM.c Goals conflict (e.g., choosing one goal will block achieving another DM.a, DM.c goal, and multiple competing goals cannot be prioritized).

Decisionmaking requires integration of a variety of types of information DM.c with complex logic.

Decisionmaking requires diverse expertise distributed among multiple DM.c, DM.e individuals or parties who may not share the same information or have the same understanding of the situation.

Competing strategies DM.c Personnel may need to unlearn or break away from automaticity of E.d, E.g trained action scripts.

Controlled actions that require monitoring action outcomes and E.f, E.j adjusting action accordingly.

Action criteria are difficult to use E.h, E.j Action requires out-of-sequence steps E.h Long-lasting, non-continuous action sequences, or long-time gap E.c, E.g between the cues for execution to initiation of the execution are necessary.

Action sequences are parallel and intermingled. E.i Action execution requires close coordination of multiple personnel at E.n different locations.

Action execution requires long sustained attention. E.d, E.e Table B-18 Cognitive Mechanisms for PIF Mental Fatigue Cognitive PIF Attribute Mechanism Identifier Sustained, high-demand cognitive activities (e.g., procedure-situation Overarching mismatches demand constant problem-solving and decisionmaking; information changes over time and requires sustained attention to monitor or frequent checking)

Long working hours with cognitively demanding tasks Overarching Sleep deprivation, exposure to noise, disturbed dark and light rhythms, Overarching and air pollution B-9

Table B-19 Cognitive Mechanisms for PIF Time Pressure and Stress Cognitive PIF Attribute Mechanism Identifier Reluctance to execute an action plan due to potential negative impacts E.b (e.g., adverse economic impact)

High time pressure due to perceived lack of adequate time to complete D.g, U.b, U.d, the task or because of training protocols that instill an artificial sense of DM.c, DM.e, E.j, time pressure and urgency for task performance E.l, E.m Mental stress concerning the high workload or task difficulty Overarching Emotional stress (e.g., anxiety, frustration) Overarching Physical stress (e.g., disturbed dark and light rhythms, and air pollution) Overarching Table B-20 Cognitive Mechanisms for PIF Physical Demands Cognitive PIF Attribute Mechanism Identifier Action execution requires high accuracy fine-motor skills, fine-motor E.l, E.m coordination, or skills of craft.

Fine or difficult motor actions, such as installing or connecting delicate E.m parts, must be performed.

The task is physically strenuous (e.g., lifting heavy objects, E.a opening/closing rusted or stuck valves, moving heavy things in water or high wind).

There is resistance to motor movement (e.g., wearing heavy clothes, E.a lifting heavy materials, opening/closing rusted or stuck valves, executing actions in water or high wind; extreme cold or heat or on unstable ground).

The task is performed in ways or locations that can impact personnel E.a safety.

B-10

APPENDIX C INSIGHTS INTO PERFORMANCE-INFLUENCING FACTORS FROM THE COGNITIVE LITERATURE In treating the contribution of PIFs to HEPs, HRA methods either model the effects of individual PIFs (e.g., THERP [3], CBDT [12]) or holistically consider the combined effects of multiple PIFs (e.g., MERMOS 1 [87], ATHEANA [10], [11]). When modeling the effects of individual PIFs, 3F some methods treat the effect in a binary fashion (i.e., a PIF is either high or low, or present or absent). Other methods use discrete levels of multipliers (i.e., the contribution of a PIF to the HEP is a multiplier to a baseline HEP).

Cognitive and behavioral scientists have experimentally measured the effects of many PIF attributes under controlled conditions. The experiments measured human error rates in cognitive tasks while systematically varying PIF attributes. Metadata studies have also synthesized the experimental results for a given PIF attribute. This section presents some examples of experimental studies where error rates in human performance were measured by varying one or more PIF attributes. Section C.1 presents four example studies. Section C.2 presents the effects of a particular PIF attribute on human errors synthesized from multiple studies.

C.1 Example Studies Demonstrating the Effects of the Performance-Influencing Factor Attributes on Cognitive Errors Example 1: The effect of incomplete information on decisionmaking in simulated pilot de-icing (by Sarter and Schroeder [88])

Twenty-seven human subjects used a flight simulator to make a decision about de-icing during icy weather. They were tested with and without an information display providing additional information about the weather. The accuracy of the information provided was varied. The results showed that providing additional accurate information improved the percentage of correct decisions on handling of icing encounters. However, performance dropped below the baseline when the information display gave inaccurate information (high uncertainty). Table C-1 shows the percentage of stall (i.e., wrong decision). The processor in this example is DM3 Acquire and select data for decisionmaking. The PIF attribute that varied is Information is unreliable or uncertain.

Table C-1 Decisionmaking Error Variation with Information Accuracy Accurate and incomplete Accurate and additional Inaccurate information information information 30 18.1 89 Stall 1 MERMOS is an acronym in French and it stands for Méthode d'Evaluation et de Réalisation des Missions Opérateurs pour la Sreté.

C-1

Example 2: The effect of detection complexity on monitoring air traffic (by Cummings and Tsonis [89])

Thirty human subjects performed simulated air traffic management tasks with a traffic timeline display. The task was to detect information in the timeline and meanwhile monitor dangerous aircraft shown in a simulated radar display next to the timeline. The number of aircraft and number of color-coded categories shown on the timeline were varied. The results showed that information misperception slightly increased as the number of aircraft increased from 10 to 20 to 30.

Information misperception increases by a factor of 2 from sequential to non-sequential arrival patterns. Detection omission error increases by a factor of 2 as the number of categories increases from 6 to 9, as shown in Figure C-1. The processor in this example is D3Perceive, recognize, and classify information. The PIF attribute is detection complexity, indicated by the number of aircraft, number of information categories, and the logic patterns of information presentation (sequential versus non-sequential arrival pattern).

Note: The horizonal axis represents the number of information categories, and the vertical axis represents detection time, percent of detection accuracy, and percent of omission errors in the upper, middle, and lower panel, respectively. Omission errors increase when the number of information categories is greater than six.

Figure C-1 The Effect of Detection Complexity on Information Detection Example 3: The effect of sleep loss on situation assessment (by Baranski, et al. [90])

The task is for groups of human subjects to assess the situation of threat on a military surveillance task by varying the time that the subjects did not sleep. The study also examined the effects of feedback information as well as supervision and peer-checking. The results in Table C-2 show that error rates in the situation assessment task increased with the number of days of no sleep, decreased with supervision and peer-checking, and had no significant change with or without feedback information.

C-2

The processor in this study is U3Integrate data and mental model in Understanding. The PIF attributes are sleep deprivation under the PIF Mental Fatigue and lack of peer-checking under the PIF Work Process.

Table C-2 Assessment Error Rate (%)

Day 2 Day 3 Full feedback 4.2 5.5 No feedback 4.5 6 Individual 6 8 Team 4.5 5.5 Example 4: The effect of unfamiliar scenarios on pilots decisionmaking (by McKinney and Davis [91])

This study examines the effects of deliberate practice on crisis decision performance. The pilots all participated in deliberate practice. The study examined the pilots making plans in response to 160 airborne mechanical malfunctions. The results show that deliberate practice enhanced performance for wholly practiced decision scenarios. Deliberate practice was not related to aggregate decisionmaking on partially practiced crisis scenariosit helped situation assessment but not decisionmaking (choosing actions). The decision error rate was 15/83 for familiar (wholly practiced) scenarios and 22/77 for unfamiliar (partially practiced) scenarios.

The macrocognitive functions in this study involve understanding and decisionmaking. The reported error rate was only for the processor DM4Make judgment or plan. The PIF attribute was Scenario is unfamiliar under the PIF Scenario Familiarity.

C.2 Synthesized Data or Evidence on the Effects of the Performance-Influencing Factor Attributes on Cognitive Errors Multitasking A dual-task paradigm is a procedure in experimental psychology that requires an individual to perform two tasks simultaneously to compare performance with single-task conditions.

Experiments show that error rates on one or both tasks are generally 1-3 times higher when they are done simultaneously compared to being done separately. For example, Drews et al.

[92] showed that students made errors about 5 percent of the time when performing speaking and action execution tasks simultaneously, compared to 4 percent when performing just one task.

Interruption and distraction Lee et al. [93] showed that drivers had a 22 percent error rate in avoiding car collisions when distracted by cell phone calls compared to a 7 percent error rate without distraction in a driving simulator study.

Bailey and Konstan [94] found that subjects had an average error rate of 22 percent in selection tasks interrupted by a reasoning task compared to an error rate of 15 percent with no interruption.

C-3

Cognitive complexity Cummings and Tsonis [89] measured the performance of simulated air traffic control management tasks (multitasking with information detection and understanding) by varying the number of airplanes managed (quantity), number of information categories (variety), and arrival patterns (predictable versus unpredictable). They found that the mean rate of omission errors was 25 percent for 10-20 aircraft and 50 percent for 30 aircraft. They also found an error rate of 30 percent for three to six information categories and 50 percent for nine categories.

Prinzo et al. [73] analyzed pilot communication errors and reported that the error rate increased nonlinearly with the complexity of the message communicated. The error rate was around 4 percent for a message complexity index (i.e., the number of messages transmitted per communication) of 1-8, 30 percent for an index of 12-20, and greater than 50 percent for an index greater than 20.

Human-system interface Cummings et al. [95] investigated the effects on driver performance of an auditory alarm scheme, reliability (measured by the percent of false alarms), and task complexity. They found that participants had a 14 percent error rate with high reliability alarms and a 42 percent error rate with low-reliability alarms.

Hameed et al. [96] examined the effectiveness of using peripheral visual and tactile cues to support task interruption management. Participants had a 0.8-percent error rate in detection tasks when there were cues to alert them of an interruption. This compared to a 16.7-percent error rate when there was no cue alert. The error rate for arithmetic tasks was 3.6 percent with the alert and 6.2 percent without the alert.

Speier et al. [97] investigated the influence of interruptions on different types of decisionmaking tasks and the ability of information presentation formats to alleviate errors. The participants error rate for symbolic tasks was 28.1 percent with a graph presentation format and 24.6 percent with a table format. The error rate for spatial tasks was 20.7 percent with a graph presentation format and 32.8 percent with a table presentation format.

Training and experience Patten et al. [74] explored the effects of experience on automobile driver performance in a driving simulator. They reported that experienced participants had a 12 percent error rate while low-experience participants had a 21 percent error rate.

Stress Keinan [98] tested the proposition that deficient decisionmaking under stress results, to a significant extent, from the individuals failure to systematically consider all relevant alternatives.

College student participants were asked to solve decision problems while being exposed to controllable stress, uncontrollable stress, or no stress at all. There was no time constraint on the performance of the task. The controllability of the stressor was found to have no effect on the participants performance. However, those who were exposed to either controllable or uncontrollable stress showed a significantly stronger tendency to offer solutions before all available alternatives had been considered. The mean error rate was 5 percent for no stress and 9 percent for high stress.

Note that the numeric values of human error rates measured in cognitive experiments cannot be directly interpreted as HEPs of cognitive failure modes because (1) the error rates are measured under special, controlled conditions and (2) an experimentally measured error rate C-4

typically involves multiple cognitive failure modes. Nevertheless, the analysis was found that the error rate data are generally convergent across different studies. For example, most studies of dual tasks show that the error rate in dual tasks is between 1 to 2 times higher than that in a single task. The results obtained in simple, controlled laboratory tasks and those in complex simulation tasks also showed consistency. This suggests that the results from laboratory studies may serve as a baseline reference for estimating HEPs in more complex, real life scenarios.

C.3 Summary The above examples are just a few out of the large body of cognitive experiments. The purpose of this discussion is not to propose direct adoption of the observed error rates for HRA, but to identify potentially useful insights and trends.

Some PIF attributes, such as task complexity, directly challenge cognitive capacity limits and can lead to high human error rates once the cognitive demand (represented by the PIF attributes) makes the cognitive mechanisms exceed their capacity limits. Other PIF attributes, such as some human-system interface features, only moderately increase error rates.

This review of the literature indicates that the major root causes of human errors in goal-directed tasks are human cognitive limits, which are generic to humans. Various factors of task context and working environment affect the cognitive limits and may lead to performance errors. This appendix presented some experimental findings in the literature to demonstrate how some of those factors may lead to human performance errors. These findings have been used in the formulation of the qualitative analysis and quantification approach in IDHEAS-G. In particular, most of the experiments reviewed suggest relatively smallnot orders of magnitudeeffects from PIFs and their additivity. The potential for large changes in HEPs arises when demands exceed resources.

C-5

APPENDIX D COGNITIVE BASIS FOR THE COMBINED EFFECT OF PERFORMANCE-INFLUENCING FACTORS ON HUMAN ERROR PROBABILITIES D.1 Overview of the combined effect of multiple PIFs An important human action modeled in HRA typically involves more than one performance influencing factor (PIF). HRA methods treat the combined effect of multiple PIFs with either holistic estimation or a modeling approach:

Holistic estimationExperts estimate the failure probability of an important human action or a cognitive failure mode for a given set of PIFs considering, but not explicitly modeling the combination of PIFs.

Modeling approachThe HEP is the product of a baseline HEP with multipliers associated with individual PIFs. (Most current HRA methods use this the approach.)

For the modeling approach, the HRA community has been controversial in how the effects of individual PIFs should be combined. The two most prevalent ways of PIF combination have been Additive and Multiplicative. Additive assumes that PIFs have no interaction, thus the combined PIF effect can be modeled with the linear sum of individual effects Multiplicative assumes that PIFs interact and the combined effect can be modeled with the multiplication of individual PIF effects A longstanding belief in the HRA community is that multiple PIFs interact to affect performance such that the combined effect of the PIFs is the multiplication of the effects of individual PIFs on HEPs. Most HRA methods use the modeling approach and adopt the Multiplicative model.

The NRC staff developed the IDHEAS-G quantification model as a modeling approach to HEPs.

The quantification model uses PIF weights as individual multipliers and uses the HEP values of the three base PIFs (Information Availability and Reliability, Scenario Familiarity, and Task Complexity) as the base HEPs. The combined PIF effect is modeled as Additive, i.e., the total multiplier being the sum of individual PIF weights. Yet, the staff recognizes that the Additive model does not address all possible PIF combinations; there are situations that the combined PIF effect is greater than the Additive effect. Thus, IDHEAS-G quantification model allows HRA analysts to model PIF interaction with an interaction factor C in the HEP calculation formula with supporting data.

The staff adopts the Additive PIF combination based on human error data in the literature. They reviewed and analyzed a variety of cognitive experimental literature in which the individual and combined effects of two or more PIFs were examined. This appendix summarized the empirical studies reviewed.

D.2 Summary of NRC staffs observations from the literature To develop the HEP quantification model in IDHEAS-G, the NRC staff identified over two hundred research papers in which human errors or task performance indicators were measured when more than one PIF varied individually and jointly. Many of those research papers are documented in IDHEAS-DATA [55]. Using the definition of PIF attribute weight in IDHEAS-G, the staff examined the individual versus combined PIF weights in the reported data with respective to fitting to the Additive, Multiplicative, or other interaction models.

D-1

Figure D-1 illustrates three ways of PIF combination observed from the data: Additive, Multiplicative, and Subtractive. In each graph, the horizontal axis represents a PIF varying from a no-impact to a poor state, the colored lines represent another PIF in no-impact or a poor state, and the vertical axis represents the resulting human error rate or a task performance indicator.

The graph on the left shows the Additive, with the two lines being parallel. The dashed line represents the situation that the combined PIF effect is less than the sum of the individual PIF effects. The graph in the middle represents the Multiplicative, with the error rate for both PIFs being greater than the sum of individual PIF effects. The graph on the right represents a rare PIF interaction, referred to as Subtractive, in which the combined PIF effect is less than one or both individual PIF effects.

Figure D-1 Illustration of Three Ways of PIF Combination The following observations were made from reviewing the data:

For the majority of the data reviewed, there was little interaction between the PIFs such that the combined PIF weight can be predicted with the addition of the individual PIF weights (as shown in the left graph). When the individual PIF weights are large, the combined weights tend to be less than the addition of the individual weights (as shown in the dashed line of the left graph).

The multiplication of individual PIF weights tends to over-estimate the combined effects measured in the studies;

PIF interaction was observed in a small portion of the data as Multiplicative or a gating effect: The additive effect of joint PIFs is only effective when the weight of one PIF is significantly high. For example, the combined effect of Task Complexity and Mental Fatigue is additive for complex tasks while Mental Fatigue has little effect when the Task Complexity is low. Such gating effects are more associated with the three base PIFs:

Scenario Familiarity, Information Availability and Reliability, and Task Complexity.

A few studies reported the Subtractive effect, meaning that adding a poor PIF to an existing PIF reduced the measured human error rates or increased human performance.

An example is adding noise to the presence of the sleep deprivation.

Some individual and combined effects of joint PIFs behave differently if both PIFs demand the same capacity-limited cognitive resources and the demand of a single PIF is already approaching the capacity limit. The combined effect is more than the addition of individual effects and reflect the catastrophic effect of exceeding the capacity limit. For example, in a dual-task experiment, if the complexity of the primary task demands working memory approaching to the limit, simultaneously performing a secondary task D-2

that also demands working memory would lead to a very high error rate, greater than the sum of the error rates of performing each task alone.

The NRC staff performed a pilot study with a small sample of the reviewed data [75]. The study calculated individual and combined PIF weights of the error rates in 23 sample datapoints and fitted the weights to the Additive rule and Multiplicative rule. The following are some preliminary findings from the analysis:

The Additive rule can roughly estimate the combined effects of PIFs on error rates.

The combined effect of some PIFs behaves differently as the PIF weights vary from low to high (e.g., Multiplicative fits better for low PIF weights while Additive fits better for higher PIF weights);

The Additive rule does not model data well if the involved PIFs demand for cognitive resources that exceed the cognitive limits.

These findings are consistent with the general observations above made from the broad literature.

D.3 Summary of meta-analysis on PIF combination in the literature The NRC staff reviewed and documented meta-analysis studies of PIF combination in the literature. The main findings of those studies are consistent with the NRC staffs observation that the Multiplicative rule was not generally supported by the data. The following is a summary of those studies:

Van Iddekinge et. al. [81] performed a meta-analysis of the interactive, additive, and relative effects of cognitive ability and motivation on performance. They analyzed the human performance data from 55 reports to assess the strength and consistency of the multiplicative effects of cognitive ability and motivation on performance. The results showed that the combined effects of ability and motivation on performance are additive rather than multiplicative.

For example, the additive effects of ability and motivation accounted for about 91% of the explained variance in job performance, whereas the ability-motivation interaction accounted for only about 9% of the explained variance. In addition, when there was an interaction, it did not consistently reflect the predicted form (i.e., a stronger ability-performance relation when motivation is higher).

Liu and Liu [99] performed regression fitting of human error data on empirical combined effects of multiple PIFs from 31 human performance papers. They calculated the multiplicative and additive effects. The median of the multiplicative effect was greater than that of the empirical combined effect, whereas the median of the additive effect was not significantly different from that of the empirical combined effect. Thus, the multiplicative model might yield conservative estimates, whereas the additive model might produce accurate estimates. It was concluded that the additive form is more appropriate for modeling the joint effect of multiple PIFs on HEP.

Mount et al. [100] studied the joint relationship of conscientiousness and general mental ability with performance to test their hypothesis of PIF interaction. This study investigated whether conscientiousness and ability interact in the prediction of job performance. The study performed moderated hierarchical regression analyses for three independent samples of 1000+

participants. Results in the study provided no support for the interaction of general mental ability and conscientiousness. The regression analysis showed that the interaction did not account for unique variance in job performance data beyond that accounted for by general D-3

mental ability and conscientiousness alone. These findings indicate that general cognitive ability does not moderate the relationship of conscientiousness to job performance.

Hancock and Pierce [80] examined the combined effects of heat and noise upon behavioral measures of human performance. Specifically, they reviewed the capabilities on a variety of neuromuscular and mental tasks with respect to personnels vulnerability to joint thermal and acoustic action. Most of the evidence indicates that such stressors do not interact significantly within the ranges experienced commonly in the industrial setting. Yet, the authors warned that various experimental and methodological inadequacies in the meager database cautioned against a simple interpretation of this apparent insensitivity.

Murray and McCally [101] reviewed human performance and physiological effects of combined stress interaction. They grouped the possible effects into four major types.

I. No effect. Combinations produce no effects greater than those of any of the included stressors alone.

II. Additive effect. Combinations produce effects greater than any single stressors, but not greater than the addition of effects from single stressors.

III. Greater than additive effect. Combinations produce effects greater than mere addition of single stress effects. This possible result is sometimes referred to as "synergistic."

IV. Subtractive effect. Combinations produce effects lower than effects produced by single stressors. This result may be referred to as "antagonistic."

These four types of outcomes seem to be likely on a theoretical basis of possible interactions among PIFs. Type I seemed most likely when the stressors included in the combination are unequal in their effects. Then, the more severe stress would dominate the results, and variables with less effect would make no detectable addition to the overall result. Type II seemed to be the most likely when the stressors are about equal in their effects, and their mechanisms of action are independent. Type III and Type IV, synergistic and antagonistic effects were rarely observed in reported experiments.

Grether [79] reviewed the studies about the effect of combined environmental factors on human errors. The reviewed environmental factors included noise, temperature, sleep deprivation, and others. The results showed that the combined effect was no more than the added single effects and could be predicted from single effects. The study suggests that the combined environmental stresses do not present a special hazard in flying that could not be anticipated from the results of single factor studies. The findings are consistent to those in Broadbents study [95] that reviewed many experiments applying different stresses to comparable subjects performing similar tasks. The study found that the experiments on the simultaneous application of two stresses show that the effects of heat appear to be independent of those of noise and sleeplessness, while the latter two conditions partially cancel each other.

Overall, the various meta-analysis of combined PIF effects and the NRC staffs observations from the data in the literature are, in general, consistent. The additive effect of PIF combination seems to be applicable for the majority of PIF weight ranges, and the added PIF weight acts as a multiplier to the base HEPs. In-depth studies are desirable to understand the nature of PIF interactions and elucidate the situations that the combined PIF effects become synergistic rather than additive, because such situations can be associated with high HEPs of important human actions.

D-4

APPENDIX E SCENARIO ANALYSIS This appendix describes how to perform scenario analysis in IDHEAS-G. The purpose of scenario analysis is to understand human performance from a holistic view of the event.

Scenario analysis includes three parts: assessment of scenario context, development of operational narrative, and identification of important human actions. This appendix provides the guidance for the first two parts in Sections E.1 and E.2, respectively, and APPENDIX F has the guidance for identification of important human actions. This appendix also provides guidance for collecting information in scenario analysis (Section E.3) and discusses HRA interaction with other technical disciplines in a PRA (Section E.4).

The scenario context provides a broad view of the conditions that affect human performance, including those that impact the systems, personnel, and the tasks to be performed. The scenario context influences event progression, the human actions and tasks to be performed, the failure modes of the tasks, and the states of the PIFs. Scenario context describes the mission and goals of the event, the objectives (i.e., expected outcomes) of the event, the system functions and high-level tasks that need to be accomplished to achieve the objectives, the systems involved, and the personnel who perform the tasks.

The purpose of the operational narrative is to develop an in-depth understanding of the event evolution. The operational narrative provides a detailed account of the event scenario, including a storytelling-style representation and timeline. The timeline provides a chronological view of the expected or potential event evolution in the scenario. The operational narrative specifies the initial condition, initiating event, and boundary conditions of the event, as well as the scenario progression and consequence.

The three parts of scenario analysis support one another to provide a holistic representation of the event. Since the information from one perspective supplements the other perspectives, the three parts should be performed iteratively to obtain an integrated understanding of the event.

E.1 Assessment of the Scenario Context Analyzing human events makes sense only within the context of the scenario. Scenario context refers to the conditions that affect human performance. For HRA, assessment of the scenario context focuses on the conditions that could affect human actions important to prevent the undesired consequence or to mitigate the event consequence. The context affects event progression, the human actions and tasks to be performed, and the failure modes of tasks. In IDHEAS-G, the PIFs are the modeling representation of scenario context. In quantifying the HEP of an event, the context is represented by the states of the PIFs.

Assessment of scenario context requires a search process. To guide a systematic search, IDHEAS-G divides scenario context into four parts: environment and situation context, system context, personnel context, and task context.

E.1.1 Development of a Human Performance Model for the Event A simple human performance model may be initially sketched to serve as the framework to assess the scenario context and to develop the operational narrative. A human performance model for an HRA event may consist of the following elements:

E-1

The goal of the event: HRA focuses on safety; therefore, the goal of an event must relate to safety. For NPP events, the mission is to operate the plant in safe status or mitigate an unsafe plant status. Specifically, the goal is to protect the reactor and containment. For nuclear fuel- or material-handling events, the goal is to prevent a radiological release and protect personnel and members of the public.

The objectives and functions: The objectives represent the desired outcomes of the event in achieving the goal. Examples in NPP operation are restoring electrical power, initiating feed and bleed, and evacuating personnel. To achieve the objectives, a set of functions must be performed. Systems, personnel, or a combination of both can perform these functions. It helps to understand human performance by identifying the functions and the nature of the work, such as performing a routine task, responding to an abnormal situation, or handling unforeseen events.

The systems: IDHEAS-G uses the term systems to broadly refer to structures, systems, and components, as well as sensors, equipment, instrumentation and controls, and HSIs. Systems encompass everything necessary to achieve the objectives.

Systems include the following:

- physical structures (and their locations) for personnel and systems to do the work

- front-line systems that perform accident mitigation functions

- systems that support primary systems or personnel; NUREG-2122 [51] defines support system as follows:

[I]n a PRA, support system failures are evaluated to determine the effect of these failures on the operability of other plant systems and components. Often one support system, such as component cooling water, provides functionality to multiple systems or components and, therefore, needs to be considered in PRA modeling to assess what happens if that capability is lost to multiple systems.

Examples of support systems include electrical power, cooling water, instrument air, and heating, ventilation, and air conditioning.

- event-related systems that do not support the mission and expected outcomes of the event, but are related to the event by sharing common resources, personnel, or physical structures

- instrumentation and controls, sensors, equipment, HSIs, and any items that support systems or personnel It is important to understand the operational concepts of the systems (i.e., how the systems are intended to work and how they are intended to interact with personnel).

The following are examples of different types of HSI:

- Personnel operate systemsSystems perform their functions as directed by personnel, such as a radiologist operating a remote-afterloading brachytherapy system to perform radiotherapy for patients.

- Personnel supervise process control systemsSuch systems, under normal operating conditions, require only occasional fine tuning of system parameters to maintain satisfactory performance, and personnel have overall responsibility for control of the system; examples are NPP control systems (e.g., feedwater control, inventory control) and medical patient-monitoring systems.

E-2

- Personnel support autonomous systemsSystems automatically perform all of the mission-critical tasks, and the major tasks for personnel are to program changes in inputs or control routines and to serve as a backup in the case of a failure or malfunction in a system component. Examples are small modular reactors and unmanned vehicles.

- Passive systemThe system operates on gravity and does not need personnel for operation (it would still need personnel for installation and maintenance, and personnel may decide to intervene in its operation).

The personnel: Personnel include all of the people who perform the tasks in an event.

Personnel may work in various structures:

- IndividualsEvery person has his or her own roles, responsibilities, and assigned tasks.

- TeamA group of individuals works collaboratively for common goals.

- OrganizationThis is a framework to outline authority and communication processes of individuals and teams. The framework usually includes policies, rules, and responsibilities for each individual in the organization.

Explicit consideration of personnel structures and team processes is important in analyzing human performance. This allows HRA analysts to systematically identify the performance challenges and opportunities for error that arise when the event involves a complex organization with individuals distributed across multiple locations with complex communication, and command and control structures. Analysis of actual emergency responses makes clear the importance of communication, cooperation, and coordination across multiple individuals distributed in time and space. The personnel structure, lines of communication, and chain of command play a critical role in successful performance.

Example 1: Human performance model for a see-and-flee eventWorkers handling nuclear materials see indications of a chemical release and flee away for personnel safety.

In this hypothetical scenario, workers are performing a routine uranium conversion process in the feed materials building. The workers on the third and fourth floors of the building are cleaning a UF6 valve with hot steam, which causes a UF6 release. The workers in the building need to flee from the building to avoid being exposed to UF6.

The focus of the analysis is that workers detect the cues of release and flee from the building through any available safety evacuation path. There could be other actions to (1) contact central control/command authority to inform them of the release, (2) inform other workers in the area, (3) secure the immediate release, if possible, and (4) minimize the spread of contamination (e.g., stop ventilation). These actions are not related to the goal of this analysis.

Goal: To protect personnel from chemical contamination.

Objective: Personnel flee the building as quickly as possible without being contaminated.

Systems: The building where workers work and the building structure through which the workers are evacuated. Other systems such as communication systems or auxiliary steam supply are not related to the goal of the analysis.

Personnel: Workers on the third and fourth floors of the building who have access to an emergency exit.

E-3

Example 2: Human performance model of a radiation brachytherapy eventApplication of radiotherapy to patients through a brachytherapy system.

A patient was prescribed a dose of 8 gray (800 rad) to the coronary artery during a Cordis Checkmate' IV brachytherapy procedure using 10 iridium-192 seeds emitting a dose of 8,991 megabecquerels (243 millicuries). In one operation, the diameter of the artery instead of the radius was incorrectly used in the treatment plan calculation. This error resulted because the physician (an authorized user) using the Cordis device was more familiar with the procedures for a Novoste' device also in use at this institution. The Novoste device uses the diameter of the artery in the dosimetry calculations, whereas the Cordis device uses the radius. The authorized user provided the wrong dimension (diameter instead of radius), which led to the calculation of an incorrect dose. As a result, the patient received an actual dose of 14.6 gray (1,460 rad) to the outer coronary artery site instead of the prescribed 8 gray (800 rad).

These are routine tasks conforming to procedures or protocols. A crew identifies the patient and prepares the patient for the treatment, and a physician operates a brachytherapy system to administer the radiotherapy to the patient.

Goal: For the patient to receive radiotherapy correctly and safely.

Objective: The right patient safely receives the correct dose of radiation at the desired locations.

Systems: Multiple systems are involved in the event:

the remote-afterloading brachytherapy system as the main operational system to execute the work

the computer or paper-based log system that documents patient information

the computer system that supports the technician (e.g., gathering information, assessing situation, and developing or modifying action plans)

the equipment (e.g., camera, oral communication systems) for the physician monitoring and communicating with the patient

the operational room where the technician operates the brachytherapy system, the patient room where the patient receives treatment, and the building structure through which the patient is moved and placed in the treatment room.

Personnel: The crew that prepares the patient for treatment may include multiple individuals, each performing their own preassigned tasks. Hand-off briefings may be needed between the individuals. Other personnel include the physician(s) who operates the remote-afterloading brachytherapy system, other doctors and experts who may be called on as needed, and the clinic or hospital as the organization with whom the personnel are affiliated. The hospital had procedures that were supposed to be followed and perhaps even second checks on calculations. The cause of the event was probably closely related to a failure to follow procedures and second-check calculations. The organization involved is the hospital; it has programmatic features in place to ensure that the proper dose was received; however, these could be violated.

E.1.2 Search for Scenario Context In performing an HRA, analysts should perform a systematic, comprehensive, and objective examination and documentation of the scenario context, without pre-existing biases that may E-4

cause them to overlook scenario-specific effects. The search for scenario context identifies the conditions that can impact the macrocognitive functions for human performance: detection, understanding, decisionmaking, action execution, and interteam coordination. The scenario context is classified into four categories: environmental, systems, personnel, and tasks. These categories are not intended to represent an exclusive classification of context. Rather, they are intended to guide the search. The NRC staff provides some probing questions to identify the context that can significantly impact the macrocognitive functions. Analysts may develop additional questions to probe the possible conditions that can affect human performance.

Environmental Context Environmental context is the performance-challenging conditions in the personnels work environment and the situation in which important human actions are performed. It includes weather, radiation, or chemicals in the workplace, and any extreme operating conditions.

Hazards such as steam, fire, toxic gas, seismic events, or flooding can introduce environmental conditions that impede personnel performance. According to NUREG/CR-5680 [45], [46], many environmental conditions can adversely affect human performance. Risk analysis typically considers the following environmental conditions:

temperature and humidity

noise

radiation or chemical contamination

light and glare

smoke and fog

high winds

standing or running water

debris

vibration

seismic aftershocks Below are some considerations for the environmental context:

Noise, smoke, and precipitation affect information detection.

Harsh environmental conditions, such as extreme heat or cold, may lead to early termination of situation assessment because personnel are unwilling to seek additional data to reconcile conflicts in the information.

Harsh environmental conditions adversely affect decisionmaking (e.g., reducing decisionmakers ability and effort in evaluating available strategies, thoroughly deliberating decisions, or mentally simulating action plans).

Environmental conditions on travel paths and at worksites can restrict personnels motor movement, reduce their motor skills, or limit the time that they can steadily perform motor activities. Examples of these conditions are wearing heavy protective clothes, high water on travel paths, high winds, extreme heat or cold, earthquake aftershocks, and chemical or other toxic contamination.

Environmental conditions such as noise or smoke can impede interteam collaboration.

Questions for probing the environmental context include the following:

Where do personnel perform actions? Are there environmental considerations?

E-5

Are there things affecting accessibility or habitability of workplace, including travel paths?

Does the workplace have good visibility needed for human actions?

Is there noise in workplace and communication pathways?

Is the work environment very cold, hot, or humid?

Is there resistance to personal or vehicles physical movement, such as strong wind or still or moving water?

Are there environmental conditions that could raise concerns of personal safety to respond to the event?

System Context System context refers to conditions that arise in systems and could negatively or positively affect human performance. For example, in an NPP fire event, some valves or breakers are designed to automatically open, close, or reset upon system disturbances. System context specifies the conditions affecting the systems capacity to perform their designed functions and subsequently lead to human failures. Identification of system context should focus on conditions that create conflicting priorities, confusion, and distractions to human performance.

Those conditions often involve non-safety systems and equipment that is not the focus of an event analysis (so they may be not modeled in the PRA). For example, NPP operators may be concerned about possible damage to major plant equipment that is not directly relevant to the PRA event scenario, failures or interruptions of non-safety power supplies that are not explicitly modeled, disruptions of low-voltage instrumentation and control power supplies that are not modeled, or investigation of false fire alarms. The scenario context should include these elements of the event scenario for a complete description of the conditions during which personnel need to perform the specific actions (modeled in the PRA) to achieve the mission.

Below are some general considerations for assessing the system context:

Systems may become unavailable or behave abnormally because of accidents, incidents, hazards, maintenance, repairs, aging, or concurrent activities to protect workers or major equipment. For example, computer systems may become temporarily unavailable because of network congestion, some sensors of NPP systems may become unreliable as the result of an electric fault, or operational system components or equipment may be disabled by problems in related systems (such as other reactor units in multiunit NPPs).

Electrical faults may reset systems or components to an undesirable status.

The designed operational range of the system, structure, or component could be exceeded and functions needed to support the component or instrument operation may be inadequate.

Structures may have degraded environmental conditions or be inaccessible because of hazards or construction activities.

Automated systems could be intentionally turned off based on a well-intentioned, but incorrect, belief by the crew.

E-6

The status of systems that are not directly associated with event mitigation, but which may divert personnel attention from the desired course of action, create conflicting priorities, introduce unexpected time delays, etc.

Situations like those listed above could impact human performance by affecting the states of PIFs such as the following:

transparency of systems to personnel

information for personnel, including information availability, completeness, reliability, and whether or not information is presented in a timely manner

transparency of equipment, instrumentation and controls

functionality of HSIs

time available for personnel to perform required actions Questions for probing the system context:

What are the safety issues and the causes (e.g., core damage caused by a loss of coolant accident)?

How do systems react to the failure of the system of interest (e.g., reactor trip and safety injection actuation)?

What are the systems, structures, and components needed to mitigate the event? What are the constraints on implementing them?

What are the system and human responses required to mitigate the safety issue? What are the setpoints for the automatic system responses?

Personnel Context The personnel context includes the conditions that challenge or facilitate the human to perform the tasks. For example, in the see-and-flee example, workers perform the tasks in a facility where accessibility to the emergency exits of the building is limited to authorized personnel; workers on floors other than 3 and 4 could not see the release when it initially occurred.

Personnel context specifies the conditions affecting individuals, teams, or organizations. The context affects personnels task performance in detecting information, understanding the situation, making decisions, executing planned actions, and interteam coordination.

Below are some considerations for the personnel context:

Availability of personnelConsider the amount and types of personnel available to respond to the event compared to the personnel needed. Personnel may become unavailable for reasons such as multiple simultaneous events, environmental effects, or duties unrelated to the event.

Operational limitations of personnelPersonnel may not perform work as expected for reasons such as physical limitations, not being prepared or trained for the type of events, or conformation to special safety or regulatory requirements.

Organizations may not have adequate infrastructure to support teamwork for reasons such as safety culture, authorization restrictions, conflict of interest or goals, or lines of communications.

E-7

Availability of personnel supportPersonnel may lack necessary support such as training, tools, procedures or protocols, or expertise for reasons such as hazards, surprise of the event, beyond-design-basis accidents, lack of experience using the supporting items, and the need to share the limited supporting items.

Environmental conditions (such as fire, smoke, flood, earthquake, noise, illumination, temperature extremes, and high radiation) that directly impact human performance may change during the evolution of the scenario.

Considerations for personnel context may also address safety culture. Safety culture is the attitude, beliefs, perceptions and values that employees share about safety. Different organizations define various safety culture metrics. In its Final Safety Culture Policy Statement

[102], the NRC defines nuclear safety culture as the core values and behaviors resulting from a collective commitment by leaders and individuals to emphasize safety over competing goals to ensure protection of people and the environment. The policy statement defines traits of a positive safety culture such as leadership safety value and actions, problem identification and resolution, and effective safety communication. Event analyses and research indicate that the extent to which an organization prioritizes safety over competing goals (e.g., cost, production, schedule) primarily has indirect effects on human performance by affecting the state or condition of other PIFs. For example, in response to perceived financial pressure on the organization, leaders may decide to defer maintenance of equipment, reduce staffing, delay or cut training, not purchase enough tools and field equipment, or otherwise limit the amount and condition of resources available to support human performance.

Personnel context may affect the states of PIFs such as:

time pressure and stress

availability of procedures, guidance, and instruction documents

staffing and decisionmakers

training, knowledge, and expertise

team and organization factors Questions for probing the personnel context include the following:

What is the personnel structure?

What are key concepts of operations (e.g., staffing, training, validation)?

Are there fitness-for-duty (fatigue, substance abuse, or illness) requirements?

What are the manpower and skillsets needed to respond to the safety issue?

What are the potential issues that could occur in the teamwork and communication areas?

Task Context The task context includes special conditions of the tasks that need to be performed, how these tasks are expected to be performed, the demands of the tasks, and the success criteria for the tasks. The conditions may change which human tasks are required, task requirements, or task difficulty. Task difficulty refers to the demand for personnel cognitive resources and collaboration. The characterization of the human-system interactions and conduct of operations specify how tasks are performed. Some aspects such as burden and pace of the tasks may be better understood from the perspective of the conduct of operations and operational experience.

E-8

Below are some general considerations for task context:

Use of computerized HSIs and supporting systems add work for personnel.

Multiple, simultaneous events may lead to multitasking, interruptions, and distraction.

Failure or unavailability of operational system components may make the event progression unpredictable.

Unusual event evolution may reduce the time available for required human actions.

Complex events often require personnel to perform tasks in distributed locations.

Personnel may need to perform additional tasks upon failures of automated systems.

Personnel may make nonrequired changes to system status or interfere with system automation with good intentions, yet the changes may lead to undesirable consequences.

Questions for probing the task context include the following:

What are the constraints in implementing the human tasks?

What is the potential task interference (e.g., sharing the same resource with other, concurrent tasks) and task dependency (e.g., tasks must be performed in sequential order, such as obtaining external permission to perform the task)?

Cues for detection: This refers to cues that would lead an operator to notice the safety issue.

- What are the cues that directly point to the safety issue?

- How are the cues generated?

- How are the cues detected and by whom and where?

- What training is related to the cues in the scenario?

- What are the key factors affecting detection of the cues?

Diagnosis and situation awareness for understanding: This refers to the information and mechanisms for the operator to understand the situation and diagnose the problem.

- What information is needed for the situation diagnosis? How is each individual piece of information generated and obtained, by whom, and where?

- What is the basis (e.g., which procedure) for making the diagnosis and situation awareness, who makes the diagnosis, and where is it implemented?

- What operator training is related to the diagnosis?

- What are the key factors affecting the diagnosis?

Decisionmaking: This uses the information based on the understanding of the situation to make decisions responding to the situation.

- What are the criteria or rules for making the decisions?

- How is the decision made and what is it based on (e.g., which procedure), by whom, and where?

- What are the competing goals and alternative options for the decision?

E-9

- What are the key factors affecting the decision?

Action: This refers to implementing the decision by interacting with the system to change the scenario direction.

- How is the task performed and what is the basis for the performance (e.g., which procedure), who performs it, and where?

- What are the success criteria of the actions?

- What are the key factors affecting performance of the action?

- Action executionAre the manual actions physically strenuous?

Interteam coordination: This refers to interactions between multiple entities (individuals, teams, and organizations) involved in the event.

- What decisionmaking authorities are involved (and what other organizational factors and interactions might come into play)?

- How are communications, resource allocations, information, and knowledge managed?

The following two examples show context generic to the scope of HRA applications for NPP internal, at-power events and for extremely hazardous events. The context identified here may not always apply to the scopes of HRA analysis. For example, teamwork may involve personnel outside the control room crew in a complicated internal, at-power fire event. These examples demonstrate that scenario context can vary greatly for NPPs.

Example 1: The generic context for HRA applications of NPP internal, at-power events includes the following:

Environmental context: Event scenarios may involve the loss of Main Control Room (MCR) ventilation, room cooling, and normal lighting. These scenarios can occur from a variety of initiating events and system failures. Fires in locations other than the MCR may result in smoke entering the MCR through ventilation systems. Seismic events may cause partial damage in the MCR (e.g., suspended ceilings, overhead light fixtures, toppling storage shelves, etc.). Some plant-specific and site-specific scenarios may result in toxic gases entering the MCR from locations inside the plant or from accidents that occur nearby the plant.

System context: The system modeled is an NPP reactor. Personnel operate the reactor systems from a main control room using HSIs. While the system and HSI functions are generally verified, they may not be "well-designed" to facilitate personnel performance in the context of a specific scenario, and the scenario-specific time margins may not be "adequate".

Personnel context: Personnel modeled are well-trained crew members. They should meet fitness-for-duty requirements and use procedures, guidance, or instructions to perform required human actions. While plants meet the minimum staffing requirement for main control room operation, there could be scenario-specific situations leading to inadequate staffing (e.g., a shift supervisor may have been called away for a mandatory drug test when an event occurs). Personnels mental fatigue and stress are scenario-specific and need objective assessment. General assumption on personnel context may overlook important factors contributing to risk.

E-10

Task context: Human actions include those allocated to control room operators as well as local operator actions. Control room operator tasks are usually prespecified in control room normal or emergency procedures while local actions outside the control room may be less prespecified and involve skill of the craft (see page A-7 of NUREG-1921 [63] for the definition of skill-of-the-craft actions). Assessment of the Interteam coordination macrocognitive function is needed for coordination of those local actions.

Decisionmaking in many control room actions may involve only the selection of a procedure-directed strategy which matches the evolving scenario conditions. However, procedure-directed strategies do not always match scenario conditions, especially in complex, rarely occurring scenarios.

Example 2: The generic context for HRA applications of NPP external hazard-induced events includes the following:

Environmental context: The work environment may be harsh or hazardous. The worksite can be inside and outside NPP control rooms. Some actions may be performed outside of a sheltered area.

System context: The modeled systems include NPP reactor systems, containment, spent fuel pool, HSIs, portable equipment, tools, and NPP structures. The systems may be damaged, degraded, or unavailable. The event may involve multiple reactor units.

Personnel context: Personnel include the trained crew as well as untrained individuals; staffing may be inadequate. Personnel work in distributed locations.

- Procedures, guidelines, and instructions may not be available or applicable, or they may be less detailed, less prescriptive, and of lesser quality.

- Fitness for duty may be impaired. Stress, anxiety, and fatigue may be significant.

Task context: Human tasks involve all of the macrocognitive functions. In particular, decisionmaking and interteam coordination can be challenging. Innovative solutions may need to be developed. Personnel may have to perform actions in complex socio-technical and even chaotic conditions. Normal command and control may be affected.

Action execution may be manually strenuous (e.g., placement of flood barriers, hauling FLEX equipment from its storage location to where it will be temporarily installed).

E.2 Development of the Operational Narrative The operational narrative is a means for the HRA analysts to develop an in-depth understanding of the scenario progression. Scenarios can be identified using the risk triplet [2]: (1) what can go wrong; (2) how likely is it to wrong; and (3) what are the consequences. Consideration of the risk triplet (i.e., scenario, frequency, and consequence) is the highest guiding principle for developing the operational narrative. Scenario identification starts with developing a baseline scenario that represents the actual event or the expected event progression. Failures of important human actions and system responses would lead to identification of new scenarios.

A baseline scenario describes the expected event evolution. Then, the baseline scenario is used as a reference scenario to identify the alternative scenarios that could have risk impacts on the goal of the event. Altogether, several representative scenarios (including the baseline scenario) may be identified, and together, they represent different potential evolutions of the event. A complete operational narrative is required for the baseline scenario. Because all the scenarios with the same initiating event share similarities in important human actions and system responses, a complete operational narrative is not generally needed for each scenario.

E-11

The operational narratives for other scenarios should emphasize differences from the baseline scenario. If a scenario contains dramatic differences in important human actions or system responses as compared to the baseline scenario, then developing a complete operational narrative for the new scenario is recommended.

The operational narrative includes two scenario documentation techniques: scenario narrative and scenario timeline. The scenario narrative is a storytelling-style documentation of the scenario progression. The scenario timeline documents a scenarios important human actions and system responses in chronological order. Figure E-1 summarizes the structure of organizing information in an operational narrative.

Scenario narrative

Overview of the event

Beginning status

Initiating event

Initial conditions

Boundary conditions

Progression and end state Baseline scenario Timeline

Date/time Operational

System response, human narrative of a response, data for human event situation awareness, and notes Deviations from the baseline Additional scenarios scenario Figure E-1 Composition of an Event Operational Narrative E.2.1 Development of the Baseline Scenario A baseline scenario is used as the reference scenario for the analysts to gain an understanding of the event progression and to identify the other risk-important scenarios. A multidisciplinary team (e.g., operations experts, system engineers, human performance experts) jointly develops the baseline scenario. Post-event analysis (or event and condition analysis) uses a failure memory approachthe failures are set to actions failed, and the successes are kept at their nominal values. The PRA should consider which actions are needed for success and then develop failure paths as needed to reach an acceptable state of completeness.

Existing PRA guidance (e.g., NUREG/CR-2300 [103]) describes how to identify and document baseline scenarios. Development of the scenario for IDHEAS-G should focus on the pieces that are relevant to the HRA. For the analysis team, the baseline scenario serves two objectives: (1) it is a means for the analysis team to establish a common understanding of the scenario and to identify the safety issues and causes, and (2) it serves as the reference scenario to identify other risk-important scenarios. To achieve the two objectives, the baseline scenario is presented in two levels of detail. The first level is a detailed scenario description, and the second level is a summarized representation of the detailed scenario description for performing a safety analysis. Two techniquesscenario narrative and timelineare discussed to document the detailed scenario description. The detailed scenario description is similar to the concept of directing a documentary film that not only documents the scenario progression but E-12

also provides more detailed descriptions for the issues related to safety. Developing the detailed scenario description helps the HRA analysts become familiar with the important human actions and systems relevant to the scenario. The summarized scenario representation is used as a model for safety analysis. It typically includes only the systems and important human actions that if not performing their designed functions would change the scenario course and consequence. Because the failures of the systems and humans to perform their designed functions would result in different scenarios, the techniques to document the analysis results typically include many scenarios developed from the same beginning state. The representative techniques are the event sequence diagram and event tree.

E.2.1.1 Scenario Narrative Scenario narrative is a storytelling technique to present the scenario. In principle, the scenario narrative should describe everything that is happening in the plant that explains the behavior of the operator in the scenario, because that is the actual context within which personnel must respond. Of course, that idea can rarely be achieved in a practical analysis. However, the narrative should describe all conditions that may have a potentially important effect on human performance, even if those conditions are not included explicitly in the PRA models. That description helps the analysts to identify and evaluate the states of relevant PIFs that account for distractions, interruptions, multi-tasking, conflicting priorities, time pressure, stress, etc. It also helps others to understand what conditions were considered by the analysts and to question the reasons for possible omissions.

The construct of the storytelling is analogous to making a documentary film that presents the scenario with various scenes. A scene or combination of scenes is used to describe a safety issue in the scenario. The story is told by presenting these scenes in a proper relation (e.g.,

causal relation or chronological order). At a high level, the story covers the beginning of the scenario, its progression, and the consequence (or end state). Before presenting the scenes, a scenario overview is provided to highlight the safety issues of the scenario. This section provides guidelines for developing narratives for the scenario overview, the beginning of the scenario, the scenario progression, and the consequence.

Scenario Overview The scenario overview documentation includes a title and a scenario summary. The title should be descriptive and provide a clue for the readers to predict the content. Therefore, the title should highlight the key safety issues and consequence. The scenario summary covers when, where, and how the event occurred; the safety issues; how the safety issues were mitigated; and the consequence.

Beginning of the Scenario The beginning of the scenario should be clearly specified at a level of detail suitable for the analysis and agreed on by the analysis team members. It is very important for all technical disciplines represented by the analysis team to apply consistent assumptions. For example, to analyze a fire event, the initial causes of the fire, location, magnitude, propagation, and the damage to systems, structures, and components are the core assumptions that should be clearly specified for the analysis. The core assumptions should be communicated to and applied by all technical disciplines represented on the analysis team. Each discipline could have discipline-specific assumptions. For example, HRA may assume that the initial fire does not cause personnel injury. The discipline-specific assumptions should not conflict with the core assumptions. The discipline-specific assumptions should be communicated to all team E-13

members to prevent potential conflict. The beginning of a scenario includes three scenes:

initial condition, initiating event, and the boundary conditions.

The initial condition describes the initial system and human conditions that have implications for the scenario progression and safety. The discussion should include environment, system, personnel, and task context. Important features to be identified include the following:

Systems, structures, and components with latent failures, unavailable (tagged out), and having historical unreliable performance (especially the ones that would affect the operators decisions and the scenario).

The facility operating modes (e.g., at-power, low-power, and shutdown in nuclear reactor operations).

Special or temporary system alignment.

Workers not in their normal locations. In the NPP fire event example [104], the shift manager and the shift technical advisor were in another building away from the main control room for shift turnover.

Operating team not in normal configuration. Temporarily, one individual may be performing dual responsibilities to fill in for a missing team member.

Personnel substitution. Temporary substitution of the individual familiar with the tasks by another individual who does not normally perform the tasks is likely to affect human performance.

Other ongoing activities performed at the same time of the initiating event that can affect the scenario.

The ASME/ANS PRA Standard [44] defines an initiating event for nuclear reactor safety as follows:

[A]n event either internal or external to that which perturbs the steady state operation of the plant by challenging plant control and safety systems whose failure could potentially lead to core damage or release of airborne fission products. These events include human-caused perturbations and failure of equipment from either internal plant causes (such as hardware faults, floods, or fires) or external plant causes (such as earthquakes or high winds).

For a nuclear fuel processing facility, the initiating event can be an external event such as a hurricane or earthquake, a facility event external to the process being analyzed, deviations from normal operations of the process, or a failure of an item relied on for safety [105].

An initiating event could be triggered by a system failure or a human error. The initiating event narrative should be described at a level of specificity such that knowledgeable readers conversant with the design of the facilities in general, but not familiar with the details of the specific facility, can generally understand the scenario (e.g., a small loss-of-coolant accident at a hot leg, a loss of offsite power event because of the grid failure, and the loss of an essential electric bus causing reactor trip because of human error in maintenance).

The boundary conditions specify the analysis scope and the assumptions applied to the analysis. This could include limiting the analysis scope to focus on the primary issues and to make simplified assumptions such as deterministic assumptions about the status of systems (e.g., damage associated with the initiating event) and personnel (e.g., personnel availability).

E-14

Scenario Flow and End State The scenario flow documents the scenario development following the given initial condition, initiating event, and boundary conditions. The purpose is to provide a good understanding of the scenario flow with sufficient details to perform a detailed HRA. The scenario flow is represented by a number of scenes. Principally, a scene describes a safety issue. A scene could have subscenes or a specific topic related to the safety issue. Collectively, the scenes cover all safety issues. Moreover, beside the safety issues that are modeled explicitly in the PRA, it is important to examine the totality of what is happening in the plant and document any conditions that may have a potentially important influence on personnel response, even if those conditions are not related directly to the equipment, human actions, and safety issues.

Restricting the focus of the narrative to describe only the scenario "safety issues" that are depicted in the PRA models may cause the analysts to overlook those influences and perhaps result in an overly optimistic perspective about the desired personnel response.

It is a good practice to analyze the scenario from the eyes of the human in the scenario. The HRA analysts need to know the mindset of the operators in different stages of the scenario (e.g., their view of the situation, their task priorities, their concerns, and their locations). The narrative includes a description of the safety issue and the responses of systems and humans to the safety issue. At a high level, these responses can be summarized using an analogy to the following macrocognitive functions:

cues for detection

diagnostic information for understanding and decisionmaking

physical actions for action execution

interaction between teams for interteam coordination The cues are the information that raises attention for detection and triggers a persons cognitive process to address the safety issue. The diagnostic information is the information required to make a diagnosis and have an appropriate situation awareness for understanding.

Decisionmaking refers to making a response decision based on the situation awareness and diagnosis. Action execution refers to steps taken implement the response decision. For each of the bullets above, the topic should be described in the context of the environment, system, personnel, and task. Table E-1 provides guidelines for the content.

E-15

Table E-1 Narrative Information Coverage of a Safety Issue Safety issue:

- What is the safety issue?

- How does it occur?

- What is the safety significance?

Cues:

- What are the cues?

- How are the cues generated?

- What are the means to detect the cues?

Diagnosis and decisionmaking:

- What is the information for diagnosis?

- How are the diagnoses made and the decisionmaking performed? What are the basis and constraints of diagnosis and decisionmaking?

- What is the information that could mislead the human to a wrong diagnosis?

Physical actions:

- What are the automatic system responses to the safety issue?

- What are the manual actions needed to mitigate the safety issue? How are the actions performed? What are the constraints on performing the actions?

Interteam coordination:

- What kinds of communication, coordination, and collaboration among different entities are required?

- What factors could have significant effects on team responses?

Scenario Timeline The scenario timeline describes the scenario events in chronological order. Each event is a force that drives the scenario direction. The final scenario direction is based on the integral effects of all driving forces. Developing a scenario timeline is essential for retrospective event analysis because the time information provides clues to probe human performance issues. For predictive event analysis, developing a timeline is beneficial to knowing the approximate scenario flow, recognizing that large uncertainties could exist in the estimated time and in the event sequence.

Constructing the scenario timeline is an effective way to develop a coherent scenario story. The system-generated data should be used as the primary data source to establish the timeline and sequence of the anchor events because this is the most objective information. The information obtained from interviews is placed next to the anchor events. The information obtained from interviews provides rich context and causal relation information, but the information accuracy is subject to the reliability and subjectivity of interviewees memory. IDHEAS-G recommends that team members discuss any risk considerations that might be missed from the event list and try to resolve the conflicting and unclear information. For conflicting, unclear, or missing information that the analysis team cannot resolve, the interviewees should be contacted to resolve the issues. After the draft scenario timeline is available, the scenario should be verified with the interviewees. The information and understanding obtained through this process provides the materials to develop the scenario narrative and scenario timeline.

For retrospective analysis, such as the ASP analysis of the H.B. Robinson nuclear power plant fire event [106], the analysis focus is on the operators possibly failing to detect the loss of RCP seal cooling and the degraded RCP seal injection condition and restoring either of them prior to RCP seal failure. The ASP analysis estimates the HEP of restoring either the RCP seal cooling or RCP seal injection based on the conditions of the events (represented by the status of PIFs)

E-16

regardless of whether the crew in the actual event succeeded or failed to perform the tasks.

The ASP analysis considers both the crew success and failure in the actual event merely as a sample of the probabilistic estimate. In general, the ASP analysis uses the failure memory approach for the modeling (failures set to true and successes set to nominal - or adjusted as needed to better represent the event). Also, ASP is a special case since it is not used to calculate a change in core damage frequency - for the SDP, a decision would need to be made about how to adjust the baseline risk model in addition to the ASP case.

The conventional timeline uses a two-column structure with the first column showing the date and time, and the second column showing all other information. The NRC staff recommends adding symbols in front of each statement in the second column to distinguish the information types. The following are instructions for using the two-column format.

Column 1Date and Time For predictive (hypothetical) event analysis, the initiating event occurs at time zero. For retrospective (actual) event analysis, the initiating event starts at the local date and time that the actual event occurred. The actual local date and time has hidden information for assessing human performance. For example, if an event happens during a Sunday night, it could imply a reduced staffing level. If incidents occurred before the initiating event, the incidents should be indicated in the timeline. In this case, these events are placed before the initiating event as part of the background information.

Column 2All Other Information: System Response, Cues, Human Response, and Notes The information in the second column is classified into four types to improve the understanding of the human-system interactions. Each information type is denoted by a bold letter as described below:

System automatic responses (S): The S indicates that the information is a system automatic response based on the setpoints or logic of the automatic component actuations or that a system failed to perform its designed function. An example is S: safety injection injected coolant into the RCS at 1,600 psig.

Information needed for human responses (I): The I indicates the information generated from a system or other source that is available for the human to diagnose the situation or make decisions. Examples are the alarms that trigger operator notification about a system abnormality.

Human responses (H): The H indicates important human cognitive activities that include detecting the cue, making a diagnosis, entering or exiting procedures, making decisions important to the scenario, and performing actions. The actions could be either physical interference with a system to change the scenario progression or the actions that should be performed and, if not performed, allow safety degradation of the scenario.

Each human response should include the task and the individual who performs the task.

For example, a reactor operators (RO) action can be denoted as H(RO). If every crew member could perform the action, it can be denoted as H(Crew).

Notes (N): The N is used to provide background, explanation, context, or supplemental information to the system automatic responses (S), key information (I), and human response (H). For example, an H(RO) is depressurize the reactor pressure vessel to a certain pressure range at a rate less than 100 °F/hr. The (N) is the task takes about 2 hours2.314815e-5 days 5.555556e-4 hours 3.306878e-6 weeks 7.61e-7 months by periodically manually opening and closing a safety relief valve.

E-17

Realizing that constructing a detailed timeline is resource consuming and may be impractical, the analysis should be done at a level of detail that is technically justifiable to capture human actions that are important to achieve the goal and objectives of the event.

E.2.2 Identification of Additional Scenarios The PRA identifies scenarios leading to undesired consequences to perform a risk assessment.

During the development of the baseline scenario, the system responses and human actions that have safety impacts are identified. As mentioned in Section 4.2, the baseline scenario could be any event sequences identified in PRA models with explicit specifications of components failures and human failure events (HFEs). Identifying additional scenarios is performed by asking what if questions on the failures and effects of the systems and human actions. Each system response and human action is a branching point in the baseline scenario. The new scenarios identified should have risk impacts, including adding new event sequences to the PRA models or affect human error probabilities of the HFEs of the baseline scenario. The PRA/HRA team would make the determination. Failure of the system or humans to perform their required functions would generate new scenarios from the branching points. Each new scenario has its own branching points which, in turn, would generate more new scenarios. This results in generating many safety-related scenarios that share the same initial condition, initiating event, and boundary conditions. The event sequence diagram [107] and event tree

[103] are two techniques to represent the scenarios. The event sequence diagram uses pivotal points to represent the branching points. The event tree uses a set of top events for all branching points of all scenarios. The difference is that, in the event sequence diagram, the same task in different scenarios is represented by two pivotal events, but in the event tree, the task is represented by the same top event. Tools (software) are available to develop event sequence diagrams and event trees and to quantify the risk.

The focus on identifying additional scenarios is to develop a high-level risk perspective. The scenario progression and the system responses and human actions that if failed would change the scenario course are identified and represented by branching points. However, performing a detailed analysis to assess the human performance is not the focus of the identification of additional scenarios.

Example: The main steam line radiation alarm is an important cue to identify the steam generator tube rupture. The question What if the radiation alarm failed? is about the probability that the human action can successfully isolate the broken steam generator.

The failure modes and mechanisms of a system response or human action failure could affect the scenario progression; therefore, the failure modes and failure mechanisms should be identified when the system responses and human actions are identified.

Example: The human action of not depressurizing or overly depressurizing the reactor pressure vessel could result in different scenario consequences. Depending on the analysis objectives, the two failure modes may need to be modeled separately.

Example: A turbine-driven auxiliary feedwater (AFW) pump failure could be a mechanical failure of the auxiliary feedwater pump or loss of control power due to depletion of direct current (DC) power. The mechanical failure of the AFW pump only affects the auxiliary feedwater system. The DC power depletion has a wide range of effects and may impact many systems in addition to AFW. Because of the significant differences affecting human responses and scenario progression, the two failure mechanisms should be modeled separately.

E-18

It may not be practical to document the scenario narrative and scenario timeline for every new scenario. However, IDHEAS-G recommends providing brief descriptions of why the new scenarios are generated (e.g., the related failure modes of system responses or human actions).

E.2.3 Scenario Analysis Based on Relevant Probabilistic Risk Assessment Models Since the 1975 publication of WASH-1400 [108], PRA modeling has become more efficient after decades of improvement. It is more likely that a new PRA model will be developed by modifying existing PRA models rather than by developing a new model from scratch (e.g., developing a fire or security PRA based on an internal events PRA).

Example: In scenarios in which the internal events operator actions are assumed failed because of impacts to the instrumentation or equipment, for the analysis of a different initiating event, the HRA analyst may need to credit additional actions. For example, the operator manually starting a pump from the main control room could be the only human action to start the pump. However, a fire scenario could fail the main control room switch. This results in a human error probability of 1.0 (certainly fail). For the fire PRA, the HRA analyst may wish to credit a local action to start the pump.

To have a good understanding of the scenario progression, even though a relevant PRA model is available, the HRA analysts still need to develop the operational narrative that includes a detailed baseline scenario (scenario narrative and scenario timeline if practical) and other representative risk-important scenarios for the new analysis. The operational narrative is necessary because it establishes familiarity with the scenario and provides the foundation for a quality HRA. Without developing the detailed baseline scenario, the foundation of the HRA becomes questionable. The existing PRA serves as a reference to facilitate the identification of representative risk-important scenarios and key systems and human responses to be modeled for the new analysis. Once the new model is available, the existing PRA model serves as a reference model to check if any important scenarios, systems, or human responses are missing from the new model.

E.3 Collection of Information for Human Reliability Analysis The quality of information collection directly affects the quality of the scenario narrative and scenario timeline, which in turn affects the HRA quality. Several NRC documents, such as NUREG/CR-2300 [103], SHARP1 [109], [110], and ATHEANA [10], [11] have provided guidance on collecting information for HRA. Taylor [111] developed the handbook for HRA scenario analysis with structured guidelines on how to collect information for HRA. This section summarizes and discusses the common techniques for HRA information collection and provides guidance on addressing biases and uncertainties associated with the information collected.

E.3.1 Techniques for Collecting Information for Human Reliability Analysis Commonly used information collection techniques include the following:

simulation data review observation of simulator exercises and review of operational documentation operating experience review talk-through and walk-through Information collection may involve all or some of these techniques, depending on the availability of the resources required for each technique. For example, HRA analysts may not always have E-19

access to simulator exercises. IDHEAS-G uses NPP events as an example to describe how each technique works, while the guidance in the description of each technique applies to nuclear-related HRA applications in general. Also discussed in this section are addressing uncertainties in the information collected and handling potential biases in information collection and interpretation.

The information collection activities typically include collecting the information in the system data logs, computing data and recorders, observing simulated exercises, reviewing related procedures, training materials, drawings, system descriptions, photos, and operating experience, walk-through and talk-through interviews (i.e., interviewing the individuals involved in or familiar with the event), and developing a coherent scenario story.

HRA includes both retrospective analysis of real events and predictive analysis of hypothetical events. For retrospective analysis of real events (e.g., the NRCs ASP program and SDP), the information collections goal is to collect the event facts to reconstruct the event with sufficient information coverage and details to perform an HRA. For predictive analysis of hypothetical events or the analysis of the normal work process, the information collections goal is to assemble reliable predictions about the scenario progression with sufficient information and details to perform an HRA. A team made up of members from multiple technical disciplines related to the analysis generally collects the information. The team needs to be familiar with the operating system and must be able to integrate the pieces of the collected information to develop an integral understanding of the context affecting human behavior and performance in the scenario under analysis. The specific technical disciplines required are analysis dependent.

The general experts include HRA, PRA, operations, and system engineering. Other domain expertise is added based on the scenarios risk considerations.

System Data For retrospective event analysis, the system data logs provide an objective time and sequence of the events that occurred (based on the recorded parameters) in the scenario. Most U.S. nuclear power stations have sequence of event recorders or other equipment that continuously records an extensive set of plant parameters. The data are recorded to monitor plant performance and to support event investigations. The events recorded in the sequence of event recorders are in chronological order. It is an objective information source used to understand the initiating event and to identify the event sequence and timing in the scenario.

The control room data strips, either on paper or digitized charts, record key system parameters before and after the initiating events. The data strips should cover the time of the whole event.

Plant computers may constantly record certain plant parameters. In addition, shift turnover logs and operator logs provide information about system status and the operator activities that may relate to the event. These information sources provide objective information for understanding the scenario. The information, if available, should be used as the anchoring events to integrate the information obtained from other sources. The operations (e.g., starts/stops and open/close) of the safety- and non-safety-related systems, structures, and components used for the event mitigation or the ones that affected the safety-related systems involved in the scenario and results should be included as the anchoring events to develop the scenario narrative and timeline.

For predictive event analysis and predictive analysis for normal operation, the term system data refers to computer simulation data that provide information about system parameters and system responses. With reasonable timing, estimations of human actions, and quality assurance of simulation fidelity, the events timing can be used as anchoring events to develop the scenario narrative and timeline.

E-20

Simulator Exercises This discussion applies to both retrospective event analysis, predictive event analysis, and predictive analysis of normal operations. Observing simulator exercises to demonstrate how the tasks are performed provides the timing information and other considerations (e.g., complexity and the needed resources) related to the task performance. The information could resolve technical questions related to the task and its effects on safety or cues to identify other safety considerations. Before observing a simulated exercise, analysts should examine whether there are any potential accident conditions under which the procedures might not match the situation as well as would be desired (e.g., potentially ambiguous decision points or incorrect guidance provided under some conditions). Information about such potential vulnerabilities will be useful later during quantification and may help identify actions that need to be modeled. While a walkdown of the control room and observations of simulator exercises and talk-throughs with crews about various accident scenarios are probably most important during the modeling phase, if time and resources allow, they may also be useful during the identification phase to help analysts understand the procedures and how the crews implement them.

Operating Experience This discussion applies to retrospective event analysis, predictive event analysis, and predictive analysis of normal operations. Operating experience from similar events in similar facilities would provide not only human behavior information for real events but also cues to identify additional safety considerations. The HRA analysts are encouraged to include operating experience relevant to the analysis in their documentation. The operating experience could be from similar events or from performing similar tasks in similar facilities.

In principle, in developing questions for each of the macrocognitive functions listed above, analysts must be aware that a team is most likely responding to the safety issue. The teamwork-related considerations such as communication, supervisor, and team coordination should be considered. A general check to ensure that the interview questions adequately cover each macrocognitive function is to examine whether the when, what, who, where, why, how, and how-much aspects are covered as comprehensively and in as much detail as practical.

For retrospective event analysis, the interviewers should try to reduce the interviewees anxiety as much as possible by clearly stating the purposes of the interview, not criticizing the individuals, and not giving an impression of interrogation. The interviewers should document the interviewees and their positions. They should start the interview with open-ended questions to let the interviewees describe the scenario from their viewpoint. During the interview, establishing a dialogue by sharing information appropriate for the conversation with the interviewee will make the interviewee comfortable speaking and recalling information. The interviewees statements can provide cues for more or specific questions. Reviewing the draft questions prepared for the interview to ensure that all questions are addressed is important.

E.3.2 Resolution of Uncertainties or Contradictions Uncertainty is an important element of HRA and a quantitative evaluation of uncertainties should be performed throughout the HRA process, to the greatest extent possible. HRA analysts often have to deal with uncertainties in information collection. They may receive contradictory information from different sources. For example, information from a talk-through with different operators may not be consistent because it may include both objective experience and subjective judgment. Taylor and Le Darz [112] interviewed HRA analysts for their experience in information collection and documented the following in the report:

E-21

Although most of the interviewees agreed that they often have to deal with uncertainty in their analyses, there was a difference of opinion regarding whether they would usually encounter contradictory information. Some interviewees stated that they would almost never receive contradictory information, whereas others said that they do sometimes get this and it is not unexpected because people can sometimes be out of date in their information. In both cases, however, the interviewees noted that the best way to resolve this is to talk to additional operators to cross check the information received. If possible, also observe the scenario to confirm the information received. Sometimes the HRA practitioner will have to make a judgement about which person is the most trustworthy information source. This requires experience, but such judgements should always be documented clearly and objectively.

Taylor and Le Darz [112] discussed sources of uncertainties in information collected:

Uncertainty can occur in scenarios that take place over a long timescale (e.g., many hours or days), when there may be no operating procedures available, or where the operating procedures are not fully developed. Uncertainty can also occur in situations where the task is particularly complex and where there may be variability in operators opinions about what actions they would take and the potential human errors that could occur.

E.3.3 Safeguarding Against Bias or Misinterpretation Information collection, especially when conducted through observation, walk-through, and talk-through, involves knowledge elicitation, interpretation, and integration. These processes are subject to bias and misinterpretation. Taylor and Le Darz [112] discussed the sources of bias and safeguarding against the bias. They noted the following in the report:

Here are two main forms of bias that could affect an HRA: the HRA practitioner could develop a bias as a result of information obtained from one or more information sources during the scenario analysis, or the plant personnel could develop a bias as a result of the information provided by the HRA practitioner during the scenario analysis (e.g.

during an interview). Equally, there can be misinterpretation on behalf of both the practitioner (in terms of understanding the information received) and the plant personnel (in terms of understanding what information the practitioner is looking for).

Some of the interviewees (the HRA practitioners interviewed) noted that it can be difficult to manage bias and misinterpretation during scenario analysis. To safeguard against this, some of the interviewees reported that they treat every analysis separately and always work on the assumption that every plant is different. Some interviewees pointed out that, although they frequently review previous analyses to learn what has been done before and to build on this previous knowledge, they do not directly copy from these previous analyses and they strive to reflect the reality of how the plant is operated now.

For example, if the previous analysis made a recommendation to improve a procedure, the practitioner will check that the improvement has actually been made and has been effective, rather than merely assuming so and crediting this recommendation in the new analysis. Thus, it is important for HRA analysts to cross check the information collected against plant specific procedures, the [updated final safety analysis report], training lessons plans, and operator logs, and the plant licensing basis as ways to safeguard against biases.

Other ways that interviewees manage bias and misinterpretation is by having more than one HRA practitioner take part during interviews (and comparing notes afterwards), and E-22

by interviewing multiple people to ensure a balanced view. A good quality assurance process should also help to manage bias and misinterpretation. If a previous analysis or other information source seems overly simplistic or conservative, then the practitioner should investigate this in more detail. However, this ability to judge simplicity or conservatism may only come with experience.

E.4 Interaction with Other Disciplines for Analysis A risk analysis team, traditionally referred to as a PRA team, requires expertise in multiple technical disciplines that generally represent the collective expertise of various people. For example, the technical disciplines of a reactor safety analysis to support a complex risk analysis may include thermal-hydraulic analysts, PRA analysts, consequence analysts, HRA analysts, seismologists, system engineers, and an operations expert who has in-depth knowledge about the plant-specific information of the reference plant used for the study. Interacting with team members to apply consistent modeling assumptions and having coherent information is very important, especially because the scope of the risk analysis, information availability, and information detail could vary significantly from one case to another. NUREG/CR-2300 [103] and NUREG-1792 [13] provide guidance on the interaction.

HRA analysts need information from the other disciplines to perform the analysis. As part of a team, one disciplines analysis results could affect the other disciplines analyses. This could occur at any stage of the analysis process. Each disciplines revised results could be fed back to the team to affect the other disciplines analyses. These team interactions are a dynamic process as stated in the PRA Procedures Guide [103]:

The event sequence diagram tends to include a significant amount of design and operational information relative to the potential success paths. Their construction is an iterative process with input from various PRA team members, particularly those who have transient analysis, operational, and simulator experience.

Therefore, the HRA analysts in the analysis process should closely communicate with the analysts from other disciplines to ensure that the team members apply the same modeling assumptions and use consistent information. This would avoid undesirable surprises at later stages of the analysis process. An example of useful communication is refining the draft scenario narrative and timeline to be consistent with inputs from other disciplines, such as PRA assumptions, and the system response timing generated by simulation of the scenario. Another example is to communicate the what if questions to the PRA analysts to facilitate integration of HRA results into a PRA model. For the HRA analysts, this process provides opportunities to broaden and deepen their understanding of the scenario. This understanding is essential to improve the analysts sensitivity in identifying human performance considerations for more reliable scenario modeling and human reliability estimates.

In addition to the interactions between the technical disciplines of the analysis team, the additional information or knowledge gained at early stages of the HRA process could affect the HRA at later stages. For example, new risk-important human actions or new scenarios may be identified because new knowledge is gained by performing a more detailed analysis or obtaining information that was overlooked. It is important to have a solid qualitative analysis to set a good foundation for later stages of the analysis process. Otherwise, the quality and efficiency of the HRA analysis are likely to become an issue.

E-23

E.5 Summary In summary, this chapter discusses scenario analysis using a human performance model, scenario context, and operational narrative to establish the foundation for performing a quality HRA. The scenario analysis provides a framework to systematically document the basic information about the scenario and enhance the HRA analysts understanding of the scenario.

The information and understanding will serve as the basis for the rest of the HRA analysis, such as identifying human failure events, analyzing tasks in the human failure events, identifying task failure modes, assessing PIFs, and analyzing time uncertainties. The outcomes of the scenario analysis also serve as a means to communicate with analysis team members from different technical disciplines to ensure a cohesive understanding of the assumptions applied to the analysis.

E-24

APPENDIX F IDENTIFICATION AND DEFINITION OF IMPORTANT HUMAN ACTIONS The important human actions (IHAs) are the human actions explicitly modeled in risk assessments and safety analysis. They are called human failure events (HFEs) in PRA [103],

items relied on for safety in integrated safety analysis [105], [113], and risk important human actions in some other places.

Before the IHA identification, the analysts need to specify the study objectives and acquire substantial information related to the system design and operations. IHAs are identified through an integrative and iterative process of divergent search and convergent screening to identify the scenarios (including an initiating event, and the components and human actions needed to respond to the event), component failure mechanisms, human error modes, dependency mechanisms, and end states (or consequences).

The master logic diagram [114] (as shown in Figure F-1) is a technique to construct the relations between end state, system functions, system, component, and component failure modes. The failure mode and effect analysis is a technique used to construct the master logic diagram to develop the component failure modes and their effects (including causing an initiating event) and to identify the systems, components, and human actions to respond to the identified initiating events. The identification of the initiating events, systems, components, and human actions to be modeled in a risk assessment or safety analysis is supported by the knowledge of system design, system operations, operating procedures and training. An iterative process refines the set for final representation in the risk assessment or safety analysis.

Figure F-1 A Master Logic Diagram Structure IHAs are the human actions explicitly modeled in the risk assessment or safety analysis and are divided into three groups: initiator, pre-initiator, and post-initiator. The initiator IHAs cause the F-1

initiating events. The pre-initiator IHAs are the maintenance and surveillance testing actions that work on the systems or components needed to mitigate the initiating events. Failure of the pre-initiator IHAs causes the systems and components to be unable to perform their designed functions to respond to the event. This type of failure is called latent failure. The post-initiator IHAs are the human actions to respond to the initiating events.

F.1 Identification of Pre-Initiator Important Human Actions Component reliability is calculated based on component performance data which include latent failures contributed by human error. Some types of maintenance and calibration errors may not be discovered by post-maintenance testing. Those errors are eventually revealed by equipment problems, including failures. Their prevalence depends on the plant-specific maintenance practices and the effectiveness of the plant-specific post-maintenance inspection and testing protocols. Most PRAs do not separately quantify these causes for equipment failure, because it is too resource-intensive to extract them from the equipment performance records.

PRAs explicitly identify, model, and quantify pre-initiator human errors with certain effects on plant safety. Examples are errors that do not restore equipment to their normal alignments after maintenance and testing activities, miscalibration of instrumentation and signal processing logic, etc. A typical PRA model may contain dozens of these pre-initiator HFEs. In some cases, depending on the plant-specific design, maintenance protocols, and testing frequencies, these types of errors can be important contributors to risk. For nuclear reactor safety, pre-initiator IHAs modeled in PRAs are typically the actions of calibration and alignment, affecting single or multiple trains or systems.

The Electric Power Research Institute (EPRI) report, Data and modeling of pre-initiator human failure events in probabilistic risk assessment [115] provides guidance to identify the mis-alignment and mis-calibration IHAs in PRA. The guidance recommends identifying pre-initiator IHAs by a review of plant planned maintenance, testing, or inspection schedules. Particularly, documents implementing the Maintenance Rule in 10 CFR 50.65 (a)(4) provide a rich data source. SHARP1 [109], [110] also provides guidance on identifying pre-initiator IHAs as well as recommendations on screening out less important pre-initiator IHAs to focus the analysis on other IHAs. The EPRI report and SHARP1 emphasize the focus of identifying pre-initiator IHAs on those affecting multiple trains and systems.

In summary, the pre-initiators IHAs to be included in the model are those affecting multiple components simultaneously or affecting a single component. They have the following characteristics:

The human actions involve a mechanism that affects redundant or diverse equipment that is required for event prevention or event mitigation (e.g., there is a general consensus that, because of its common cause potential of mis-calibration, the common mis-calibration of a whole group of sensors (e.g., all level sensors) is most important).

Thus, it is normal to see PRA models with common mis-calibration events included in the plant logic model (e.g., for high high-level containment pressure or for refueling water storage tank level) [109], [110].

The IHAs, affecting either multiple or single components, have all of the following characteristics:

- They cause the systems or components to not be able to perform their required functions in the scenario.

F-2

- The failure or degradation of the systems and components affects the event prevention or event mitigation.

- The failure or degradation of the systems and components is unlikely to be detected before the initiating event.

F.2 Identification of Initiator Important Human Actions The initiating event frequencies are calculated based on operating experience, which includes the contribution of human error and the other causes. Therefore, the initiator IHAs are typically not modeled separately. If the analysts want to separately model the initiator IHAs, the analysts should ensure that there is no overlap between the initiating event caused by human error and by other causes. For nuclear reactor safety, the initiator IHAs, if separately modeled, typically involve the support systems that support multiple frontline components and systems. Failure mode and effect analysis is an effective approach (e.g., using fault tree analysis) to identify the support systems failure modes causing initiating events. The initiator IHAs are identified by finding the opportunities for human action to cause the specified failure modes [116].

F.3 Identification of Post-Initiator Important Human Actions The post-initiator IHAs are identified by reviewing operating procedures, interviewing operating personnel, conducting walk-throughs, observing simulator training, and reviewing operating experience. Because of the dynamic nature of human-system interactions, identifying the representative scenarios following an initiating event requires a significant combination of knowledge of the responses and reliabilities of systems and components, operator training, and procedure instructions. For reactor safety, the ASME/ANS PRA standard [44] had the following three high-level requirements relevant to the identification of the post-initiator IHAs:

(1) A systematic review of the relevant procedures shall be used to identify the set of operator responses required for each of the accident sequences.

(2) Human failure events shall be defined that represent the impact of not properly performing the required responses, in a manner consistent with the structure and level of detail of the accident sequences.

(3) Recovery actions (at the cutset or scenario level) shall be modeled only if it has been demonstrated that the action is plausible and feasible for those scenarios to which they are applied.

F.4 Identification of Errors of Commission An IHA could be desired or undesired. The desired IHAs mitigate the event. Failure to successfully perform the desired IHAs is a human error, an error of omission (EOO). The undesired IHAs degrade the event. Performing the undesired IHAs is a human error, an error of commission (EOC). The previous three sections have discussed the identification of the desired IHAs. This section discusses the identification of the undesired IHAs (EOCs).

NUREG-1880 [11] defines EOC as follows:

A human failure event resulting from an overt, unsafe action, that, when taken, leads to a change in plant configuration with the consequence of a degraded plant state.

Examples include terminating running safety-injection pumps, closing valves, and blocking automatic initiation signals.

F-3

NUREG-1921 [63] describes the undesired action as a subset of EOC in a fire scenario:

An undesired action is defined as a well-intentioned operator action that is inappropriate for a specific context and that unintentionally aggravates the scenario. Undesired responses consist primarily of shutting down or changing the state of mitigating equipment in a way that increases the need for safe shutdown systems, structures, and components. The key criterion in identifying undesired operator actions is that the action leads to a worsened plant state (e.g., turning a transient initiating event into a consequential loss of coolant accident). For example, spurious indications occur when electrical cables routed through a zone in which the fire is postulated are shorted, grounded, or opened as the cable insulation is burned. These instrument wires feed alarms and control indications that act as cues for operator actions. Therefore, an undesired action can be triggered through a false cue that tells the operator to take an action that is potentially detrimental to safe shutdown. For example, an action is classified as undesirable if the operators conclude, from false cues, that the safety injection (SI) termination criteria are met and then shut down SI when it is inappropriate to do so.

NUREG-1921 emphasizes that EOCs are well-intentioned actions. This is a key principle for EOC identification. EOCs that occur at random are typically not modeled because that would require additional analysis effort without yielding practical benefit. However, well-intended actions that lead to undesirable risks need to be modeled because they occur for a rational justification and can dominate the outcomes of event responses. Operational experience with nuclear power plants has shown that operators may delay or inhibit a necessary function of the standby safety systems for various reasons in responding to initiating events.

ATHEANA [10] and the Fire Human Reliability Analysis Guidelines [63] discuss in detail the identification of the undesired IHAs (EOCs). The discussions are summarized in the following items:

The action directly disables the system, subsystem or component needed to provide the system function required in the scenario. As a result, the EOC alters the event scenario progression in a way that is not anticipated by the PRA models for the nominal event sequence and is not evaluated by the identified EOO HFEs. For example, an EOC might cause a transient scenario to develop into a consequential LOCA scenario. This context is different from inadvertently disabling or failing to start equipment that is needed to mitigate conditions in the nominal scenario.

There is a rational justification to indicate that the EOC is well-intentioned. The common situations are (1) there are competing goals and (2) personnel cannot fully evaluate the consequences of the decided action, or personnel do not understand the systems and consequences of the decided action.

The unintended (slip type) human errors have EOO and EOC considerations that need to be analyzed separately. For example, if the wrong pump is switched off because the pump switches are close together, two analyses are needed because, first, the intended pump was not switched off (an EOO), and second, an unintended pump was switched off (an EOC). Whether the EOC should be explicitly modeled depends on the EOCs impact on the scenario. The EOC should be modeled explicitly if it has a cascading effect on the scenario progression. If the EOC affects only the workers performance (e.g., it increases workload), then the EOC does not need to be explicitly modeled.

F-4

F.5 Composition of Important Human Actions An IHA typically comprises several human cognitive and physical activities and could be jointly performed by more than one person. The end point of the IHA is the human physical actions to deliver the desired function (for intended IHAs) or to negatively affect the scenario (for the unintended IHAs). An IHA comprises the activities starting from the worker sensing the need to do something (e.g., perceiving an alarm or receiving a work order to perform a routine task) that starts a sequence of cognitive and physical activities to the end point of the IHA. The cognitive and physical activities between the two points include diagnosis of the situation, communication, movement to different physical locations, performing probing actions to verify hypotheses, deciding the appropriate procedure to implement, recovering from error, etc. Explicitly modeling the activities individually is impractical and unnecessary. The task analysis in IDHEAS-G provides guidance to model IHA reliability.

In some situations, IHAs are grouped to reduce the total number of IHAs in the risk assessment or safety analysis to control the model size (e.g., the event tree in PRA) for practical assessment. An example is, in an extended loss of alternating current power (ELAP) event, to shed the DC load and use the portable generator to charge the station battery. The plant has two levels of DC load shed: (1) an initial DC load shed after a station blackout event is declared; and (2) a deep DC load shed after an ELAP event is declared. The human actions and effects are summarized as the following (Figure F-2):

If operators fail to perform the initial DC load, the station battery will deplete within 2 hours2.314815e-5 days 5.555556e-4 hours 3.306878e-6 weeks 7.61e-7 months .

If they successfully perform the initial DC load shed but no other actions, the station battery will deplete within 5 hours5.787037e-5 days 0.00139 hours 8.267196e-6 weeks 1.9025e-6 months .

If they successfully perform the initial DC load shed and the deep DC load shed but no other actions, the station battery will deplete within 7 hours8.101852e-5 days 0.00194 hours 1.157407e-5 weeks 2.6635e-6 months .

If they successfully use the portable diesel generator to charge the essential batteries but without fuel replenishment, the station batterys operation time is extended for an additional 12 hours1.388889e-4 days 0.00333 hours 1.984127e-5 weeks 4.566e-6 months .

When successfully using the portable diesel generator to charge the essential batteries with fuel replenishment, the station battery can operate throughout the whole scenario.

In the above example, the human actions may be combined to have fewer IHAs to balance the analysis effort and gain risk insights. This requires evaluating different failure effects on scenario progression, demands on components and systems, effects on human behavior, and other factors to ensure that the IHA combination does not significantly affect the risk profile.

F-5

The initial DC load shed The portable generator is performed in time. is deployed in time.

The DC system The deep DC load shed The generator fuel is DC Power Available survived the hazard. is performed in time. replenished in time. Duration (hr)

ELAP The whole scenario Yes Yes Yes Yes Yes 19 No No 7

No 5

No 2

No 0

Note: The time information in the above tree is only to illustrate human actions effects. The values do not represent any plant.

Figure F-2 Example of DC Availability in an ELAP Event Related to Human Error in Shedding the DC Load F.6 Summary This appendix describes IHA identification as summarized below:

Failure to perform the desired IHA is an EOO. Performing the undesired IHA is an EOC.

The pre-initiator and initiator IHAs are typically not explicitly modeled. If modeled in reactor risk assessment, the pre-initiator IHAs typically are the human actions of calibration or system alignment and affect single or multiple trains or systems; the initiator IHAs typically are the human actions of working on the support system that could affect trains or systems.

The identification of undesired IHAs (EOCs) focuses on the well-intended human actions that have a significant negative impact on scenario safety.

An IHA includes all the cognitive and physical activities from sensing a need to act to the end point of the physical actions to achieve the IHAs function. The task analysis in IDHEAS-G provides guidelines to model the IHAs reliability.

F-6

APPENDIX G TASK ANALYSIS Task analysis was long recognized as an important component of HRA. Task analysis is well understood by human factors professionals. Many task analysis techniques have been developed for various purposes of human factors engineering. This appendix provides a specific, logical framework and vocabulary for performing task analysis for HRA.

In Kirwan and Ainsworths book, A Guide to Task Analysis [117], task analysis is described as follows:

Task analysis covers a range of techniques used by ergonomists, designers, operators and assessors to describe, and in some cases evaluate, the human-machine and human-human interactions in systems. Task analysis can be defined as the study of what an operator (or team of operators) is required to do, in terms of actions and/or cognitive processes, to achieve a system goal. Task analysis methods can also document the information and control facilities used to carry out the task.

In IDHEAS-G, the purpose of task analysis is to identify and characterize tasks within a human action. HRA quantification is performed on the critical tasks of an important human action.

Task analysis identifies the tasks critical to the success of the event and analyzes the cognitive activities in the tasks. The cognitive activities in a task are assessed according to the five macrocognitive functions: detection, understanding, decisionmaking, action execution, and teamwork. Task analysis is performed and documented with several graphic representations:

A task diagram identifies the task sequences or paths that lead to the success.

A timeline represents all of the tasks from the beginning of the important human action and key timing information such as the onset of important cues and time available for the tasks.

A teamwork diagram illustrates the required interteam collaborative activities (e.g.,

communication, coordination, and cooperation).

Many task analysis techniques and methods have been developed for various human factors applications. Most of those were developed for purposes other than HRA (such as interface design, staffing, and training program design). IDHEAS-G integrates existing task analysis methods to develop the HRA task analysis guidance presented in this appendix. The overall structure and the outcomes of this HRA task analysis guidance are organized for performing HRA quantification analysis. The guidance is focused on the structure and outcomes, along with high-level guidance for the process of achieving the outcomes. Section G.1 introduces the definitions of tasks and critical tasks. Section G.2 describes task identification and graphic representations. Section G.3 includes the guidance for characterizing critical tasks in an important human action. Section G.4 introduces task analysis methods in general and several task analysis techniques that have been demonstrated useful for HRA.

G.1 Task Structure and Critical Tasks G.1.1 Task Structure A task usually refers to a clearly defined piece of work, sometimes of short or limited duration, assigned to or expected of a person. A task is a set of related human activities to achieve a G-1

common goal. In IDHEAS-G, the term task refers to a clearly defined piece of the important human action required to achieve the success criteria of that action. In short, an important human action can be divided into a set of tasks. An entire human action may be performed by different people or teams, at various locations, and in different time intervals. Breaking an important human action into discrete tasks can facilitate the analysis of human performance and quantitatively assess the likelihood of human errors in performing the tasks.

Not all of the tasks in an important human action are essential to the success of the action.

Some required tasks are confirmatory, and incorrectly performing them would not necessarily lead to failure of the important human action. Personnel may also perform secondary tasks that do not necessarily relate to the success criteria of the important human action. The critical tasks are the ones that are essential to the success of the important human action, and failure of any of the critical tasks will fail the important human action. Thus, each critical task represents an opportunity for failure. HEP quantification of an important human action is performed for the critical tasks (i.e., the HEP of an important human action is the combined HEP of all of the critical tasks (see Equation (4.2)).

Figure G-1 shows the IDHEAS-G task structure (i.e., how a task is represented in relation to the various levels of analysis in IDHEAS-G). A task is at the lowest level of IDHEAS-G qualitative analysis: scenarios have one or more important human actions, and an important human action is divided into multiple tasks. On the other hand, a critical task is the basic element on which HRA quantification analysis is performed. The HEP of an important human action is the combined HEPs of all the critical tasks in the important human action. A task consists of cognitive activities, which are achieved through macrocognitive functions. Failure of the macrocognitive functions is represented with various cognitive failure modes. The HEP of a critical task is the combined HEPs of all its cognitive failure modes (see Equation (4.3)).

Human Event IHA 1 IHA 2 IHA 3 Critical Critical Critical Critical Critical Task Task 1 Task 2 Task 3 Task Macrocognitive functions required for the task Figure G-1 IDHEAS-G Task Structure G.1.2 Definition of a Task An important human action can be broken into tasks at any arbitrary level of granularity. One traditional question in performing an HRA is the level of detail at which an important human action should be decomposed into tasks. A related question is how to define the boundaries of the tasks in an important human action. In human factors and HRA practice, the level of G-2

dividing tasks depends on the specific application and analysis purpose. There are no universally applicable criteria for task breakdown. IDHEAS-G provides the following general guidance on dividing and identifying tasks:

What is a task?

- A task constitutes a recognizable and consequential unit of human activities;

- A task needs to be performed by humans to achieve a desired plant status;

- Successful performance of the action portion of the task will alter the scenario course toward safer plant status

Boundaries between tasks can be distinguished by any of the following:

- Clearly defined goal

- Clearly defined initial or entry state

- Clearly defined ending or exit state (i.e., consequences or outputs)

The scope of a task is as follows:

- A task may be represented with one or several macrocognitive functions

- A task does not have to involve physical manipulations. A task could be information collection, situation assessment, decisionmaking, or coordination of teams.

The basis of task breakdown for HRA quantification is as follows:

- The tasks in an important human action for HRA quantification should be broken down to the level where data or evidence (experts, job performance behavior, models) of human performance measures exists for HEP estimation.

- In the IDHEAS framework, an important human action should be broken into tasks at a level that retains the important human action context and can be represented with macrocognitive functions.

Note that the basis for task breakdown in HRA also serves as the basis for HEP estimation.

One problem in HRA quantification is mismatches between the data used to estimate HEPs and the task analysis. For example, an HRA method may provide the HEP values of the failure modes based on data of personnel performing individual steps of a procedure, while HRA analysts may apply the HEP values to a task that consists of many steps of a procedure or several procedures. Thus, task breakdown in an important human action should be based on the data or evidence used for probability estimation.

Example: Level of detail of task breakdown. In a NPP flooding event, one important human action is to cool down the reactor using portable pumps. One task of the important human action is setup of the portable pump, which includes clearing of debris and connecting the pump.

Clearing of debris could involve use of heavy or light equipment. Connecting the pump could involve fine-motor skills. Because flooding is a rare event, there is little data or evidence available to estimate the time required for and the failure probability of setup of a portable pump when the worksite is flooded. However, data and experts are available, respectively, for estimating the likelihood of failing to operate debris-removal equipment and the likelihood of failing to connect a pump. Therefore, the task of setup of a portable pump should be further broken into two tasks, both of which are critical for the important human action.

G-3

G.1.3 Critical Tasks Critical tasks are identified through analyzing the consequences of failing the tasks, opportunities for error correction, and the level of human involvement in the tasks. Reviewing existing documentation is usually the first step in identifying critical tasks. The critical tasks for an important human action may have already been defined in training programs, quality assurance documents, fault tree analysis, safety integrity level assessment, and other formats.

Yet, to use the documentation to achieve an inventory of critical tasks requires good knowledge of safety assessment techniques, such as the ability to understand and interpret the documents and relate the information to the tasks.

While the primary criterion for a critical task is the importance of the task to the success of the important human action, identification of critical tasks may also consider the error recovery opportunities. Because there may be opportunities for the operating personnel to recover from an error within the time window, the task analysis may also identify these opportunities. During a scenario, the cues to an important human action may occur at different times (e.g., additional alarms). Also, additional cues generated by expected scenario progression may be credited for error recovery opportunities. For tasks involving manipulation of systems, the error correction opportunities may arise from monitoring system feedback (e.g., indications that the system is not responding as would be expected if the intended action had been completed correctly). In general, tasks in safety-critical operational systems such as nuclear reactors typically have multiple error recovery opportunities as the result of redundancies in system design, procedures, and personnel work processes.

G.1.4 Uncertainty in Identification of Critical Tasks Identification of critical tasks deals with uncertainty in judging task criticality and error recovery credibility. This uncertainty is associated with information incompleteness, contradictory information obtained from various sources, and analysts mindset or misinterpretation.

Uncertainty can particularly occur in scenarios that take place over a long timescale (e.g. many hours or days), when there may be no operating procedures available, when the task is particularly complex, and when personnels opinions about what actions they would take may vary [111], [117]. Uncertainty should be discussed with other HRA and PRA analysts and with operations personnel to agree on how to address it in the HRA. An HRA should always document uncertainty in identification of critical tasks.

G.2 Identification and Graphic Representations of Critical Tasks For HRA quantification analysis, it is important to capture all of the critical tasks, their relations, cues for the tasks, consequences of failing a task, and timing information. Graphic representations of the tasks and their relationships help identify critical tasks and organize the outcomes of task analysis. A thorough task analysis should identify and represent the tasks in an important human action and key information for important human action quantification in three formats of graphic representation: a task diagram, a timeline, and a teamwork diagram.

G.2.1 Task Diagram A task diagram is a tree or network structure with nodes, branches, and paths that systematically represent the tasks and their relations to the systems operational states.

Developing a task diagram is a good place to start a task analysis because it makes the analyst think through the process and identify variability and challenges. A task diagram begins with the presence of the cues of the important human action and ends with the important human action G-4

success criteria. In between the beginning and end points of the diagram are the sequences, paths, or branches of tasks required to change the states of the systems to achieve success.

The relationship among the tasks is important for combining the HEPs of individual critical tasks into the HEP of the important human action. For example, some tasks can be carried out in any order or in parallel, and some tasks must be performed in a linear sequence; for others, the relationship is conditional (e.g., if such a condition exists for task A, perform task B). Some tasks may involve coordinated actions among crew members or between control room crew members and personnel in the field. Figure G-2 illustrates four task diagrams representing important human actions from simple to complex, as described below:

(a) A simple important human action may have only one task, meaning that an individual or a coherent team performs the entire action in a set time period and the status of PIFs remain the same for different parts of the action.

(b) An important human action may consist of tasks that form a single path to the success point. Failing a critical task breaks the success path.

(c) A complex important human action may consist of multiple intermingled paths and branches of tasks.

(d) A simple or complex important human action may have parallel, alternative paths to success. While performing the tasks along any of the paths will lead to the success, the failure modes and context can be quite different. Thus, different paths can result in different HEPs for the same important human action.

Simple Enter Task 1 Goal (a)

HFE the HFE achieved Single (b) Task 1 Task 2 Task 3 Task 4 task path Task 1 Task 2 Task 3 Task 4 Parallel or (c) intermingled Task 5 Task 6 Task 7 Task 10 tasks/paths Task 8 Task 9 Task 1 Task 2 Task 3 Task 4 Alternative (d) tasks/paths Task 5 Task 6 Task 7 Task 8 Task 9 Task 10 Task 11 Note: The red font denotes a critical task.

Figure G-2 Types of Task Relations Illustrated with Task Diagrams Example: Task diagram in the see-and-flee event. As shown in Figure G-3, the important human action has two parallel task paths for third floor and fourth-floor workers, respectively.

Each path has two critical tasks: detect the chemical release and decide to flee and flee from the building. Yet, the cues for the two paths are different. The chemical release occurred on G-5

the third floor, so the workers perceived smoke as the direct cue for contamination. The fourth-floor workers either perceived the smoke later or received the direction to flee from other personnel. The event may also be represented with a different task diagram by combining the two critical tasks on each path, detect the chemical release and decide to flee and flee from the building, into one critical task. The diverse ways of breaking down the tasks in the important human action result in the same set of macrocognitive functions; therefore, they are equivalent for the purposes of the quantification analysis.

3rd floor workers Detect the Flee from the chemical release building and decide to flee Chemical All workers release are outside 4th floor workers Detect the Flee from the chemical release building and decide to flee Figure G-3 An Example Task Diagram for the See-and-Flee Event G.2.2 Timeline A timeline represents the evolution of an important human action. For time-critical actions, the timeline can be used to determine and represent the overall time available and the tasks required to achieve a system state, the sequence and intervals at which tasks occur, the duration of a task, and the timing when the cues become available. Parallel timelines may be developed for scenarios that involve several individuals or teams working simultaneously and independently.

A timeline represents the occurrence of cues, critical tasks, and important transitions of system states along the timeline. It also represents noncritical tasks along the timeline for assessing their impact on personnel performing critical tasks. Noncritical tasks may include such things as peer-checking, routine monitoring, or maintenance activities. The timing information of concurrent noncritical tasks determines the status of some PIFs such as multitasking, interruptions, and distractions. In addition, noncritical tasks may share resources (e.g.,

personnel, tools, job aids) with the critical tasks.

Furthermore, estimates of time allow a determination of whether there is adequate time to perform the tasks. While precise timing of individual tasks is not required, it is important to identify the ordering of critical points (e.g., when does a system parameter reach a critical value that triggers a response, and when do personnel perform noncritical tasks in relation to the timing of performing critical tasks). Such timing information can be used to estimate the probability distribution functions of time available and time required for the important human action. The probability distribution functions are used to estimate HEPs attributable to having inadequate time to perform an action (details are described in Chapter 5, Time Uncertainty Analysis).

Example: Timeline for a see-and-flee event. In this hypothetical scenario, the workers on the third and fourth floors of the building are cleaning a uranium hexafluoride (UF6) valve in a 55-gallon drum on a decontamination pad with hot steam, which causes a UF6 release. The G-6

workers in the building need to flee from the building as quickly as possible to avoid exposure to UF6. Workers detect the cues of contamination and flee from the building through any available safety evacuation paths. This scenario may have other actions such as actions to (1) contact central control/command authority to inform them of the release, (2) inform other workers in the area, (3) secure the immediate release if possible, and (4) minimize the spread of contamination (e.g., stop ventilation).

The important human action see and flee is that workers handling nuclear materials see indications of chemical release on the third floor and flee for personnel safety. The goal of the action is to protect personnel from chemical contamination. This is an urgent action but is not time critical because there is no specification of the time available for workers to complete the action. However, the avoidance of a health effect from the release may impose a constraint on the available time to leave the area.

A timeline is developed to show the order of cues and tasks, as illustrated in Figure G-4. The timeline also indicates the potential overlap of third- and fourth-floor workers fleeing from the building. The information associated with the timeline includes:

Beginning point: Chemical release occurs

t0: Cues for chemical release are present

t1: Third-floor workers detect the release

t2: Third-floor workers begin to flee

t3: Fourth-floor workers detect the release

t4: Fourth-floor workers begin to flee

Ending point: All of the workers are outside the building Notice that the estimated time periods for the flee tasks starting at the third and fourth floors may overlap in time. That means that the workers on both floors may share the same resources (staircases, doors) in fleeing.

t0: Cues t1: 3rd floor t2: 3rd floor t3: 4th floor t4: 4th floor available workers workers workers workers detect the start to flee detect the start to flee cue cue All Chemical workers release t0 t1 t2 t3 t4 are outside Note: The array bar represents time; the thinner horizontal bars represent the estimation of time required to complete the task indicated.

Figure G-4 An Example Timeline for the See-and-Flee Event G.2.3 Interteam Coordination Diagram For events that involve collaborative teamwork across multiple entities, a teamwork diagram is used to represent the task sequences of the teams and the required teamwork activities, such as communication, coordination, command and control, distribution of decisionmaking, and authorization chains. A teamwork diagram delineates how the various teams work together.

The information represented in a teamwork diagram can be used to analyze and quantify the failure of the interteam coordination macrocognitive function involved in the critical tasks.

G-7

Example - Nuclear power plant crisis management teamwork. This example is adapted from Le Bot et al. [118]. In an NPP emergency, different operational states of the facility are anticipated. Because of the severity of the situation, an in-house emergency plan is initiated, leading to the mobilization of a multi-level crisis organization:

locally: local command post, local crisis team, shift manager

nationwide: national technical support team, national command post, AREVA crisis team, Institute for Radiological and Nuclear Safety (IRSN) crisis team (regulatory), and SEPTEN crisis team (another utilitys team of experts).

Figure G-5 shows the teamwork diagram for the situation.

Figure G-5 An Interteam Coordination Diagram in NPP Crisis Management [118]

G.3 Characterization of Critical Tasks G.3.1 Task Characterization for Human Reliability Analysis Once the critical tasks are identified, detailed characterization of each critical task should be documented. The objective of task characterization is to specify the context of the critical tasks for quantitative analysis. The characterization of a task determines the states of PIFs that are relevant to the task. The characterization also includes identifying cognitive activities involved in the task. The activities determine the macrocognitive functions and cognitive processes required for the task, thus these are the basis for determining the CFMs applicable to the task.

Characterization of a critical task should include, but is not limited to, the characteristics listed in Table G-1.

G-8

Table G-1 Task Characterization for HRA Task characteristics Description Task goal The expected outcome of the task with respect to the desired system states (e.g., reach hot shutdown within 3 hours3.472222e-5 days 8.333333e-4 hours 4.960317e-6 weeks 1.1415e-6 months , flee from the building).

Specific requirements Specifications for the task goal such as timing requirements or how the task goal should be achieved (e.g., monitoring parameters at a certain time interval, using secondary cues when the primary cues are not available, cooling down RCS within a certain rate).

Cues and supporting The cues to initiate the task and key information needed to perform the information task. A cue could be an alarm, an indication, a procedure instruction or others (e.g., onsite report). The supporting information is in addition to the cue required to perform the task.

Procedures Available procedures, guidance, or instructions designed for the task.

Personnel Types of personnel needed for the task, minimum staffing required, special skillset required.

Task support Job aids, reference materials, tools, and equipment needed.

Location Where the task is performed, special environmental factors at the location.

Cognitive activities Cognitive activities involved in the task that place demands on their corresponding macrocognitive functions.

Concurrent tasks Concurrent tasks that compete for personnels cognition and resources (e.g., tools, job aids).

Interteam Interteam collaborative activities required for the task and coordination requirements for communication facilities (e.g., equipment, tools, considerations devices, etc.).

G.3.2 Assessment of Cognitive Activities Involved in a Task Cognitive activities in a task are assessed to determine the macrocognitive functions needed for achieving the task. Performing a critical task involves the successful performance of one or more specific cognitive and behavioral activities, such as collecting data and comparing data to a decision criterion. In operational documents and domain expert interviews, tasks are generally described in terms of human behaviors with respect to systems. Such descriptions usually provide information about what macrocognitive functions are involved in a task. In the cognition model described in Chapter 2, IDHEAS-G provides a taxonomy of cognitive activities for each macrocognitive function. This taxonomy can be used to assess cognitive activities involved in a task and subsequently identify the macrocognitive functions involved. Table G-2 summarizes the taxonomy.

G-9

Table G-2 Taxonomy of Cognitive Activities Macrocognitive Types of cognitive activities function Detection

Detect cues (through carefully monitoring, searching, inspecting, or comparing, etc.)

Acquire information (checking, reading, communicating/chatting, computing, etc.)

Understanding

Maintain situational awareness

Assess status based on indirect information

Diagnose problems and resolve conflicts in information

Predict or form expectations for the upcoming situation development Decisionmaking

Make a go/no-go decision for a pre-specified action

Select among multiple options or strategies

Change or add to a preexisting plan or strategy (e.g.,

changes of personnel, criteria, sub-goals, etc.)

Develop a new strategy or plan Action Execution

Execute cognitively simple actions

Execute cognitively complex actions

Execute long-lasting actions

Execute control actions

Execute fine-motor actions

Execute physically strenuous actions Interteam coordination

Communication

Cooperation

Coordination (including command and control)

G.4 Task Analysis Techniques Many task analysis techniques are available to obtain the expected outcomes described in Sections G.1, G.2, and G.3. Selection of a task analysis technique for HRA depends on the nature of the event being analyzed and resources available for conducting task analysis [111].

Kirwan and Ainsworth [117] examined many task analysis techniques and recommended 10 techniques alone or in combination for HRA purposes. Taylor and Le Darz [112] piloted five task analysis techniques in an HRA case study and demonstrated that a combination of different task analysis techniques is typically more effective than the use of a single technique. A combination of techniques can provide comprehensive insight into how personnel perform the task and interact with other people and systems, and the factors that could influence the likelihood of personnel making errors in task performance. This section introduces several task analysis techniques that have proven useful for HRA.

G.4.1 Hierarchical (Functional) Task Analysis and Tabletop Analysis Hierarchical task analysis is usually depicted graphically. It describes the task in terms of its top-level goals and the individual operations or actions below each goal. The main task goals in a hierarchical task analysis may be developed based on initial information from the important human action definition and subject matter experts. Hierarchical task analysis is often performed with the tabletop technique, which is a talk-through with subject matter experts to G-10

identify the main functions or objectives of the event, types of personnel and systems associated with the functions, and expected tasks for the functions.

Tabular task analysis is a task description technique that records information in a columnar tabular format. The column titles will vary depending on the purpose and focus of the analysis.

It is usually used in conjunction with the hierarchical task analysis to further investigate and provide more detail about the tasks described in the analysis. The tabular task analysis format can be a useful tool for screening tasks and human errors and for documenting task characterization. A tabular task analysis may be developed to collate the information collected, identify knowledge gaps, and identify areas of focus. The tabular task analysis may be updated several times throughout the HRA as additional information and clarification are received. The tabular task analysis can document the information needed to identify CFMs, quantify HEPs, and document the links between the evidence to substantiate quantification calculations.

Both analysis techniques involve consulting with a group of experts who understand the systems to define and assess aspects of those systems. The discussions are typically directed around some basic framework (e.g., procedures). This technique can create detailed task information and can analyze that information in a problem-solving and explanatory way. Before consulting experts, analysts may gather information via document review to gain a basic understanding of response operations (e.g., necessary tasks, equipment, and personnel).

Experts are then asked targeted questions aimed at validating analysts understanding of response operations, gathering missing information, identifying gaps, and gaining a deeper understanding of specific aspects of response operations.

Example: Task diagram developed through a hierarchical (functional) task analysis with the tabletop analysis technique (from NUREG-2180 [119]):

HFE Definition: Respond to Very Early Warning Fire Detection Systems in Nuclear Facilities (in-cabinet, fire suppression strategy)

Response operations primarily involve four types of personnel: (1) main control room operators, (2) field operators, (3) digital instrumentation and control (DI&C) technicians, and (4) fire brigade. Main control room operators are responsible for detecting an alert, using the correct alarm response procedure, dispatching personnel to the fire location, and, on alarm, activating the fire brigade. The field operator is responsible for serving as the initial fire watch (with suppression capabilities) and opening cabinets. The DI&C technician is responsible for gathering necessary equipment, traveling to the fire location, and using the equipment to find the incipient fire source. The fire brigade is responsible for suppression duties once it arrives on the scene. A tabletop analysis was performed for these four personnel groups or teams. The results are represented in the task diagram as shown in Figure G-6 (from Figure 9-2 in NUREG-2180).

G-11

VEWFD Response Operations Event Field Operator Technician Fire Brigade Operational MCR Response Response Response Response Goal Detects Alert Travels to fire Retrieves Component begins location equipment to degrading locate incipient Begins using fire source Alarm Begins serving Response as posted fire Procedure watch at Travels to fire Alert identified bank location sounds in of cabinets MCR Consults MCR Begins using computer to equipment to determine fire locate affected location (i.e., cabinet bank of Locate cabinets) degraded component and Opens affected de-energize cabinet before flaming Component Continues to Degrade Dispatches FO fire occurs to fire location Uses equipment to locate Dispatches tech degrading to fire location component Continues Communicates monitoring the information MCR computer about screen during degrading the field component to investigation FO Maintains Maintains communication communication with FO with MCR throughout throughout investigation investigation Communicates information Makes decision about degraded to de-energize component to MCR Instructs FO to De-energizes de-energizeg Component Figure G-6 Generic Depiction of Operations in Response to an In-cabinet Aspirating Smoke Detection VEWFD (Very Early Warning Fire Detection Systems)

Alert Followed by Alarm where a De-energization Strategy is Being Used G.4.2 Crew Response Diagram The crew response diagram (CRD) is a part of the qualitative analysis guidance in the IDHEAS At-Power Application [22]. It applies to events with procedure-driven human responses. A CRD graphically represents and documents the task sequence in an important human action. A CRD starts with the first cue for personnel to perform the expected responses to succeed in the G-12

important human action. Following the cue, the responses that must be successfully completed to make the important human action successful are identified. A node is identified for each significant transition point in the response, such as entering a procedure, transitioning to another procedure, deciding how to respond to the situation, and execution. Each node also has a branching point followed by a success branch and a failure branch. The sequences leading to the success of the important human action are referred to as success paths. One or several tasks may be associated with each node. If the consequence of failing a task leads to the failure branch of the node, the task is identified as a critical task.

A CRD analysis also includes developing a timeline in parallel. The timeline captures the following:

the operational system status trajectory in terms of the timing of cues and other system process parameters that are required to trigger personnel to respond to the situation, to perform the right actions, and to realize an opportunity for error recovery

the time at which personnel are expected to reach critical steps in the procedure or the critical tasks are performed The objective of developing the CRD is to identify the expected crew response paths within the important human action that lead to success. This includes providing information to understand the path progression (e.g., procedure transitions). There could be more than one success path with or without explicit procedure transition paths. Identifying the success paths is beneficial for understanding the variations in performing the important human action. Trying to identify all possible paths (based on procedure instruction) may be time consuming and thus not practical for some PRA applications. The HRA analysts should maintain a balance between the number of success paths identified and the amount of effort to be invested in the analysis. Identifying too many success paths could make it more difficult to maintain adequate situational awareness.

The following describes the process of developing a CRD.

(1) Develop a Crew Response Diagram

Identify the operating procedures or training materials that apply to the scenario of analysis. The focus is on identifying the procedures (titles and identifications) and key parts of the procedures (procedure steps, foldout pages and checklists, etc.) that guide personnel to respond to the actions.

Determine the relevant cues and their timing. Cues could be alarms, plant indications, field reports, and procedure instructions, etc. to trigger personnel responses. Computer simulation of system responses could provide timing information for certain cues (e.g.,

tank water level is below a certain threshold or pressure is exceeding a certain level in NPP reactor systems). It is particularly important to identify the cues that lead personnel to enter the correct procedure(s) for the important human action. For a complete understanding of the scenario context and conditions that may affect PIF assessments, it is also important to identify cues that may divert attention, cause confusion, or distract personnel from the desired procedural response and their timing relative to the relevant cues. If EOCs are modeled in the PRA, it is also important to identify specific cues that may trigger EOC actions (e.g., as an alternative to the desired response), and their timing.

Identify the proceduralized and non-proceduralized responses that lead to success. In some cases, no written procedure applies to the important human action. Personnel G-13

would rely on training, engineering judgment, and skill-of-the-craft to implement the expected responses in the important human action. The identification of such responses requires understanding the systems and interviewing operations staff.

(2) Identify and Define the Critical Tasks in the Crew Response Diagram The purpose of this stage is to identify and analyze the tasks critical to the success of the important human action. Failure of any of these tasks will fail the important human action.

When constructing a CRD, the critical tasks are represented under the response nodes. There is considerable flexibility in the number of nodes to be included in the diagram versus the numbers of critical tasks under the nodes. A node could include one or several critical tasks.

There is a tradeoff between the number of response nodes in the diagram and the number of critical tasks for each node. An analyst may choose to use a response node to represent only one critical task or to cluster several critical tasks in one response node.

Identification of critical tasks begins with reviewing available procedures, guidelines, or instructions for the human response of the node. Each individual step in the procedures or guidelines may be viewed as a subtask; subtasks may involve different cognitive activities (e.g.,

a step may direct the crew to collect information, to verify plant status, to perform a plant state assessment, to make a decision such as transferring to another procedure or branch of a procedure, or to execute the required manipulations). A critical task is identified by grouping the subtasks according to a common goal. If no procedure is available, critical tasks are identified by analyzing what personnel have to do to achieve the expected response.

(3) Identify Potential Recovery Opportunities Common practice in PRA for NPPs is to consider recovery opportunities for important human actions. One advantage of a CRD is that it helps identify potential recovery opportunities. Each of the critical tasks represents an opportunity for failure. This is represented on the CRD as a downward arrow. The purpose of this step is to explore the possibilities for recovery given a failure at one of the nodes of the CRD. This step identifies the opportunities for error correction (i.e., for recovery of the failure to correctly perform the task(s) represented by the node).

The critical tasks represented in the diagram nodes include not only manipulations but also information collection, assessment and response selection tasks. The opportunities for recovery can come from a number of sources. Information collection, assessment and response selection tasks are usually associated with a procedure entry, procedure transfer, or initiation of an action. No matter the reason for failure at a node, the assumption is made that, following the failure to take the correct path, the operators are still using their procedures.

Consequently, the error correction opportunities relate to subsequent procedure steps (or steps in other applicable procedures) that have the potential to place the crew on an alternative success path or that act as additional cues to perform the correct task or perform the correct procedure transfer. In addition, plant conditions may evolve and generate new alarms or key parameter changes that crews would normally be monitoring and would serve as cues for identifying the need for a different response.

For manipulation tasks, the error correction opportunities will primarily arise from a monitoring activity that is capable of detecting that the plant is not responding as would be expected if the intended action had been completed correctly. These opportunities focus on the crews detection and assessment of the plant feedback.

G-14

(4) Develop Timeline in Parallel with the Crew Response Diagram In parallel with the development of the CRD, a timeline should be developed for each success path identified on the diagram, as necessary, to support assessment of the feasibility and error recovery of the responses. The timeline indicates the expected time of the occurrence of the plant cue (plant event) and the expected time for the completion of the crew tasks. The timeline could be critical to specifying the states of PIFs for estimating HEPs.

In summary, the CRD organizes the outcome of the task analysis. The graphic representation of the CRD illustrates the success path, failure paths, and recovery paths. The supporting information for each node should also be summarized and documented as a part of task characterization. On the other hand, while CRDs may be a good choice for proceduralized responses (e.g., NPP EOPs), it may not be the best tool for other responses that are less linear or have multiple success paths. For complex scenarios with parallel important human actions and much coordination, an integrated timeline showing key decision and communication points may be a better option than a CRD. The ATHEANA method [10], [11] provides guidance for constructing such an integrated timeline. Nevertheless, a CRD is a good place to start because it forces the analyst to think through the process and identify variability and challenges.

Example: CRD for an important human action in an NPP fire event. This event has multiple important human actions that are defined as HFEs in the PRA model. This example demonstrates the CRD for one of the HFEs: HFE-3.

Event overview: At 18:52 with the plant operating in Mode 1 at approximately 100-percent power, an electrical feeder cable failure caused an arc flash and fire on a nonvital electrical bus.

The electrical bus failed to isolate because of a breaker failure, and the fault persisted much longer than design expectations. The effects were widespread throughout the electrical systems. The electrical isolations and automatic repowering also created time sequences that caused inadvertent equipment actuation and damage. The fault condition reduced voltage to reactor coolant pump (RCP) B, causing an automatic reactor trip on RCS loop low flow.

Pressurizer level and pressure decreased due to RCS cooldown, resulting in an automatic SI.

Multiple equipment malfunctions further complicated plant response.

Loss of RCP seal injection and cooling: Within the first minute of the initiating event, RCP seal cooling (via component cooling water (CCW)) is lost due to the closing of flow control valve (FCV) 626, the CCW thermal barrier outlet isolation valve. FCV-626 closed due to an inaccurate high-flow signal when the flow sensor lost power during electrical realignments resulting from the fault.

Approximately 27 minutes into the event, chemical and volume control (CVC) Valve 310A fails open. When valve CVC valve 310A fails open, the charging flow is diverted from the RCP seals to the RCS and RCP seal injection becomes inadequate (there is some injection flow, but it is inadequate to fulfill its safety function). As a result, the RCP seals begin to heat up.

Expected operator response: With both RCP seal cooling from CCW unavailable and seal injection inadequate, the appropriate crew response would be to restore seal cooling from CCW to the RCP thermal barrier heat exchangers. For successful recovery, operators would have to reopen FCV-626 from the control room before voiding within the RCPs occurs. Based on Westinghouse calculations, the RCP seals will experience voiding conditions approximately 19 minutes after all RCP seal cooling and injection are lost.

HFE-3 definition: Failure to restore CCW to the RCP thermal barrier heat exchangers by reopening FCV-626.

G-15

Crew Response Diagram for HFE-3: Figure G-7 shows the CRD for HFE-3, which is followed by a description of the nodes of the CRD.

Identify loss of Enter Path-1 Reenter Path-1 Identify LOSC from Transfer to Open FCV-626 seal injection and upon automatic Enter EPP-4 upon automatic CCW and enter Step 10 at at Step 10 enter AOP-018 reactor trip safety injection APP-001-D1 Step 2 Section C 0 1 2 3 4 5 6 OK Fail to open FCV-626, execution R Fail to transfer to 7 Step 10 of AOP-018 Fail to enter AOP-018 Figure G-7 Crew Response Diagram for HFE-3 in the Example of NPP Fire Event Node 0: Enter Path-1 upon automatic reactor trip The crew enters Path-1 (Emergency Procedure Flow Path) upon automatic reactor trip at 18:52.

This node is for information only. No critical tasks are associated with it because the symptom pf a reactor trip is vivid to the operators and the operators have routinely trained to detect and respond immediately (by memory) to reactor trip events.

Node 1: Enter EPP-4 The crew enters End Path Procedure (EPP) 4, Reactor Trip Response from Path-1, when they decide that no SI is required. No critical tasks are associated with this node because transition to EPP 4 is a natural choice without good alternatives based on the event timing and condition, and the operator responding to the event entered EPP 4.

Node 2: Reenter Path-1 upon automatic SI The crew reenters Path-1 at 19:00 upon automatic SI due to a rapid cooldown. No critical tasks are associated with this node because of lack of good alternatives based on the event timing and condition, and the operator reentered Path-1 in the event.

Node 3: Identify loss-of-seal cooling (LOSC) from CCW and enter procedure APP-001-D1 One step in Path-1 will direct the crew to check the RCP thermal barrier cooling water low-flow annunciator. This provides an opportunity for the crew to recognize loss-of-seal cooling from CCW due to the closure of FCV-626. The critical tasks associated with Node 3 include:

Check the RCP thermal barrier cooling water low-flow annunciator per the diamond step in Path-1 Node 4: Identify loss-of-seal injection and enter Abnormal Operating Procedure (AOP)-018, Section C The procedure step for identifying loss of seal injection is listed below:

IF FCV-626 has failed closed, THEN PERFORM the following:

G-16

VERIFY RCP seal injection flow 6 gallons per minute (gpm) to 20 gpm.

ATTEMPT to reopen FCV-626 IF FCV-626 will NOT reopen, THEN INITIATE action to restore FCV-626 to OPERABLE status and CONTACT Engineering for assistance.

Node 4 has a critical task:

verify RCP seal injection flow rate between 6 and 20 gpm Node 5: Transfer to Step 10 at Step 2 Check elapsed time since all RCP seal cooling was lost - GREATER THAN 15 MINUTES IF RCP Seal Cooling is NOT OR can NOT be restored in less than 15 minutes, THEN go to Step 3.

Go to Step 10.

Node 5 has one critical task: check elapsed time since all RCP Seal Cooling was Lost Node 6: Open FCV-626 at Step 10 Step 10 of Section C of AOP-018 is the procedural direction for opening FCV-626. It takes 1 minute to transfer to Step 10 and 1 minute to open FCV-626 (i.e., 19:28). Thus, the RCP seal cooling is restored within 15 minutes after it was lost.

The critical tasks associated with Node 6 include:

Open FCV-626 at Step 10, which is a simple execution.

Node 7: Recovery of Nodes 3 and 4 At Node 3, the crew may choose not to enter APP-001-D1 if there is not enough manpower. A later step asks the crew to check if at least one charging pump is running. This step may reinforce the crews decision that restoring seal cooling from CCW is not a priority because charging pump B is providing sufficient seal injection. In this case, the crew, based on training, would enter Section C of AOP-018 upon RCP B and RCP A high bearing temperature alarms.

At Node 4, the crew may fail to recognize loss of seal injection when they see the seal injection flow rate is below 6 gpm at APP-001-D1 Step 4. Similar to recovery in Node 3, RCP B and RCP A high bearing temperature alarms give the crew an opportunity to enter AOP-018. The RCP B high bearing temperature alarm, reinforced by the low seal injection flow, should make the crew realize the loss of seal injection.

CRD Timeline: Figure G-8 illustrates the timeline for HFE-3. The green color represents actual events or boundary conditions, and the cyan color represents predicted events and estimates.

G-17

Figure G-8 Timeline for the Example HFE in the NPP Fire Event The following information was obtained through discussion with plant staff.

It will take 30 minutes for the crew to reach the diamond step after the reactor trip (i.e.,

19:22).

It takes about 1 minute to check RCP seal injection flow rate at Step 4 of APP-001-D1.

It takes about 2 minutes to transfer to Section C of AOP-018 from APP-001-D1 (i.e.,

19:25).

It takes about 1 minute to reach Step 2 after entry into Section C of AOP-018.

It takes about 1 minute to transfer to Step 10 from Step 2.

It takes about 1 minute to open FCV-626 at Step 10.

According to the guidance of AOP-018, the crew needs to open FCV-626 within 15 minutes after all seal cooling is lost, which is within the 19-minute window specified for HFE-3.

The analysts should try to obtain information to assess uncertainty about operator response time especially if the time-required and time-available in this event are close. The timeline shown in Figure G-8 is the timing of the crew in the actual event. A risk analysis is an analysis of all crews instead of a specific crew. The time differences between fast crews, normal crews, and slow crews could be large enough that the systems change their statuses within the time interval. Therefore, the system status could be different at the time when the fast crews and the slow crews make the same decisions. The difference could result in different crew responses G-18

and, potentially, new scenarios. The analysts should identify and document the uncertainty analysis.

G.5 Collecting Data and Information for Task Analysis Collecting information for task analysis often uses a combination of data collection techniques, such as examination of operational documents, interviews with personnel (talk-through), walk-through on the site with interactive observations of real or simulator operations, and collection of data through constructive questionnaires. Analysts may choose one or a combination of the techniques based on the specific application, availability and accessibility to operational sites and personnel, and personal training and experience. Moreover, analysts should examine how tasks are carried out in practice, not only how they should be carried out according to written procedures or guidance. Many books and guidance documents for task analysis have detailed descriptions of the techniques. Among those, the Handbook for HRA Scenario Analysis [111]

provides practical guidance on how to collect information. IDHEAS-G gives analysts the flexibility to choose task analysis and information collection techniques.

G.6 Summary This appendix provides high-level guidance on task analysis. The purpose of task analysis is to identify and characterize critical tasks. The outcome of task analysis for the same important human action may represent various levels of task breakdown or diverse ways of grouping simple activities into a single task (i.e., defining task boundaries differently). Note that performing a task analysis to a greater level of detail would not necessarily lead to a greater number of CFMs being relevant for any individual important human action since the CFMs apply to different macrocognitive functions. In any case, the end result of the qualitative analysis is a clear definition of the tasks and activities required for success of the important human action.

Given this and an understanding of the cognitive characterization for each task, HRA analysts can screen which of the CFMs are relevant for each task and assess the states of relevant PIFs.

G-19

APPENDIX H IDENTIFICATION OF COGNITIVE FAILURE MODES Reliability analysis uses failure modes to quantify error probabilities. Failure modes refer to the classification of the ways, or modes, in which something might fail. HRA uses human failure modes as the taxonomy to describe the ways that failure of important human actions may occur and quantify their likelihood. Existing HRA methods use various classification schemes for human failure modes. The schemes are based on task objectives (e.g., close a valve, line up pumps), the ways tasks are performed (e.g., use HSIs, use procedures), the cognitive nature of tasks (e.g., diagnosis, action), or representative (generic) tasks.

IDHEAS-G uses macrocognitive functions and their processors to model critical tasks.

Consequently, the failure of a task can be modeled with the failure of macrocognitive functions and the processors. IDHEAS-G thus has a set of cognitive failure modes (CFMs) that can adequately represent the failure of any human task within the application scope of an HRA method. This appendix provides guidance on how to use the IDHEAS-G CFMs in HRA.

H.1 A Basic Set of Cognitive Failure Modes The cognition model of IDHEAS-G is the basis for developing a set of CFMs. The cognitive basis framework describes how a critical task is achieved, as illustrated in Figure H-1(a):

Any critical task can be achieved through the five macrocognitive functions: detection, understanding, decisionmaking, action execution, and interteam coordination.

Each macrocognitive function is achieved through a set of processors that are carried out through cognitive mechanisms.

PIFs affect the capacity limits of the cognitive mechanisms.

This framework delineates the success path of cognitively achieving a critical task. IDHEAS-G uses the same taxonomy to represent the ways that a critical task might fail, as illustrated in Figure H-1(b):

The cognitive failure of a critical task is the result of failure of any macrocognitive function.

The failure of a macrocognitive function is the result of erroneous cognitive processes, which can result from the failure of any processors of the macrocognitive function.

Failure of the processors is the result of ineffectiveness of one or several cognitive mechanisms.

PIF attributes challenge the capacity limits of the cognitive mechanisms, which lead to the cognitive mechanisms being less effective or ineffective; therefore, the PIF attributes increase the likelihood of errors in the processors.

The above framework for cognition failure suggests two classification schemes of CFMs: a high-level classification based on the failure of the macrocognitive functions and a lower-level classification based on the failure of the processors.

H-1

(a) Macrocogniton Taxonomy (b) Cognition Failure Taxonomy Failure of the Failure of the Macrocognitive Processors Macrocognitive Processors Functions Functions Detection D1-D5 Failure of Failure of D1 D2 D5 Detection Understanding U1-U5 Failure of Failure of U1 U2 U5 Understanding Failure of Critical Task Decisionmaking DM1-DM6 a Critical Task Failure of Failure of DM1 DM2 DM6 Decisionmaking Action execution E1-E5 Failure of Failure of E1 E2 E5 Action execution Interteam coordination T1-T7 Failure of Interteam Failure of T1 T2 T7 coordination Note: The symbol represents the union (i.e., OR logic) of the events.

Figure H-1 (a) Macrocognition Taxonomy and (b) Cognition Failure Taxonomy H.1.1 Failure of the Macrocognitive Functions as High-Level Cognitive Failure Modes According to the cognition failure taxonomy in Figure H-1(b), failure of an important human action can be represented by the failure of one or more macrocognitive functions of the critical tasks. This classification scheme results in five CFMs, as described in Table H-1.

Table H-1 Failure of the Macrocognitive Functions as the High-Level CFMs Macrocognitive Function Cognitive Failure Mode Detection Failure of detecting cues/information Understanding Failure of understanding/assessing the situation Decisionmaking Failure of making decisions/planning actions Action execution Failure of executing planned actions Interteam coordination Failure of interteam coordination This set of high-level CFMs constitutes a complete representation of cognition failure. They are, theoretically, nonoverlapping because each macrocognitive function is defined with its own scope. They link to the PIFs, but are not specific enough. For example, two tasks that demand the same macrocognitive function may involve different sets of processors, which leads to different sets of PIFs and may result in different HEPs. However, if the two tasks are modeled with the failure of the same macrocognitive function, then their HEPs would be the same.

Therefore, the failure of the macrocognitive functions as CFMs are not specific enough to differentiate changes in the context.

H.1.2 Failure of the Processors as Cognitive Failure Modes Another classification scheme is to use the failure of the processors as a set of middle-level CFMs. As shown in Figure H-1(b), the failure of the processors of the five macrocognitive functions forms a complete representation of macrocognition failure. The CFMs do not overlap H-2

because the processors are intended to be nonoverlapping. These CFMs have explicit links to a limited set of cognitive mechanisms. The first column in Table H-2 through Table H-5 and all the rows in Table H-6 list the processor failures for each of the macrocognitive functions, respectively.

Theoretically, the failure of the processors forms a set of CFMs that are complete, nonoverlapping, and specific. However, these CFMs are not observable to support HEP estimation. They represent the ways that the internal cognitive process of achieving a macrocognitive function might break down. Most of those internal processes are not behaviorally observable, and they are not directly related to data or evidence of existing human performance measures. Thus, it is difficult to assess the effects of PIFs on a CFM to estimate HEPs.

H.1.3 Behavioral Characteristics of Processor Failure as Detailed Cognitive Failure Modes To make the processor failure observable, a set of characteristics of processor failure should be developed to represent the behaviorally observable ways that processor failure occurs. These characteristics constitute a set of detailed, observable CFMs. They represent processor failure and make use of data and evidence for HEP estimation.

The criteria for developing the set of such detailed CFMs are the same as those for any failure modes: completeness, nonoverlapping, specificity, and observability. The detailed CFMs are most useful when they are application-specific, i.e., they describe characteristics of human actions for the given HRA application. Because IDHEAS-G is independent of any application, the NRC staff developed a generic set of detailed CFMs. They can either be used directly or adapted to a specific HRA application.

The generic detailed CFMs are based on the following aspects of a processor:

The processor is not achievable.

The processor is achievable, but personnel do not perform it.

The processor is achievable, but personnel perform it incorrectly.

Several examples are provided below to demonstrate the development of the detailed CFMs.

Example 1The processor failure D1 is Fail to initiate detection. Three middle-level CFMs can represent this middle-level CFM:

D1-1 Detection is not initiated (e.g., skip steps of procedure for detection, forget to check information, fail to realize the need to check information, fail to check the right information)

D1-2 Wrong mental model for detection (e.g., incorrect planning on when, how, or what to detect)

D1-3 Fail to prioritize information to be detected Example 2The processor failure D2 is Fail to select, identify, or attend to sources of information. The following two detailed CFMs can represent this middle-level CFM:

D2-1 Fail to access the source of information

D2-2 Attend to wrong source of information H-3

D2-1 represents The processor is not achievable, and D2-2 represents The processor is achievable, but personnel perform it incorrectly. The processor is achievable, but personnel do not perform it is represented by D1-1, Detection is not initiated.

Example 3The processor failure D3 is Fail to perceive, recognize, or classify information.

The following detailed CFMs represent this middle-level CFM:

D3-1 Unable to perceive information

D3-2 Key alarm not perceived

D3-3 Key alarm incorrectly perceived

D3-4 Fail to recognize that primary cue is not available or misleading

D3-5 Cues not perceived

D3-6 Cues misperceived (e.g., information incorrectly perceived, fail to perceive weak signals, reading errors, incorrectly interpret, organize, or classify information)

D3-7 Fail to monitor status (e.g., information or parameters not monitored at proper frequency or for an adequate period of time, fail to monitor all of the key parameters, and incorrectly perceiving the trend of a parameter)

D3-1 represents the situation The processor is not achievable. The remaining detailed CFMs represent various types of detection activities: responding to alarms or alerts, checking cues or information, and monitoring system status, which correspond to the processor being achievable, but either the personnel do not perform it or they perform it incorrectly.

Example 4The processor failure E4 is Fail to perform the planned action. The following detailed CFMs represent this middle-level CFM:

E4-1 Fail to follow procedures (e.g., skip steps in procedures)

E4-2 Fail to execute simple action

E4-3 Fail to execute complex action (e.g., execute a complex action with incorrect timing or sequence, execute actions that do not meet the entry conditions)

E4-4 Fail to execute physically demanding actions

E4-5 Fail to execute fine-motor actions

E4-6 Fail to perform status checking required for executing critical steps of a task This example demonstrates that a processor failure can be further divided to represent distinctive types of cognitive activities. Here E4, Fail to perform the planned action, can be represented by three detailed CFMs: E4-1, Fail to follow procedures; E4-2, Fail to execute simple action; and E4-3, Fail to execute complex action. Depending on the specific HRA application, analysts may use E4-3 to represent failure of complex actions in general, or they may choose to use the more specific attributes of E4-3. For example, if an HRA involves assessing important human actions required for maintenance or NPP shutdown, the specific attributes of E4-3 can better model the failure of various manual actions. E4-4 and E4-5 emphasize physically demanding actions, such as repairing or installing equipment in a flooding H-4

hazard. E4-6 models the failure of a specific type of tasks in which execution of some critical steps require monitoring status to meet specific criteria.

The right column of Table H-2 through Table H-5 shows the full set of detailed CFMs, which represent the behaviorally observable failures of the processors, for the detection, understanding, decisionmaking, and action execution macrocognitive functions, respectively.

The different CFM levels represent human failures from macrocognition to very detailed behavioral aspects of the underlying cognitive processes. Notice that Table H-6 does not develop detailed CFMs for interteam coordination. Compared to other macrocognitive functions, the failure modes of interteam collaborative activities and their relation to PIFs are not well addressed in the literature. Also, the purpose of developing detailed CFMs is to have behaviorally observable failure modes for HRA, while the processor failures of interteam coordination are already behaviorally observable.

Table H-2 Detection CFMs High-Level CFM: Failure of Detection Middle-Level Detailed CFMs for Detection CFMs Fail to initiate D1-1 Detection is not initiated (e.g., skip steps of procedure for detection detection, forget to check information, fail to realize the need to check information, fail to check the right information, no mental model for detection)

D1-2 Wrong mental model for detection (e.g., incorrect planning on when, how, or what to detect)

D1-3 Failure to prioritize information to be detected Fail to select, D2-1 Fail to access the source of information identify, or D2-2 Attend to wrong source of information attend to sources of information Fail to perceive, D3-1 Unable to perceive information recognize, or D3-2 Key alarm not perceived classify D3-3 Key alarm incorrectly perceived information D3-4 Fail to recognize that primary cue is not available or misleading D3-5 Cues not perceived D3-6 Cues misperceived (e.g., information incorrectly perceived; failure to perceive weak signals; reading errors; incorrectly interpret, organize, or classify information)

D3-7 Fail to monitor status (e.g., information or parameters not monitored at proper frequency or for adequate period of time, failure to monitor all of the key parameters, and incorrectly perceiving the trend of a parameter)

H-5

Table H-2 Detection CFMs (continued)

High-Level CFM: Failure of Detection Middle-Level Detailed CFMs for Detection CFMs Fail to D4-1 Fail to self-verify the perceived information against the detection verify the criteria perceived D4-2 Fail to peer-check the perceived information information Fail to D5-1 The detected information not retained or incorrectly retained communicate (e.g., wrong items marked, wrong recording, and wrong data entry) the acquired D5-2 The detected information not communicated or miscommunicated information Table H-3 Understanding CFMs High-Level CFM: Failure of Understanding Middle-Level Detailed CFMs for Understanding CFMs Fail to assess U1-1 Incomplete data selected (e.g., critical data dismissed, critical data or select data omitted)

U1-2 Incorrect or inappropriate data selected (e.g., failure to recognize the applicable data range or recognize that information is outdated)

Incorrect U2-1 No mental model exists for understanding the situation mental model U2-2 Incorrect mental model selected U2-3 Failure to adapt the mental model (e.g., failure to recognize and adapt mismatched procedures)

Incorrect U3-1 Incorrectly assess situation (e.g., situational awareness not integration of maintained, and incorrect prediction of the system evolution or data and upcoming events) mental model U3-2 Incorrectly diagnose problems (e.g., conflicts in data not resolved, under-diagnosis, failure to use guidance outside main procedure steps for diagnosis)

Fail to iterate U4-1 Premature termination of data collection (e.g., not seeking additional the data to reconcile gaps, discrepancies, or conflicts, or failing to revise understanding the outcomes based on new data, mental models, or viewpoints U4-2 Failure to generate coherent team understanding (e.g., assessment or diagnosis not verified or confirmed by the team, and lack of confirmation and verification of the results)

Fail to U5-1 Outcomes of understanding miscommunicated or inadequately communicate communicated the outcome H-6

Table H-4 Decisionmaking CFMs High-Level CFM: Failure of Decisionmaking Middle-Level Detailed CFMs for Decisionmaking CFMs Inappropriate DM1-1 Incorrect decision model or decisionmaking process (e.g., incorrect decision model about who, how, or when to make decision, decision goal is not supported by the decision model or process)

DM1-2 Incorrect decision criteria Incorrect goals DM2-1 Incorrect goal selected or priorities DM2-2 Unable to prioritize multiple conflicting goals Data are DM3-1 Critical information not selected or only partially selected (e.g., bias, under- under-sampling of information) represented DM3-2 Selected information not appropriate or not applicable to the situation DM3-3 Misinterpretation or misuse of selected information Incorrect DM4-1 Misinterpret procedure judgment or planning DM4-2 Choose inappropriate strategy or options DM4-3 Incorrect or inadequate planning or developing solutions (e.g., plan wrong or infeasible responses, plan the right response actions at wrong times, fail to plan configuration changes when needed, plan wrong or infeasible configuration changes)

DM4-4 Decide to interfere or override automatic or passive safety-critical systems that would lead to undesirable consequences Failure to DM5-1 Unable to simulate or evaluate the decisions effects (e.g., fail to simulate or assess negative impacts or unable to evaluate the pros and cons) evaluate the decision/ DM5-2 Incorrectly simulate or evaluate the decision (e.g., fail to evaluate strategy/plan the side effects or components, or fail to consider all key factors)

DM5-3 Incorrect dynamic decisionmaking Failure to DM6-1 Decision incorrectly communicated communicate or authorize DM6-2 Decision not authorized the decision DM6-3 Decision delayed in authorization H-7

Table H-5 Action Execution CFMs High-Level CFM: Failure of Action Execution Middle-Level Detailed CFMs for Action Execution CFMs Fail to assess E1-1 Action is not initiated action plan and E1-2 Incorrect interpretation of the action plan (e.g., wrong criteria equipment/tool preparation or coordination)

E1-3 Wrong action criteria E1-4 Delayed implementation of planned action E1-5 Incorrect addition of actions or action steps to manipulate safety systems outside action plans (e.g., error of commission)

Fail to develop E2-1 Fail to modify, adapt, or develop action scripts for a high-level action or modify action plan scripts E2-2 Incorrectly modify or develop action scripts for the action plan Fail to E3-1 Fail to coordinate the action implementation (e.g., fail to coordinate coordinate team members, errors in personnel allocation) action E3-2 Fail to coordinate activities that must be performed in a sequential implementation or integrated manner E3-3 Fail to check the entry conditions for initiating the action execution Fail to perform E4-1 Fail to follow procedures (e.g., skip steps in procedures) the planned E4-2 Fail to execute simple action action E4-3 Fail to execute complex action (e.g., execute a complex action with incorrect timing or sequence, execute actions that do not meet the entry conditions)

E4-3A Fail to execute control actions E4-3B Fail to execute long-lasting actions E4-4 Fail to execute physically demanding actions E4-5 Fail to execute fine-motor actions E4-6 Fail to perform status checking required for executing critical steps of a task Fail to verify or E5-1 Fail to adjust action by monitoring, measuring, and assessing adjust action outcomes E5-2 Fail to complete entire action scripts or procedures (e.g., omit steps after the action criteria are met)

E5-3 Fail to record, report or communicate action status or outcomes Table H-6 Interteam Coordination CFMs High-Level CFM: Failure of Interteam Coordination T1 Fail to establish or adapt the interteam coordination infrastructure T2 Fail to manage information T3 Fail to maintain shared situational awareness Middle-Level T4 Inappropriately manage resources CFMs T5 Fail to plan or make interteam decisions or generate commands T6 Fail to implement decisions or commands T7 Fail to control the implementation H-8

In summary, IDHEAS-G provides a basic set of CFMs at three levels: failure of macrocognitive functions (high-level CFMs), processor failure (middle-level CFMs), and detailed observable CFMs. A specific HRA may choose to use the CFM levels as appropriate depending on the purposes of the HRA and data available to assess the HEPs of the CFMs. A specific HRA may also choose to use the combination of different levels as needed. For example, if the event being analyzed consists of mainly planned manual actions without complex situation assessment and decisionmaking, the analysis may use the detailed CFMs for failure of action execution and use the failure of macrocognitive functions (high-level CFMs) for the rest.

H.2 Development of Application-Specific Cognitive Failure Modes Theoretically, the basic set of CFMs can be used to perform HEP estimates for any HRA application. However, assessing the full set of processor failures (middle-level CFMs) or detailed CFMs can be very time-consuming. Therefore, a subset of the basic CFMs may be selected or adapted from the basic set to meet the need of a specific HRA application. For example, the IDHEAS At-Power Application [22] uses 14 detailed CFMs to model control room crew failures.

Developing a set of application-specific CFMs from the basic set in IDHEAS-G is a trade-off between method generalization and specification. An application-specific CFM set is a subset or a simplified version of the basic set. This means that some CFMs in the basic set are eliminated or simplified. Such a simplified CFM set generally works for events that meet the assumed scope of the application. However, there may be scenarios where one or several assumptions do not apply; therefore, some eliminated CFMs can be critical for the specific scenario. Thus, an application-specific CFM set should be reviewed for its applicability to the event being analyzed. When the application-specific CFM set is not adequate for an outlier scenario, HRA analysts should revisit the basic set of IDHEAS-G CFMs to add cognitive failure modes as necessary.

H.2.1 Selection and Adaptation of the Cognitive Failure Modes from the IDHEAS-G Basic Set The selection and adaptation of the CFMs should comply with the general criteria: the CFMs should adequately represent possible failures of the tasks in the application (completeness), be nonoverlapping, link to PIFs (specificity), and be behaviorally observable. Special attention is needed to ensure the criterion of adequate representation because a subset of the CFMs from the basic set may result in gaps in representing all of the possible cognition failures. The fundamental principle for developing an application-specific subset of CFMs is that a rationale must be provided if a CFM is not selected for the application-specific set.

IDHEAS-G recommends the following process for selecting and adapting the basic set of CFMs:

(1) Select the applicable CFMs related to the macrocognitive functions. If a macrocognitive function is not needed for performing the tasks or is not necessary to model because the cognitive activities involving those functions are very simple and straightforward, all of the CFMs associated with that macrocognitive function are not included in the specific set.

ExampleElimination of CFMs associated with macrocognitive functions. The human tasks in the see-and-flee example in APPENDIX G do not involve interteam teamwork; thus, the CFMs associated with interteam coordination are not selected. The goal of the see-and-flee action is to flee the workplace upon detecting the chemical release. The cognitive activities involving understanding are very simple and straightforward H-9

knowing that it is a chemical release by seeing the smoke. Thus, the CFMs associated with the understanding and interteam coordination macrocognitive functions are not selected.

(2) Select applicable processor failures. This is done by examining every processor of the selected macrocognitive functions. Not all of the processors in a macrocognitive function are needed to achieve the tasks in a specific application. The context of environment, systems, tasks, and personnel together guide the selection.

ExampleElimination of processor failures. The processors of detection are examined against the context for the see-and-flee example:

D1Personnel need to have a mental model of the signs of chemical release.

D2Personnel do not need to attend to the source of the signs because personnel flee regardless of where the source of the release is.

D3Personnel need to perceive the signs.

D4Personnel do not need to iteratively verify the signs of release.

D5Personnel do not need to retain or communicate the detection of the signs.

Therefore, D1 and D3 are selected for the set of CFMs.

(3) Select and adapt detailed CFMs. Depending on the nature of the HRA application, the detailed CFMs for a given application can be derived from the basic set in various ways:

Eliminate: Some detailed CFMs may not apply and, therefore, can be eliminated.

For example, the IDHEAS At-Power Application [22] assumes that decisionmaking is limited to choosing and implementing the strategies specified in procedures; therefore, many detailed CFMs for the decisionmaking macrocognitive function can be eliminated. A caution is that some eliminated processors may be present in a specific scenario of event analysis. The elimination may lead to gaps in CFM representation and result in underestimation of the HEP. Once again, the principle for eliminating a CFM from the basic set is that a rationale must be provided for elimination.

Merge: The attributes of some processors may not be behaviorally distinguishable, or the effects of the PIFs on these detailed CFMs are indistinguishable. Thus, the detailed CFMs can be merged into a single CFM for HEP estimation. A caution for merging detailed CFMs is that the combined detailed CFMs may become insensitive to differences in event context.

Split: An HRA application may involve very specific types of human actions. In this case, it may be beneficial to split a detailed CFM into more specific failure modes. A caution for splitting detailed CFMs is that information about event context or PIFs may get lost when CFMs are too detailed.

Refine and adapt: The description or definition for the basic set of detailed CFMs is generic and may not well match the given HRA application, or the definitions may be difficult to interpret in the context of the application. They can be refined or adapted with more application-specific descriptions. A caution in refinement is that the scope of a detailed CFM may change with the new definition and that H-10

may lead to inadequate representation of cognition failure. That may also result in underestimation of HEPs.

H.2.2 Define the Application-Specific Cognitive Failure Modes If application-specific CFMs are used, the analyst should clearly define the meaning and applicability of the CFMs. The CFM definition should be described with respect to the context of the HRA application. The IDHEAS At-Power Application [22] defines the meaning of the CFMs specific to control room crew tasks along with the applicability in control room proceduralized task performance. Below is an example adapted from Chapter 6 of NUREG-2199, Vol. 1 [22].

Example: Summary of NUREG-2199 definition of the CFM Key Alarm Not Attended To This CFM represents the failure to respond to a key alarm. A key alarm is one that is the first indication of the need for a response, and in this context it is considered to be unexpected.

Furthermore, a key alarm is not necessarily a single alarm, but instead it could be multiple annunciators that form a recognizable pattern. It is expected that the response for a key alarm is well trained and essentially automatic. Failure includes both the failure to perceive the alarm and failure to understand the alarm. For those alarms for which the response is memorized, simple, and ingrained (e.g., pressing the scram control on receipt of a scram alarm), this could also include the failure to act. For alarms that lead to entering a procedure (such as an alarm response procedure), any actions in that procedure should be addressed separately using appropriate CFMs.

This CFM applies to a task for which the principal cue is an alarm and a failure to respond would lead to the failure of the important human action being modeled. This CFM applies to important human actions where: (1) the alarm is the principal cue and is sufficient for a correct assessment of the plant status so that the required response is unambiguous for a nominal situation, or (2) the alarm is a trouble alarm that leads to entry into an alarm response procedure. This CFM does not apply to alarms that serve as reminders associated with parameters that are being monitored.

Finally, the resulting set of specific CFMs for a given HRA application should be viewed as an adequate representation of failures of human tasks for the assumed scope and context of the application. There may be unusual scenarios that have special system, task, or personnel features outside the assumed contexts. In that case, the basic set of IDHEAS-G CFMs should be reexamined to consider adding detailed CFMs that are applicable to the special scenarios.

H.3 Determination of the Cognitive Failure Modes A prerequisite for identifying CFMs is the characterization of the critical tasks in terms of the specific activities identified as essential for success and their requirements, since this will be used to identify the relevant CFMs. The outcomes of the task analysis, along with the outputs of the scenario analysis, provide the structured context for the important human action or critical tasks being analyzed. From the operational narrative, definition of the important human action, and the description of the critical tasks and activities needed for success, HRA analysts identify the CFMs applicable to the critical task.

The rationale for identifying potentially applicable CFMs should be clearly documented. It is recommended to use probing questions to capture the rationale. Table H-7 provides reference questions for identifying failures of the macrocognitive functions. The first column of the table contains the reference questions. The questions probe the types of cognitive activities required to achieve the critical task, and such information is documented as the output of task analysis.

H-11

For each macrocognitive function, if the answer to the question is yes for the critical task being evaluated, that CFM is determined to be applicable to the critical task. Those for which the answer is no are screened out from consideration of the HEP because the CFMs are not applicable. Table H-7 has a third column (not shown) for HRA analysts to document their justification if the answer to a question is no.

Table H-7 Reference Questions for Identifying Failures of the Macrocognitive Functions Applicable to a Critical Task Applicable high-level Reference questions CFM if YES to the question Does the success of the task require monitoring and detecting Failure of detection cues, checking and acquiring information, or recognizing and responding to alarms?

Does the success of the task require maintaining situational Failure of understanding awareness, assessing status, diagnosing problems, resolving conflicts in information, or making predictions for the upcoming situation development?

Does the success of the task require selecting among multiple Failure of options or strategies, making changes or additions to a preexisting decisionmaking plan or strategy, or developing a new strategy or plan?

Does the success of the task require executing planned physical Failure of action actions that manipulate the systems or equipment being operated execution or the personnel working on the task (e.g., traveling from one working area to another)?

Does the success of the task require multiple teams and require Failure of interteam communication, coordination (including command and control), or coordination cooperation between the teams?

The reference questions for identifying failure of processors (middle-level CFMs) or detailed CFMs should be developed and phrased with respect to the context of the event. Table H-8 provides some example reference questions for detailed CFMs. Because these questions are adapted from the IDHEAS At-Power Application [22], they were phrased in the context of NPP control room operation. The first column of the table describes the macrocognitive functions and only those involved in the critical task need be addressed. For example, if the critical task being evaluated does not involve action execution, then none of the detailed CFMs for action execution would apply. The second column contains the detailed CFMs for the relevant macrocognitive function. Then, for each relevant macrocognitive function, if the answer to the question in the third column is yes for the critical task being evaluated, that detailed CFM is determined to be applicable to the critical task. Those for which the answer is no are screened out from consideration of the HEP because the CFMs do not apply.

H-12

Table H-8 Example Reference Questions for Identifying Detailed CFMs in NPP At-Power Internal Event Applications (adapted from NUREG-2199, Vol. 1)

Macrocognitive Detailed CFM Reference Question Function Does the critical task include failure to Key Alarm Not Attended to respond to an alarm as the primary cue for success?

Detection Data Misleading or Not Available Does success require acquiring data Wrong Data Source Attended to or information?

Critical Data Misperceived Critical Data Dismissed or Does success require data collection to Understanding Discounted assess plant status?

(Status Does success require evaluating Assessment) Premature Termination of Critical changes of critical plant parameters Data Collection or seeking additional information?

Does success require a decision Misinterpret Procedures (e.g., transfer to another procedure or Decisionmaking initiate action)?

(Response Planning)

Does the procedure allow a choice of Choose Inappropriate Strategy strategies?

Does success require responding Delay Implementation when a critical value is reached (given that the value has been recognized)?

Critical Data Not Does success require monitoring for a Checked/Monitored with critical plant parameter as a cue to Action Execution Appropriate Frequency initiate response?

Fail to Initiate Execution Fail to Correctly Execute Does the task require action on plant Response (Simple Task) systems?

Fail to Correctly Execute Response (Complex Task)

In summary, IDHEAS-G provides a basic set of CFMs. The definitions of the CFMs are cognition-based and system neutral so that they can be applied to a wide range of HRA uses (e.g., fuel and radioactive material safety, spent fuel pool and dry cask storage, radioactive medicine, new reactor construction, before and after core damage, actions taking place inside and outside of the main control room). Application-specific sets of CFMs can be developed from the basic set to calculate the HEP of important human actions for a given HRA application, such as NPP at-power, internal event applications. For a specific application, HRA quantification may exclude the CFMs that are not relevant to the application. This allows for flexibility while ensuring that the HRA quantifications for different applications all have a common cognitive basis. The basic set of CFMs adds another flexibility by offering three levels of CFMs. An HRA application may choose to use one level or the combination of different levels of CFMs based on the purpose of the HRA and the human error data available for HEP estimation.

H-13

APPENDIX I ASSESSMENT OF PERFORMANCE-INFLUENCING FACTORS IDHEAS-G provides a structure of PIFs consisting of PIFs and their attributes. IDHEAS-G uses PIFs to model the context of an important human action. The modeling includes selecting applicable PIFs based on the context and assessing the applicability of the PIF attributes. This appendix first introduces the process of assessing PIFs and then provides a generic example to discuss PIF assessment in the context of NPP control room actions in internal, at-power events.

I.1 The Process of Assessing Performance-Influencing Factors Assessment of the PIFs affecting an important human action may take the following steps:

(1) Select PIFs to represent the context of the important human action.

The IDHEAS-G PIF structure provides 20 PIFs. The IDHEAS-G PIFs described in Chapter 3 of this report are defined and organized in a way that is intended to systematically account for all elements of human cognitive performance. Therefore, in principle, all 20 PIFs are always relevant to every HFE. Because PIF assessment is made for the applicable CFMs of the critical tasks in an HFE, each CFM is associated with a subset of the 20 PIFs. Analysts should systematically evaluate all the PIFs relevant to the CFM. An analyst may judge that some PIFs, including their attributes, may not have a significant influence on altering the nominal expected human cognitive performance during the scenario that is being evaluated. The analyst must then provide the rationale and justification for why those PIFs do not merit further detailed evaluation.

(2) Select PIF attributes relevant to the CFMs.

Elimination of some PIF attributes may be necessary so that the total number of PIF attributes associated with a CFM is manageable for the purposes of HEP estimation.

The PIF attributes that do not contribute significantly to the CFM may be eliminated. A rationale must be provided for the elimination of the PIF attributes.

(3) Represent contexts that positively affect human performance While IDHEAS-G defines context as the conditions that challenge or positively affect human performance, the PIFs are defined as neutral. The PIF attributes all have a negative impact on HEPs. The contexts that positively affect human performance are represented by alleviating some PIF attributes. For example, the baseline state of training means that training is good enough and would not increase HEPs. The context that training is better than the baseline means that the training may alleviate some PIF attributes in procedures, guidance, and instructions; teamwork and organizational factors; or other PIFs.

(4) Identify additional PIFs and PIF attributes.

The list of PIFs and PIF attributes in Chapter 3 represents the NRC staffs present state of knowledge. New PIFs and/or PIF attributes may be needed for new scenarios, applications, technologies, or conduct of operations. Whenever a new PIF or PIF attribute is identified, it should be assessed against the cognitive mechanisms to establish its links to the CFMs.

I-1

I.2 Demonstration of Assessment of Performance-Influencing Factors in Nuclear Power Plant Control Rooms The PIFs and their attributes are generalized across various systems, events, and applications.

The description of the PIFs and their attributes is intended to be generic without reference to a specific HRA application. Thus, it is not obvious how they are related to a specific application or event. This section documents some considerations for PIFs in NPP control room operations as an example to discuss PIF assessment in a specific application, i.e., NPP control room actions in internal, at-power events. The information is intended to facilitate the readers understanding of PIFs. The example does not provide a systematic assessment of all PIFs associated with each macrocognitive function. Instead, this example assesses only some example PIFs that can significantly affect human performance. Because the example does not provide a specific scenario to allow a systematic identification of scenario context, this example is not intended to be a reference of how an analyst should systematically consider and evaluate the PIFs in a practical analysis. Readers of this example should not use it as evidence for why it is acceptable to quickly focus attention on only a couple of "obvious" PIFs, without the systematic assessment of scenario-specific influences.

I.2.1 Performance-Influencing Factor Considerations for Detection The main control room crew constantly monitors a set of plant parameters key to plant safety.

Examples for a pressurized water reactor during normal operation include sub-cooling margin, RCS temperature and pressure, SG water levels and pressures, and pressurizer water level.

During a severe accident situation, reactor pressure vessel water level; core exit temperature; containment temperature, pressure, and hydrogen concentration; electric power supply; and water supply may become important to the decisionmakers. Examples of important parameters in a boiling-water reactor include reactor pressure vessel water level and pressure, containment pressure and temperature, suppression pool water level and temperature, hydrogen concentration, electric power supply, and water supply. If the instrument is available and performs its designed functions, then detection would mainly depend on whether the operator is checking the correct instrument, whether there are HSI issues, or other considerations that could affect the detection.

During abnormal or emergency events, the AOPs and EOPs provide step-by-step instructions to handle the event. The procedures explicitly specify the plant parameters to check and the decision criteria. These parameters are directly indicated by plant instrumentation. Examples of procedure instructions are check if the pressurizer pressure is greater than X psig, check if a valve is open, and check if the pressurizer pressure is stable or increasing. In this case, there is no need to integrate different information (shown in other indicators) to determine the values or status of the parameters.

The tables in APPENDIX B list the following PIFs, which have one or more attributes that are linked to the Detection macrocognitive function.

Work Place Accessibility and Habitability

Work Place Visibility

Noise

Cold / Heat / Humidity (overarching effects)

System and I&C Transparency to Personnel

Human-System Interface

Staffing

Procedures, Guidance, and Instructions I-2

Training

Team and Organization Factors

Work Processes (overarching effects)

Information Availability and Reliability

Multi-tasking, Interruptions, and Distractions

Task Complexity

Mental Fatigue (overarching effects)

Time Pressure and Stress Thus, these 16 PIFs are relevant to the Detection macrocognitive function. An analyst must then examine each PIF in the context of the specific scenario that is occurring and the specific HFE that is being evaluated to determine whether the PIF merits special attention (i.e., whether it might improve or adversely affect the expected baseline human performance). The example in this section considers only the PIFs for "Human-System Interface" and "Procedures, Guidance, and Instructions" for the purpose of understanding the PIFs in a specific application.

The same is true for the discussion of the rest of the macrocognitive functions in this example.

Human-System Interface Detection may not be prompted by procedures, but rely on an operators response to the presence or onset of the information, such as response to the onset of an alarm. Detection of the presence or onset of information is vulnerable to the saliency of the signal relative to all other information available. For example, the unique dynamics of the electrical fault in the fire event at the H.B. Robinson Steam Electric Plant closed a component cooling water flow control valve (FCV-626) [104]. The operators did not expect this closure because the valve remained open in simulator exercises of similar scenarios. In this event, the operators did not detect the valve closure until later in the scenario. In the event, several hundred alarm tiles were lit and created high noise for operators. The signal (the alarm indicating that FCV-626 was closed) was relatively weak compared to the noise. This example shows the difficulty of noticing a plant abnormality when the operator is not expecting it, and the information (signal) is presented among many other irrelevant pieces of information (noise).

Situations may occur where instrumentation is not available (or does not exist) to directly indicate system status. Examples are identifying a ruptured SG, determining whether the reactor pressure vessel has been breached in a severe accident, and identifying the release path of a radioactivity release event. In such situations, detection requires that operators calculate or derive a parameter from several pieces of information. The HSI may provide only the low-level information (e.g., main steamline radiation and SG water levels) for the operator to integrate to generate the desired high-level information (e.g., tube rupture of a certain SG).

Main control rooms with advanced digital instrumentation and controls may have sophisticated logic to provide direct indications of plant parameters that are not directly indicated in conventional control rooms.

Some information may be displayed only in a location that is not constantly monitored, which may reduce opportunities to detect the information. For example, although most information is available in the main control room, some information may be detected only on remote panels outside the control room.

HSIs are generally designed to facilitate information perception. Yet, the HSI may impede information perception in various scenarios such as the following:

Some indicators are calibrated only to operate within a certain range. The individual has to know the range to correctly interpret the information.

I-3

Digital displays may use display format (e.g., change font color and blinking fonts) to indicate the parameters status (e.g., not reliable).

Procedures, Guidance, and Instructions In most NPP abnormal events, procedures explicitly instruct the operators to detect specific parameters. Yet, procedures do not cover all of the pieces of information that operators should detect, especially during events with system, component, or instrument failures.

I.2.2 Performance-Influencing Factor Considerations for Understanding To correctly understand a situation, operators must form a holistic picture, including plant status, task priorities, and potential conflicts among any concurrent tasks. Therefore, correct understanding relies on having the information about the current plant status and having a mental model of the plant responses to predict near-term plant status (instead of current plant status). For example, operators may use currently available information to estimate the time to exceed a plant threshold (e.g., time to exhaust the essential batteries, time to core damage, and time to containment reaching a high-pressure status).

Assessing plant status requires integrating multiple plant symptoms, available information, and considerations of indication reliability to predict event evolution. The cognitive activities involve integrating multiple pieces of information and reasoning to reach a conclusion. For example, to determine the reactor pressure vessel water level in a loss of instrumentation power situation, personnel would calculate the reactor pressure vessel water level with input from decay heat, injection flow rate, vessel geometry, and leakage flow, etc. Personnel assess the water level by integrating multiple pieces of information.

Instrument and ControlIndicator Failure In certain situations, an indicator may fail to display a correct value. The operator is trained not to rely on a single indication for information but to check other redundant or relevant indications to reach a confident conclusion. Typically, redundant indications or other information is available for the operator to conclude that an indication has failed. Therefore, success of the detection macrocognitive function means to detect the correct value (or status) instead of the face value (or face status, as shown in a failed indicator). In other words, successful detection also includes confirming that the information is valid.

If an indicator presents off-scale indications (e.g., above the upper bound or below the lower bound), the indications may be informative depending on the diagnosis need. For example, even if the exact parameter value of an indicator is not available for off-scale indication, the display provides the information that the parameters value is above the upper bound setting of the indicator. Other forms of instrumentation failure could include the following:

A miscalibrated tool is used to calibrate the instrument.

The indicator operates outside of its operational condition. For example, water level measurement is sensitive to the water density of the indicators sensing and reference lines. In an adverse environment, the water in the reference line may be evaporated, boiling, or have a significant density change because of changes in surrounding temperature. In these situations, the indication is no longer reliable. EOPs generally include additional guidance for operators when there are adverse environmental conditions that adjust the thresholds for instrument readings.

I-4

Information Availability and Reliability When there is a large-scale instrumentation failure (e.g., due to the loss of instrumentation power such as in the H.B. Robinson fire event [104]), many indications may be unavailable or unreliable. In this condition, assessing plant status can be challenging because of incomplete or inaccurate information.

Procedures, Guidance, and Instructions Even though procedures are available to assist operators in diagnosing an abnormal event, procedures can never address every possible situation. Human judgment is needed to apply the procedure properly and understand the situation. For example, in a pressurized-water reactor, soon after the accumulator actuation following a loss-of-coolant accident, the RCS temperature will decrease for a brief time and then rise again. The operator should know the RCS temperature trend to respond to the loss of heat sink EOP when it asks the operators if there is sufficient RCS heat removal capability. The accumulators provide only temporary cooling, so operators must understand that the RCS does not have enough cooling to follow the procedure correctly.

In implementing procedures, operators are expected to constantly maintain awareness of the scenario progression and look ahead in the procedures to maintain awareness of the procedural path. The control room crew independently applies its mental models to understand the event in parallel to implementing procedures. The operators know the plant operational history and the current specific plant configurations. This knowledge helps them to apply the procedures correctly. The consistency between the operators mental model and procedural instructions is a key factor affecting the operators confidence in following the procedure. A lack of consistency between the operators mental model and the procedure increases the likelihood that operators will decide to deviate from the procedure. Therefore, HRA analysts should not heuristically conclude that the diagnosis or understanding of the event is straightforward simply because procedures are available. Instead, they should check the consistency between the procedure instruction and the event specifics.

Scenario FamiliarityCognitive Bias Previous similar symptoms or misleading indicators can bias operators understanding of the plant situation. For example, in the Three Mile Island Nuclear Station accident, operators were unable to resolve conflicts between plant parameters because their mental model was biased by a misleading indicator about the status of a pressurizer power-operated relief valve. Moreover, they had not been trained to recognize that a high pressurizer water level may indicate a steam bubble in the reactor vessel (i.e., under certain conditions, the pressurizer is not a reliable indicator of reactor system water inventory). Also, bias may lead operators to believe that they have already reached an adequate understanding and to prematurely terminate data collection needed to fully diagnose or understand a problem.

I.2.3 Performance-Influencing Factor Considerations for Decisionmaking An NPP has many procedures that provide response plans for a wide range of incidents and accidents. The AOPs and EOPs typically provide clear step-by-step instructions as response plans for situations. For example, the Steam Generator Tube Rupture Procedure in a pressurized-water reactor specifies the response plan for isolating the broken SG(s), including adjusting the ruptured SG power-operated relief valve controller setpoints, checking that the ruptured SG relief valves are closed, closing steam supply valves from the ruptured SG(s) to turbine-driven auxiliary feedwater pumps, verifying that blowdown isolation valves from the I-5

ruptured SG(s) are closed, and closing the ruptured SG main steamline isolation and bypass valves. After completing these actions, the procedure guides the operator to check the ruptured SG water level to ensure that the actions were performed properly. However, the SG water level will not confirm that the actions to isolate were all completed; SG water level will continue to increase until RCS pressure is reduced. Other actions would have opposite impacts on the water level (e.g., isolating auxiliary feedwater supply compared to isolating blowdown).

There are typically different levels of procedure use and compliance outlined at nuclear plants.

EOPs are generally continuous procedures to which operators are expected to adhere.

However, other types of procedures have a different level of compliance expectations (and many typical actions that are considered skill of the craft). Unless they have strong justifications, the operators are expected to follow the response plan as specified in the procedures. This expectation does not mean that operators would implement a procedure exactly as instructed by the procedure. The operator is permitted a certain flexibility in implementing procedure instructions. For example, if a procedure-specified component is not available in the scenario, the operator must choose an alternative to achieve the same plant function. In some situations, the operators may decide to deviate from the procedure instructions. For example, if the operator senses that the procedure is guiding the crew towards a destination that is inconsistent with the operators perception of the situation or in a direction that the operator is hesitant to go (e.g., the decision has high economic consequences), the operator may choose to deviate from the response planning as specified in the procedure. An example is the decisionmakers choice to deviate from the procedure instructions in the Davis-Besse Nuclear Power Station loss of all feedwater event [120].

Procedures, Guidance, and Instructions In situations where there is no appropriate procedure to provide a response plan, the operator must develop the response plan, on the scene, based on knowledge (e.g., implementing a creative alignment to remove decay heat when the preplanned options are not available). In such situations, the response plan is likely to be less comprehensive and more prone to error than if a suitable procedure were available.

Training Operators may not have adequate training on how to recognize the discrepancy between the situation and procedures and, subsequently, how to modify response plans. In addition, operators may not have adequate training on recognizing the urgency of responding to some plant abnormalities.

I.2.4 Performance-Influencing Factor Considerations for Action Execution The plant operators are trained and licensed or certified to operate plant components and special equipment. Therefore, the operator can be assumed to have the required knowledge, skills, and abilities (e.g., to know the action location and how to perform the action) to complete the actions. The action reliability could depend on some of the factors discussed below.

Task Complexity Actions can be as simple as turning a switch in the control room, navigating through the computer user interface, or manually opening valves. On the other hand, actions can be as complex as performing multiple procedure steps to maintain control of a plant parameter, or they may require precise timing coordination. Actions can be short, such as pushing a button, or long, such as performing an event procedure (e.g., the SG tube rupture procedure), slowly depressurizing the reactor pressure vessel with monitor-and-control types of actions (e.g., a I-6

cooldown rate not exceeding 100 degrees Fahrenheit per hour), or setting up a portable pump to inject coolant into SGs.

Environment and Situation Factors Actions could be performed in an air-conditioned and well-lit workplace (e.g., the main control room) or at an onsite location with poor lighting (e.g., shedding direct current load in a station blackout event), ergonomically challenging displays and controls, and harsh environmental conditions (e.g., high temperature, high humidity, and high radiation levels). The work environment includes not only the environment of the worksite but also the travel path to the worksite. Harsh environmental conditions in the travel path and at the worksite could prevent the actions from been executed. Work in high radiation areas would have time limitations on how long an individual can stay in the work area. Entering certain areas of the plant requires wearing protective clothing. These aspects of the work environment may affect human performance.

Human-System InterfaceSystem Feedback Actions on components typically provide immediate feedback to confirm that the action has been successfully performed. For example, when changing a valves position by turning the valves control switch, the valve position may be confirmed by noting that the valve position indication light changes color or noting a change in the flow rate. The effects of some actions may take longer to appear. For example, injecting coolant into a boiling SG would initially result in a decrease in the SG water level (because of condensing the steam bubbles) followed by a later increase in water level. When coolant is injected into an overheated core, it may take a while before the reactor pressure vessel temperature begins to trend down and the indications show an increase in water level.

Staffing After the Fukushima Daiichi event, U.S. plants procured portable equipment to mitigate hypothetical extended loss of heat sink and loss of ultimate heat sink events. The mitigating strategies are referred to as FLEX strategies. Implementing the strategies requires team effort to clean debris in the equipment transportation route and staging locations, move the portable equipment to the staging locations, and set up and operate the portable equipment. Performing the mitigation strategies requires not only a sufficient number of personnel, but also certain skills to operate the equipment. In an extreme event when site accessibility is limited, lack of staffing and skillsets could limit the success of the required actions.

In summary, assessment of PIFs should start with the systematic evaluation of the 20 PIFs in the IDHEAS-G PIF model. HRA analysts first select the PIFs applicable to the important human action being analyzed and justify not selecting some PIFs. For every applicable PIF, HRA analysts assess the PIF attributes based on the information obtained through scenario and task analysis. Finally, new PIFs or PIF attributes may be needed for new scenarios, applications, technologies, or conduct of operations.

I-7

APPENDIX J QUANTIFICATION OF HUMAN ERROR PROBABILITY IDHEAS-G models an important human action with time uncertainty, critical tasks, applicable CFMs, and PIFs. An HEP model is needed to calculate the HEPs of the CFMs for any given combination of PIFs. An HEP model should include the following elements:

quantitative modeling of PIF attributes

a method or rule for combining the effects of multiple PIFs on the HEP of a CFM

estimated HEP distributions of the CFMs for given PIFs There could be multiple approaches to each of these elements. Selecting an approach is often a tradeoff between the thoroughness and resource-demands of an HRA. Thus, in developing a HEP model for a specific HRA application, the analyst needs to consider the purpose of the HRA application, the resource demands of the model, and the data available to support the model. This appendix discusses various approaches to these elements.

J.1 Approaches to Quantitatively Model Performance-Influencing Factors The effect of an individual PIF on HEPs generally follows a logarithmic function. However, it is difficult to define the states of a PIF along a continuum. Defining the states of a PIF is a tradeoff between the number of PIF states (simplicity) and the practicality of the quantitative relationship to estimate the HEP. Below are some typical approaches for defining the states of a PIF:

1) Binary states: A PIF is simply modeled as the good or poor state. This is simple for modeling, but it introduces great subjectivity in HRA analysts judgment of the good versus poor state. The difference in judgment, in turn, can introduce great variability in the resulting HEPs.

2) Multiple states: Representing a PIF with multiple states is to make finer samples along the PIF continuum compared to binary states; therefore, the variability in resulting HEPs can potentially be reduced. Yet, multiple states lead to an exponential increase in the number of combinations of PIF states. Depending on the method of calculating HEPs, this could make it impractical to develop a method that can estimate HEPs for numerous combinations of PIF states.

3) Continuous scale: Some PIFs can be represented with a continuous scale. For example, cold or heat can be represented with temperature, visibility can be measured by the luminance of ambient lights at the worksite, and information availability and reliability can be measured by the percentage of information missing. Using a continuous, measurable scale reduces the variability in subjective judgment. The challenge is the practicalitywhether an HRA analyst can obtain the measurements and whether the quantitative relationship between the measurements and resulting HEPs is known.

In choosing an approach to model PIFs, the analyst should consider the following:

1) The states of a PIF should be explicitly defined so that they can be assessed unambiguously.

2) The effect of the PIF states on resulting HEPs can be estimated or empirical data are available to support the estimation of the effect.

J-1

3) The number of PIF state combinations should be manageable for an HEP model because the number of combinations increases exponentially with the number of PIF states.

J.2 Approaches to Combine the Effects of Multiple Performance-Influencing Factors A CFM is affected by multiple PIFs. Many PIFs interact, which affects the CFMs. There are two approaches for modeling the combined effects of multiple PIFs.

Holistic approach. This approach estimates the HEPs by considering all the impacts of the applicable PIFs together; that is, the HEPs are estimated for the given combination of states of the multiple PIFs. Decision trees are often used to represent PIF combinations in the holistic approach. Each branching point of the tree represents one PIF, and the different branches represent the states of a PIF. Thus, each path of a tree represents one possible combination of PIF states. HEPs are estimated for every path. This approach avoids explicitly modeling the interaction of PIFs. However, the paths of a decision tree exponentially increase as the number of PIFs and PIF states increases. Moreover, there may not be any empirical evidence supporting the estimation of the HEPs of all the paths.

When using the holistic approach, the analyst should consider the following:

1) The number of PIF attributes should be limited. The combination of too many PIF attributes may result in HEPs that are insensitive to individual PIFs.

2) The number of PIF combinations exponentially increases with the number of PIF attributes.

3) Data or evidence and experienced domain experts should be available for eliciting expert judgment of the HEPs.

4) Adding new PIF attributes to an existing HEP model is difficult because a new PIF attribute changes all of the PIF combinations; hence, all of the HEPs in the model need to be re-estimated.

Individual approach. This approach first considers the effects of individual PIFs, then combines the individual effects. The number of PIFs or PIF attributes does not limit this approach.

However, the potential dependency of the PIFs does challenge this approach. If the states of one PIF depend on the states of another PIF, the combined effect would double count the individual effects. Another challenge is the rule of combining the individual effects. HRA methods prevalently use multiplication of individual effects. Yet, the combination rule highly depends on how the effect of a PIF on HEPs is modeled. IDHEAS-G proposes to model the effect as the percent of increase in a HEP that results when a PIF changes from its base state to a poor state. Empirical data suggest that the combined effect of multiple PIFs can be roughly modeled by adding such defined individual effects.

Analysts using the individual approach should consider the following:

1) whether the effect of individual PIFs on HEPs is clearly defined

2) whether data or evidence is available for estimating the effect of individual PIFs to the HEPs

3) whether there is empirical data or evidence supporting the rules of combining individual effects for the definition of PIF effects used J-2

J.3 Estimation of Human Error Probability Distribution A HEP can be interpreted as the number of errors in performing a human action divided by the number of times the action is performed. In the real world, there is not adequate data to precisely compute the HEPs of rare events. As a common HRA practice, the HEPs of human failure modes in an HRA method are typically estimated by characterizing what is known about the parameter in terms of a probability distribution that represents the current state of belief in the possible values of the parameter. The next three sections briefly describe three approaches for estimating HEPs: data integration, Bayesian estimation, and expert judgment.

J.3.1 Data Integration There are general mathematical or statistical approaches for dealing with uncertain, aggregated, and/or truncated/censored data. Those approaches can be as relatively simple as calculating the mean of the numeric values of a data set, or the weighted average by some weighting rules, or as sophisticated as multi-variable fitting. However, the confidence obtained from integrating a set of data to generate a single representative value or probabilistic distribution depends on the sample size and quality of the data set. For example, if the numeric values of the data are not continuously distributed or fall in a binary distribution, the mean of the numeric values does not represent the data set.

As of 2019, the data the NRC staff generalized in IDHEAS-DATA (i.e., the collection of tables described in Sections 6.2.1, 6.2.2, and 6.2.3) are limited and typically do not form a continuous distribution when there are multiple data points for one HEP or PIF weight. Moreover, there are missing data points for many PIF attributes. Therefore, when the NRC staff integrated the data for the IDHEAS-ECA method in 2019, it could not apply a single approach to all the base HEPs and PIF weights. The NRC staff used aggregation, interpolation, reasoning, and engineering judgment on a case by case basis. Below are some general strategies the NRC staff used in the integration:

1) Multiple data points for a base HEP or PIF weight The human error data are first evaluated for their uncertainties and applicability in the source documents. The NRC staff considered that the NPP operational data that were systematically collected for HRA had the highest applicability while cognitive experiments performed in research laboratories with students had the least applicability.

The NRC staff used high-applicability data to anchor a base HEP or PIF weight and used other data points to adjust the uncertainties in the high-applicability data points.

For the multiple data points that have about the same level of applicability and certainty, the NRC staff used the mean human error rates of the data points as the base HEP.

2) No data point was excluded from being used to establish base HEPs and PIF weights, but there were data points excluded from being used on the combined effects of several CFMs and/or PIF attributes.

When there were multiple data points with combined effects of two or three CFMs or PIF attributes, the NRC staff performed data fitting to get the best-fit base HEPs or PIF weights. When there were only a few data points or a variety of CFMs and PIFs involved in the data points, the NRC staff combined the data points to estimate the range and then used the mean value as the base HEPs or PIF weights.

J-3

3) No data points directly applicable to assess PIF weights The available data in IDHEAS-DATA do not have numeric human error information for many attributes in the PIFs such as Work Process or Team and Organization Factors.

Yet, there have been studies demonstrating that those attributes impact human performance in measures other than human error rates, such as increasing personnel workload or reducing situational awareness. The NRC staff assigned the PIF weight as 1.1 or 1.2 for those attributes, pending for future updates as relevant human error data become available.

4) Consistency checking and adjustment with benchmark values After the initial base HEPs and PIF weights are developed, they are checked for internal consistency against the literature that ranks the likelihood of certain types of human errors and the contribution of various PIFs. The NRC staff also used reported rates of human events and estimated HEPs from the NRC 2018 FLEX HRA expert elicitation as benchmarks to check and adjust some base HEPs and PIF weights within their uncertainty ranges.

J.3.2 Bayesian Estimation When limited numerical data are available in the form of the number of failures in a given number of demands, the HEP distribution can be estimated through Bayesian analysis.

NUREG-2122 [51] discusses the definition of Bayesian analysis:

In a PRA, Bayesian analysis is commonly used in the computation of the frequencies and failure probabilities in which an initial estimation about a parameter value (e.g.,

event probability) is modified based on actual occurrences of the event. The parameter value may have a probability distribution associated with it. Thus, the event probability to be determined is based on a belief, rather than on occurrence ratios. Any actual occurrence or lack of occurrence of the event is used to measure consistency with the original hypothesis, which is then modified to reflect this evidence. The modified or updated hypothesis is the most meaningful estimate of the parameter.

The initial hypothesis is called the prior. The prior should be as relevant as possible to the parameter value in question. The final parameter estimate will depend on the prior chosen to a certain extent. For example, industry average (generic) data may be used as the prior. Non-informative priors can be used if no basis for making an educated guess exists. The prior is modified by actual observations of the event occurrences (e.g., plant-specific data) to calculate the posterior or best estimate of the parameter.

The process is called Bayesian update.

Bayesian analysis is used when occurrences of an event are sparse or nonexistent, such that probability estimates using the proportion of actual event occurrences (frequentist approach) are not reliable. It also can be used to produce a probability distribution for the parameter in question.

In risk analysis, both frequentist and Bayesian analysis may be used. Frequentist analysis is used when the occurrence data is sufficiently abundant, Bayesian analysis is used otherwise.

The terms Bayesian analysis, Bayesian estimation, and Bayesian statistics are used interchangeably.

J-4

Bayesian estimation (or inference) requires specifying a prior distribution and a likelihood function. 1 Prior distributions can be broadly classified as informative or noninformative. In PRA, 4F the likelihood function is used to model the process that gives rise to the datadata are observable [121]. In general, an informative prior is based on historical data on the distribution of the parameter of interest. Specifically, in the context of an HRA, an informative prior can also be based on expert judgement. To perform a Bayesian update on a parameter of interest, the data in the update need to be independent from the data used to develop the informative prior.

A noninformative prior is one that expresses ignorance as to the true value of the parameter being estimated. When a noninformative prior is used the intent is to let the data dominate the posterior distribution that results from the Bayesian update. The noninformative prior should reflect any information that is known independent of the data used in the Bayesian update [121].

The Jeffreys noninformative prior is often used when data are scarce.

The frequently used probability distribution functions for the prior include the normal, lognormal and beta distributions. It is generally considered that normal or lognormal distribution is better in modeling physical phenomena while the beta distribution is better for modeling probabilities.

Yet, there have been no data-based studies comparing the applicability of these functions in modeling HEPs. In fact, a discrete probability distribution, without a pre-defined range or analytical form, may be used to characterize the uncertainty when one of the cited functions does not adequately represent the evidence or expert assessments. In many cases, the most informative representation of the uncertainty in a parameter value may be provided by a discrete probability distribution that does not have a defined analytical form. Discrete probability distributions are used extensively in PRAs. Experience has also shown that discrete probability distributions are often the best and most efficient format to represent the uncertainty from an expert elicitation process.

J.3.3 Expert Judgment When numerical data are unavailable or sparse, the judgment of experts with knowledge in the specific technical field is used to arrive at best estimates of the distribution of the probability of a parameter or basic event. The process of obtaining these estimates typically is called expert judgment elicitation, or simply expert judgment or expert elicitation, and it is usually used when studying rare events. Ideally, this approach provides a probability distribution with values for the central tendency of the distribution and of the dispersion of the distribution, which represents the expert or best available knowledge about the probability of the parameter or basic event.

As new experiential or empirical data become available, the data can be used to verify or modify the expert information, or the experts can use the new data to update their judgment. As additional information becomes available, the Bayesian approach provides a methodology to account for new information, without having to repeat the expert elicitation process. As the influence of the evidence increases, the influence of the prior based on expert elicitation decreases.

Guidance for Formal Expert Judgment Expert judgment is the information provided by experts in response to a technical question

[122]. It represents an informed opinion or belief about the state of knowledge of a technical issue based on experts training and background. The NRC has used expert elicitation to inform 1 As noted by Kelly and Smith [121], a likelihood function is also referred to synonymously as a stochastic model, probabilistic model, or aleatory model.

J-5

many important regulatory decisions. Notably, expert elicitation has been an essential part of the NRCs use of PRA methods. In response to SRM COMGEA-11-0001, Utilization of Expert Judgment in Regulatory Decision Making [123], the NRC staff developed practical guidance for conducting formal expert elicitation to support decisionmaking activities [124], [125]. The guidance describes the basic principles and process of conducting expert elicitation, along with lessons learned from piloting the guidance. The basic principles were generalized from NUREG/CR-6372 [52], [53], developed by the Senior Seismic Hazard Analysis Committee, which describes a process to guide the performance of probabilistic seismic hazard analysis through formal expert judgment. The following is a brief summary of the basic principles and process of expert elicitation.

1. Basic Principles for Eliciting Expert Judgment The ultimate objective of conducting an expert elicitation is to appropriately represent the center, body, and range of the technical communitys views about a technical problem. When expert judgment is used to support decisionmaking, the elicitation should be performed in a manner that ensures confidence in the results. As such, expert elicitation should conform to the following principles, regardless of the scale, level of effort, and the method or procedures used for the elicitation process:

1) Representation of technical communityThe resultant expert judgment should represent the overall communitys views and beliefs about the state of knowledge for the technical problem. The expert panel should (1) be an adequate sample of the overall technical community, (2) have sufficient breadth of knowledge that it can evaluate the available data, and (3) include leaders in the technical field who can capture the communitys degree of consensus and diversity.

2) Independent intellectual ownershipWhile the project sponsors have legal ownership of the project deliverables, the expert panel collectively has intellectual ownership of the results (i.e., the panel is responsible for the robustness and defensibility of the results).

To ensure intellectual ownership, all inputs to the elicitation should be shared with every expert. To maintain the independence of intellectual ownership, expert judgment must be based on the experts knowledge and expertise, not the positions of the project sponsors or organizations with which the experts are associated. Each expert should also maintain independence from the other experts on the team to avoid (or mitigate) a group-think bias risk.

3) Avoidance of conflicts of interestTo minimize bias in the elicitation, careful consideration should be given to potential conflicts of interest before selecting experts.

Experts should be free from direct and potential conflicts of interest to the extent practical. In all cases, potential conflicts of interest or even the appearance of conflicts of interest should be disclosed up front.

4) Breadth of state of knowledgeThe expert panel should evaluate a range of data and models that are representative of the overall technical community to obtain the range of knowledge and interpretations of the technical issue.

5) Interaction and integrationTo represent the knowledge and interpretations of the technical community, experts should interact with each other as they accumulate and evaluate existing knowledge and make interpretations. Experts should make their interpretations based on the integration of their own knowledge and inputs from other experts. The final results should be the integration of the individual judgments to represent the center, body, and range of the state of knowledge.

J-6

6) Structured processAn expert elicitation should employ a structured process to facilitate interaction and integration and reduce biases in the outcomes.

7) TransparencyOften the results of an expert elicitation serve a range of users with diverse needs. To ensure that the results are used appropriately, the information generated must be documented in a transparent way. Transparency includes the input data and models that were considered, the process used, the results obtained, and the caveats and limitations of the inputs, process, and results. Transparency also helps to demonstrate the stability and integrity of the results.

2. Expert Elicitation Process A structured and systematic process should be used that encompasses all of the basic principles. This section describes a recommended systematic expert elicitation process that consists of 10 steps across four phases. Figure J-1 illustrates the recommended expert elicitation process.

Final documentation Planning and preparation Pre-elicitation work Elicitation and sponsor review Assemble and Define the expert disseminate the elicitation dataset Elicit expert judgments Document the Familiarize and process and results, Form the expert and conduct refine the technical panel sponsors technical issues Integrate expert review judgments Develop the project Conduct training and plan piloting Participatory peer review Figure J-1 Diagram of a Formal Expert Elicitation Process Phase 1: Planning and preparationThe purpose of this phase is to ensure that the elicitation problem is sufficiently defined to address the regulatory application of interest; that the project team, expert panel, and elicitation process are adequate to address the elicitation problem; and that the experts are provided with necessary information before the actual elicitation.

Step 1. Define the expert elicitation (objective and expected outcomes, intended use of the outcomes, technical issues, boundary conditions).

Step 2. Form the expert panel.

Step 3. Develop the project plan.

J-7

Phase 2. Pre-elicitation workThe purpose of this phase is to ensure that the dataset is compiled with the involvement of the expert panel, and all of the team members understand the project, the technical problems, the individuals role and responsibilities, and the theories of probabilities and uncertainties.

Step 4. Assemble and disseminate the dataset.

Step 5. Become familiar with and refine the technical issues.

Step 6. Conduct training and piloting.

Phase 3. ElicitationThe purpose of this phase is to elicit expert judgments through interactive workshops. The expert panel interacts to evaluate the data and models, make interpretations, form initial judgments, and integrate the judgments to represent the distribution of views in the technical community.

Step 7. Elicit expert judgments through face-to-face, facilitated, and structured workshop(s).

Step 8. Integrate expert judgments to find the center, body, and range of the communitys state of knowledge.

Phase 4. Final documentation and sponsor reviewThe purpose of this phase is to develop final documentation of the process and results and have the technical staff of the sponsor organization review the documentation for regulatory assurance.

Step 9. Document the process and results and conduct sponsors technical review.

All-Phase. Participatory peer reviewThis is not a separate phase. Rather, the purpose of this all-phase activity is to ensure that participatory peer review occurs in all phases of the expert elicitation process.

Step 10. Conduct participatory peer review The above expert elicitation guidance provides a structured, systematic approach to conducting expert elicitation. The guidance has been piloted in several PRA modeling developments, including HEP estimation in the development of the IDHEAS At-Power Application [22].

Depending on the intended use of the expert judgment and available resources, the staff may choose to implement an expert elicitation process at varying levels of effort. Nevertheless, regardless of the detailed implementation of the process, expert elicitation of HEPs using IDHEAS-G should follow all of the basic principles described.

It is important to recognize that the general principles of expert elicitation described in this section apply to a formal expert elicitation process described above and any form of information elicitation of experts' estimates with uncertainties, such as the derivation of information and estimates from interviews with plant personnel (e.g., probability distributions for the timing of events, time required to perform specific tasks, etc.). The process shown in Figure J-1 is resource demanding and is not needed for the elicitation of expert information during interviews with plant personnel. Yet, the basic principles of the elicitation process should be followed when estimates of parameter values and associated probabilities are derived from subject matter experts such as plant personnel.

J.4 Assessment of Data and Evidence in Human Error Probability Estimation In a Bayesian approach, regardless of the quantity of data available (even massive amounts of data), expert judgment is still needed to consider the applicability of the data, whether there are J-8

gaps in the data, and where there may be uncertainties in the data. Whether Bayesian computation or expert elicitation is the chosen approach, the data and evidence used for estimating HEPs should describe the human errors associated with the tasks or CFMs at the same level as those in the IDHEAS-G quantification model. Because HRA data are rare, estimating HEPs often requires using available data from different sources. First, the data need to be assessed to determine the tasks represented by the data and their applicability to the generic tasks and CFMs in IDHEAS-G. In addition, the context of the data needs to be assessed to ensure that it is used appropriately for corresponding combinations of PIFs.

ExampleMisuse of data because of various levels of tasks. Operating experience data in NPPs may be collected for human errors in specific procedural steps or for types of HFEs.

Such data cannot be directly used for HEP estimation of IDHEAS-G CFMs because the CFMs describe human errors in performing critical tasks that may constitute many procedural steps.

ExampleMisrepresentation of data. One implicit assumption in HRA is that the HEPs are for average crews. In reality, human performance varies greatly from crew to crew. While this crew variability can be represented with the distribution of an estimated HEP, the available operating experience data can be biased and thus does not represent the average or the full range of crew performance.

J.5 Documentation and Communication of Estimation of Human Error Probability Distributions and the Assumptions Made for the Estimation IDHEAS-G provides a set of CFMs, PIFs, and a HEP model to estimate HEPs of important human actions. When using IDHEAS-G to calculate the HEP of an important human action, HRA analysts need to verify the following:

1) The critical tasks identified for the HFE are at the same level of detail as those assumed in the HEP estimation.

2) The CFMs identified are within the same scope as those assumed in the HEP estimation.

3) The states of the PIFs are consistent with the definitions of states used in the HEP estimation.

Communicating the assumptions with HEP distributions ensures that HRA analysts properly use the HEP values in the quantification. An effective way of communicating is by using examples to demonstrate the assumptions.

In summary, when estimating HEPs, analysts should comply with the following guidance:

1) HEP distributions should be estimated with a Bayesian approach.

2) Data or evidence used in the Bayesian approach should be assessed for applicability and uncertainties.

3) Using expert judgment for HEP estimation should comply with the seven basic principles described in Section J.3.3.

4) Assumptions made for HEP estimation should be documented and communicated along with the estimated HEP distributions.

J-9

APPENDIX K IDHEAS-G TREATMENT OF DEPENDENCY BETWEEN HUMAN FAILURE EVENTS The HEP of an important human action is a conditional probability. In other words, the HEP is estimated based on the assumed specific conditions or context in which the important human action is performed. The context of an important human action in a PRA model (i.e., a human failure event (HFE)) may change significantly as a result of the performance of its preceding important human action. Dependency represents the effects of success or failure of an important human action on the performance of the subsequent action. WASH-1400 [108] refers to dependency as coupling, and THERP [3] refers to it as dependence. Dependency may exist when actions performed earlier in a scenario affect subsequent actions or when actions overlap in time.

THERP evaluates dependency between steps of an important human action by assessing the dependency level between two consecutive HFEs and adjusting the HEP of the dependent HFE accordingly. The dependency level is assessed by the commonalities of the two HFEs.

Existing HRA methods adapt the THERP dependence model. THERPs dependence model provides a gross approach to address dependency in HRA. Dependency evaluation should consider how an HFE occurs and how the specific failure affects subsequent HFEs. IDHEAS-G models HFEs with cognitive failure modes (CFMs) to distinguish different failure modes, which enable precise modeling of dependency. IDHEAS-G models HFE context with specific PIF attributes and quantifies the effects of PIF attributes on specific CFMs. This provides a structure to quantify dependency effects. Thus, IDHEAS-G is capable of expanding existing dependency evaluation by modeling the mechanisms underlying dependency. The NRC staff developed the IDHEAS-G Dependency Model. The process of implementing this model is described in Chapter 4. This appendix describes the basis and details of the IDHEAS-G Dependency Model.

K.1 Modeling Human Failure Event Dependency in PRA What is dependency between HFEs? Why is there dependency? In practice, multiple sequential HFEs are often evaluated throughout the progression of an event scenario. A potentially important consideration for the analysis of the reliability of an HFE is the performance of the preceding HFE. If personnel have taken the appropriate actions for preceding HFEs, it can be evidence that they understand what is happening and have developed an appropriate response strategy (at least, up until that point). Conversely, if the scenario involves errors or failures to perform the required tasks in a timely manner preceding to the HFE being evaluated, it can be evidence that the personnel are struggling with establishing an adequate understanding of the plant status or making an effective decision or timely implementation of the recovery or mitigative actions. Thus, how the preceding HFEs are performed and the consequences of their performance affect the context and reliability of performing the HFE being evaluated.

K.1.1 Dependency in Pre-Initiators and initiators Dependency could occur in pre-initiators (e.g., such as valves or switches left in a wrong position, calibration errors, or use of incorrect fuel, lubricant or additives), initiators (e.g., the human actions related to the supporting system initiating events), and post-initiators [126], [127].

NUREG-1792 [13] states the following:

K-1

Dependencies among the pre-initiator HFEs, and hence the corresponding HEPs in an accident sequence, should be quantitatively accounted for in the PRA model. This is particularly important so that combined probabilities are not inadvertently too optimistic, resulting in the inappropriate decrease in the risk-significance of human actions and related accident sequences and equipment failures. In the extreme, this could result in the inappropriate screening out of accident sequences from the model because the combined probability of occurrence of the events making up an accident sequence drops below a threshold value used in the PRA to drop sequences from the final risk results.

K.1.2 Dependency in Post-Initiators NUREG-1792, Good Practices for Implementing [HRA], describes dependency as follows:

Dependencies among the post-initiator HFEs, and hence the corresponding HEPs in an accident sequence should be quantitatively accounted for in the PRA model by virtue of the joint probability used for the HEPs. This is to account for the evaluation of each sequence holistically, considering the performance of the operators throughout the sequence response and recognizing that early operator successes or failures can influence later operator judgments and subsequent actions. This is particularly important so that combined probabilities that are overly optimistic are not inadvertently assigned, potentially resulting in the inappropriate decrease in the risk-significance of human actions and related accident sequences and equipment failures. In the extreme, this could result in the inappropriate screening out of accident sequences from the model because the combined probability of occurrence of the events making up an accident sequence drops below a threshold value used in the PRA to drop sequences from the final risk results.

Several existing HRA methods, such as THERP, SPAR-H, and Fire HRA, explicitly identify and evaluate dependency among post-initiator HFEs and adjust HEPs of the HFEs (from independent HEPs to become dependent HEPs) accordingly.

K.1.3 Dependency Between Pre- and Post-Initiators PRA should also quantitatively account for dependencies between the initiator and post-initiator HFEs. HFEs that contribute to the occurrence of an initiating event and HFEs responding to that initiating event may or may not be performed by different people, and dependency could exist between these two types of HFEs. Such dependence may have a very important influence on the overall evaluation of human performance throughout the scenario. For example, an initiating event involves failure of a normally-operating cooling water pump, followed by personnel failure to start the standby pump before temperatures exceed equipment trip setpoints. The loss of cooling may last for an extended period of time, depending on the plant-specific design, equipment trip setpoints, etc. The reasons why the personnel did not or could not start the standby pump, and the time during which those reasons persisted, can affect the other actions to cope with the loss of cooling event. That is especially true for actions that are required within a relatively short time after the initiating event occurs.

In practice, the modeling of pre-initiator, initiator, and post-initiator HFEs are conceptually the same. The actions are performed in response to system cues to achieve the desired system functions, and they must be accomplished within their functionally-determined time window.

Also, they all can be modeled with the five macrocognitive functions in IDHEAS-G. Therefore, the same techniques for identifying and evaluating sources of dependence among post-initiator HFEs are applicable to identify and evaluate dependencies between pre-initiator HFEs and between initiator HFEs and post-initiator HFEs. It is important for PRA and HRA to K-2

quantitatively account for the dependency between initiator and post-initiator HFEs without overlooking or even ignoring it.

K.2 Advancing the State of Practice for Dependency Evaluation While developing IDHEAS-G, the NRC staff made initial efforts to advance the state of practice for dependency analysis. There were multiple sessions of formal and informal knowledge elicitation from the NRC staff and the broad HRA/PRA technical community on dependency analysis. Existing HRA methods, practices, and experience with treating HFE dependency were also reviewed. Through those efforts, the NRC staff identified the key areas and strategies for improvement in evaluating dependency. Furthermore, the NRC staff developed the IDHEAS-G dependency model to address those areas and extend dependency evaluation in existing HRA methods.

K.2.1 Areas of Improvement in Existing Dependency Modeling and Practices Cognitive foundation for dependency The dependency factors used for dependency evaluation should tie to mechanisms for dependency. Consider an event in the IDHEAS-G cognitive framework where a response involves different macrocognitive functions: personnel have a correct understanding but fail to execute the first response correctly, which results in the HFE. In practice, if personnel fail the first HFE because of an incorrect situation assessment, the personnel may carry the incorrect understanding of the situation to the performance of subsequent HFEs. This phenomenon could cause systematic errors on the subsequent HFEs. Therefore, cognitive modeling is needed to accurately model the dependency.

Explicit consideration of different portions of a human failure event Dependency evaluation in current practices is applied in a PRA process. The evaluation assesses the dependency levels between the HFEs within a cutset of a PRA model. In this practice, the dependency between two HFEs is denoted by a single dependency level. In reality, the dependency exists between the elements of two HFEs. The elements could be the critical tasks to be performed to achieve the HFEs objectives (e.g., the occurrence of an HFE causes a change to the tasks to be performed to achieve the subsequent HFEs objectives), the macrocognitive functions required for the critical tasks, the time availability of the HFE, or the PIFs. For more precise dependency modeling, an area for improvement is to explicitly model the dependency between the elements of HFEs.

Basis for dependency factors Dependency evaluation needs the basis for why and how the HEP of a dependent HFE would increase given the presence of dependency factors. Existing dependency models use a set of factors to assess the dependency level between HFEs. The dependency level is determined by the combination statuses of the factors. These factors (e.g., same people, same location, and same cue) provide little explanations and justifications about how the dependency occurs.

Thus, an area of improvement is to model the underlying mechanisms or causes of dependency between two HFEs.

Basis for human error probability adjustment Lastly, none of the existing dependency approaches were empirically based. The approaches were based on the belief that some commonalities (similarity and proximity) between two HFEs may increase the chances of human errors. epin [128] compared several dependency K-3

approaches and the results showed large differences in the calculated HEPs for the same events within the same PRA. The differences are the consequence of the subjectivity in dependency treatment. An area of improvement to address the issue is to provide more specific guidance supported by empirical examples to reduce the subjectivity.

K.2.2 Insights on Advance Existing Dependency Approaches With the considerations described above, the NRC staff recognized that simply improving individual parts of the current dependency approaches, such as providing detailed guidelines on assessing dependency factors, would not address the fundamental limitations in existing dependency approaches due to the lack of fundamental technical basis. Thus, the NRC staff proposed a new framework to model dependency. The new framework is described as the following:

Dependency evaluation needs to identify dependency at the cognitive process level and cognitive and behavioral science should inform the treatment of dependency.

Modeling the effect of dependency should be consistent with the modeling an HFE.

Thus, dependency evaluation should include the analysis of the HFE context and definition, the critical tasks, and the CFMs and PIFs affected by dependency.

IDHEAS-G provides a foundation to advance dependency evaluation through its HFE modeling structure and cognitive modeling elements. Therefore, the authors of this report developed the IDHEAS-G dependency model as described in the next section.

K.3 IDHEAS-G Dependency Model This section describes the IDHEAS-G Dependency Model. Figure K-1 illustrates an overview of the model. It consists of three parts: identifying dependency context, modeling dependency context, and adjusting the HEP (or calculating the dependent HEP). The following subsections discuss the three parts.

1. Identify the dependency context Cut set with

Consequential dependency multiple HFEs (HFE1, HFE2)

Resource-sharing dependency

Cognitive dependency HFE1 and HFE2 are 2. Model the dependency context HFE2lHFE1 independent All no Any Are there changes to HFE2s:

Definition? Time required and time available?

P(HFE1,HFE2)= yes?

Feasibility? CFMs?

P(HFE1)*P(HFE2) Critical tasks? PIF attributes?

Yes

3. Calculate P(HFE2lHFE1) based on P(HFE1,HFE2) =

context changes to HFE2 and using P(HFE1)*P(HFE2lHFE1) same method as individual HFEs HFE2lHFE1 means the occurrence of event HFE2 given the occurrence of event HFE1, where HFE1 is the first event and HFE2 is the second event.

Figure K-1 Overview of the IDHEAS-G Dependency Model K-4

K.3.1 Identification of Dependency Context Dependency context refers to the clue that results from the failure of an HFE and changes the scenario. The clue specifies the changes to the context of subsequent HFEs. The following definitions of the three types of dependency facilitate the search for dependency context.

Consequential dependency The outcome of an HFE directly affects the performance of subsequent HFEs. THERP [3]

refers to this as direct dependence. The outcome of the preceding HFE may affect various elements of the subsequent HFEs, including HFE definition (e.g., HFE feasibility), the critical tasks that must be performed, the applicable CFMs, the time availability, and the applicable PIFs. Some general examples of consequential dependency include:

Taking a longer time to complete a task results in less time available to complete the subsequent tasks. For example, taking longer to diagnose the problem would result in less time available to perform actions to solve the problem.

A task failure results in failure to perform subsequent tasks because the tasks must be performed in sequence. For example, repair work requires a specific part, and if the part is not delivered, the repair work cannot be completed.

A task failure results in the subsequent tasks being implemented on the wrong subject.

For example, using a wrong patient record could result in applying the wrong radiation dose and location to the patient in radiation therapy.

A task failure increases the workload of performing the subsequent tasks. For example, additional actions may be required to mitigate a situation caused by an automatic system actuation triggered by the mistake of the preceding actions.

The occurrence of an HFE affects the subsequent HFEs because it produces an incorrect mental model of the scenario, wrong preconceived expectations, or a bias that steers personnel to an inappropriate, preferred course of action.

The discussion in Section K.4 provides an empirical example of consequential dependency in an NPP event.

Resource-sharing dependency Resource-sharing dependency occurs when tasks in an HFE share the same resources with the subsequent HFEs. For example, containment spray and emergency core cooling systems (i.e.,

centrifugal charging pumps and safety injection pumps) share the same water source, or there is limited manpower to perform multiple tasks. In a resource-sharing situation with a narrow margin of sufficient resources to meet all demands, occurrence of an HFE may have taken more than the allocated resources; therefore, it results in less than normally available resources for the subsequent HFEs. The decisionmakers would need to consciously decide to either take a risk to perform all the tasks in subsequent HFEs by carefully maneuvering the limited resources or they may decide to apply the resources to one critical task to ensure its reliable completion while sacrificing the other critical tasks. Such dependency could modify the feasibility of subsequent HFEs, the critical tasks to be performed, and the relevant CFMs and PIFs. Some general examples of resource-sharing dependency are:

K-5

The personnel performing Task A are expected to support performance of Task B after completing Task A. If Task A cannot be performed in time, then Task B must be performed with less manpower than expected.

Tasks performed simultaneously require close coordination to achieve success. A task failure may increase the difficulty of performing the other tasks.

Cognitive dependency Cognitive dependency refers to the dependency in the cognitive flow of two consecutive HFEs.

The cognitive flow includes detecting information, understanding the situation, making response decisions, executing the response decisions, and coordinating responses of different teams involved. To successfully accomplish a critical task, all the cognitive activities of the task need to be performed successfully. The information detected is used to understand the situation.

Incorrect information detection would result in incorrectly understanding the situation. Incorrect understanding of the situation would lead to incorrect decisions, which, in turn, results in incorrect or ineffective action execution. When assessing the HEP of a critical task, IDHEAS-G models the conditional HEPs of individual cognitive failure modes. That is, the HEP of failure of detection assumes that the information presented for detection is correct. The HEP of failure of understanding the situation is assessed under the condition that all critical information is correctly detected. The HEP for failure of decisionmaking is assessed on the condition of having the correct understanding of the situation. The HEP for failure of action execution is based on having the correct decision on response. Finally, the HEP of failing interteam coordination is under the condition that personnel correctly perform the other four macrocognitive functions. However, these conditions may be interrupted by the dependency between the CFMs of two HFEs.

Cognitive dependency occurs when an HFE creates a biased mindset or expectation of the situation. As such, personnel may fail the subsequent HFEs due to having the wrong mental models for initiating and performing the cognitive activities of the critical tasks. This consideration is most relevant if the crew or personnel perform the subsequent actions using the same procedures and having the same training. Cognitive dependency is often not apparently

'consequential', and the sources of that dependence may be subtle. Therefore, some sources of cognitive dependency have been overlooked or not evaluated thoroughly in high-level assessments. For example, personnel believe that the test instrument has been correctly calibrated while it was not, and they use the instrument to calibrate the setpoints of equipment in both HFEs. Another example is that personnel skip performing independent peer check because they believe that the other teammate is highly experienced so he or she would not make a mistake. This type of dependency is particularly important for human actions recovering an HFE because it is highly likely that the recovery action is performed in the context like the one that fails the original HFE.

The cognitive dependency context may be implicitly modeled in THERP through its dependency assessment guidelines such as similarities among personnel with respect to all relevant factors. Cognitive dependency is addressed in the Fire HRA guidelines (NUREG-1921) for the identification and evaluation of important factors that affect dependencies.

A special instance of cognitive dependency is complacency dependence. When an undesired consequence can be prevented by either of two individuals or teams performing two HFEs, the individual or team relies on the other party to take the action. Consequently, neither party performs it. In other words, personnel performing one HFE creates a biased mindset that certain actions have already been performed by others, regardless of the success or failure of the other HFE. This kind of dependency can be prevalent in complex, dynamic scenarios. Yet, K-6

it can also be easily overlooked because there is no apparent consequence. In particular, complacency dependence does not require that the individuals or teams be in the same physical location. Some general examples of cognitive dependency include:

HFE1 occurs due to the failure of detecting a critical piece of information. HFE2 has a critical task that requires detecting the same information. The context of HFE2 does not introduce opportunities to question the correctness of the critical information in HFE1; thus, the PIFs that led to the failure of detection in HFE1 are the same for HFE2. Then, the incorrect detection of HFE1 would result in failure of detecting the same information in HFE2. The focus of this kind of dependency is on the human tendency to seek convenience because the information is readily available (availability heuristics). The following are some contextual factors contributing to the common cause dependency in failure of detection:

The procedure or process of performing HFE2 does not clearly direct personnel to check the information. The process refers to normal operation requirements such as information gathering in shift turnover. Personnel likely have the tendency to use the information acquired in HFE1.

The information is not readily available.

This is a high pace or complex scenario.

Personnel fail HFE1 due to failure in understanding the situation. HFE2 also requires the same understanding. If there is no vivid and strong alternative information to alter the context of HFE2, the same PIFs that led to an incorrect understanding in HFE1 would lead to incorrect understanding of the situation in HFE2. An example of cognitive dependency that involves failure of understanding is in the Three Mile Island accident, where the operators believed that the RCS was full of water due to a misunderstanding based on the water level in the pressurizer that indicated it was in a condition known as liquid solid). The understanding led to the actions of tripping the safety injection. The incorrect understanding of plant status caused the operators to systematically defeat the automatic safety functions which led to the undesired consequence. The following are the context factors for common cause dependency due to failure of understanding:

There is no new information (including system feedback) that is vividly available (e.g., in procedure or shift turnover) to personnel to alter the incorrect understanding in HFE1.

This is a high pace or complex scenario.

Supervisor, overseer, or independent checker (e.g., shift technical advisor) does not act independently.

With the correct understanding of the situation, personnel make a wrong decision in HFE1 due to a certain mindset; for example, production over safety, incorrectly prioritizing safety concerns, or the decisionmaker is influenced by stakeholders who have different considerations from the operators. The same mentality in decisionmaking could cause dependent decision failure in HFE2. The following context factors likely contribute to the common cause dependency in failure of decisionmaking:

The decision has high economic consequence, and an alternative option that has potential to avoid or mitigate the economic consequences exists.

K-7

The decision is strongly influenced by outside stakeholders who have different considerations from the decision makers.

A source of decisionmaking dependence may arise due to the development and adherence to a single strategy to mitigate an evolving scenario. For example, personnel in the Technical Support Center (TSC) may develop a particular strategy and course of action to mitigate the evolution or the consequences of a severe event. Experience has shown that once a strategy is developed, personnel tend to adhere to that strategy, despite possible indications that it may be flawed. They may be reluctant to question or revise the strategy until something dramatic occurs, which then compels them to re-evaluate their original rationale. In the interim, multiple decisionmaking human failure events (HFEs) may occur while personnel attempt to implement the faulty strategy.

Personnel fails to execute the action in HFE1, and HFE2 requires executing the same action. Personnel have the tendency to execute the action in the same way as they performed the preceding, similar action. Therefore, the PIFs resulting in failure of action execution in the HFEs would fail the same action execution in HFE2. An example is using mis-calibrated test equipment to calibrate a group of transmitters. The following context factors likely contribute to this kind of common cause dependency:

The tasks are performed by the same person or same group of individuals.

The actions are within the same task order or the same procedures.

System, component, or equipment does not provide feedback on what caused the failure of HFE1.

K.3.2 Modeling Dependency Context in IDHEAS-G Structure IDHEAS-G models the context of an HFE at progressively more detailed levels to quantify the HEP. IDHEAS-G explicitly delineates how each level of context modeling contributes to the final HEP of an HFE. Figure K-2 illustrates the structured context in IDHEAS-G. The various levels of context modeling are described as the following:

HFE definitionThe HFE definition defines the context specifically applicable for the event, such as its beginning and desired end states, time criticality and availability, who performs the action, and whether the HFE is feasible.

Time uncertaintiesFor time-sensitive important human actions, uncertainties in time available and time required contribute to the HEP of the HFE.

Critical tasks and task characterizationThe HFE definition is propagated to identify and characterize critical tasks in the event.

Macrocognitive functions required for a task and the CFMs of a critical taskThe next level of analysis focuses on the macrocognitive functions involved and, therefore, determines the CFMs applicable to the critical tasks.

Applicable PIFs and their statusThe HEPs of the CFMs are determined by the status of the applicable PIF attributes.

Assume that two HFEs, HFE1 and HFE2 are consecutively performed. One or several types of dependency context become applicable to HFE2 because of the failure of HFE1. The dependency context may change the definition of HFE2, its critical tasks, applicable CFMs, and applicable PIF attributes. The IDHEAS-G framework allows analysts to systematically model K-8

the dependency context by examining the changes to each level of context modeled for HFE2.

The HEP of HFE2 can then be recalculated based on the changes. Figure K-2 illustrates the process of modeling dependency context.

To model the dependency context of HFE2 caused by the failure of HFE1, analysts may systematically examine the changes in HFE2 context:

Are there changes to the HFE definition (e.g., beginning or ending states, personnel, location, etc.)?

Does the occurrence of HFE1 make HFE2 infeasible?

Does the occurrence of HFE1 change the time availability for HFE2?

Are the critical tasks of HFE2 different?

Are there new CFMs of the critical tasks?

Are there changes in PIF attributes applicable to the CFMs?

If the answers to all the questions above are no, then HFE2 is deemed to be independent of HFE1. If the answer to any of the questions is yes, then HFE2 is dependent of HFE1. The changes are then documented for HEP adjustment.

Success Success OK HFE 1 HFE 2 (or other events)

Failure Failure Use HEP 2 HFE 2 definition Time uncertainty Task 1 Critical task 2 Task 3 No Interteam Change?

Action Detection Understanding Decisionmaking execution coordination CFMs CFMs CFMs CFMs CFMs Yes Adjust PIF 1 PIF 2 PIF 3 PIF 4 PIF 5 HEP 2 Figure K-2 Illustration of Dependency Adjustment of HEP Based on IDHEAS-G Context Structure K.3.3 Recalculate the HEP of the Dependent HFE After the dependency context is identified and modeled in the IDHEAS-G framework, estimation of the probability of the dependent HFE can be based on the modeling of the dependency context, i.e., the changes from the original assessment of HFE2 identified any of the following:

HFE definition, feasibility, time availability, critical tasks, CFMs, or PIFs. That is, the dependent HEP is estimated or calculated by applying the same method of estimating HEPs of individual HEPs to the modeled dependent context.

K-9

Notice that different types of dependency sources use the same process of recalculating the dependent HEP of HFE2. The purpose of identifying dependency types serves as a screening process for HRA analysts to search for dependency context and focus the modeling of dependency context on pertinent items of change. Once the dependency context is modeled in the IDHEAS framework, the HEP calculation is based on the modeled context, regardless of the source of the context.

K.3.4 An Example Illustrating the IDHEAS-G Dependency Model This section presents an example to demonstrate identifying and modeling dependency. Figure K-3 shows a diagram of the two consecutive HFEs. The scenario involves the use of a portable diesel generator in an Extended Loss of AC Power (ELAP) event. The response strategies include a critical human action where a portable diesel generator is used to power the electrical buses. The action includes three critical tasks: (1) transporting, (2) connecting, and (3) starting and operating the portable diesel generator. HFE1 is defined as the failure to use the portable diesel generator. The occurrence of HFE1 could be due to the failure of any of the three critical tasks. HFE2 is for any action to be performed if HFE1 occurs. That is, HFE2 evaluates actions that are needed to correct cognitive or implementation errors that contribute to failure of HFE1 and to achieve the defined functional success criteria within the available time window. The HEPs, if they were to be hypothetically assessed as independent, are P1 for HFE1 and P2 for HFE2. If there is no dependency between the HFEs, the combined HEP would be P1 x P2, as shown in Figure K-3(a). However, there could be various instances of dependency that change the context of HFE2 due to the failure of HFE1, thus change P2 to P2DEP, as shown in Figure K-3(b). Subsequently, the combined HEP for the whole event would be P1 x P2DEP.

(a) Context Success HFE1 S of HFE1 Failure (P1)

Context Success HFE2 S of HFE2 Failure (P2)

F (P1xP2)

(b) Context Success HFE1 S of HFE1 Failure (P1)

Dependent Success Context of HFE2 S HFE2 Failure (P2DEP)

F (P1xP2DEP)

Figure K-3 (a) Diagram for Modeling Hypothetically Independent HFEs. (b) Diagram for Modeling Dependent HFEs.

K-10

Next, the following discussion provides a set of hypothetical instances of dependency between the two HFEs. Each instance represents one or more types of dependency context:

consequential dependency, cognitive dependency, and resource-sharing dependency.

Instance 1: Consequential dependency Some of the critical tasks for HFE2 could differ after failure of HFE1. Due to the consequences of HFE1, personnel may decide to take an alternative option to achieve the success criteria of HFE2. The alternative option involves different critical tasks. The HEPs for the new set of critical tasks should be recalculated. This instance demonstrates that the dependency effects are evaluated by the change of the critical tasks of HFE2.

Instance 2: Consequential dependency Failure of HFE1 results in additional critical tasks in HFE2. Personnel fail the critical task of transporting the generator in HFE1. The failure blocks the transport route. Therefore, making the route accessible is an additional critical task to HFE2. This new critical task increases the time required for HFE2. Moreover, HFE1 results in a reduction in the time available for HFE2.

Performing HFE1 takes excessively longer than expected. This may alter the timeline of HFE2 and reduce the time available for performing HFE2 (by the same or a different set of personnel).

On the other hand, HFE1 may fail early so it takes less than the expected time, then there is more available time for HFE2. In either way, the HEP of HFE2 attributed to time uncertainties should be recalculated based on the new time uncertainty analysis. Therefore, the new HEP of HFE2 includes the HEPs of the three original critical tasks (i.e., transport, connect, and start and operate the portable diesel generator), the new critical task (i.e., make the transport route accessible), and the recalculated time uncertainty HEP ( ). This instance demonstrates that the dependency effects are evaluated by the change in the HFE time availability and the new critical task.

Instance 3: Consequential dependency Failure of HFE1 results in different CFMs or PIFs in HFE2. The inaccessible transportation route would result in failure of transporting the generator. When the pre-specified route is not accessible, personnel need to assess the event situation to determine a new route. This adds a new CFM, failure of decisionmaking, to the critical task of transporting the generator in HFE2.

In addition, use of the new route may change some PIFs of HFE2, such as visibility or resistance to physical movement. Those subsequently change the HEP of the HFE2 critical tasks. This instance demonstrates that the dependence effects are evaluated by changes in CFMs and PIFs.

Instance 4 - Consequential dependency Failure of HFE1 alters the definition of HFE2 and makes HFE2 infeasible. Personnel fail the critical task of connecting the generator. The procedure of connecting the generator has many steps and several steps are irreversible. Personnel make an error in those steps and create a situation in which the generator can no longer be connected. In this case, HFE2 become infeasible and its HEP becomes 1. This instance demonstrates that the dependence effects are evaluated by changing the HFE feasibility.

Instance 5 - Cognitive dependency Cognitive dependency can occur to the same personnel performing the same task. Personnel fail the critical task of connecting the generator in HFE1. Assume that skipping key procedure steps is the only failure for HFE1 and HFE2, and there is nothing to explicitly alert personnel K-11

that, "You skipped steps in the procedure. That is why the bus is still de-energized." It is reasonable to assume, because of behavior inertia, HFE2 has a higher failure probability in the condition of HFE1 failure than HFE1 success. Behavior (decision) inertia is the tendency to repeat previous choices independently of the outcome, which can give rise to perseveration in subobtimal choices [129]. The behavior inertia is a cognitive dependency in IDHEAS-G. In this instance, the cognitive dependency causes the conditional probability of skipping a key procedure step in HFE2 to be higher than in the general situation. Failure of performing a key procedure step would increase the error probability of HFE2.

Instance 6 - Cognitive dependency Cognitive dependency can occur in different personnel or the same personnel. Personnel may fail starting or operating the portable generator in HFE1 due to the specific power loading requirements that differ from the plants normal generators. The same or a different set of personnel that perform HFE2 and rely on the same mental model of their normal generators more likely to fail HFE2. The HEP of HFE2 should account for the effect of wrong mental models due to common procedures, training, and experience of the two HFEs.

Instance 7 - Resource-sharing dependency HFE1 and HFE2 share the same set of personnel. For example, six people are normally available to perform these actions, and they may divide the tasks among the group. However, during this particular scenario, with the failure of HFE1, only three people are available to perform all of the tasks in HFE2. Those limited resources require that analysts must carefully evaluate HFE2, because the tasks must be performed in series, and they must be performed by the same personnel.

Another example is that HFE1 and HFE2 share the same critical tools. They both need to use the same truck to transport the generator that is attached to a trailer. Failure of transporting the generator in HFE1 results in some damage to the truck; thus, the truck cannot haul the trailer in the normally expected way. Personnel may find a different truck or figure out some other way to haul the trailer, but the untested or unpracticed way increases the difficulty of hauling the trailer and increases the chances of failure. This instance demonstrates that the dependency causes the changes of the critical tasks to implement HFE2 and changes of the applicable PIFs in implementing the critical tasks.

These instances continue to indicate that the different context for HFE2 can result from consequential dependency, cognitive dependency, and resource-sharing dependency. Notice that modeling the dependency is based on the changes in the context for HFE2 without specifying the direction of the changes. Generally, the changes are in the negative direction in that they result in a higher HEP compared to that without the dependency evaluation, i.e., the errors in performing HFE1 reduce the reliability in performing HFE2 (or increase the error probability of HFE2). However, the changes can be positive, especially with consequential dependency and cognitive dependency. The errors in performing HFE1 increase the reliability of performing HFE2. For example, quickly declaring that HFE1 cannot be accomplished successfully makes more time available for HFE2 (compared to the time available for HFE2 if it takes a longer time to declare HFE1 as unsuccessful). In the notion of cognitive dependency, the cognitive effects from a preceding failure might result in a lower HEP for the subsequent HFE, compared to a hypothetical, independent assessment of action HFE2.

K-12

K.4 Discussion K.4.1 Positive Dependency HFE dependency currently considered in PRA focuses on the effect of an HFE (HFE1) on a subsequent HFE (HFE2), i.e., the failure-failure dependency. There could be a potential dependency of the success of HFE1 on HFE2. In particular, if personnel have taken the appropriate actions for HFE1, there is empirical evidence that they understand what is happening and have developed an appropriate response strategy (at least, up until that point in the scenario). Therefore, in principle, the evaluation of some PIFs for HFE2 should account for that "positive" (i.e., increase the probability of success) cognitive coupling. For example, if the PIF scenario familiarity is evaluated as Unfamiliar under the assumption that HFE1 fails, then the success of HFE1 can change that PIF to familiar. The HEP of HFE2 may be lower than the "independent" value when HFE2 is evaluated without considering the preceding personnel successes. Modeling this kind of success-failure dependency is outside of current PRA approaches. For the same notion, IDHEAS-G recommends providing explicit justifications to credit the dependence effects.

K.4.2 Minimum Joint Human Error Probability A topic related to dependency is the minimum joint HEP of related HFEs. The conditional probability of a PRA failure event sequence can be reduced by implementing more event mitigation options, as long as these options are not completely dependent on each other. In theory, the sequences conditional probability can be infinitely reduced by applying more and more mitigation options. However, operational experience does not support that theory. For an HFE that can be accomplished by successfully performing any of multiple available manual actions, operational experience suggests that an HFE reliability threshold exists. NUREG-1792

[13] states that the total combined probability of all the HFEs in the same accident sequence or cutset should not be less than a justified value.

NUREG-1792 suggests establishing reliability threshold values (or a minimum joint HEP). The primary purpose in adopting a minimum or limiting value is to recognize that there may be causes of human failure that have not been thought about, or that are not accounted for in the particular HRA method used. In this way, the limiting value is one way to treat completeness uncertainty of the unknown unknown kind [130]. On the other hand, setting the minimum joint HEPs to be too conservative would affect the risk profile of the PRA model and increase regulatory burden on the NRCs licensees.

NUREG-1792 suggests minimum joint HEPs without specifying their applicable conditions, and the values lack a strong data basis.

K.5 Empirical examples of dependency in NPP events This section provides some real event examples of HFE dependency. These examples are intended to help understand and identify dependency context in complex scenarios.

Example 1: Consequential dependency.

On April 17, 2005, at 8:29 a.m., Millstone Power Station, Unit 3, a four-loop pressurized-water reactor, experienced a reactor trip from 100-percent power. The trip was caused by an unexpected A train safety injection (SI) actuation signal and main steamline isolation [131]

caused by a spurious Steam Line Pressure Low Isolation SI signal. As a result of the main steam isolation signal, the main steam isolation valves and two of the four main steamline K-13

atmospheric dump valves automatically closed. With the closure of the main steam isolation valves, the main steamline safety valves opened to relieve secondary plant pressure. Control room operators entered EOP E-0, Reactor Trip or Safety Injection, and manually actuated the B train of SI and actuated the B main steam isolation train in accordance with station procedures. Both motor-driven AFW pumps started to maintain steam generator levels. The turbine-driven AFW pump attempted to start but immediately tripped on overspeed. Operators were dispatched to investigate the cause of the turbine-driven AFW pump trip.

At approximately 8:42 a.m., the shift manager noted that a B main steam safety valve had remained opened for an extended period of time. In consultation with the unit supervisor and shift technical advisor, the shift manager declared an alert based on a stuck open main steam safety valve. The crew determined that the stuck open main steam safety valve represented a non-isolable steamline break outside containment. The main steam safety valves were in fact functioning as designed to relieve post-reactor-trip decay heat with a main steamline isolation signal present. In this event, the main steam safety valves closed once the operators took positive control of decay heat removal by remotely opening the atmospheric dump bypass valves.

At 8:45 a.m., because of the addition of the inventory from the safety injection, the pressurizer reached water solid conditions and the pressurizer power-operated relief valves cycled many times to relieve RCS pressure and divert the additional RCS inventory to the pressurizer relief tank. No pressurizer safety valve actuations occurred, and the pressurizer relief tank rupture diaphragm remained intact. At approximately 8:59 a.m., the operating crew transitioned from EOP E-0 to ES-1.1, Safety Injection Termination. The safety injection was reset, the crew terminated safety injection at 9:12 a.m., and normal RCS letdown was reestablished at 9:20 a.m. Millstone Unit 3 entered hot shutdown at approximately 7:03 p.m.

Discussion: The Millstone event shows consequential dependence. The operator failed to perform timely control of the RCS temperature and SG pressure (an upstream important human action), which caused a chain reaction affecting the performance of the subsequent important human actions (e.g., resetting SI and hot shutdown of the reactor). In this event, failure to promptly control the RCS temperature and SG pressure caused the main steam safety valves to be open for a prolonged period. This misled the operator to believe that the main steam safety valve was stuck open so that the operator had to perform additional tasks such as deciding whether the main steamline break procedure should be entered and evaluating the emergency activation level for the stuck-open main steam safety valve event. This, in turn, slowed the pace of implementing the procedure to terminate SI. The delay contributed to the pressurizer becoming liquid-solid, which, in turn, added tasks such as having crew debriefs to communicate the situation and decide whether the pressurizer block valves should be closed. The SI was not terminated until 30 minutes after the initiating event.

This event shows the consequential dependency of an important human action (timely control of RCS temperature and SG pressure) on subsequent important human actions (termination of SI).

Example 2: Cognitive dependency.

On March 5, 2002, the Davis-Besse Nuclear Power Station discovered a cavity in the reactor pressure vessel head. The cavity was discovered following a plant shutdown for a refueling outage, during which the plant was conducting inspections for vessel head penetration nozzle cracking in response to NRC Bulletin 2001-01 [132]. Upon further examination, the station found that the cavity extended completely through the 16.8-cm (6.63-inch) thick carbon steel reactor pressure vessel head down to a thin internal liner of stainless-steel cladding. This implied that immediately before the plant shutdown for refueling, the stainless-steel cladding K-14

was acting as the primary system pressure boundary over the region of the cavity. In this case, the cladding withstood the primary system pressure over the cavity region during operation.

However, the cladding is not designed to perform this function.

Discussion: The NRC inspection report [133] concluded that the reactor pressure vessel head cavity was likely a result of multiple years of acid corrosion. At some point in the latter half of the 1990s, the combination of flange leakage and leakage through vessel head penetration nozzle 3 caused the formation of the wastage cavity discovered in March 2002. This implies that there could have been more than five opportunities (five refueling outages) to identify the problem. However, the NRC inspection report states the following:

Information gained through interviews of the Davis-Besse Nuclear Power Station and NRC staff indicated that a mindset had developed that boric acid corrosion on the RPV head would not result in significant wastage because of the elevated temperature of the RPV head, resulting in dry boric acid deposits. Given this mind set, there was a presumption that boric acid deposits would not be a concern because the corrosion rates would be extremely low.

This example shows cognitive dependencythe station did not appreciate that boric acid would corrode the head, so it failed to act on the head corrosion.

Example 3: Resource-sharing dependency While in Mode 3, on May 7, 2004, the Palo Verde Nuclear Generating Station licensee was simultaneously testing the atmospheric dump valve and boron injection systems in Unit 1. The simultaneous performance of these tests caused a loss of letdown due to high regenerative heat exchanger outlet temperature. The loss of letdown occurred because operators were using a single charging pump for the boron injection test and increased letdown flow to accommodate the RCS heatup following atmospheric dump valve partial stroke testing. Operators restored letdown within 2 minutes. Subsequently, a pressurizer level transient occurred to a level greater than 56 percent, requiring entry into Technical Specification 3.4.9, Condition A, for 23 minutes

[134].

Discussion: The NRC inspection report [134] concluded that operators elected to perform a combination of tests that caused the loss of letdown flow and pressurizer level transient above the Technical Specification limits. This issue involves aspects associated with poor decisionmaking, questioning attitude, awareness of plant conditions, and communications between personnel performing simultaneous tests. This example shows resource-sharing dependency because the tests required close coordination to complete them without causing the loss of letdown flow and pressurizer level transient.

K.6 Summary This appendix describes the IDHEAS-G dependency model along with some insights for improving dependency analysis in HRA. The model is capable of systematically identifying changes to the context of an HFE that result from the failure of the preceding HFE, modeling the changes at different levels of IDHEAS-G process, and re-estimating the HEP based on the changes. This approach does not rely on similarities between the HFEs. The context of an HFE can be altered by the failure of its preceding HFE even though the two HFEs are performed by different people at different locations and time. In short, IDHEAS-G can treat dependency based on changes to the HFE context. Future research should validate this dependency model with real PRA examples and improve the model.

K-15

APPENDIX L UNCERTAINTY ANALYSIS AND DOCUMENTATION Identifying and understanding uncertainties are important parts of the process of achieving technical acceptability in risk-informed activities. Regulatory Guide 1.200 [135] notes the following:

An important aspect in understanding the base PRA results is knowing the sources of uncertainty and assumptions and understanding their potential impact. Uncertainties can be either parameter or model uncertainties. Assumptions can be related either to PRA scope and level of detail or to model uncertainties. The impact of parameter uncertainties is gained through the actual quantification process. The assumptions related to PRA scope and level of detail are inherent in the structure of the PRA model.

The requirements of the applications will determine whether or not they are acceptable.

The impact of model uncertainties and related assumptions can be evaluated qualitatively or quantitatively.

The NRCs guidance for treatment of uncertainty in PRA, NUREG-1855 [136], classifies three types of uncertainty according to their source: model uncertainty, parameter uncertainty, and incompleteness uncertainty. NUREG-1855 provides detailed guidance for treating uncertainties in PRA. IDHEAS-G adopts NUREG-1855 guidance on treating the three types of uncertainty in HRA. It is recommended that every step of the IDHEAS-G process include identifying sources of the three types of uncertainty and analyzing the impact whenever possible. The sources and potential impacts should be documented along with the outputs of each step.

L.1 General Guidance on Treatment of Uncertainty IDHEAS-G describes an HRA process that includes four stages, which are described in Chapter 4. Treatment of uncertainties is a part of the integrative stage, but it should be performed at every stage. The following is the general guidance for treatment of uncertainty in IDHEAS-G:

Treatment of uncertainty should be performed at every IDHEAS-G stage.

Treatment of uncertainty includes identifying sources of uncertainty; analyzing the impact, if possible; and documenting uncertainties.

Treatment of uncertainty should be performed with all three types: model, parameter, and incompleteness uncertainty.

L.2 Specific Guidance for the Three Types of Uncertainty This section includes specific guidance for each type of uncertainty. The guidance is adopted from NUREG-1855 [136].

L.2.1 Model Uncertainty Model uncertainty relates to the uncertainty associated with some aspect of the HRA process that can be represented by any one of several different approaches, none of which is clearly more correct than another. Consequently, uncertainty is introduced in the HRA results because there is no consensus about which resulting HRA outcomes most appropriately represent the aspect of the event being modeled.

L-1

Key model uncertainty is related to an issue for which no consensus approach exists and where the choice of approach is known to influence the HRA outcomes (e.g., introduction of new HFEs, alternative choice of critical tasks, and introduction of new CFMs and PIFs). Model uncertainty may result from a lack of knowledge about how systems and personnel behave under certain conditions. Model uncertainty can arise for the following reasons:

The phenomenon being modeled is itself not completely understood (e.g., behavior of gravity-driven passive systems in new reactors, human behavior with a new conduct of operations, and some system and human behaviors under extreme operating conditions).

For some phenomena, other data or information may exist, but need to be interpreted to infer system and personnel behavior under conditions different from those in which the data were collected (e.g., personnel behavior in response to actual severe accidents versus the behavior in severe accident response exercises).

The nature of system failure modes is not completely understood or is unknown (e.g.,

failure modes of digital instrumentation and controls).

The choice of approaches used to estimate HEPs in the application-specific quantification method (e.g., Bayesian computation versus expert judgment for estimating HEPs, use of a decision tree or other means for estimating the combined effects of multiple PIFs).

Treatment of model uncertainty includes identifying and evaluating the sources of uncertainty and related assumptions that are key to the specific application. When performing an HRA, analysts typically make assumptions about the constituent parts of model uncertainty. For transparency and traceability of an HRA, the assumptions should be made explicit and be clearly documented. For key uncertainties, HRA analysts should understand and document how reasonable alternatives would affect the final outcomes of the HRA and PRA and the decisions that led analysts to determine which of the outcomes of the HRA should be applied.

As indicated in the last bullet above, the analytical form of the quantification model and its related assumptions are also a source of model uncertainty. For example, three different quantification models (i.e., THERP, CBDT, and the proposed IDHEAS-G quantification model) may produce significantly different HEP estimates for the same human action performed in the context. The reason could be due to differences in how each quantification model models the relationship between PIFs and HEPs. To address this kind of uncertainty, analysts may first examine the scope and assumptions of the alternative models for their applicability to the HFEs being analyzed, and use the different applicable models to quantify a set of HFEs that have the same scenario context to determine the range of estimated HEPs and to identify whether the use of a particular model might introduce a systematic bias in the analysis results. Analysts may examine the results from available benchmark studies that have performed similar comparisons.

ExampleModel uncertainty in extreme long-lasting scenarios. Helton et al. [137] discussed some key sources of model uncertainty in extreme long-lasting scenarios of NPP severe accidents and how IDHEAS-G addresses the uncertainties. They noted the following:

Performing HRA for severe accident scenarios with current HRA methods is challenging, particularly for long-running scenarios. A typical HRA includes identifying critical [important] human actions or HFEs in the [probabilistic safety assessment (PSA)] scenarios, failure modes of the [important] human actions, L-2

and the context factors that challenge the success of the [important] human actions. For severe accident scenarios, identification of these elements introduces uncertainties that are not adequately addressed in existing HRA methods. These uncertainties combine to influence what (operationally) will be done or attempted and stems broadly from lack of standardization in requirements (e.g., qualifications, team composition, training) for operator response for this context. Some of the contributing uncertainties include:

Key [important] human actions to be included in HRAUnlike in emergency operating procedures (EOPs) where key [important] human actions are specified in procedures, key [important] human actions following core damage and in very long accident scenarios can vary with differences in scenario progression and mitigation strategies. Though severe accident management (and other) guidelines do provide structure for this response, novel solutions could introduce new [important] human actions outside PSA models. Assumptions on the ending point of PSA simulation can also affect identification of key [important] human actions.

Additionally, changes in the composition and role of the response organization and changes to the decisionmaking process that can evolve during long-duration scenarios are not typically modeled in traditional PSAs.

Success criteria for key [important] human actionsHRA is to assess the likelihood that operating personnel can successfully perform the required actions. Uncertainties in PSA simulations propagate to the success criteria of the [important] human actions. Unlike in EOPs where the success criteria for [important] human actions are more clearly understood, severe accident management strategies often cannot explicitly define the success criteria for [important] human actions. This is particularly true when the actions involve decisionmaking based on incomplete, erroneous, or unreliable information, and this uncertainty directly interacts with uncertainties discussed in the preceding and proceeding sections.

Context factors that challenge [important] human actionsContext factors in very-long accident scenarios are far beyond those typically modeled in existing HRA methods. As an example, accumulated responsibilities of operational personnel in numerous ongoing activities other than performing key actions, although not explicitly modeled, can impact human reliability of the modeled key actions. Meanwhile, various environmental factors impose uncertainties to human reliability, and staffing level and composition vary during a very-long scenario.

Assessment of such context factors may vary with different choices of simulation end-time.

Time uncertaintiesTime available for [important] human actions is the most important factor in determining the feasibility and reliability of

[important] human actions. Time availability is determined by the system time allowed for completing the key [important] human action and the time required by operating personnel to perform the action. Existing HRA methods typically treat the system time available for an [important] human action as a constant value from thermal-hydraulic simulation. While this L-3

approach may be adequate for normal and emergency operation, it is less satisfying for post-core damage and long-running scenarios. In these cases, assessment of both system time available and time required for

[important] human actions can vary more widely with different assumptions made in the PSA, and different underlying uncertainties and variations within a particular PSA accident sequence. Essentially, just the time factor alone can dominate the uncertainties in HRA results.

IDHEAS-G provides a systematic framework and guidance for defining HRA scope, developing an operational narrative of the PRA scenario progression, identifying important human actions in the scenario, identifying contexts that challenge important human actions, performing time uncertainty analysis, and quantifying HEPs for the important human actions. IDHEAS-G provides a structured process for identifying important human actions in complicated, long-lasting scenarios. It also has explicit guidance on treating time uncertainties in important human actions. The overall IDHEAS-G framework can assist HRA analysts in determining the analysis scope and assumptions.

L.2.2 Parameter Uncertainty Parameter uncertainty is the uncertainty in the values of the parameters of an HRA model given that the model has been determined to be appropriate. Parameter uncertainty relates to the uncertainty in the computation of HEPs. Probability distributions that relate to the analysts degree of belief in the values of these parameters can characterize the uncertainty.

Treatment of the uncertainty includes identifying the sources of parameter uncertainties, characterizing the uncertainty through probability distribution, evaluating the impact on the resulting human error probabilities, and propagating these uncertainties through the PRA model to characterize the uncertainty in the numerical results of the analysis. In this manner, the impact of the parameter uncertainties on the numerical results of the PRA can be assessed integrally.

A specific source of this type of uncertainty in the context of the IDHEAS methodology might arise from the analysts' assessment of PIF attributes. For example, suppose that a PIF attribute is characterized with a simple binary present vs. absent construct or a scale with three states of "low, medium, high". In a particular scenario, the analysts may be uncertain whether to assign a "medium" or "high" state to that PIF attribute. That assignment could have a significant numerical impact on the respective HEP. In this situation, the range of numerical effects from this source of parameter uncertainty can be quantified by evaluating the HEP for both PIF states. In a more comprehensive assessment, the analysts can assign probabilities that each state applies. The overall HEP, and its uncertainty distribution, can then be the probabilistically-weighted combination of the results from the two assessments. Alternatively, if the HEPs are significantly different and the significance propagates to the PRA model, then the PRA model may treat the context corresponding to the two states of the PIF attribute as two separate HFEs.

IDHEAS-G describes three constituent parts of the HEP of an important human action: is the error probability associated with time uncertainty, is the error probability of failure of the macrocognitive functions for the critical tasks of the important human action, and adjustment to the HEP according to dependencies between important human actions. IDHEAS-G requires estimation of and with probability distributions. Regarding combining multiple parameters involved in an HRA process, NUREG-1855 [136] states the following:

When the parameters are combined algebraically to evaluate the PRA numerical results or some intermediate result such as a basic event probability, these uncertainty L-4

distributions can be mathematically combined in a simple way to estimate the uncertainty of those numerical results.

L.2.3 Incompleteness Uncertainty Incompleteness uncertainty relates to risk contributors that are not accounted for in the HRA/PRA model. This type of uncertainty may further be categorized as either being known but not included in the model, or unknown. Both known and unknown types of uncertainty are important. The sources of completeness uncertainty are as follows.

The scope of the HRA/PRA does not include some classes of initiating events, hazards, modes of operation, or system failure modes. Some contributors or effects may be knowingly left out of the model for a number of reasons. For example, methods of analysis have not been developed for some issues, and these gaps must be accepted as potential limitations of the technology. Thus, the impact on actual risk from unanalyzed issues cannot be explicitly assessed. A specific example is that the decisionmakers managing a severe accident may vary greatly in their experience, competencies, and methods for making decisions.

The level of analysis may have omitted phenomena, failure mechanisms, or other factors because their relative contribution is believed to be negligible. For example, the resources to develop a complete HRA model may be limited, which could lead to a decision not to model certain contributors to risk (e.g., time of the day when a task is performed).

Some phenomena or failure mechanisms may be omitted because their potential existence has not been recognized or no agreement exists on how a PRA/HRA should address certain effects, such as the effects on risk arising from interpersonal differences or organizational cultures.

Incompleteness uncertainty expresses the limitations in the scope of the HRA model. The limitations in scope can result in uncertainty about the full spectrum of risk contributors in a PRA model or other HRA applications. The treatment of incompleteness uncertainty identifies key uncertainties and documents them. To account for unknown risk factors, PRA/HRA models consider them in the range of the key parameter probability distribution and typically establish some conservative lower limits in the important outcomes. For example, minimum HEP values are assumed for CFMs when all the PIF attributes are in a nominal state.

IDHEAS-G considers incompleteness uncertainty in both qualitative analysis and quantification.

In the guidance for scenario analysis, the HRA analysts are to work with the PRA team to determine the analysis scope, the event boundary conditions and termination criteria, and the systems that should be included in the analysis. The HRA and PRA teams explicitly or implicitly perform some kind of screening analysis to demonstrate that a particular item (e.g., a hazard group, an initiating event, a component failure mode) can be eliminated from further consideration in a PRA being used to support a risk-informed application. This screening can be accomplished by showing that either the item has no bearing on the application (qualitative screening) or that the contribution of the item to the change in risk associated with the application is negligible (quantitative screening). Furthermore, the process of identifying important human actions is a qualitative screening process because it identifies only those important human actions that have impacts on safety.

L-5

L.3 Summary Uncertainty analysis is not an add-on step to the HRA process defined in IDHEAS-G. Instead, it is a part of all HRA stages. Without properly addressing uncertainties, the resulting outcomes of HRA may be misinterpreted or inappropriately used in their associated applications. In addition, systematically identifying and documenting sources of uncertainties can help in reconciliation of analyst-to-analyst variability in the resulting HRA outcomes.

L-6

APPENDIX M DEMONSTRATION OF THE IDHEAS-G HUMAN RELIABILITY ANALYSIS PROCESS This appendix uses two examples to demonstrate the implementation of the IDHEAS-G process for performing a Human Reliability Analysis (HRA). One example analyzes an actual event and the other analyzes a hypothetical event. The examples demonstrate how the information collected in one stage of the process is used in other stages. The demonstration does not intend to perform a comprehensive HRA.

As a reminder, the IDHEAS-G HRA process is structured as follows:

Stage 1Scenario analysis o Operational narrative Event overview (for hypothetical event) or review for (actual event)

Initiating event Boundary conditions Scenario timeline Baseline scenario Deviation scenarios Operational experience review o Context analysis Environment and situation context System context Personnel context Task context o Identification and definition of important human actions (IHAs)

Stage 2Modeling of IHAs o Task analysis Task diagrams Identification of critical tasks Characterization of critical tasks o Applicable cognitive failure modes (CFMs) of every critical task o Performance-influencing factors (PIFs) relevant to every critical task o Time uncertainty analysis of the IHA, if it is time critical

Stage 3Human error probability (HEP) estimation of every IHA o Calculation of ; the HEP attributed to time uncertainties, if the IHA is time critical o Calculation of ; the HEP attributed to the applicable CFMs of all the critical tasks

Stage 4Integrative analysis o Dependency analysis of IHAs (or human failure events (HFEs)) in a probabilistic risk assessment (PRA) cutset o Analysis and documentation of uncertainties in the HRA process Note that the two examples below do not include Stage 4 because it involves the integration of an HRA to the PRA while the scope of the examples is to demonstrate the IDHEAS-G process.

M-1

M.1 Analysis of an Actual Event This example uses the fire event that occurred at the H. B. Robinson Steam Electric Plant on March 28, 2010 [104] to demonstrate how to apply IDHEAS-G for an HRA. The analysis is based on readily available information, such as the U.S. Nuclear Regulatory Commission (NRC) augmented inspection team report [104], the NRC Accident Sequence Precursor (ASP) program analysis report [138], and other related documents (including the H.B. Robinsons corrective action report and an earlier site visit that included interviewing the operators and instructors).

The following acronyms are used in this section:

AO: auxiliary operator AOP: abnormal operating procedure ARP: alarm response procedure ASME/ANS: American Society of Mechanical Engineers/American Nuclear Society BOP: balance of plant CCW: component cooling water CFM: cognitive failure mechanism CVCS: chemical and volume control system EAL: emergency activation level EDG: emergency diesel generator EPP: end path procedure gpm: gallons per minute HEP: human error probability IHA: important human action LCV: level control valve MCC: motor control center MCR: main control room MOV: motor operated valve MSIV: main steam isolation valve MSR: moisture separator reheater PIF: performance-influencing factor RCP: reactor coolant pump RCS: reactor coolant system RHR: residual heat removal RNO: response not obtained M-2

RO: reactor operator RWST: refueling water storage tank SI: safety injection SM: shift manager SS: shift supervisor STA: shift technical advisor SUT: start-up transformer Tavg: average temperature of hot leg and cold leg ( = ( + )2)

UAT: unit auxiliary transformer VCT: volume control tank M.1.1 Scenario Analysis The scenario analysis includes three parts: operational narrative, event context, and human performance model of the event.

M.1.1.1 Operational Narrative Event review Event title: H.B. Robinson Steam Electric Plant electric fault with a near miss of reactor coolant pump (RCP) seal damage The event title provides a concise highlight of the analysis:

Where: H.B. Robinson plant

What happened: an electric fault

What was the consequence or risk: a near miss event of RCP seal damage The real event had two fire events. The second fire was triggered after the first fire was extinguished and the reactor was stabilized. The following discussion only covers the events related to the first fire.

On March 28, 2010, at 18:52 local time, the H.B. Robinson Steam Electric Plant Unit No. 2, a Westinghouse three-loop reactor, was operating in Mode 1 at 99.3-percent power with a power reduction in progress to a scheduled refueling outage. A fault occurred in a 4,160 volt (V)

(referred to as 4 kilovolt (kV) for the remainder of this chapter) feeder cable from Bus 4 to Bus 5 due to a cable insulation failure. When the fault occurred, Breaker 24 (see Figure M-1) did not clear the fault as expected and remained closed throughout the event. The breaker failure to open led to internal damage to the Unit Auxiliary Transformer (UAT) and a lockout of the UAT on fault pressure that caused the Main Generator lockout relay to operate.

As a result, the fault persisted on 4 kV Buses 4 and 5 while the time over-current protection for 4 kV Bus 4 Feeder Breaker 20 began to time. The voltage for 4 kV Buses 4 and 5 became significantly depressed due to the fault, and the B Reactor Coolant Pump (RCP) motor slowed, which actuated the low Reactor Coolant System (RCS) flow reactor protection logic for the B RCS loop resulting in a reactor trip. The fault current was initially fed from the UAT. After three to four seconds, the internal failure of the UAT tripped the fault pressure protection, which M-3

locked out the UAT and the Main Generator. The fast transfer from the Main Generator lockout signal opened Breaker 20 and closed Breaker 19, which transferred the fault from the UAT to the Start-Up Transformer (SUT). Following the transfer of the fault to the SUT, voltage for 4 kV Bus 3 became significantly depressed resulting in actuation of the loss-of-voltage relays for the 480 V E-2 safety bus. The 480 V E-2 safety bus then separated from 4 kV Bus 3, the B Emergency Diesel Generator (EDG) started automatically and connected to the 480V E-2 safety bus, and the load sequencer operated as designed. After several seconds, the time over-current relays for Breaker 19 actuated and tripped the breaker, which cleared the fault and ended the first electrical fault event. The entire sequence of automatic protective actions occurred in 20 seconds (18:52:22 to 18:52:42). After this point, the plant electrical configuration was as follows (see Figure M-1):

4 kV Buses 1, 2, and 3 were powered from the SUT

480 V E-2 safety bus was powered by the B EDG

4 kV Buses 4 and 5 were de-energized

480 V Buses 3, 4, and 5 were de-energized

MCC 4, 8, 11, 12, 13, 14, 15, 17, and 21 were de-energized M-4

2 Main To 220KV Switchyar Generator To 115KV SPAN BUS Main SUT UAT 12 20 17 7

BUS 2 BUS 1 BUS 3 BUS 4 BUS 5 4kV 4kV 10 19 24 BUS 1 BUS 2A BUS 2B BUS 3 BUS 4 BUS 5 MCC MCC MCC MCC MCC MCC 1, 19, 22 2, 7 3, 20 4, 8 11, 12, 13, 14, 21 15, 17 E-1 E-2 DS BUS 480V A B DSDG EDG EDG Instrument Buses 1 2 3 4 6 7 8 9 MSIVs FCV-626 Figure M-1 A Simplified Electric Diagram After the disruption to the electric plant and reactor trip, a series of equipment problems and operator performance issues increased the overall significance of the event. The following equipment conditions existed after the reactor trip:

When the 480 V E-2 safety bus momentarily lost power, flow control valve (FCV)-626, the component cooling water (CCW) return valve from the RCP thermal barrier heat exchanger closed and isolated CCW flow to all the RCP thermal barriers. This condition went undetected by the operating crew for a period of 39 minutes (18:52 to 19:31).

When Motor Control Center (MCC) 4 deenergized, all Moisture Separator Reheater (MSR) Drain Tank Alternate Drain valves and MSR Timer valves failed open, providing a M-5

flow path for the main steam to reach the main condenser via the MSR Shutoff valves and MSR tubes. This flow of main steam resulted in a cooldown of the RCS. The RCS cool down led to an automatic Safety Injection (SI) due to low pressurizer pressure (19:00). The SI injected to the RCS for approximately 12 minutes with a maximum flow of approximately 260 gallons per minute (gpm). Additionally, power was unavailable to the MSR Shutoff valves, preventing the valves from being remotely closed from the main control room (MCR). A loss of Instrument Bus 3 occurred during the restoration of the B battery charger to the B battery. The loss of Instrument Bus 3 caused channel failures, which satisfied the high steam flow coincident with low RCS average temperature (Tavg) logic resulting in the automatic closure of the main steam isolation valves (MSIVs), which stopped the RCS cool down at 19:25. The Tavg decreased to 442 degrees Fahrenheit (°F). This represents an average cool down rate of 105 °F per hour.

Note: The above paragraphs provide a detailed account of the initial condition, initiating event, and boundary conditions.

During the event, the expected automatic actions on low volume control tank (VCT) level to swap charging pump suction to the refueling water storage tank (RWST) did not occur due to the control module was not properly configured (a latent failure). This condition went undetected by the operating crew for a period of 49 minutes (19:00 to 19:49). Review of plant indications revealed that the remaining charging pump was no longer delivering flow to the RCS or RCP seals after 37 minutes (19:37). RCP seal cooling was maintained through manual action to re-open FCV-626 at 19:31.

At the time of reactor trip, the MCR was manned with a shift supervisor (SS), reactor operator (RO), and balance of plant (BOP) operator. The shift manager (SM) and shift technical advisor (STA) were at a pre-shift turnover meeting with the previous crew at another building a few minutes walking distance from the MCR and with the telephone line connected to the MCR.

After the reactor trip, the SS entered the emergency operating procedure (EOP) PATH-1, then transferred to end path procedure (EPP)-4, Reactor Trip Response from PATH-1, due to no SI signal. Operators started charging pumps B and C at 18:53, as directed by EPP-4. The SM and STA rushed back to the MCR by hearing the MCR crew announcing the reactor trip through the telephone and a loud steam relief noise. On their way back to MCR, they saw that the 4KV Bus 5 was on fire. They entered the MCR at about 18:56 and announced that the 4KV Bus 5 was on fire. Before this point, the MCR crew (i.e., SS, RO, and BOP operator) did not know there was a fire. After this point, the BOP operator was in abnormal operating procedure (AOP)-41 (i.e., response to fire event) working with the onsite fire brigade to extinguish the fire.

The SS and RO continued the EPP-4. At 19:00, an automatic SI occurred due to low RCS pressure caused by the cooldown. Following the SI, the operating crew entered the PATH-1 procedure from EPP-4. The SM was assessing the emergency activation level (EAL) with the STAs assistance.

Note: The above description illustrates the scenario in which crew responded to the situation.

The RCP seals integrity was starting to be challenged. The RCP seals were cooled by RCP seal injection (by charging flow) and RCP seal cooling (by CCW). Immediately tripping the RCPs with combination of in time restoration of either the seal injection or seal cooling would protect the RCP seals from damage.

RCP Seal Injection - Charging Flow The charging flow degraded in this event. The problems included reduced charging flow and limited suction source. At 19:18, valve 310A in the chemical and volume control system (CVC-M-6

310A), which provides charging flow to Loop 1, failed opened. This caused an increased flow to the RCS; thus, reducing the charging system back pressure and reducing injection flow to the RCP seals. The CVC-310A failure was due to several leaks in the instrumentation air. The CVC-310A could not be closed manually in this event.

The charging flow withdrew water from the Chemical and Volume Control System (CVCS) to inject into the RCS. In normal situations, RCS water would return to the CVCS through the letdown. In this event, the steam demand (from the MSR valves failing open) caused a rapid decrease in pressurizer water level which triggered a letdown isolation. This caused RCS water not to return to the CVCS. In addition, when the B and C charging pumps were manually started, the B was in manual at the minimum speed and C was in automatic. With the C charging pump in automatic, the lower pressurizer level resulted in an automatic increase in charging flow. The letdown isolation and increase in charging flow reduced the level of the VCT in the CVCS.

When the VCT level decreased below 24.4 inches (62 centimeters (cm)) at 18:57, the makeup control system should have automatically recovered the water level. This did not happen due to the earlier electric fault that disarmed the automatic VCT makeup. When the VCT level reached 12.4 inches (31.5 cm) at 19:00, the charging suction should, but did not, automatically transfer to the RWST due to a latent failure in incorrect configuration of a control module related to the water makeup. The VCT level was indicated at 0 at 19:12. There was still net positive suction head available to the charging pump below the 0 indication. It was estimated the remaining water could support the B charging pump operation until 19:37. Operators did not identify the issue until 19:46.

Reactor Coolant Pump Seal CoolingComponent Cooling Water The FCV-626 was located in the combined CCW return from the three RCP thermal barrier heat exchangers. In its normal open position, it allowed CCW flow to pass through the thermal barrier heat exchangers, providing backup cooling to the RCP seals in the event of a loss of the primary cooling flow (seal injection) from the charging pumps. The FCV-626 closed when power to the 480 V E-2 safety bus was transferred to the EDG. The valve remained closed for approximately 39 minutes before the operators recognized the condition, reopened FCV-626 at 19:31, and restored CCW cooling to the RCP thermal barrier heat exchangers.

In this event, the RCP seal cooling water was restored about six minutes before the complete loss of the degraded seal injection. The RCP seals remained intact in this event.

Note: The above paragraphs describe the main scene of the integrity of RCP seals. This includes two sub-scenes, one about the RCP seal injection and another about RCS seal cooling.

At 19:31, operators reached a step in PATH-1 (see the red diamond in Figure M-2) to check if the RCP Thermal Barrier Cooling Water Hi/Lo Flow Alarm Illuminated. The RO noticed the increasing RCP temperature alarms and yellow annunciators for RCP parameters that triggered a knowledge-driven diagnosis about RCP abnormality. This led to knowing of the FCV-626 closure. The RO opened FCV-626. This provided RCP seal cooling to the RCP seals.

M-7

Figure M-2 PATH-1 Procedure on the Step of Checking RCP Thermal Barrier Cooling Water Hi/Lo Flow Alarm Illuminated.

At 19:49, the SS and RO reached a step in PATH-1 to verify at least one charging pump running and then to establish charging flow as necessary. They recognized the problems of the VCT level and charging flow. The RO identified the VCT level low and that the RWST swap should have occurred. The RO believed that the swap did occur and turned the switches to level control valve (LCV)-115B CLOSED and LCV-115C OPEN, which was an incorrect action. The RO left the suction of the charging pumps aligned to the VCT. An Auxiliary Operator (AO) was then dispatched to perform the pre-start checks on the C Charging Pump. The AO communicated with the MCR about the abnormality of the charging pump. The STA heard the crew conversation about the charging flow, so he pulled out plots in a monitor display, which showed no charging flow. The STA then went to the control board and identified that the charging pump suction was not properly aligned to the RWST and announced this to the entire operating crew. The RO repositioned the switches from LCV-115B CLOSED to AUTO and LCV-115C OPEN to AUTO but obtained no response from the valves. The RO then realigned the charging pump suction to the RWST by placing the LCV-115B to OPEN and the LCV-115C to CLOSE. The C charging pump was then started by the RO to provide RCP seal injection.

The restoration of the RCP seal injection did not affect the RCP seal integrity because the RCP seal cooling was restored 22 minutes before the restoration of the charging flow.

Note: The above paragraphs describe the restoration of the RCP seal cooling or seal injection to prevent RCP seal damage which was a key driver of risk in the event. The end consequence was an RCP seal damage near miss because the CCW was restored to the seal cooling before a complete loss of the seal injection.

M-8

Initial Condition Unit 2 was operating in Mode 1 at 99.3-percent power and reducing power toward a scheduled refueling outage. No significant equipment was out of service. The C CCW pump and A and C charging pumps were running. An SS, RO, and BOP operator were present in the MCR.

The SM and STA were in a building near the MCR for a shift turnover. A telephone line was connected between the MCR and the shift turnover room. The SS was a staff crew member who does not perform stand watch on a regular basis, but performs stand watch only to satisfy the minimum licensing requirement to maintain the SSs senior reactor operator license.

Initiating Event An electrical fault occurred on a 4 kV feeder cable and caused a fire that resulted in reactor trip.

Boundary Conditions

1. When MCC 4 was de-energized, all MSR Drain Tank Alternate Drain valves and MSR Timer valves failed open, providing a flow path for the main steam to the main condenser via the MSR Shutoff valves and MSR reheater tubes. This flow of main steam caused an RCS cooldown. Additionally, power was unavailable to the MSR Shutoff valves, preventing the valves from being remotely closed from the MCR.

2. The FCV-626 closed during the electric power transition. The closure of FCV-626 isolated the RCP seal cooling from the CCW.

3. An instrumentation air leak caused CVC-310A to fail open, thereby reducing the charging flow rate to the RCP seal injection.

4. The VCT automatic makeup at 24.4 inches (62 cm) was not functional because of the electric fault.

5. A latent failure caused the charging suction swapping from the CVCS to the RWST at 12.4 inches (31.5 cm) VCT level to not be functional.

6. The loss of the 4kV Buses 4 and 5 and the subsequent de-energization and re-energization of the Bus 3 in a short time interval affected the MCR indications including losing some indications through the entire event (e.g., the reactor rod bottom lights) and temporarily losing some indications that were restored automatically after Bus 3 was re-energized. It was estimated that about a half of the control panel indications were lost when Buses 3, 4, and 5 were de-energized. Most of the indications returned to operation after Bus 3 was re-energized.

7. Following the first electric fault, the configuration of the electric plant was as follows:

UAT inoperable

4 kV Buses 1, 2, and 3 were powered from the SUT

480 V E-2 safety bus was powered by the B EDG

4 kV Buses 4 and 5 were de-energized

480 V Buses 3, 4, and 5 were de-energized

MCC 4, 8, 11, 12, 13, 14, 15, 17, and 21 were de-energized

8. The event occurred on a Sunday night (March 28, 2010). This site was in an off-shift staffing level.

M-9

M.1.1.2 Scenario Timeline Table M-1 shows the scenario timeline of the H.B. Robinson fire event up to the termination of safety injection using the two-column format and symbols in the second column that are described in APPENDIX E.

Table M-1 Scenario Timeline of the Fire Event at the H. B. Robinson Steam Electric Plant Scenario overview: On March 28, 2010, at 18:52 local time, the H.B. Robinson Unit 2, a Westinghouse three-loop reactor, experienced a fault in a 4 kV cable, which induced a fire event that caused a reactor trip, subsequent SI actuation, and an Alert emergency declaration. During this event, two separate fires occurred approximately four hours apart.

Latent failures, equipment malfunction, unexpected equipment responses, and human performance issues caused a reduction in RCP seal injection capabilities and a loss of RCP seal cooling. The event did not cause damage to the RCP seals because the RCP seal cooling was restored in time.

Time S: System Responses (hh:mm) H(abc): Human Responses; abc: the individuals position.

N: Notes I: System generated information 3/28/2010 The initiating event, an electric fire, occurred.

18:52 M-10

Table M-1 Scenario Timeline of the Fire Event at the H. B. Robinson Steam Electric Plant (continued) 18:52+ (S) Reactor, turbine, and main generator successfully tripped automatically.

(S) Charging pump A was de-energized when its electric bus was de-energized.

(S) Charging pump C was de-energized and FCV-626, which provides flow from the thermal barrier heat exchangers, closed when power to the E-2 safety bus was transferred to the B EDG.

(S) CCW pump C was de-energized and subsequently started on the sequencer.

(S) CCW pump B started when Instrument Bus 4 was de-energized (S) MSR Drain Tank Alternate Drain valves and MSR Timer valves failed open, providing a flow path for the main steam to the main condenser causing an RCS cooldown.

(S) B RCP tripped (S) FCV-626 closed (S) RCS letdown closed H(SS) Entered the PATH-1 EOP and determined that an SI actuation was not required (I) Annunciator RCP Thermal Barrier Cooling Water Hi or Low Flow Alarms triggered. (N) This annunciator was a key indication in the PATH-1 procedure to cue the loss of RCP seal cooling.

(I) Within 20 seconds after the reactor trip (18:52:22 to 18:52:42), the loss of the 4 kV Buses 4 and 5 and the subsequent de-energization and re-energization of the Bus 3 in a short time interval affected the MCR indications including the loss of some indications throughout the entire event (e.g., the reactor rod bottom lights) and the temporary loss of some indications that were restored automatically after Bus 3 was re-energized. It was estimated that about a half of the control panel indications were lost when Buses 3, 4, and 5 were de-energized. Most of the indications returned to operation after Bus 3 was re-energized. The indicator states confused the operators and slowed down the procedure progress. For example, the rod bottom lights were off.

The operators were not able to identify if all the rods were inserted. The startup rate indication was reading zero instead of negative (i.e., indicating the reactor power was trending down) and the digital RCS temperature recorders did not have readings. The RO had to use the less familiar indications (e.g.,

temperature reading on another control panel) to proceed through the procedure.

H(SM and STA) Rushing back to the MCR from the shift turnover room 18:53 H(RO) Started the charging pumps B (in manual minimum flow) and C (in auto) per PATH-1.

H(SS and RO) Monitored cool down due to the MSR shut-off valves failing open, expecting an SI signal.

M-11

Table M-1 Scenario Timeline of the Fire Event at the H. B. Robinson Steam Electric Plant (continued) 18:56 H(SM and STA) The SM and STA entered the MCR and announced a fire on Bus-5.

H(SS) Entered EPP-4, Reactor trip response.

H(BOP) Entered AOP-041, Response to fire event, to coordinate firefighting with the fire brigade.

(N) After this point, the SS and RO run PATH-1. The BOP runs AOP-041. The SM assessed the emergency activation level. The STA assisted the SM and performed an independent check of the plant status.

18:57 (S) VCT level reached 24.4 inches (62 cm). The automatic VCT makeup failed to operate.

18:58 (S) The pressurizer level was off indicating scale low 19:00 (S) SI actuated due to low pressure (S) The VCT level reaches 12.4 inch (31.5 cm). The charging suction automatic swap to the RWST failed.

H(SS&RO) Transited back to PATH-1 H(Fire brigade) Used chemical fire extinguisher to extinguish Bus 5 fire 19:19 (S) CVC-310A failed open causing an increase in charging flow but reduction in the flow for RCP seal injection 19:24 (I) B RCP high bearing temperature alarm 19:26 (S) The MSIVs closed on loss of Instrument Bus 3 which was lost during the restoration of the B Battery Charger. This stopped the RCS cool down, and the RCS temperature and pressurizer pressure immediately begin to recover.

(N) The MSIV closure had a positive impact on the event but was not a purposeful action by the MCR crew.

19:30 (I) A RCP Bearing High Temperature Alarm H(Fire Brigade) An offsite fire department with a fire engine arrived at the site.

19:31 H(SS and RO) PATH-1 directed the check if the RCP Thermal Barrier Cooling Water Hi/Lo Flow Alarm Illuminated. The operator identified that the RCPs were in trouble.

19:32 H(RO) Opened FCV-626. This restored RCP seal cooling.

(N) The actions to open the FCV-626 was knowledge driven, not guided by procedure.

19:33 (I) B RCP #1 Seal Leak-off High Temperature alarm (I) C RCP Bearing High Temperature Alarm 19:34 H(SS and RO) Entered AOP-018 RCP Abnormal Conditions due to RCP high temperature trends and alarms.

H(BOP) Reintegrated with the SS and RO after completing the coordination with the fire brigade to extinguish the fire.

19:37 (S) Loss of all charging flow.

19:46 - H(RO) Stopped B Charging Pump based on recognition that the VCT is empty 19:49 and swap over to the RWST had failed to occur.

H(RO) Aligned charging suction to the RWST but performed incorrectly.

H(STA) Identified the charging suction was not aligned to the RWST. Made it known to the crew.

19:50 H(RO) Realigned Charging Pump Suction to RWST 19:53 H(RO) C Charging Pump Started to restore RCP seal injection.

20:26 H(SS) Entered EPP-7 SI Termination 20:44 H(RO) Terminated SI.

M-12

M.1.1.3 Relevant Operating Experience This section discusses an actual event with a similar context as the event being analyzed. On March 12, 1968, San Onofre Nuclear Generating Station, Unit 1, experienced a fire in a cable tray in the No. 2 480 V switchgear room [139]. At the time of the fire incident, the unit was operating at 380 megawatt-electric when, at 12:21 a.m., several alarms were received in the control room including:

Intake Structure Hi Level

480 V System Ground

Station direct current (DC) Bus Ground or Low Voltage

Hydraulic Stop Gate Trouble

"Sphere Heating and Ventilating System Trouble.

At 12:25 a.m., the annunciator panels for the "turbine-generator first out, auxiliary, and electrical boards" were lost. An auxiliary operator reported smoke in the No. 2 480-V switchgear room.

At 12:27 a.m., operators observed blue arcing above the east door window of the No. 2 480-V switchgear room.

At 12:32 a.m., fire was observed in three cable trays above the east door.

The reactor was tripped at 12:34 a.m., and began unit shutdown actions at 12:37 a.m. The No. 2 480-V bus was cleared by over-current relay operation.

At 12:35 a.m., assistance was requested from the closest external fire department, which happened to be a Marine Corps Fire Department.

At 12:45 a.m., 24 minutes after the first control room alarms were received, the Fire Department arrived on the scene. The electric motor driven fire pumps would not start. Therefore, the gasoline engine driven backup emergency fire pump was started (12:56 a.m.).

The fire was declared extinguished at 1:00 a.m., 39 minutes after the initial control room alarms.

During cooldown efforts following the fire, it was determined that the coolant boron concentration was decreasing instead of increasing as expected, so the cooldown was suspended for 3 hours3.472222e-5 days 8.333333e-4 hours 4.960317e-6 weeks 1.1415e-6 months and 40 minutes until the problem was diagnosed and fixed.

Post-fire investigation revealed that power and/or control circuits were affected for residual heat removal (RHR) suction and discharge valves, the CCW heat exchanger outlet valve, the South primary plant makeup water pump, and three annunciator panels. Damaged cables rendered the following equipment electrically inoperable:

Safety injection recirculation valves

West recirculation pump and discharge valve

Electric auxiliary feedwater pump

Safety injection train valves (West train motor operated valves (MOVs))

Refueling water pump discharge valve to recirculation system The following equipment was lost due to the relay cutout of the No. 2 480 V bus:

West RHR pump

South transfer pump

Boric acid injection pump

Boric acid storage tank heaters & boric acid system heat tracing M-13

South primary plant makeup pump

Flash tank bypass valve

East and West flash tank discharge pumps

Center component cooling water pump

Several other MOVs M.1.1.4 Identify Additional Scenarios Because the point of the H.B. Robinson event analysis is to calculate the conditional probability of RCP seal failure based on the actual event conditions, the what-if questions are focused on the structures, systems, and components (SSCs) and human failures that would fail the RCP seals. In this event, the RCP seal failure probability is dominated by the operating crew performance. Therefore, the what-if question is what-if the operators failed to open FCV-626 in time (to restore CCW flow to cool RCP seals).

M.1.2 Context Analysis The event context includes task context, environment and situation context, system context, and personnel context. The task context discusses how the personnels tasks are performed at a high-level. The environment and situation context describes the conditions that facilitate or challenge personnel performing the tasks. The system context discusses the operability constraints of SSCs important to performing the tasks identified in the task context. The personnel context discusses personnel and organizational constraints on performing the tasks.

M.1.2.1 Task Context Before the event, the plant was at 99.3-percent power and reducing power to prepare for a scheduled refueling outage. After the initiating event, the operators focus was to bring the plant to a hot-shutdown state. The event required the MCR crew to implement the abnormal and emergency operating procedures to protect reactor safety and coordinate the MCR crew, onsite fire brigade, and offsite fire brigade to extinguish the fire. The objective for protecting reactor safety was responding to a reactor trip with an uncontrolled cooldown due to steam leakage by implementing the PATH-1 procedure to bring the reactor to a safe and stable state (hot shutdown). The critical tasks were to restore either the RCS seal cooling or seal injection in time to prevent a RCP seal damage and to terminate SI to stabilize RCS. The tasks were performed by the SS and RO. The BOP operators main responsibility was to coordinate with the fire brigade for firefighting and join the SS and RO to implement procedures after extinguishing the fire. The SMs main tasks were to monitor the event recovery activities and assess the EAL. In this event, the first fire did not meet the criterion for an EAL declaration.

Only the second fire, which was not within the scope of this demonstration, required an Alert EAL declaration. The fire in the 4-kV room was extinguished by 19:04. There was a report of smoke at the intake area that turned out to be a false alarm. The less important MCR activities included paging the offsite emergency response staff to respond to the event and requesting local fire brigade assistance to fight the fire. The timeline of these tasks is shown in Figure M-3.

M-14

19:00 20:00 18:52 20:50 19:31 Restore RCP seal cooling 19:04 Extinguish the first fire 19:19 19:53 Restore RCP seal injection Degraded 1 20:44 Terminate SI

1. A complete loss of RCP seal injection Note: The beginning of each task, if any, is the time that the problem occurs.

Figure M-3 The Critical Tasks and Their Completion Time in the H.B. Robinson Event M.1.2.2 Environment and Situation Context The event being analyzed occurred in the MCR; thus, the working environment had no special challenges (e.g., heat or coldness, visibility, high noise) to human performance. MCR operators were in a challenging situation at the time of the event because there were two fires. Both fires were extinguished by fire brigade coordinating with the MCR crew. In the first half hour of the event, the BOP operator was implementing the fire procedure to extinguish the fire in coordination with the fire brigade and was not involved in implementing the emergency procedure (PATH-1). The SS and RO implemented the emergency procedures. This could disrupt and distract the crew.

M.1.2.3 System Context The main system unavailability affecting the event included the following:

At the beginning of the event (19:52), a non-isolable pathway was formed from the main steam line to the main condenser causing the RCS cooldown

CVC-310A failed open causing an increased charging flow rate and a reduced flow rate for RCP seal injection

The VCT automatic makeup was not available

The automatic swap of charging suction from the VCT of the CVCS to the RWST was not available

The FCV-626 closure due to the electric power transition in the beginning of the event was not expected by the operating crew The other key system context affecting human performance was the electric power transition at the beginning of the event affecting the MCR human-system interface. Within 20 seconds after the reactor trip (18:52:22 to 18:52:42), the loss of the 4 kV Buses 4 and 5 and the subsequent de-energization and re-energization of the Bus 3 in a short time interval affected the MCR indications, including the loss of some important indications throughout the event (e.g., the reactor rod bottom lights) and temporarily the loss of some indications that were restored automatically after Bus 3 re-energized. It was estimated that about a half of the control panel M-15

indications were lost when Buses 3, 4, and 5 were de-energized. Most of them returned to operation after Bus 3 was re-energized. The loss of indicators confused the operators and slowed down the procedure implementation. For example, the rod bottom lights were off. The operators were not able to identify if all rod were inserted. The startup rate indication was reading zero instead of negative (i.e., indicating the reactor power was trending down) and the digital RCS temperature recorders did not have readings. The RO had to use the less familiar indications (e.g., temperature reading on another control panel) to obtain the information.

The MCR computer showed that a good portion of the parameters were not reliable (by showing text in different colors and blinking). The STA felt that the inputs to the safety function tree shown in the computer were not reliable. The STA used the hardcopy of the safety function tree and walked to the control panel to obtain plant parameters to determine the tree status. This slowed down the STA in performing his responsibilities.

The control panel indicators were relatively small. The SS, from his standing position, was not able to see clearly the needle indications. The RO had to stand close to the indicators to read the indications. When all control panel parameters were available, the operators normally do not need to rely on these indicators for information. In this event, because some primary indications were not available, the SS and RO had to rely on the indicators. This and the combination of the unavailability of some important indications slowed down the pace of the SS and RO in implementing the PATH-1 procedure.

M.1.2.4 Personnel Context The event occurred on Sunday night. The staffing was at an off-schedule level. Immediately before the event, a SS, RO, and BOP operator were inside the MCR. The SM and STA were in a building near the MCR for a pre-shift turnover. A telephone was connected between the MCR and the shift turnover room. There was a senior reactor operator (SRO) on the site serving as the fire brigade incident commander and four fire brigade members. The offsite fire department with a fire engine arrived on site at 19:30 to assist the firefighting.

The SS was a staff crew member who did not perform stand watch on a regular basis. The RO, BOP operator, and SM were experienced operators. The BOP operator was an SRO temporarily performing the BOP responsibility on the shift. The STA had been an STA for a few months.

M.1.3 Identification of Important Human Actions The key risk consideration of this event was that the RCP seals were close to failure. RCP seal failure would result in a loss-of-coolant accident (LOCA). There was a relevant PRA model available. The PRA information could be used to identify the important human actions (IHAs) required in an RCP seal failure event. Figure M-4 shows a PRA fault tree on RCP seal failure.

The top event of the fault tree is loss of seal cooling, a main mechanism that fails the seals.

The seal cooling can be provided by CCW (seal cooling) or charging pump (seal injection, shown as FAILURE OF CVC COOLING TO RCP SEALS in Figure M-4). In this event, the seal cooling was initially lost and the seal injection was degraded then completely lost. The seal cooling was restored when the seal injection was in a degraded state. Restoring seal cooling in this event requires re-opening FCV-626 (circled in Figure M-4). Regarding the seal injection, because CVC-310A failed open because leakage of instrumentation air, within the interested time window of this event analysis, the seal injection is determined cannot be restored. From an operational perspective (based on procedure), the operator was required to trip RCPs to protect RCP seals. Based on the analysis above, the IHA in this event is to re-opening FCV-626 to restore seal cooling in time to prevent RCP seal failure.

M-16

Figure M-4 A PRA Fault Tree on RCP Seal Failure Definition of the Important Human Actions The system responses relating to prevent a RCP seal failure were the following:

The FCV-626 closure occurred at the beginning of the event (18:52). This stopped the RCP seal cooling.

Approximately 27 minutes into the event (19:19), CVC-310A failed open. CVC-310A failing open diverted the charging flow from the RCP seals to the RCS. RCP seal injection became inadequate (there was some injection flow, but it was inadequate to fulfill its safety function). As a result, the RCP seals began to heat up (27 minutes after FCV-626 closure or 19:19). The RCP seal injection failure was not recoverable in this event.

For a successful recovery, operators needed to re-open FCV-626 to restore the RCP seal cooling. Tripp RCPs early would extend the time available for the operator to restore the RCP cooling before a seal failure. Open FCV-626 and trip RCPs were simple actions performed from the MCR. A 13 minutes time window was used as the time available for the operator to restore the RCP cooling from when all RCP seal cooling and injection were lost. The 13 minutes time available was determined based on studies performed by Westinghouse [106], [140], [141]. Therefore, FCV-626 needed to be re-opened by 19:32 (= 19:19 + 13 minutes) to prevent RCP seal failure. Note that some people have argued the 13 minutes is a conservative estimate. Because the conservatism is not an emphasis of this demonstration, this example uses 13 minutes as the time available for operator to re-open FCV-626.

The IHA Re-opening FCV-626 is defined as follows:

Success criteria for the IHA - reopen FCV-626 in less than 13 minutes after CVC-310A failed open.

M-17

Beginning and ending points of the IHAThe IHA started when FCV-626 failed open in the beginning of the event and ended when FCV-626 was re-opened.

Cue for initiating the IHAFCV-626 status indication and the alarm RCP thermal barrier Hi or Lo flow alarms illuminated were available at the same time of FCV-626 failing closed.

M.1.4 Modeling Important Human ActionsTask Analysis In analyzing the risk of the event posed to a typical operating crew, one applies the same initial condition, initiating event, and boundary conditions of the actual event. Even though the actual event (e.g., the H.B. Robinson fire event) did not progress to an undesired consequence (i.e.,

core damage), a different crew in the same event may have resulted in an undesired consequence. Therefore, the analysis of the event here represents a typical crews responses to the given initial condition, initiating event, and boundary condition instead of the crew responses in the actual event.

M.1.4.1 Identification of Critical Tasks Task analysis should identify critical tasks and the macrocognitive functions (i.e., detection, understanding, decisionmaking, action execution, and interteam coordination) involved. In this event, the cues included the following:

The valve FCV-626 position indication (18:52; 0 minute after 1) 5F

The annunciator RCP thermal barrier cooling water hi or low (18:52; 0 minute)

The annunciator RCP B high bearing temperature (19:24; 32 minutes)

The annunciator RCP A high bearing temperature (19:30; 38 minutes)

The annunciator RCP C high bearing temperature (19:33; 41 minutes)

The annunciator RCP B seal leakoff high temperature (19:33; 41 minutes)

The annunciators RCP A and C seal leakoff high temperature (not mentioned in the event)

These cues could be detected by the operators situation awareness or directed by the PATH-1 procedure step RCP Thermal Barrier Cooling Water Hi or Lo Flow Alarms Illuminated. The cues were direct indications of RCP abnormality that should lead the operators to enter AOP-018, RCP Abnormal Conditions. AOP-018 would lead operators to identify the FCV-626 closure. Otherwise, the alarm response procedures (ARPs) of any of the above alarms requires the operators to enter AOP-018.

Once entering AOP-018, the expected procedure path included the following steps: 1 to 6 and 25 to 34. Steps 2, 3, 4, and 34 were opportunities to avoid a seal failure. These steps are discussed below:

Step 2, response not obtained (RNO): to perform a cross check of all RCP parameters to determine the cause of the indicated high leakoff flow. This required the operators to perform knowledge-driven diagnoses to identify the causes.

Steps 3 and 4: to trip all RCPs. This action would delay RCP seal failure.

Step 34: to open FCV-626.

1 The time lapse from the point of FCV-626 closure.

M-18

Therefore, after entering AOP-018, the actions to prevent a seal failure would most likely be performed from steps 2 to 4. In the actual event, after detecting the alarms, without entering the AOP-018, the operator opened the FCV-626. It was up to the analysts to decide whether the knowledge-driven behavior could be credited. This demonstration considers only the operators entering AOP-018 to solve the problem. In summary, the critical tasks included the following:

Critical task 1detecting the alarms pointing to RCPs being abnormal

Critical task 2entering AOP-018, and

Critical task 3reopening FCV-626 in time.

The above critical tasks are performed in a sequence. Failure of any of them will certainly cause failure of the subsequent tasks. Figure M-5 shows the task diagram in which the label F is for failure and S is for success. Failure of any of the three critical tasks leads to the failure of the IHA.

Figure M-5 Task Diagram to Prevent RCP Seal Failure M.1.4.2 Characteristics of the Critical Tasks The three critical tasks (i.e., detecting RCP abnormality alarms, entering AOP-018, and reopening FCV-626) are characterized in Table M-2, Table M-3, and Table M-4, respectively.

M-19

Table M-2 Task Characteristics of Detecting RCP Abnormality Alarms Task Goal Detect RCP abnormal alarms.

Specific None Requirement Detect any of the alarms below:

- FCV-626 valve position indication (18:52)

- Annunciator RCP thermal barrier cooling water hi or lo flow (18:52)

- Annunciator RCP B high bearing temperature (19:24)

Cue and - Annunciator RCP A high bearing temperature (19:30)

Supporting - Annunciator RCP C high bearing temperature (19:33)

Information - Annunciator RCP B seal leakoff high temperature (19:33)

PATH-1 has a step to check the annunciator RCP thermal barrier cooling water hi or lo flow alarms. The other alarms have to be detected by self-awareness.

PATH-1 instructs to check RCP thermal barrier cooling water hi or lo flow Procedure alarm.

Personnel SS and RO Task Support None Location Main control room Cognitive Activity Detectiondetecting and responding to the alarms The focus of the SS and RO is to implement PATH-1. Interruptions Concurrent include answering emergency phone calls, intermittent alarms, and Tasks responding to the subsequent events caused by the electric power failure.

The SS and RO implement PATH-1. This scenario does not have a Teamwork typical three-member crew to implement the procedure. The BOP Consideration operator is implementing the fire procedure in coordination with onsite fire brigade.

A couple hundred alarms are triggered within a few minutes after the Others initiating event.

M-20

Table M-3 Task Characteristics of Entering AOP-018 Task Goal Enter AOP-018 Verbatim following PATH-1 procedural instruction would not lead to Specific entering AOP-018. Operator enters the procedure based on relating the Requirement perceived alarms to abnormal RCPs Cue and Supporting Any alarm listed in Table M-2 could trigger the operator to enter AOP-018.

Information Procedure AOP-018, RCP Abnormal Conditions Personnel SS and RO Task Support None Location Main control room Understandingunderstand that the RCP is in trouble, specifically Cognitive Activity knowing that detecting the low RCP seal injection flow rate leading to realize that the charging flow does not reach to the RCP seals.

Concurrent Continue to implement PATH-1. The AOP-018 and PATH-1 are to be Tasks implemented in parallel.

SS and RO implement PATH-1. This scenario does not have a typical Teamwork three-member crew to implement the procedure. The BOP operator is Consideration implementing the fire procedure in coordination with onsite fire brigade.

Operators know the entry condition to the AOP-018. There is a competing Others priority of implementing the PATH-1 procedure and AOP-018.

Table M-4 Task Characteristics of Reopening FCV-626 or Tripping the RCPs Task Goal Reopen FCV-626 and trip the RCPs to prevent RCP seal damage Specific None Requirement Cue and Supporting Instructed by AOP-018.

Information Procedure AOP-018, RCP Abnormal Conditions Personnel SS and RO Task Support None Location Main control room Cognitive Activity Action execution Continue to implement PATH-1. The AOP-018 and PATH-1 are to be Concurrent Tasks implemented in parallel.

SS and RO implement PATH-1. This scenario does not have a typical Teamwork three-member crew to implement the procedure. The PATH-1 procedure Consideration has to be paused to implement AOP-018.

Others None M-21

M.1.5 Cognitive Failure Modes and Performance-Influencing Factors Sections M.1.1 and M.1.2 discuss the high-level context of the scenario. M.1.3 discusses the identification of the important human actions. M.1.4 discusses the critical tasks of the important human actions. This section identified the PIFs of the critical tasks based on the foundation established in the previous sections.

Critical Task 1Detect RCP Abnormal Alarms The dominant CFM is expected to be D1fail to initiate detection. The applicable PIFs include:

Information availability and reliability: The indications for this cue are genuine.

However, the electric fault causes many indications to be momentarily unavailable.

Some indications become available after the electric transition, and others remain unavailable throughout the event. The display reliability from the crews perspective is questionable.

Human-system interface: The signal (cue) is weak or masked because there are simultaneously hundreds of alarms on the alarm panels. There are also salience considerations about the information having a similar appearance with the surrounding information, that is, the alarm tiles relating to the cue are in the alarm panels with other similar alarm tiles.

Scenario familiarity: The MCR indications do not show a recognizable event pattern to the operating crew. Also, the operators expectation on information detection is biased, that is, when the crew was trained in the simulator for similar scenarios, the FCV-626 does not close. The crew would not expect the FCV-626 closure in this event; therefore, the operators do not have the motivation to check for the information.

Critical Task 2Enter AOP-018 The dominant CFMs to fail to enter AOP-018 after successful detection of RCP abnormal alarms are:

U3-2Incorrectly diagnose the problem: The operators fail to associate the alarms to the loss of RCP seal cooling and seal injection.

U3-3Fail to use guidance outside the main procedure steps: Verbatim following PATH-1 will not guide the operators to identify the problem because PATH-1s instructions are to check if any charging pump is running instead of checking if there is sufficient RCP seal charging flow. This requires knowledge-driven diagnosis to associate the alarms to the troubled RCP seals.

The applicable PIFs include:

Procedures, guidance, and instructions: The involved procedures are inadequate to develop the proper mental model of the situation. The Crew has to rely on knowledge to develop a mental model.

Scenario familiarity: The scenario is unfamiliar with an unexpected combination of equipment malfunctions.

M-22

Critical Task 3Open FCV-626 Once entering AOP-018, the procedure path of steps 1, 2 RNO, and 10 should be the path to open the FCV-626. The dominant CFMs are:

E4-1Fail to follow procedures, e.g., skip steps in procedures: This refers to the situation that the operators fail to follow the procedure path of steps 1, 2 RNO, and 10.

E 4-2Fail to execute simple actions: This refers to the situation in that the operator fails to turn the switch to the correct position or turns an incorrect switch.

The cognitive failure probability (Pc) can be calculated by the probabilistic addition of the critical tasks HEPs. A critical tasks HEP is calculated based on the cognitive failure modes and PIFs.

Calculating Pc is not demonstrated in this report. Demonstrations on calculating Pc are available in the IDHEAS-ECA report [78].

M.1.6 Time Uncertainty Analysis The timeline of an IHA includes two high-level elements: time available ( ) and time required

( ). The time required includes three time-segments: Tdelay, Tcog, and Texe, where:

Tdelay is the time from the abnormality start until the time at which the cues to the abnormality are available for operators to detect;

Tcog is the time starting when the cues are available for detection until the response decision is made; and

Texe is the time taken for the operators to complete the needed action(s) after the response decision is made.

The time available is estimated as 40 minutes. This starts at the initiating event and ends at 13 minutes after the CVC-310A failed open. Tdelay is zero because the cue to the FCV-626 closure is immediately available after the initiating event.

Calculating Tcog is complicated in this event. The cues (FCV-626 closure indication and the alarms related to RCP abnormality such as RCP thermal barrier high temperature and high RCP bearing temperature) can be detected by operator self-awareness or instructed by the PATH-1 procedure to check these alarms. The ASME/ANS PRA standard [44] supporting requirement HR-H2 states that the cues (e.g., alarms) that alert the operator to the recovery action are adequate provided that procedures, training, or skill of the craft exist. This supporting requirement is consistent with the two different paths to detect the cues. These two paths of cue detection are discussed below separately.

Early Self-Awareness Cue Detection Even though the cues, specifically the RCP thermal barrier HI or LO flow alarms, were available right after the initiating event, the electric fault severely affected the MCR information display that made early alarm detection difficult in the scenario. Several hundred alarms were triggered within a short period of time following the initial electric fault. Most of the secondary side and some of the primary side indications failed dark, and some displays went dark and came back to power again during the electric transition. The early alarm detection is not credited (not modeled) without additional manpower to support operator tasks given the evidence shown by the operator responses in the actual event.

M-23

Procedure-Guided and Late Self-Awareness Cue Detection The PATH-1 procedure has a step explicitly checking the cue. Based on the scenario, PATH-1 is expected to be entered after the reactor trip then the crew is transferred to EPP-4 Reactor Trip Response due to no SI signal. The PATH-1 procedure will be reentered when the SI signal occurs at 19:00. It is estimated that the crew would reach to the procedure step to check the cue RCP thermal barrier HI or LO flow alarms illuminated at 30 minutes after the initiating event. The PATH-1 step after detecting the cue is to check that at least one charging pump is running. By following the procedure instruction, the RCP thermal barrier hi or lo flow alarms illuminated alarm would likely lead the operator to realize the lack of RCP seal cooling because, following the alarm detection, PATH-1 instructs the operator to check if at least one charging pump is running. The answer is yes. There are two charging pumps running but little injection flow is available for RCP seal injection because CVC-310A failed open. At about the same time, the RCP B high bearing temperature alarm is triggered (19:24, 32 minutes after the initiating event). The operator has to make a conscious decision to enter the alarm response procedure APP-001-D1 that leads to check the less than 6 gpm seal inject flow rate, which, in turn, would lead the operator to AOP-018 RCP abnormal conditions Section 3 loss of seal injection. In AOP-018 Section 3, the expected procedural path is the following:

Step 1: Answer yes to Check RCP thermal barrier cooling water low flow alarm illuminated

Step 2 RNO: Answer no to All RCPs seal cooling was lost greater than 15 minutes

Step 10: to ensure FCV-626 is open.

The time estimation is the following:

30 minutes for the crew to reach the PATH-1 RCP thermal barrier hi or low flow alarms illuminated after the initiating event (i.e. 19:22).

1 minute to check RCP seal injection flow rate at Step 4 of APP-001-D1.

2 minutes to transfer to Section C of AOP-018 from APP-001-D1 (i.e. 19:25).

1 minute to reach Step 2 after entry of Section C of AOP-018.

1 minute to transfer to Step 10 from Step 2.

1 minute to open FCV-626 at Step 10.

Therefore, the estimated time to open FCV-626 based on procedure instruction is 36 minutes after the initiating event.

If the operator does not enter AOP-018 from APP-001-D1, the later alarms including RCP A high bearing temperature (19:30), RCP C high bearing temperature (19:33), and RCP B high leakoff (19:33) are likely to catch operators attention to enter AOP-018 because, at that time, the alarm tiles are relatively static. The new blinking alarms added into the alarm panels are likely to be detected. After detecting these alarms, the operator is expected to enter AOP-018.

With the detection of the alarms at 19:33, the FCV-626 will be opened at 19:38 (46 minutes after the initiating event).

The 36 and 46-minute estimates discussed in the above two paragraphs in this analysis represent two equally likely paths. As a result, the 36 and 46 minutes are used to represent the 5th and 95th percentiles of a normal distribution, respectively, for the uncertainty distribution of the time required. This results in a mean of 41 minutes, and a standard deviation of 3 minutes 2.4 seconds. With the time available of 40 minutes, this generates a (the HEP due to time M-24

uncertainties) of 0.629. The HEP of the important human action is the total probability of Pc and Pt.

In summary, this event analysis demonstrates the IDHEAS-G process for analyzing risks of an actual event. This is performed by applying the system behaviors in an actual event on a typical operating crew to analyze the crew and system responses in the event. The actual event has specifications on the event context such as instrumentation issues (many control panel indications were unavailable and computer displays of plant parameters were not reliable), the staffing (SM and STA are not in the MCR when the event occurs, and the BOP operator is detached from implementing emergency procedures with the SS and RO because the BOP operator is implementing fire procedure with the fire brigade), latent failure (CVCS failed to automatically swap to RWST), training-specifics (the crew biasedly assumed that FCV-626 remains open based on their previous simulator training on similar scenarios), and procedure-specifics (the PATH-1 instruction to ensure the charging pump is running but not to ensure that there is charging flow to the RCP seals) are not typically modeled for the analysis of hypothetical events. These considerations are factored into the identification of CFMs and the assessment of the states of the PIFs.

M.2 Analysis of a Hypothetical Event This section provides an example analysis of a hypothetical event. This demonstration uses NPP core damage as the undesired consequence. Post core damage scenarios are not included in the analysis of this event.

The following are some commonly used acronyms in this section:

AC: alternating current BDB: beyond-design-basis CFM: cognitive failure mode CST: condensate storage tank DC: direct current EDG: emergency diesel generator ELAP: extended loss of AC power ERO: emergency response organizations FLEX: diverse and flexible coping strategies FO: field operator FSG: FLEX support guidelines HCTL: heat capacity temperature limit HEP: human error probability HPCI: high-pressure core injection IHA: important human action LOOP: loss of offsite power M-25

MCR: main control room OSC: operation support center PIF: performance-influencing factor PSP: pressure suppression pressure RCIC: reactor core isolation cooling RO: reactor operator RPV: reactor pressure vessel SBO: station blackout SFP: spent fuel pool SM: shift manager SRV: safety relief valve SS: shift supervisor SSC: structure, system, and component STA: shift technical advisor TSC: technical support center VAC: voltage alternating current M.2.1 Scenario Analysis M.2.1.1 Operational Narrative Event Review Event title: A beyond-design-basis seismic event induces an extended loss of AC power at a General Electric Type (GE) 4 boiling-water reactor (BWR) with Mark 1 containment A nuclear power station with a single unit on site, a GE Type 4 BWR with Mark 1 containment, experiences a hypothetical beyond-design-basis (BDB) earthquake causing an extended loss of alternating current (AC) power (ELAP) event. All of the installed AC power sources (offsite AC, emergency diesel generators, and emergency station black out AC line) are lost following the earthquake. The installed AC power cannot be restored throughout the event. Because of the loss of all AC power for an extended period of time, the crew has to implement FLEX strategies to mitigate the event.

Initial condition The reactor is at 100-percent power steady state operation. Plant staffing is at the normal operation level. All components and systems are available and are in their normal alignment for full power operation.

Initiating event A beyond-design-basis (BDB) earthquake causes an ELAP event.

M-26

Boundary conditions The following are the plant conditions immediately after the earthquake:

1. Reactivity Control: The reactor tripped automatically and successfully as designed (i.e.,

all rods inserted, no anticipated transient without scram (ATWS)).

2. Reactor Water Inventory Control, Reactor Pressure Control, and Heat Removal From the core:

a. The reactor pressure boundary remains intact.

b. The emergency core cooling system is available. The high-pressure core injection (HPCI) system and the reactor core isolation cooling (RCIC) system automatically actuate taking water from the condensate storage tank (CST) after reaching the set points.

c. The reactor pressure vessel (RPV) safety relief valves (SRVs) cycle open to maintain RPV pressure.

3. Containment Integrity: Containment is automatically isolated and is intact.

4. Electrical: An ELAP event with the following specifics:

a. Off-site power is lost due to grid damage and cannot be immediately repaired

b. All emergency diesel generators fail at load-sequencing and cannot be immediately restarted or repaired

c. The station has an alternate AC source by a station blackout line for connecting to a nearby hydraulic power station. This AC source is not available due to the hydraulic power station being damaged by the earthquake.

d. The AC buses being fed by station batteries through inverters remains intact.

e. The station batteries and connections to the AC buses remain intact.

f. Without shedding the DC power, the DC power is available for two hours. The site implements a two-stage DC load shed to prolong DC power availability. The initial DC load shed is implemented in the station blackout (SBO) procedure to extend the DC availability to five hours. The deep DC load shed is implemented in the ELAP procedure that extends the DC availability to seven hours.

5. Instrumentation:

a. The instrumentation air is lost due to the loss of AC power. The nitrogen cylinders connected to SRVs remain intact.

b. The AC powered communication system is not available. The radio communication is available but requires setting up a portable antenna to cover the whole site area. Cell phone communication is not available.

c. The DC-powered MCR indications function as designed.

6. Plant service systems: The plant service systems (e.g., service water and component cooling water) are not available due to loss of AC power.

7. Radioactivity release: No immediate radiation indications.

8. Spent fuel pool (SFP) and dry cask: The SFP and dry cask remain intact. No indication of damage except for the loss of the normal SFP cooling system due to the SBO.

9. Concurrent events:

M-27

a. No independent concurrent events, e.g., no active security threat nor severe weather.

b. No large fire triggered by the earthquake.

c. Some workers were injured by the earthquake that required calling the offsite ambulance for assistance.

10. Structure, system, component, and accessibility:

a. The main turbine and main generator trip automatically and successfully.

b. All seismic class I structures remain intact. All other buildings suffer various degrees of damage.

c. Personnel need the master key (can be obtained from the main control room) to enter the restricted areas to perform mitigation actions due to the loss of access control system.

d. The FLEX equipment remains intact. Fallen structures and debris are in the FLEX equipment transportation paths. The debris needs to be removed to position the portable FLEX equipment in their designated operating locations.

11. Staffing:

a. The event occurs during normal work hours. The site has sufficient manpower to operate the MCR, technical support center (TSC), and operation support center (OSC).

A few minutes after the main earthquake ceases, the main control room (MCR) crew recognizes that the reactor, the main turbine, and the main generator trip automatically, HPCI and RCIC start automatically, offsite power is lost, and the emergency diesel generators (EDGs) fail to start automatically (a SBO). The RO tries to manually start the EDGs from the MCR control panels but the EDGs do not start. The SS enters the RPV control procedure (because of the reactor scram) and the loss of offsite power (LOOP) procedure (because of the loss of offsite power). The RPV control procedure is the default entry procedure for a reactor scram event.

The site has an integrated LOOP-SBO-ELAP procedure with the entry condition of a LOOP.

Following the RPV control procedure, the reactor operator (RO) maintains the RPV pressure and water level. Following the LOOP procedure, the shift supervisor (SS) instructs the transmission operator to configure the emergency SBO line to feed the stations electric system.

The emergency SBO line cannot be established because the hydraulic power station connecting to the line is damaged.

Note: Because this is a hypothetical event, the scenario narrative describes the baseline scenario with the emphasis of developing a most likely scenario as the baseline scenario. The less-likely but possible component and instrumentation failures could occur affecting operator performance will be discussed in the scenarios deviated from the baseline scenario (using the what-if approach). Some what-if questions are identified in section M2.1.4. The details of the deviation scenarios are not discussed in this example.

The SS enters the SBO procedure shortly after not being able to restart EDGs from the MCR.

Guided by the SBO procedure, the SS calls for field operators to come to the MCR to obtain a master key (needed to access the restricted area because the accessibility is affected by the loss of AC power) and for task assignment. Field operators are dispatched to restart the EDGs locally. The field operators quickly assess the EDGs status in the EDG rooms and conclude that the EDGs cannot be restarted within an hour and report the conclusion back to the MCR.

Field operators are also sent to perform the initial DC load shed to prolong DC availability, connect backup nitrogen cylinders to prolong the SRVs operation, and assess the FLEX M-28

equipment staging locations and transportation routes to prepare for FLEX equipment deployment.

The SS enters the ELAP procedure after knowing from the field operators that the AC power cannot be restored within an hour, an entry condition to enter the ELAP procedure. The ELAP procedure instructs the crew to activate FLEX mitigation strategies and mobilizes FLEX equipment (e.g., deep DC load shed, use FLEX generators to power emergency buses, and use the FLEX pump for core cooling), defeat HPCI and RCIC trip logic to prolong their operations, vent containment to protect containment integrity, inhibit the automatic RPV depressurization function to conserve RPV inventory and ensure ability to depressurize when needed, etc. The SS works with the two reactor operators in the MCR to implement the procedure.

After the earthquake, the SM oversees the event mitigation activities and assesses the EAL. A SBO (loss of all AC power for more than 15 minutes) is a site area emergency which requires the mobilization of the emergency response organizations (ERO) that include the TSC and OSC. There is sufficient manpower onsite to operate the TSC and OSC to coordinate with the MCR for event mitigation. The primary job of shift technical advisor (STA) is to perform independent assessment of the plant condition. Communicators in the MCR communicate with the local government, NRC, and onsite/offsite communication (e.g., request for offsite fire brigade support). The non-technical staff is evacuated when accessibility to leave the site is available.

M.2.1.2 Scenario Timeline Table M-5 shows the timeline of the baseline scenario using the two-column format and symbols in the second column that are described in APPENDIX E. The timeline is developed based on [142] with some modification for this demonstration.

M-29

Table M-5 Timeline of the Baseline Scenario Scenario overview: A GE Type 4 BWR with Mark 1 containment, single reactor site experiences a hypothetical beyond-design-basis (BDB) earthquake that caused an extended loss of AC power (ELAP) event. All of the installed AC power sources (offsite, emergency diesel generators, and emergency station black out AC line) are lost and cannot be restored following the earthquake. The site implements FLEX strategies to mitigate the event.

Time S: System Responses (hh:mm) H(abc): Human Responses; abc: The individuals position N: Notes I: System generated information 00:00 S: An extended loss of AC power (ELAP) event caused by a beyond-design-basis earthquake.

N: Assume that the main ground motion lasts for five minutes.

00:00+ S: Reactor Scrams & Turbine trips S: HPCI and RCIC start automatically on inch (-122-cm) signal.

00:05 H(MCR): Enter RPV control procedure based on the reactor scram.

H(MCR): Enter SE-11 LOOP/SBO/ELAP procedure based on the loss of offsite power.

H(RO): Shutdown HPCI; N: Because there is no indication of a loss of coolant accident event, as long as RCIC is in service, HPCI operation is not required. The operator shuts down HPCI to conserve DC power.

H(SS): Requests field operators to report to the MCR for task assignments and to obtain master keys. N: Performed in accordance of SE-11 instruction. The master key is necessary to enter the restricted area because the security system is disabled with the loss of all AC power.

00:10 H(SS): Enter the Primary Containment Control procedure because the containment pressure is greater than 2 psig.

M-30

Table M-5 Timeline of the Baseline Scenario (continued) 00:15 H(MCR): Distribute master keys in MCR to field operators (FO). N: in accordance of SE-11 instruction.

H(FO*): Locally start the EDGs per procedure X, attachment A (Texe = 45 min.). N:

The EDGs cannot be started based on the event assumptions.

H(FO): Perform initial DC load shed per SE-11, Att. T (Texe = 45 min.); N: The DC load shed is performed at the cable spreading room, reactor building and turbine building. N: Without shedding DC load, the battery is expected to last for two hours. This initial DC load shed extends the essential battery availability to five hours.

H(SM): Declare a site emergency based on MS1 Loss of all Off-Site and all On-Site alternating current (AC) power to emergency busses for 15 minutes or longer.

H(SM): Declare to mobilize the emergency response organization.

00:20 H(RO): Open SRVs to reduce RPV pressure to 500 psi, then control RPV cooldown and depressurization at a rate close to but not exceeding 100°F/hr until the RPV pressure is between 200 and 300 psi. N: This process takes two hours to reach to the desired pressure.

00:30 H(FO): Commence opening RCIC/HPCI room doors per SE-11 Att.U. N: This action is taken to prolong RCIC/HPCI availability to prevent a high temperature trip.

01:00 H(SM): Declare a general emergency based on MG1 Prolonged loss of all Off-Site power all On-Site AC power to emergency busses H(SS): Enter ELAP procedure because the AC power is not expected to be restored within one hour.

H(FO): Commence deep DC load shed per FSG-012 ELAP direct current (DC)

Load Shed (Texe = 30 min.). N: With deep DC load shed, the batteries are expected to last for seven hours.

H(FO): Establish backup nitrogen per FSG-044 (Texe = 30 min.); N: The backup nitrogen prolongs the RPV SRVs availability to depressurize the RPV.

H(FO): To commence debris removal and deploy FLEX equipment. N: The FLEX equipment cannot be deployed until the debris on the transportation routes is removed.

H(FO): Defeat RCIC trips and isolation logic (FSG-043); N: This action prolongs RCIC operation.

H(FO): Commence communication antenna deployment and opening hatches and doors (FSG-020, FSG-033); N: This action facilitates communication.

H(RO): Commence containment venting with torus pressure greater than 2 psig; N: This action protects containment.

H: TSC and OSC in operation.

01:30 H(FO): Complete the deep DC load shed.

03:00 H(FO): Deploy FLEX generator to charge 480 VAC emergency buses 04:00 H(FO): Complete deployment of portable fans to supply cooling air flow to the RCIC rooms per FSG-042; N: This action prolongs RCIC operation.

05:00 H(FO): Commence battery room venting per FSG-031; N: This action prevents battery performance from being affected by high room temperatures.

05:30 H(FO): Complete installation of SFP hoses on the refueling floor per FSG-042.

05:45 H(FO): Commence control room venting per FSG-030.

M-31

Table M-5 Timeline of the Baseline Scenario (continued) 06:00 H(FO): Commence deployment of portable pump that allows for makeup to the RPV, torus, and SFP. N: The pump staging locations are pre-specified.

07:00 H(FO): Implement portable generator to power the safety related 480 VAC.

12:00 H(FO): Commence makeup to the SFP from the FLEX Pump based on lowering SFP level.

30:00 H(FO): Commence injection into torus.

24:00 - H(MCR): Continue to maintain critical functions of core cooling (via RCIC),

72:00 containment (via hardened vent opening and FLEX pump injection to torus), and SFP cooling (FLEX pump injection to SFP).

FO: Field operator could be equipment operator, fire brigade, chemist, digital Instrumentation and control technician, reactor protection technician, health physics, and security personnel, etc.

The FOs in the table above represent multiple individuals. However, the same individual could be assigned to perform more than one task in sequential.

M.2.1.3 Relevant Operating Experience The operating experience described below is a direct quote from the executive summary of The official report of The Fukushima Nuclear Accident Independent Investigation Commission [143].

On March 11, 2011, the Great East Japan Earthquake triggered an extremely severe nuclear accident at the Fukushima Daiichi Nuclear Power Plant (NPP), owned and operated by the Tokyo Electric Power Company (TEPCO). When the earthquake occurred, Units 1, 2, and 3 of the Fukushima Daiichi plant were in at-power operation; and Units 4 to 6 were undergoing periodic inspections. The emergency shut-down feature, or SCRAM, went into operation at Units 1, 2 and 3 immediately after the commencement of the seismic activity. The seismic event caused a loss of the offsite power to the Daiichi NPP. The emergency diesel generators automatically started as designed. The tsunami caused by the earthquake flooded and totally destroyed the emergency diesel generators, the seawater cooling pumps, the electric wiring system and the DC power supply for Units 1, 2 and 4, resulting in loss of all power - except for an external supply to Unit 6 from an air-cooled emergency diesel generator at about 50 minutes after the earthquake. In short, Units 1, 2 and 4 lost all power. Unit 3 lost all AC power, and later lost DC power before dawn of March 13, 2011. Unit 5 lost all AC power.

The tsunami did not damage only the power supply. The tsunami also destroyed or washed away vehicles, heavy machinery, oil tanks, and gravel. It destroyed buildings, equipment installations and other machinery. Seawater from the tsunami inundated the entire building area and even reached the extremely high-pressure operating sections of Units 3 and 4, and a supplemental operation common facility (Common Pool Building). After the water retreated, debris from the flooding was scattered all over the plant site, hindering movement. Manhole and ditch covers had disappeared, leaving gaping holes in the ground. In addition, the earthquake lifted, sank, and collapsed building interiors and pathways, and access to and within the plant site became extremely difficult. Recovery tasks were further interrupted as workers reacted to the intermittent and significant aftershocks and tsunami. The loss of electricity resulted in the sudden loss of monitoring equipment such as scales, meters and the control functions in the central control room. Lighting and communications were also affected. The decisions and responses to the accident had to be made on the spot by operational staff at the site, absent valid tools and manuals.

M-32

M.2.1.4 Identify Additional Scenarios The risk-important SSCs and human actions mentioned in the scenario timeline are the candidates of drafting the first set of what-if questions. Additional what-if questions are asked based on the newly identified scenarios. The identification of the risk-important SSCs and human actions are dependent on the scope and assumption of the analysis. For example, the baseline scenario assumes the reactor tripped automatically. The reactor could fail to trip automatically. It is the analysts decision on whether to include the question what if the reactor fails to trip automatically? Below are some examples of what-if questions:

HPCI and RCIC:

o What if HPCI and RCIC fail to start initially after the earthquake? This leads to a new scenario that has a new human action to black start HPCI/RCIC.

o What if HPCI and RCIC fail during operation (for reasons such as tripping due to high operation temperature, and other trip logic not bypassed)?

DC power:

o What if operators fail to shed the DC load (including initial DC load shed and deep DC load shed)?

o What if operators fail to charge the DC power with the portable generators?

o What if additional diesel fuel is not available for long-term operation of the portable diesel generator?

o What if the essential battery fails due to high room temperature?

SRVs:

o What if SRVs fail stuck open?

o What if SRVs cannot be used to depressurize the RPV?

Portable pumps:

o What if portable pumps are not available for cooling the reactor core and SFP?

o What if pumps fail during operation?

Containment o What if containment cannot be vented or vented at different times of the scenario?

o What if the containment vent is left open after venting?

o What-if the containment penetrations are not isolated automatically and cannot be isolated manually?

TSC operation o What-if the TSC diesel generator that provide electricity for TSC fail to operate?

M-33

M.2.2 Context Analysis M.2.2.1 Task Context Immediately after the earthquake the operators immediate attention is to ensure sufficient core cooling and to restore electricity for a safe unplanned shutdown. The earthquake is expected to affect the MCR displays that slows down the procedure implementation pace. This also reduces the likelihood for operators to detect plant malfunctions in which parameters are not mentioned in the procedures.

The MCR primary tasks to protect the reactor safety are guided by the following three procedures:

RPV control procedure. The procedure entry condition is the reactor scram that occurs during the earthquake.

Primary containment control procedure. The procedure entry condition is the containment pressure greater than 2 psig. Based on the plant simulation model calculation, this entry condition is reached at about ten minutes after the earthquake.

The LOOP/SBO/ELAP procedure. The entry condition is the loss of all offsite power that occurs during the earthquake.

The key tasks to protect reactor safety are identified in the scenario timeline. The tasks are regrouped based on their objectives:

Ensure core cooling. Specific tasks include:

o Prevent tripping RCIC. Specific activities include (the time zero is when the main earthquake occurs. Time is specified in hh:mm.):

Open RCIC/HPCI room doors (at 00:30)

Defeat RCIC trips and isolations (at 01:00)

Use fan to cool RCIC room (at 04:00) o Vent containment to reduce torus temperature increase rate (01:00). RCIC pumps fail at high torus temperature.

o Align portable FLEX pump to cool the core (at 06:00)

Prolong DC power availability: Specific tasks include:

o Shutoff HPCI (at 00:05) o Perform initial DC load shed in SBO procedure (at 00:15) o Perform deep DC load shed in ELAP procedure (at 01:00) o Vent battery rooms (at 05:00) o Use FLEX generators to charge DC battery (07:00) o Refuel FLEX generators (later scenario)

Control RPV pressure. Specific tasks include:

M-34

o Depressurize RPV to between 200 and 300 psig (at 00:20) o Disable RPV automatic depressurization system o Align Nitrogen bottles to RPV SRVs (01:00)

Protect containment. Specific tasks include:

o Vent containment (01:00)

Protect SFP o Align FLEX pump for SFP makeup (04:00)

Protect MCR work environment o Commence MCR ventilation (05:45)

Overarching o Remove debris o Distribute master keys to the field operators to enter the controlled area because the security system is affected by the loss of all AC power o Setup antenna to support field communication In addition to the above safety related tasks, the following tasks are expected:

Survey earthquake damage to the site

Try to restore the installed AC power. This includes offsite AC power, EDGs, and SBO line. Based on the event assumption, the installed AC power cannot be restored.

Make the EAL declaration and mobilize the ERO

Evacuate non-technical staff

Communicate with field operators, TSC, and OSC

Communicate with the offsite response center, local government, and NRC

Answer emergency phone calls for things such as personnel injury.

Most of these tasks are ex-control room actions. These tasks are to be performed at multiple locations, by different groups of people, and in various overlapping time periods. They require extensive coordination and communication. Many of these tasks are rarely performed and personnel need to perform their tasks under unfamiliar locations and infrastructure. Some tasks may be inter-dependent or need to be performed in certain orders. Some tasks may be highly physical demanding.

Even though the site has sufficient manpower to perform the above tasks, the MCR is the control center of all of these operations. The MCR monitors the task performance status and is involved in all important communications. The MCR operating crew is expected to be fully occupied until there is a good prospective that the core cooling can be maintained to safely shutdown the reactor.

M-35

M.2.2.2 Environment and Situation Context Accessibility The earthquake causes structural damage affecting accessibility to perform mitigation strategies. For plant-specific analysis, the site map would be used to assess the earthquake damage to each risk-important SSCs. This demonstration is not a plant specific analysis. The accessibility to perform mitigation actions is simply assumed accessible but subject to different degrees of delay. It is assumed that the debris in transportation paths needs to be removed to transport the FLEX equipment from the onsite FLEX storage building to their operating locations. The loss of AC power affects the access control system. The field operators need to come to the MCR to obtain a master key to enter the restricted area.

Visibility With DC load shed, some ex-control room work sites may have weak ambient lights thus the visibility may impact personnels task performance.

Noise With the earthquake damage and various rescues and restorations going on, noise can be high in worksites and affect aural signal detection and communication.

Humidity and Temperature In SBO, the ventilation and cooling systems to the main control room, technical support center, and operation support center are lost. The operators open doors and windows, if available, to reduce the humidity and rising temperature.

M.2.2.3 System Context The following discusses the constraint in SSC to perform mitigation actions.

HPCI/RCIC The HPCI and RCIC are the only available installed systems to maintain core cooling. The HPCI is shutoff by operators to conserve DC power. This makes RCIC as the only system for core cooling. To maintain the RCIC in operation, the following conditions have to be met:

RPV pressure is greater than 150 psig (to drive the turbine-driven RCIC pump)

The DC power is used to automatically trip RCIC at high RPV water level and start RCIC at low RPV water level. The DC power also enables operators to control the steam flow rate to the RCIC pump and RCIC injection flow rate to the RPV.

Sufficient condensate storage tank (CST) or torus inventory (for initially sucking water from the RWST then automatically swapping to the torus at low CST level). The automatic swap requires DC power. If DC power is not available, the valves can be manually opened locally.

Low torus temperature. Water temperatures greater than 230 °F would fail the RCIC pump due to cavitation. The torus water temperature increases mainly due to absorbing the steam dumped from HPCI, RCIC, and the RPV SRVs. Venting containment is a strategy to reduce the torus temperatures increase rate.

Defeat RCIC trip signals, e.g., high temperature isolation and low suction pressure.

M-36

DC Power The DC power is critical to operate RCIC and to display plant status. Without DC power, the plant staff has to black start the RCIC. Without DC power, the operators have no indication to control the RPV level.

After the ELAP, the DC power capacity is two hours. The initial DC load shed instructed by the SBO procedure extends the capacity to five hours. The deep DC load shed instructed by the ELAP procedure extends the capacity to seven hours. Using the FLEX generator to charge the batteries enables DC availability as long as the batteries are charged.

RPV SRVs The plant is designed to use a combination of SRVs and safety valves (SVs) to maintain RPV pressure. Each SRV is self-actuating at the set relieving pressure but may also be opened from the main control room, the remote shutdown panel, or the alternative shutdown panel. If an SRV fails to pilot-open at the set pressure (this requires DC power and pneumatic pressure) the SRV is designed to mechanically open at a higher pressure to act as a safety valve.

Operating the SRV to depressurize the RPV requires DC power (to operate control valves and sense parameters) and pneumatic pressure (to move the pilot valves). A nitrogen accumulator is attached to an SRV to provide pneumatic energy for relief valve operation after the normal pneumatic supply is not available (due to the loss of AC power). The nitrogen accumulator can provide five valve operations. To extend SRV operation, backup nitrogen cylinders need to be connected to the SRVs. The SRVs can be opened only when the RPV pressure is greater than 100 psig. They close automatically when the RPV pressure is less than 50 psig.

Torus Temperature and Inventory The torus is critical to absorb energy dumped from the RPV and filter radioactivity released from the SRVs. Three operational boundaries are needed to maintain torus functions: the heat capacity temperature limit (HCTL), pressure suppression pressure (PSP), and torus temperature. The HCTL (characterized by torus temperature, torus water level, and RPV pressure) specifies the conditions where the torus has sufficient capacity to absorb the heat dumped from the RPV. The PSP (characterized by torus pressure and torus water level) specifies the conditions where the torus is capable of suppressing containment pressure. RCIC and HPCI pumps fail when the torus temperature exceeds 350 °F.

Human-System Interface The DC-powered important parameter displays and component controls in the MCR are assumed functioning as designed. The TSC has a diesel generator for TSC operation. The diesel generator is functional in the baseline scenario.

Communication Equipment The event involves high coordination and frequent communication between main control room, technical support center, operation support center, and field operators. The communication equipment is DC powered separately from the essential DC power.

Emergency Lighting The normal lighting is not available because of the SBO. The emergency lighting is powered by batteries. The local in-door lighting relies on the emergency light and the flashlight carried by the operator. Once the batteries are drained, the emergency lighting is lost.

M-37

M.2.2.4 Personnel Context When the event occurs, there is sufficient manpower onsite to run the MCR and to establish the technical support center (TSC) and the operation support center (OSC) for event mitigation.

Personnel injury may occur, and the non-technical staff will be evacuated from the site. The injuries might directly affect in-plant operators who are needed to support mitigation of this scenario. Injuries to other plant personnel may also require direct involvement by operations supervisors to coordinate efforts to stabilize their conditions and to coordinate their medical evacuation. Concerns about injuries to co-workers throughout the plant may also distract the attention of operators and supervisors and cause delays in their performance of the desired responses. However, in the baseline scenario, it is assumed that the site has sufficient workers qualified to perform the required tasks. These are assumed not to affect event mitigation.

Some actions such as connecting hoses or removing debris may be performed by non-licensed personnel.

Since this event is caused by a very severe earthquake, it is certain that the area surrounding the plant has sustained extensive damage. Thus, the workers naturally would be concerned about possible harm or the unknown status of their families and loved ones. These concerns could affect the workers performance.

Procedures are available for all mitigation actions. FLEX equipment operating manuals and procedures are available. Simulator training for LOOP and SBO events is conducted annually.

The training on responding to an ELAP event is conducted in classroom and tabletop settings.

The field operators are trained to operate the FLEX equipment. Yet, such training is infrequent compared to EOP training and is not included in plants Systematic Approach of Training (SAT)

Drills to mobilize the TSC and OSC are conducted annually.

After declaring an Alert, the SM serves as the emergency response director who has the authority to direct event mitigation. Multiple teams and organizations are involved. There are well established responsibilities, authorities, and lines of communication among the parties.

However, all the involved parties have not been drilled together in a beyond-design-basis scenario.

M.2.3 Identification and Definition of Important Human Actions For a long and complex event, there are many human actions that are important to the safety goals in mitigating the event consequences. Examples include shedding DC load (interact with DC power supply), depressurizing RPV (interact with SRVs), venting containment, and deploying FLEX generators and pumps, etc. Additional IHAs may be identified in the other scenarios of the same initiating event. Examples are manually transferring RCIC suction from the CST to the torus if it is not automatically transferred.

This demonstration uses the example important human action of deploying a FLEX generator to power the 480 VAC emergency buses and, in turn, to charge the essential batteries. The following is the definition:

The action starts right after ELAP is declared. The ELAP procedure directs personnel to deploy the FLEX generator. The action ends at the 480VAC emergency buses being powered by the FLEX generator. The success criterion of the action is correctly operating FLEX generator to power the 480 VAC emergency buses within the specified time. The FLEX generator is loaded on a trailer in the FLEX storage building. It needs to be transported from the FLEX storage M-38

building to its operating location, staged, and connected properly. There are FSGs to guide the action.

Note that the debris in the transportation route needs to be removed before the action can be performed. Removing debris is considered as a separate important human action because it is performed by a different group of people and affects the deployment of all FLEX equipment.

M.2.4 Modeling of Important Human ActionsTask Analysis M.2.4.1 Identification of Critical Tasks The action begins at following the ELAP procedure to deploy the FLEX generator and ends at the 480VAC emergency buses being powered by the FLEX generator. Figure M-6 shows the task diagram depicting the success (labeled as S) and failure (labeled as F) paths of the action. The cue of starting a FLEX generator deployment to power the 480VAC emergency buses is explicitly stated in the ELAP procedure. Deploying a FLEX generator to power the 480VAC emergency buses is a part of the decision of declaring ELAP. Personnel would follow the instructions unless there were strong reasons otherwise. In US nuclear power plants, deviating from an emergency procedure instruction has to be agreed with by two SROs.

Possible reasons for not deploying a FLEX generator are not having sufficient manpower or the FLEX generator transportation routes or set up is not accessible. None of these reasons exist in this analysis.

Start - Order to Transport and S S Operate FLEX S deploy FLEX Connect stage FLEX generator to 1S generator per FLEX generator generator power 480 VAC ELAP procedure F F F 2F 3F 4F Figure M-6 Task Diagram to Deploy a FLEX Generator Deploying the FLEX generator to power the 480 VAC emergency buses starts with the MCR giving the order to the OSC manager to deploy a team to implement the order. The field crew needs to communicate with the MCR crew to specify the generator operating locations and to align the emergency buses to be powered by the FLEX generator. Prior to deploying a FLEX generator, the SBO procedure instructs the crew to assess the FLEX equipment deployment location, and if needed, remove the debris in the equipment transportation routes. Performing FLEX location assessment and debris removal is modeled separately.

Therefore, the critical tasks for this action are transporting and staging the generator, connecting the generator to the emergency buses (including alignment of the buses), and operating the generator. The action is broken down into these three critical tasks because they are performed at different locations, by different groups of personnel, with different procedures.

M.2.4.2 Characteristics of the Critical Tasks The task characterization for each critical task is summarized in Table M-6, Table M-7, Table M-8, and Table M-9, respectively.

M-39

Table M-6 Task Characteristics of Detect ELAP Procedure Instruction to Deploy a FLEX Generator Task Goal Detect the ELAP procedure instruction to deploy a FLEX generator Specific None Requirement Cue and Supporting An ELAP procedure step explicitly states to deploy a FLEX generator.

Information Procedure ELAP procedure Personnel SS Task Support None Location Main control room Cognitive Detection Activity SS concurrently implements the following three emergency procedures: (1)

Concurrent ELAP procedure; (2) RPV control procedure; and (3) containment control Tasks procedure. Aftershocks are likely to occur. MCR staff make emergency phone calls to offsite staff and local resources (e.g., ambulance).

Teamwork SS and two ROs implement procedures. STA performs independent Consideration situation assessment. SM oversees the event mitigation.

The ELAP procedure is part of the LOOP/SBO/ELAP procedure. The Others symptoms to enter the procedure and to transfer to the ELAP procedure are vivid.

Table M-7 Task Characteristics of Transporting the FLEX Generator Transport the generator from the FLEX equipment building to the Task Goal specified location and stage it properly Specific Need to be able to communicate with the OSC.

Requirement Cue and Supporting A procedure-instructed task Information Procedure FSG Personnel Non-licensed personnel Task Support None From the FLEX equipment building to the specified onsite generator Location location.

Cognitive Activity Action execution Concurrent Tasks No concurrent task for the personnel performing the transportation Teamwork Coordination with OSC and MCR on clearing the travel path and Consideration staging the generator.

M-40

Table M-8 Task Characteristics for Connecting the FLEX Generator Task Goal Correctly power the 480 VAC emergency buses Specific Certain steps of aligning the buses and connecting cables have to be Requirement performed in the exact order as specified in the FSG.

Cue and Supporting The OSC specifies the individuals to perform the task.

Information Procedure FLEX support guideline (FSG)

Personnel Field operators Coordinate the MCR operators to align the 480 VAC emergency buses Task Support to the FLEX generator Location Onsite building or shelter Cognitive Activity Action execution Concurrent Tasks A team dedicated to the task Teamwork Based on the available onsite field operators, the team members may Consideration not have previously worked together for this type of task Others The FSGs instructions are clear.

Table M-9 Task Characteristics of Operating the FLEX Generator to Power the 480 VAC Emergency Buses Start the generator and continuously power the 480 VAC emergency Task Goal buses Specific Starting the generator and continuously powering the 480 VAC emergency Requirement buses may require opening/closing certain breakers Cue FSG instructed task Procedure FLEX support guideline (FSG)

Personnel Field operators Location Onsite building or shelter Cognitive Action execution Activity Concurrent Powering the 480 VAC emergency buses requires continuous monitoring.

Tasks The personnel performing this task may have other concurrent tasks.

M.2.5 Cognitive Failure Modes and Performance-Influencing Factors Transporting and staging the FLEX generator The primary failure modes are the following:

E3-1Delayed implementation: The action cannot be performed prior to the FLEX equipment transportation route being accessible. The crew may be reassigned to other tasks while waiting for the transportation path to be cleared.

E4-2Fail to execute skill-of-craft actions: This refers to failing to hock up the trailer to the transportation vehicle or operate the vehicle through very rough travel paths (even though the debris is removed).

M-41

The applicable PIFs are the following:

Accessibility/habitability of workplace including travel paths: The FLEX generator transportation path and deployment location may have accessibility issues. This example does not show event and site-specific analyses to assess the accessibility.

Workplace visibility: Low visibility of the travel path and worksites Connecting the FLEX generator The primary failure modes are the following:

E1-1Incorrectly assess or interpret the action plan (e.g., errors in personnel allocation, equipment/tool preparation, or coordination): This action requires coordinating with the MCR to align the emergency buses to be powered by the FLEX generator and is not performed until the FLEX generator transportation path and the generator operating locations are accessible.

E4-1Fail to follow procedures, e.g., skipping steps in procedures or performing the steps in wrong order: This refers to failing to implement the FSGs.

The applicable PIFs are the following:

Scenario familiarity: Implementation of the FLEX strategy and equipment is rarely performed. Non-planned situations are likely to occur in a beyond-design-basis event.

Staffing: Forming a temporary team to deploy a FLEX generator to power the 480 VAC emergency buses is not routinely practiced. The assigned crew members collectively may not have the needed knowledge, skills, and abilities to perform the task.

Team and organization factors: The FLEX generator team needs to communicate with the MCR to align the emergency buses to be powered by the FLEX generator and the OSC to know the debris removal status and details in setting up the FLEX generator.

Communication can be difficult because of an unfamiliar communication protocol or less than adequate common mental models of various parties. In addition, the communication could be challenged be the unavailability of AC powered communication equipment.

Training: Staff are under-trained for the types of actions. This is a once-in-a-lifetime event. The site does not emphasize training on this type of event as much as responding to more frequent events.

Procedures, guidance, and instructions: The procedure for aligning buses and connecting the generator may not have adequate detail. The specifications on some steps may not match the situation.

Starting and continuously operating the FLEX generator The primary CFM is the following:

E4-1 Fail to follow procedures (e.g., skip steps in procedures): This refers to failing to start and operate the generator as specified by the corresponding FSG.

M-42

The applicable PIFs are the following:

Scenario familiarity: Implementation of the FLEX strategy and equipment is rarely performed. Non-planned situations are likely to occur in a beyond-design-basis event.

Starting and operating a FLEX generator may require manipulations that are different from those for normal diesel generators.

Staffing: Forming a temporary team to deploy a FLEX generator to power the 480 VAC emergency buses is not practiced routinely. The assigned crew members collectively may not have the needed knowledge, skills, and abilities to perform the task.

Training: Staff are under-trained for the types of actions. This is a once-in-a-lifetime event. The site does not emphasize training on starting and operating the generator as much as responding to more frequent events.

Multitasking, interruption, distraction: Personnel monitoring the status for continuous operation of the generator may have other main tasks.

The cognitive failure probability (Pc) can be calculated by probabilistic addition of the critical tasks HEPs. A critical tasks HEP is calculated based on the cognitive failure modes and PIFs.

Calculating Pc is not demonstrated in this report. Demonstrations of calculating Pc are available in the IDHEAS-ECA report [78].

M.2.6 Time Uncertainty Analysis Time available for performing the action The success criteria of deploying the FLEX generator to power the 480 VAC emergency buses are specified below and shown in Table M-10:

Shed the DC load: If the initial DC load is successfully performed, the time available is five hours; otherwise the time available is two hours.

Given a successful initial DC load shed, the time available is seven hours if the deep DC load shed is performed successfully; otherwise the time available is five hours.

Table M-10 shows that if the initial and deep DC load sheds were successfully implemented, then the personnel have seven hours to deploy the FLEX generator to prevent a loss of DC power. If the initial DC load shed is successfully implemented but the deep DC load shed fails, the available time to deploy the FLEX generator is five hours. It is assumed that the deep DC load shed would fail if the initial load shed fails. In this case, the available time to deploy the FLEX generator is only two hours.

Table M-10 Time Available for Successful Deployment of FLEX Generator Initial DC Load Shed Time available (Hours)

Success Fail Success 7 N/A Deep DC Load Shed Fail 5 2 The system window, TSW, for time available is two hours for failing the initial DC load shed. TSW is five hours if the initial DC load shed is successfully performed but the deep DC load shed is failed. TSW is seven hours if both the initial and deep DC load shed are performed successfully.

M-43

The time starts at the initiating event. Both the initial DC Load Shed and Deep DC Load Shed are performed by hands without the need of using special equipment.

The ELAP procedure gives commands to remove debris from the FLEX equipment transportation routes and deploy a FLEX generator to power the 480 VAC bus. These two commands are estimated to be given at one hour after the initiating event. The time required for debris removal needs to be estimated based on the site and event-specific information. This demonstration does not perform the level of analysis needed to estimate the required time for debris removal.

Therefore, if debris removal is not needed, the time available for deploying the FLEX generator would be (TSW - one hour for entering ELAP procedure).

If debris removal is needed, the time available for deploying the FLEX generator would be (TSW - one hour for entering ELAP procedure - the time required for removing debris).

In fact, if TSW is two hours and debris removal is needed, the action becomes infeasible because there is not enough time to perform it.

Time Required The time required for deploying a FLEX generator is plant specific and can vary greatly with the situation and environmental conditions. For this demonstration, it is assumed that the time taken to deploy the FLEX generator to power the 480 VAC emergency buses is represented by a normal distribution with a mean of 45 minutes and a standard deviation of 15 minutes, and the deployment cannot be started until debris is removed.

Assume that debris removal is represented by a normal distribution with a mean of two hours (i.e., three hours after the initiating event) and a standard deviation of 30 minutes, , the failure probability of powering the 480 VAC emergency buses before battery depletion due to insufficient time is calculated as the probability of the time required being greater than the time available. Because the sum of two normal distributions is still a normal distribution, the uncertainty distribution of the combination of the time required to remove debris and to deploy a FLEX generator to power the emergency buses is a normal distribution. The mean and standard deviation of the new normal distribution are calculated by the equations below. The new distribution has a mean of 3 hour3.472222e-5 days 8.333333e-4 hours 4.960317e-6 weeks 1.1415e-6 months 45 minutes (i.e., 1 hour1.157407e-5 days 2.777778e-4 hours 1.653439e-6 weeks 3.805e-7 months to enter ELAP procedure, 2 hours2.314815e-5 days 5.555556e-4 hours 3.306878e-6 weeks 7.61e-7 months to remove debris and 45 minutes to deploy and establish the FLEX generator to power the emergency buses). The standard deviation of the new distribution is 54 minutes (i.e., square root of the 45 minutes square pluses 30 minutes square).

µ+ = 1 + µ + µ 2

+ = ( )2 +

0.97 if TSW is two hours

0.08 if TSW is five hours

1.6E-4 if TSW is seven hours The important human actions HEP is the total probability of Pc and Pt.

M-44

M.3 Summary Two human events are analyzed and documented in this appendix to demonstrate the HRA process with IDHEAS-G. The analysis develops a baseline event scenario and identifies the IHAs, analyzes the critical tasks of an IHA and their CFMs, and discusses the assessment of relevant PIFs based on the assumptions made for the operational narrative and event context.

The outcomes of the analysis constitute the basis to calculate the HEP of the IHAs in the event.

The analysis also demonstrates how to assess the HEP of an IHA attributing to uncertainties in time available for performing the action and time required (i.e., ). The analysis also demonstrates the identification of the cognitive failure modes applicable to the critical tasks and the assessment of PIFs relevant to the tasks. These are the inputs to a HEP model for calculating the HEP attributing to cognition failure (i.e., ).

M-45

NUREG-2198 The General Methodology of an Integrated Human Event Analysis System (IDHEAS-G) May 2021 J. Xing, Y. J. Chang, J. DeJesus Segarra Technical Division of Risk Analysis Office of Nuclear Regulatory Research U.S. Nuclear Regulatory Commission Washington, D.C. 20555-0001 Same as above This report describes a human reliability analysis (HRA) methodology developed by the U.S. Nuclear Regulatory Commission (NRC) staff, the General Methodology of an Integrated Human Event Analysis System (IDHEAS-G).

IDHEAS-G was developed in response to the staff requirements memorandum (SRM) M061020, dated November 8, 2006, to the Advisory Committee on Reactor Safeguards (ACRS). The SRM directed the ACRS to, work with the

[NRC] staff and external stakeholders to evaluate different human reliability models in an effort to propose a single model for agency use or guidance on which model(s) should be used in specific circumstances. IDHEAS-G is intended to be a human-centered, general methodology used to develop application-specific HRA methods by the NRC. It integrates the strengths of existing HRA methods and enhances HRA in: (1) application scope, (2) scientific basis, (3) HRA variability, and (4) data for HRA. An example of the use of the IDHEAS-G framework is the development of Research Information Letter 2020 IDHEAS for Event and Condition Assessment (IDHEAS-ECA) and its associated software tool. IDHEAS-ECA has proven to be a useful HRA method for supporting the NRCs risk-informed decisionmaking processes.

Human reliability analysis Human error probability Human performance Performance influencing factors Macrocognition

NUREG-2198 The General Methodology of An Integrated Human Event May 2021 Analysis System (IDHEAS-G)

NUREG-2198, the General Methodology of an Integrated Human Event Analysis System (IDHEAS-G)

Text

SUMMARY

SUMMARY

Navigation menu

Search