ML19011A431
| ML19011A431 | |
| Person / Time | |
|---|---|
| Issue date: | 01/16/2019 |
| From: | Office of Nuclear Regulatory Research |
| To: | |
| Nathan Siu 415-0744 | |
| Shared Package | |
| ML19011A416 | List:
|
| References | |
| Download: ML19011A431 (38) | |
Text
Human Reliability Analysis Lecture 5-2 1
Key Topics
- HRA importance
- General description
- Fundamental model
- Methods
- Validation
- Challenges 2
Overview
Resources A. Kolaczkowski, et al., Good Practices for Implementing Human Reliability Analysis, NUREG-1792, April 2005.
J. Forester, et al., Evaluation of Human Reliability Analysis Methods Against Good Practices, NUREG-1842, September 2006.
J. Forester, et al., The International HRA Empirical Study: Lessons Learned from Comparing HRA Methods Predictions to HAMMLAB Simulator Data, NUREG-2127, August 2014.
J. Forester, et al., The U.S. HRA Empirical Study: Assessment of HRA Method Predictions against Operating Crew Performance on a U.S.
Nuclear Plant Simulator, NUREG-2156, June 2016.
A.M. Whaley, et al., Cognitive Basis for Human Reliability Analysis, NUREG-2114, January 2016.
3 Overview
Other References A. Poucet, Human Factors Reliability Benchmark Exercise: Synthesis Report, EUR 1222 EN, Ispra Joint Research Centre, Commission of European Communities, August 1989.
E. Lois, et al., International HRA Empirical Study - Phase 1 Report: Description of Overall Approach and Pilot Phase Results from Comparing HRA Methods to Simulator Performance Data, NUREG/IA-0216, Vol. 1, November 2009.
A. Bye, et al., International HRA Empirical Study - Phase 2 Report: Results from Comparing HRA Method Predictions to Simulator Data from SGTR Scenarios, NUREG/IA-0216, Vol. 2, August 2011.
V.N. Dang, et al., International HRA Empirical Study - Phase 3 Report: Results from Comparing HRA Methods Predictions to HAMMLAB Simulator Data on LOFW Scenarios, NUREG/IA-0216, Vol. 3, December 2014.
H. Blackman, N. Siu, and A. Mosleh, Human Reliability Models: Theoretical and Practical Challenges, Center for Reliability Engineering, University of Maryland, College Park, MD, 1998.
4 Overview
Other References A. D. Swain and H.E. Guttmann, Handbook of Human Reliability Analysis with Emphasis on Nuclear Power Plant Applications: Final Report, NUREG/CR-1278, August 1983.
P. Moieni, et al., A PC-based human reliability analysis (HRA) software, Proceedings ANS International Topical Meeting on Probabilistic Safety Assessment (PSA 93), Clearwater Beach, FL, January 26-29, 1993.
D. Gertman, et al., The SPAR-H Human Reliability Analysis Method, NUREG/CR-6883, August 2005.
U.S. Nuclear Regulatory Commission, Technical Basis and Implementation Guidelines for A Technique for Human Event Analysis (ATHEANA), NUREG-1624, Rev. 1, May 2000.
J. Xing, et al., An Integrated Human Event Analysis System (IDHEAS) for Nuclear Power Plant Internal Events At-Power Application, NUREG-2199, Vol. 1, March 2017.
Y.J. Chang and J. Xing, The general methodology of an Integrated Human Event Analysis System (IDHEAS) for human reliability analysis method development, Proceedings International Conference on Probabilistic Safety Assessment and Management (PSAM 13), Seoul, Korea, October 2-7, 2016.
S. Lewis and S. Cooper, EPRI/NRC-RES Fire Human Reliability Analysis Guidelines: Final Report, EPRI 1023001/NUREG-1921, July 2012.
A. Lindeman and S. Cooper, EPRI/NRC-RES Fire Human Reliability Analysis Guidelines -
Qualitative Analysis for Main Control Room Abandonment Scenarios, Supplement 1, EPRI 3002009215/NUREG-1921, Supplement 1, August 2017.
5 Overview
Human Actions and NPP PRA Operational decisions and actions have played an important role in every major NPP accident and incident
- Occurrence and progression
- Successes and failures 6
PRAs that dont account for human contributions are not useful (for most applications)
HRA Importance
Example Events Browns Ferry 1 & 2 cable fire (1975)
Worker ignites polyurethane foam, starts cable fire Fire suppression delayed 7+ hours (reluctant to use water)
Operators achieve safe shutdown using non-safety system Davis-Besse loss of feedwater (1985)
Operator error causes loss of feedwater Multiple malfunctions => feed and bleed cooling directed by procedures, would have major economic consequences Shift supervisor chooses to wait for recovery of AFW (which is successful)
Fukushima Dai-ichi Unit 1 (2011)
Operators close isolation condenser (little effect given accident conditions)
Operators perform numerous non-proceduralized actions (e.g., scavenge car batteries to supply power) in attempts to save plant Ex-control room actions hampered by site conditions (tsunami alerts, aftershocks, damage, dark, radiation, )
7 HRA Importance
What is HRA?
- In the context of NPP PRA: A structured approach used to identify potential human failure events and to systematically estimate the probability of those events using data, models, or expert judgment (NUREG-2122)
- Human Failure Event: interface with rest of PRA model:
- Terminology used to emphasize connection with NPP PRA model (basic events), avoid connotation of blame (e.g., when time available is insufficient)
- Includes errors of omission, errors of commission
- Can be included at scenario level (event trees) or system level (fault trees) 8 General Description
HRA General Process Activities Qualitative analysis Modeling Quantification Supports overall model construction
- Initiating event identification
- Accident scenario modeling
- Systems modeling Not just a quantification activity 9
General Description
HRA Dimensions and Descriptors Time Pre-initiator Initiator Post-initiator Space Within control room Outside control room Organization Control room crew Field operators Emergency response organization Implicit Actions addressed by other PRA model elements (e.g., initiating event frequencies, loss of offsite power recovery, common cause failure probabilities)
Pre-initiator decisions affecting fundamental plant design (e.g.,
flood barrier height) and operations (e.g., resources for training)
Out-of-scope for NPP PRA Sabotage Terrorism 10 General Description
Typical HFE Level of Detail
- Macro-level crew actions, e.g.,
- Isolate faulted steam generator
- Initiate bleed and feed cooling
- Recover a failed pump
- Micro-level modeling (e.g., put control switch X in pull-to-lock position) can support HFE; need to consider micro-level recoveries as well as failures 11 General Description
How Things Work
- Task-oriented view
- Diagnosis and Planning
- Action
- Cognitive view
- Detecting/Noticing
- Sensemaking/Understanding
- Decision Making
- Action Execution
- Teamwork (communication/coordination) 12 Macrocognitive functions (NUREG-2114)
Fundamental Model
Naturalistic Decision Making 13 From NUREG-2114, per F.L. Greitzer, et al., Naturalistic decision making for power system operators, International Journal of Human-Computer Interaction, 26(2-3), 278-291, 2010. doi:10.1080/10447310903499070 Fundamental Model
How Things Can Fail 14 Real-world contextual elements and PIFs* can include:
Specific conditions (e.g., problematic components, mixed crews)
Scenario dynamics (e.g., shift changes, multiple system shocks)
Economic concerns Social behaviors and relationships
- Usually referred to as Performance Shaping Factors (PSFs)
Fundamental Model
Human Error Probability (HEP)
Quantifies aleatory uncertainty Is subject to epistemic uncertainties Is a function of the task, the scenario context leading up to the task, and the relevant PIFs Underlying assertion: human actions are predictable (in a probabilistic sense)
Performance of specific tasks, often with specific procedures and training Bounded rationality: operators/staff are trying to do the right thing Note: HEP functional behavior on PIFs is usually assumed to be multiplicative, but other data might support additivity Fundamental Probabilistic Model 15
=,,
Fundamental Model
HRA Approaches 16 Holistic Analysis (ATHEANA, MERMOS)
- Analyze context and develop operational story / narrative
- Identify situations deviating from the base story that lead to undesired actions
- Estimate the HEPs of the deviations PIFs Strengths - Preserves context; uses expert ability to integrate complex information Limitations - Level of effort; subjectivity and variability HFE Tasks + Context (plant situation, scenario, and crew factors)
Decomposition-Based Analysis (THERP, SPAR-H, CBDT, etc.)
- Decompose HFE into tasks, possibly subtasks / steps
- Analyze PIFs for the lowest decomposition level
- Calculate HEP of every part, combine HEPs for the event Strengths - Transparency; consistency Limitations - Formulaic; loss of context, interactions, non-linearities Task 1 Task 2 Task 3 Subtasks / Task steps Methods
Technique for Human Error Rate Prediction (THERP)
Widely-used HRA method, based on research started in 1976 Task-oriented, focus on rule-based behavior (but also includes a time-reliability correlation for diagnosis)
Task successes and failures represented with HRA event tree Tables used to quantify task success/failure probabilities
- Some empirical basis
- Considerable expert judgment Provides modifiers for dependent actions 17 NUREG/CR-1278 Methods
Human Cognitive Reliability/Operator Reliability Experiment (HCR/ORE)
- Extension of HCR method (which was based on skill/rule/knowledge base categorization of actions)
- Focused on probability of non-response
- Non-response = failure to diagnose OR failure to initiate response in a timely manner
- Normalized correlations for groups of HFEs (human interactions) categorized by cue-response characteristics.
- Analyst estimates median response time and time window; model provides non-response probability.
- Has no floor for very large time margins
Cause-Based Decision Tree (CBDT)
Originally a supplement to HCR/ORE, now a standalone method in the EPRI HRA Calculator Eight decision trees used to develop non-response probabilities, considering multiple PIFs (e.g., training quality, procedures, human-machine interface)
Initial non-response probabilities modified by a time-based recovery factor and added to the probabilities of execution failure 19 1)
Relevant data not available 2)
Data not attended to 3)
Data errors 4)
Data misleading 5)
Procedure steps missed 6)
Misinterpretation of instructions 7)
Errors in interpreting logic 8)
Deliberate violations Methods
Standardized Plant Analysis Risk - HRA (SPAR-H)
Developed to support SPAR models, event and condition assessments Derived from THERP, multiple PIFs (PSFs) aggregated into eight groups based on information processing model 20 1)
Available time 2)
Stress and stressors 3)
Complexity 4)
Experience and training 5)
Procedures (including job aids) 6)
Ergonomics and human-machine interface 7)
Work processes 2-7 2-5,7 2-7 1-4, 6-7 Methods
SPAR-H Worksheets 21 Methods
A Technique for Human Event Analysis (ATHEANA)
- Development started in support of low power and shutdown PRA (different conditions from at-power);
evolved into general method
- Does not use pre-established list of PIFs (PSFs)
- Holistic quantification via expert judgment; emphasizes involvement of knowledgeable plant staff (operations and training) 22 Methods
Integrated Human Event Analysis System (IDHEAS)
Staff response to Commission direction to evaluate the different human reliability models in an effort to propose a single model for the agency to use or guidance on which model(s) should be used in specific circumstances General methodology + application modules At-power Event and condition assessment Decomposition-based, cognitive focus Supported by extensive review of human cognition literature (psychology, cognition, behavioral science, human factors) to identify relevant functions, mechanisms, and factors 23 Methods
Qualitative analysis HEP quantification Define HFE: fail F&B HFE Feasible?
PRA scenario Task analysis 45 min R
R R
Enter FR-H1 Decide F&B Xfr FR-H1 Step 10 Manual Rx Trip Total LOFW E-0 to ES-01 Implement F&B FR-H1 Steps 10-13 1
2 3
4 OK Fail: execution Fail: no decision to establish F&B Fail: no entry to FR-H1 and no F&B 1
2 3
4 5
7 6
8 9
IDHEAS At-Power HEP 1 HEP 2 HEP 3 HEP 4
Context Character a Context Character b Context Character m Tasks Critical Task 1 Critical Task 2 Critical Task K
Failure Modes
Failure Mode 1 Failure Mode 2 Failure Mode N Enter FR-H1 Data Misleading Methods
IDHEAS-G 25 Example Task:
Identify Ruptured SG (as part of an action to isolate the ruptured SG)
Cognitive Activities:
Detect any one of:
unexpected rise in any SG NR level high radiation level from any SG sample high radiation from any SG steamline high radiation from any SG blowdown Understand that any one signal provides indication of the faulted SG. Note:
The HRA-specified context includes successful reactor and turbine trip, energization of all AC buses, SI actuated, AFW available.
The specified context does not explicitly address the possibility of confounding signals and demands (e.g., alarms from unrelated SSCs not modeled in the PRA but demanding operator response.)
Macrocognitive Functions:
Detection Understanding Tasks are accomplished through the performance of various cognitive activities. These cognitive activities exercise general macrocognitive functions.
Methods
IDHEAS-G 26 Example Macrocognitive Function:
Detection Cognitive Process Elements:
Establish mental model Select, identify, attend to information sources Perceive, recognize, classify information Verify, modify detection outcomes Retain, document/record, communicate outcomes Cognitive Mechanisms:
Sensing Perception of sensing stimuli Vigilance maintenance
Capacity Limits:
Mismatch between sensory system and signal Weak signal Reduced vigilance due to sustained cognitive activities Performance Influencing Factors:
Human-system interface Environmental factors Stress, time pressure, and anxiety Mental fatigue
Macrocognitive functions are accomplished through a set of cognitive processes (elements) and cognitive processes are accomplished by cognitive mechanisms.
Performance influencing factors affect how well the cognitive mechanisms are executed by challenging capacity limits for these mechanisms.
Methods
IDHEAS-G 27 Example Task:
Identify Ruptured SG (as part of an action to isolate the ruptured SG)
Macrocognitive Function:
Detection Understanding Proximate Causes:
Failure to perceive information Failure to attend to source of information
Cognitive Mechanisms:
Sensing Perception of sensing stimuli Vigilance maintenance
Performance Influencing Factors:
Human-system interface Environmental factors Stress, time pressure, and anxiety Mental fatigue
- Note: from a systems point of view, a task is modeled as a series system with a very large number of potential single-point failures.
Task failure can be caused by failure of any single cognitive mechanism (which propagates through the cognitive process/macrocognitive function/cognitive activity causality chain).* Each potential failure of a cognitive process is a potential proximate cause for macrocognitive function failure.
Methods
HRA Guidance Many methods and viewpoints, but general agreement on high-level model and good practices NUREG-1792: high-level guidance, e.g.,
- Perform field observations and discussions
- Use screening values during initial quantification
- Account for dependencies among HEPs
- Evaluate the reasonableness of the HEPs NUREG-1842: evaluation of several methods against these good practices Various documents for specific applications, e.g., NUREG-1921 (fire HRA) and NUREG-1921 Supplement 1 (fire HRA, main control room abandonment) 28 Methods
Ispra Benchmark Exercises (1986-1988)
European Commission Joint Research Centre Comparison of methods and modeling 15 teams, multiple methods Test and maintenance
- Failure to detect check valve failure, failure to restore system
- Good agreement on qualitative characterization (key human error interactions and failure mechanisms), divergence on modeling and quantification
- Some variance reduction when using a common model Complicated transient
- LOOP, 2/4 EDGs fail to start, partial CCF of EFW valves
- Differences in modeling (scope of analysis, aggregation) and quantification
- Large method-to-method and team-to-team differences 29 Validation Validation
International HRA Empirical Studies OECD/NEA Halden Reactor Project Comparisons of analysis results with data from HAMMLAB simulator to identify strengths and weaknesses 14 operator crews, 13 HRA teams, blind study Operational transients:
- Steam generator tube rupture (SGTR), loss of feedwater (LOFW)
- Base case and complex, multiple HFEs with varying difficulty Findings include:
- Large variations in how crews followed procedures
- Large variations in HEPs; many rankings dont reflect difficulty
- Some analyses dont strongly differentiate across HFEs
- Methods that emphasize mechanisms and contextual factors provide richer (and often predictive) narratives, but not necessarily better HEPs 30 Validation
Study Process 31 NUREG-2127 Validation Challenges Include:
Differences between HAMMLAB simulator and home plant Characterizing crew behaviors (e.g., drivers for performance)
Statistically small sample Defining failure for intermediate HFEs
US HRA Empirical Studies Similar to international study but using a US PWR (simulator and crews).
Also addressed concerns regarding Lack of testing of team-to-team variability in using the same method Inability of analysis teams to visit simulator, interview crews 4 crews, 9 HRA teams Operational transients:
LOFW followed by SGTR Loss of component cooling water and RCP seal water SGTR Findings include:
Less variability vs. HAMMLAB study and Ispra: HRA team learning? Better practiced with US crews? Plant visit?
Qualitative analyses can be improved HRA improvements should focus on aiding analysts finding and characterizing contextual factors and mechanisms causing cognitive failures 32 Validation
Comparing Predictions with Performance 33 NUREG-2127 NUREG-2156 Validation
Technical Challenges
- Complicating factors
- Specific conditions (e.g., pre-accident conditions including problematic components; specific crew on shift including makeup crews)
- Scenario dynamics (e.g., mindset established by specific evolution, shift changes, multiple system shocks, changes in local environment, external directions)
- Additional crew concerns (e.g., economic impact of action, offsite environment)
- Social behaviors and relationships (e.g., trust within crew, between organizational elements, group behavior) 34 Challenges
Technical Challenges (cont.)
Data from actual incidents
- Statistically sparse, arguably unique characteristics for each event
- Extremely rich qualitative information for a few events Data from other simulator exercises: transferability to HRA/PRA
- Design and operational differences
- Data collection protocols Technology advances affecting human performance
- Advanced control rooms
- Smart/distributed technology
- Remote operations 35 Challenges
Socio-Organizational Challenges
- Multiple technical disciplines with varying goals, views on the meaningfulness of a PRA-oriented HRA, views on needed rigor
- Interdisciplinary trust
- HRA developers: academic/professional reward system =>
proliferation of HRA methods
- PRA analysts: need for now answers => development of good enough methods, resistance to change
- PRA users: discomfort with large uncertainties =>
dismissal/discounting of results and insights
- Science critics: weaknesses in current methods/models => house of cards view on PRA and RIDM affecting willingness to help 36 Challenges
Grand Challenge - Incorporating Organizational Factors Long-recognized as an important influence
- Culture and climate
- Resources
- Direct involvement in events Scope >> current PRA scope
- Time
- Organizations (functions and structure)
- Space
- Technical disciplines Data
- Availability
- Quality Non-monotonic effects 37 Challenges
Non-Monotonic Effects: Examples
- Good safety culture can reduce worker risk but increase plant-level risk
- Pre-emptive reactor trip on loss of communications with diver
- Reluctance to send workers to hazardous areas
- Forceful leadership can overcome organizational inertia but can also stifle important views 38 Challenges