ML19011A431: Difference between revisions

From kanterella
Jump to navigation Jump to search
(Created page by program invented by StriderTol)
 
(Created page by program invented by StriderTol)
Line 16: Line 16:


=Text=
=Text=
{{#Wiki_filter:Human Reliability AnalysisLecture 5-21 Key TopicsHRA importanceGeneral descriptionFundamental modelMethodsValidationChallenges2Overview ResourcesA. KolaczkowskiNUREG-1792, April 2005.NUREG-1842, September 2006.Learned from Comparing HRA Methods Predictions to HAMMLAB NUREG-2127, August 2014.Method Predictions against Operating Crew Performance on a U.S. NUREG-2156, June 2016.NUREG-2114, January 2016.3Overview Other ReferencesA. PoucetEUR 1222 EN, IspraJoint Research Centre, Commission of European Communities, August 1989.Phase 1 Report: Description of Overall Approach and Pilot Phase Results from Comparing HRA Methods to NUREG/IA-0216, Vol. 1, November 2009.Phase 2 Report: Results from NUREG/IA-0216, Vol. 2, August 2011.Phase 3 Report: Results from Comparing HRA Methods Predictions to HAMMLAB Simulator Data on LOFW NUREG/IA-0216, Vol. 3, December 2014.H. Blackman, N. Siu, and A. Mosleh, Human Reliability Models: Theoretical and Practical Challenges, Center for Reliability Engineering, University of Maryland, College Park, MD, 1998.4Overview Other ReferencesNUREG/CR-1278, August 1983.P. Moieni-Proceedings ANS , Clearwater Beach, FL, January 26-29, 1993.D. Gertman-NUREG/CR-6883, August 2005.NUREG-1624, Rev. 1, May 2000.Internal Events At-PowerNUREG-2199, Vol. 1, March 2017.Proceedings International Conference on Probabilistic Safety Assessment and Management (PSAM 13), Seoul, Korea, October 2-7, 2016.-EPRI 1023001/NUREG-1921-RES Fire Human Reliability Analysis Guidelines EPRI 3002009215/NUREG-1921, Supplement 1, August 2017.5Overview Human Actions and NPP PRAOperational decisions and actions have played an important role in every major NPP accident and incidentOccurrence and progressionSuccesses and failures6human contributions are not useful (for most applications)HRA Importance Example EventsBrowns Ferry 1 & 2 cable fire (1975)Worker ignites polyurethane foam, starts cable fireFire suppression delayed 7+ hours (reluctant to use water)Operators achieve safe shutdown using non-safety systemDavis-Besseloss of feedwater (1985)Operator error causes loss of feedwaterMultiple malfunctions => feed and bleed cooling directed by procedures, would have major economic consequencesShift supervisor chooses to wait for recovery of AFW (which is successful)Fukushima Dai-ichiUnit 1 (2011)Operators close isolation condenser (little effect given accident conditions)Operators perform numerous non-proceduralizedactions (e.g., scavenge car batteries to supply power) in attempts to save plantEx-control room actions hampered by site conditions (tsunami alerts, 7HRA Importance What is HRA?to identify potential human failure events and to systematically estimate the probability of those events -2122)Terminology used to emphasize connection with NPP PRA time available is insufficient)Can be included at scenario level (event trees) or system level (fault trees)8General Description HRA General ProcessActivitiesQualitative analysisModelingQuantificationSupports overall model constructionInitiating event identificationAccident scenario modelingSystems modelingNot just a quantification activity9General Description HRA Dimensions and DescriptorsTimePre-initiatorInitiatorPost-initiatorSpaceWithin control roomOutside control roomOrganizationControl room crewField operatorsEmergency response organizationImplicitActions addressed by other PRA model elements (e.g., initiating event frequencies, loss of offsite power recovery, common cause failure probabilities)Pre-initiator decisions affecting fundamental plant design (e.g., flood barrier height) and operations (e.g., resources for training)Out-of-scope for NPP PRASabotageTerrorism10General Description Typical HFE Level of DetailMacro-level crew actions, e.g.,Isolate faulted steam generatorInitiate bleed and feed coolingRecover a failed pumpMicro-level modeling (e.g., put control switch X in pull-to-lock position) can support HFE; need to consider micro-level recoveries as well as failures11General Description Task-oriented viewDiagnosis and PlanningActionCognitive viewDetecting/NoticingSensemaking/UnderstandingDecision MakingAction ExecutionTeamwork (communication/coordination)12Macrocognitivefunctions(NUREG-2114)Fundamental Model Naturalistic Decision Making13From NUREG-2114, per F.L. GreitzerInternational Journal of Human-Computer Interaction, 26(2-3), 278-291, 2010. doi:10.1080/10447310903499070Fundamental Model How Things Can Fail14Real-world contextual elements and PIFs* can include:Specific conditions (e.g., problematic components, mixed crews)Scenario dynamics (e.g., shift changes, multiple system shocks)Economic concernsSocial behaviors and relationshipsFundamental Model Human Error Probability (HEP)Quantifies aleatory uncertaintyIs subject to epistemic uncertaintiesIs a function of the task, the scenario context leading up to the task, and the relevant PIFsUnderlying assertion: human actions are predictable (in a probabilistic sense)Performance of specific tasks, often with specific procedures and trainingBounded rationality: operators/staff are trying to do the right thingNote: HEP functional behavior on PIFs is usually assumed to be multiplicative, but other data might support additivityFundamental Probabilistic Model15Fundamental Model HRA Approaches16Holistic Analysis(ATHEANA, MERMOS)HFEAnalyze context and develop operational story / narrativeIdentify situations deviating from the base story that lead to undesired actionsEstimate the HEPs of the deviationsPIFsStrengths Preserves context; uses expert ability to integrate complex informationLimitations Level of effort; subjectivity and variabilityHFETasks + Context (plant situation, scenario, and crew factors)Decomposition-Based Analysis(THERP, SPAR-H, CBDT, etc.)Decompose HFE into tasks, possibly subtasks / stepsAnalyze PIFs for the lowest decomposition level Calculate HEP of every part, combine HEPs for the eventStrengths Transparency; consistencyLimitations Formulaic; loss of context, interactions, non-linearitiesTask 1Task 2 Task 3Subtasks / Task steps Methods Technique for Human Error Rate Prediction (THERP)Widely-used HRA method, based on research started in 1976Task-oriented, focus on rule-based behavior (but also includes a time-reliability correlation for diagnosis)Task successes and failures represented with HRA event treeTables used to quantify task success/failure probabilitiesSome empirical basisConsiderable expert judgmentProvides modifiers for dependent actions17NUREG/CR-1278Methods Human Cognitive Reliability/Operator Reliability Experiment (HCR/ORE)Extension of HCR method (which was based on skill/rule/knowledge base categorization of actions)Focused on probability of non-responseNon-response = failure to diagnose OR failure to initiate response in a timely manner-response characteristics.Analyst estimates median response time and time window; model provides non-response probability.Included in EPRI HRA Calculator18Methods Cause-Based Decision Tree (CBDT)Originally a supplement to HCR/ORE, now a standalone method in the EPRI HRA CalculatorEight decision trees used to develop non-response probabilities, considering multiple PIFs (e.g., training quality, procedures, human-machine interface)Initial non-response probabilities modified by a time-based recovery factor and added to the probabilities of execution failure191)Relevant data not available2)Data not attended to 3)Data errors 4)Data misleading5)Procedure steps missed6)Misinterpretation of instructions7)Errors in interpreting logic8)Deliberate violationsMethods Standardized Plant Analysis Risk HRA (SPAR-H)Developed to support SPAR models, event and condition assessmentsDerived from THERP, multiple PIFs (PSFs) aggregated into eight groups based on information processing model201)Available time2)Stress and stressors3)Complexity4)Experience and training5)Procedures (including job aids)6)Ergonomics and human-machine interface7)Fitness for duty8)Work processes2-72-5,72-71-4,6-7Methods SPAR-H Worksheets21Methods A Technique for Human Event Analysis (ATHEANA)Development started in support of low power and shutdown PRA (different conditions from at-power); evolved into general methodFocuses on HFE context, identification of error-forcing conditions (EFCs)Does not use pre-established list of PIFs (PSFs)Holistic quantification via expert judgment; emphasizes involvement of knowledgeable plant staff (operations and training)22Methods Integrated Human Event Analysis System (IDHEAS)reliability models in an effort to propose a single model for the agency to use or guidance on which model(s) should be used in specific General methodology + application modulesAt-powerEvent and condition assessmentDecomposition-based, cognitive focusSupported by extensive review of human cognition literature (psychology, cognition, behavioral science, human factors) to identify relevant functions, mechanisms, and factors23Methods Qualitative analysis HEP quantificationDefine HFE: fail F&BHFE Feasible?PRA scenario Task analysis45 minRRREnterFR-H1Decide F&BXfrFR-H1Step10Manual Rx TripTotalLOFWE-0 toES-01ImplementF&BFR-H1Steps10-131234OKFail: executionFail: no decisionto establish F&BFail: no entry to FR-H1and no F&B123457689IDHEAS At-PowerHEP 1HEP 2HEP 3HEP 4ContextCharacter aContextCharacter bContextCharacter mTasksCritical Task 1Critical Task 2Critical Task KFailure ModesFailure Mode 1Failure Mode 2Failure Mode NEnter FR-H1Data MisleadingMethods IDHEAS-G25ExampleTask:Identify Ruptured SG (as part of an action to isolate the ruptured SG)Cognitive Activities:Detect any one of:unexpected rise in any SG NR levelhigh radiation level from any SG samplehigh radiation from any SG steamlinehigh radiation from any SG blowdownUnderstand that any one signal provides indication of the faulted SG. Note:The HRA-specified context includes successful reactor and turbine trip, energization of all AC buses, SI actuated, AFW available.The specified context does not explicitly address the possibility of confounding signals and demands (e.g., alarms from unrelated SSCs not modeled in the PRA but demanding operator response.)Macrocognitive Functions: DetectionUnderstandingTasks are accomplished through the performance of various cognitive activities. These cognitive activities exercise general macrocognitivefunctions.Methods IDHEAS-G26ExampleMacrocognitive Function: DetectionCognitive Process Elements:Establish mental modelSelect, identify, attend to information sourcesPerceive, recognize, classify informationVerify, modify detection outcomesRetain, document/record, communicate outcomesCognitive Mechanisms:SensingPerception of sensing stimuliVigilance maintenanceMismatch between sensory system and signalWeak signalReduced vigilance due to sustained cognitive activitiesPerformance Influencing Factors:Human-system interfaceEnvironmental factorsStress, time pressure, and anxietyMental fatigueMacrocognitivefunctions are accomplished through a processes are accomplished by cognitive mechanisms. Performance influencing factors affect how well the cognitive mechanisms are executed by challenging Methods IDHEAS-G27ExampleTask:Identify Ruptured SG (as part of an action to isolate the ruptured SG)Macrocognitive Function: DetectionUnderstandingProximate Causes:Failure to perceive informationFailure to attend to source of informationCognitive Mechanisms:SensingPerception of sensing stimuliVigilance maintenancePerformance Influencing Factors:Human-system interfaceEnvironmental factorsStress, time pressure, and anxietyMental fatigue*Note: from a systems point of view, a task is modeled as a series system with a very large number of potential single-point failures.Task failure can be caused by failure of any single cognitive mechanism (which propagates through the cognitive process/macrocognitivefunction/cognitive activity causality chain).* Each potential failure of a macrocognitivefunction failure.Methods HRA GuidanceMany methods and viewpoints, but general agreement on high-NUREG-1792: high-level guidance, e.g.,Perform field observations and discussionsUse screening values during initial quantificationAccount for dependencies among HEPsEvaluate the reasonableness of the HEPsNUREG-1842: evaluation of several methods against these good practicesVarious documents for specific applications, e.g., NUREG-1921 (fire HRA) and NUREG-1921 Supplement 1 (fire HRA, main control room abandonment)28Methods IspraBenchmark Exercises (1986-1988)European Commission Joint Research CentreComparison of methods and modeling15 teams, multiple methodsTest and maintenanceFailure to detect check valve failure, failure to restore systemGood agreement on qualitative characterization (key human error interactions and failure mechanisms), divergence on modeling and quantificationSome variance reduction when using a common modelComplicated transientLOOP, 2/4 EDGs fail to start, partial CCF of EFW valvesDifferences in modeling (scope of analysis, aggregation) and quantificationLarge method-to-method and team-to-team differences29ValidationValidation International HRA Empirical StudiesOECD/NEA HaldenReactor ProjectComparisons of analysis results with data from HAMMLAB simulator to identify strengths and weaknesses14 operator crews, 13 HRA teamsOperational transients:Steam generator tube rupture (SGTR), loss of feedwater (LOFW)Base case and complex, multiple HFEs with varying difficultyFindings include:Large variations in how crews followed proceduresdifficultyMethods that emphasize mechanisms and contextual factors provide richer (and often predictive) narratives, but not necessarily better HEPs30Validation Study Process31NUREG-2127ValidationChallenges Include:Differences between HAMMLAB simulator and home plantCharacterizing crew behaviors (e.g., drivers for performance)Statistically small sampleintermediate HFEs US HRA Empirical StudiesSimilar to international study but using a US PWR (simulator and crews). Also addressed concerns regardingLack of testing of team-to-team variability in using the same methodInability of analysis teams to visit simulator, interview crews4crews, 9 HRA teamsOperational transients: LOFW followed by SGTRLoss of component cooling water and RCP seal waterSGTRFindings include:Less variability vs. HAMMLAB study and Ispra:HRA team learning? Better practiced with US crews? Plant visit?Qualitative analyses can be improvedHRA improvements should focus on aiding analysts finding and characterizing contextual factors and mechanisms causing cognitive failures32Validation Comparing Predictions with Performance33NUREG-2127NUREG-2156Validation Technical ChallengesComplicating factorsSpecific conditions (e.g., pre-accident conditions including problematic components; specific crew on shift including makeup crews)Scenario dynamics (e.g., mindset established by specific evolution, shift changes, multiple system shocks, changes in local environment, external directions)Additional crew concerns (e.g., economic impact of action, offsite environment)Social behaviors and relationships (e.g., trust within crew, between organizational elements, group behavior)34Challenges Technical Challenges (cont.)Data from actual incidentsStatistically sparse, arguably unique characteristics for each eventExtremely rich qualitative information for a few eventsData from other simulator exercises: transferability to HRA/PRADesign and operational differencesData collection protocolsTechnology advances affecting human performanceAdvanced control roomsSmart/distributed technologyRemote operations35Challenges Socio-Organizational ChallengesMultiple technical disciplines with varying goals, views on the meaningfulness of a PRA-oriented HRA, views on needed rigorInterdisciplinary trustHRA developers: academic/professional reward system => proliferation of HRA methodsPRA users: discomfort with large uncertainties => dismissal/discounting of results and insights36Challenges Grand Challenge Incorporating Organizational FactorsLong-recognized as an important influenceCulture and climateResourcesDirect involvement in eventsScope >> current PRA scopeTimeOrganizations (functions and structure)SpaceTechnical disciplinesDataAvailabilityQualityNon-monotonic effects37Challenges Non-Monotonic Effects: ExamplesGood safety culture can reduce worker risk but increase plant-level riskPre-emptive reactor trip on loss of communications with diverReluctance to send workers to hazardous areasinertia but can also stifle important views38Challenges}}
{{#Wiki_filter:Human Reliability Analysis Lecture 5-2 1
 
Overview Key Topics
* HRA importance
* General description
* Fundamental model
* Methods
* Validation
* Challenges 2
 
Overview Resources
* A. Kolaczkowski, et al., Good Practices for Implementing Human Reliability Analysis, NUREG-1792, April 2005.
* J. Forester, et al., Evaluation of Human Reliability Analysis Methods Against Good Practices, NUREG-1842, September 2006.
* J. Forester, et al., The International HRA Empirical Study: Lessons Learned from Comparing HRA Methods Predictions to HAMMLAB Simulator Data, NUREG-2127, August 2014.
* J. Forester, et al., The U.S. HRA Empirical Study: Assessment of HRA Method Predictions against Operating Crew Performance on a U.S.
Nuclear Plant Simulator, NUREG-2156, June 2016.
* A.M. Whaley, et al., Cognitive Basis for Human Reliability Analysis, NUREG-2114, January 2016.
3
 
Overview Other References
* A. Poucet, Human Factors Reliability Benchmark Exercise: Synthesis Report, EUR 1222 EN, Ispra Joint Research Centre, Commission of European Communities, August 1989.
* E. Lois, et al., International HRA Empirical Study - Phase 1 Report: Description of Overall Approach and Pilot Phase Results from Comparing HRA Methods to Simulator Performance Data, NUREG/IA-0216, Vol. 1, November 2009.
* A. Bye, et al., International HRA Empirical Study - Phase 2 Report: Results from Comparing HRA Method Predictions to Simulator Data from SGTR Scenarios, NUREG/IA-0216, Vol. 2, August 2011.
* V.N. Dang, et al., International HRA Empirical Study - Phase 3 Report: Results from Comparing HRA Methods Predictions to HAMMLAB Simulator Data on LOFW Scenarios, NUREG/IA-0216, Vol. 3, December 2014.
* H. Blackman, N. Siu, and A. Mosleh, Human Reliability Models: Theoretical and Practical Challenges, Center for Reliability Engineering, University of Maryland, College Park, MD, 1998.
4
 
Overview Other References
* A. D. Swain and H.E. Guttmann, Handbook of Human Reliability Analysis with Emphasis on Nuclear Power Plant Applications: Final Report, NUREG/CR-1278, August 1983.
* P. Moieni, et al., A PC-based human reliability analysis (HRA) software, Proceedings ANS International Topical Meeting on Probabilistic Safety Assessment (PSA 93), Clearwater Beach, FL, January 26-29, 1993.
* D. Gertman, et al., The SPAR-H Human Reliability Analysis Method, NUREG/CR-6883, August 2005.
* U.S. Nuclear Regulatory Commission, Technical Basis and Implementation Guidelines for A Technique for Human Event Analysis (ATHEANA), NUREG-1624, Rev. 1, May 2000.
* J. Xing, et al., An Integrated Human Event Analysis System (IDHEAS) for Nuclear Power Plant Internal Events At-Power Application, NUREG-2199, Vol. 1, March 2017.
* Y.J. Chang and J. Xing, The general methodology of an Integrated Human Event Analysis System (IDHEAS) for human reliability analysis method development, Proceedings International Conference on Probabilistic Safety Assessment and Management (PSAM 13), Seoul, Korea, October 2-7, 2016.
* S. Lewis and S. Cooper, EPRI/NRC-RES Fire Human Reliability Analysis Guidelines: Final Report, EPRI 1023001/NUREG-1921, July 2012.
* A. Lindeman and S. Cooper, EPRI/NRC-RES Fire Human Reliability Analysis Guidelines -
Qualitative Analysis for Main Control Room Abandonment Scenarios, Supplement 1, EPRI 3002009215/NUREG-1921, Supplement 1, August 2017.
5
 
HRA Importance Human Actions and NPP PRA Operational decisions and actions have played an important role in every major NPP accident and incident
* Occurrence and progression
* Successes and failures PRAs that dont account for human contributions are not useful (for most applications) 6
 
HRA Importance Example Events
* Browns Ferry 1 & 2 cable fire (1975)
  - Worker ignites polyurethane foam, starts cable fire
  - Fire suppression delayed 7+ hours (reluctant to use water)
  - Operators achieve safe shutdown using non-safety system
* Davis-Besse loss of feedwater (1985)
  - Operator error causes loss of feedwater
  - Multiple malfunctions => feed and bleed cooling directed by procedures, would have major economic consequences
  - Shift supervisor chooses to wait for recovery of AFW (which is successful)
* Fukushima Dai-ichi Unit 1 (2011)
  - Operators close isolation condenser (little effect given accident conditions)
  - Operators perform numerous non-proceduralized actions (e.g., scavenge car batteries to supply power) in attempts to save plant
  - Ex-control room actions hampered by site conditions (tsunami alerts, aftershocks, damage, dark, radiation, )
7
 
General Description What is HRA?
* In the context of NPP PRA: A structured approach used to identify potential human failure events and to systematically estimate the probability of those events using data, models, or expert judgment (NUREG-2122)
* Human Failure Event: interface with rest of PRA model:
  - Terminology used to emphasize connection with NPP PRA model (basic events), avoid connotation of blame (e.g., when time available is insufficient)
  - Includes errors of omission, errors of commission
  - Can be included at scenario level (event trees) or system level (fault trees) 8
 
General Description HRA General Process
* Activities
  - Qualitative analysis
  - Modeling
  - Quantification
* Supports overall model construction
  - Initiating event identification
  - Accident scenario modeling
  - Systems modeling
* Not just a quantification activity 9
 
General Description HRA Dimensions and Descriptors
* Time
* Implicit
  - Pre-initiator          - Actions addressed by other PRA
  - Initiator                model elements (e.g., initiating
  - Post-initiator            event frequencies, loss of offsite power recovery, common cause
* Space                        failure probabilities)
  - Within control room    - Pre-initiator decisions affecting
  - Outside control room      fundamental plant design (e.g.,
flood barrier height) and
* Organization                operations (e.g., resources for
  - Control room crew         training)
  - Field operators
* Out-of-scope for NPP PRA
  - Emergency response
                            - Sabotage organization
                            - Terrorism 10
 
General Description Typical HFE Level of Detail
* Macro-level crew actions, e.g.,
  - Isolate faulted steam generator
  - Initiate bleed and feed cooling
  - Recover a failed pump
* Micro-level modeling (e.g., put control switch X in pull-to-lock position) can support HFE; need to consider micro-level recoveries as well as failures 11
 
Fundamental Model How Things Work
* Task-oriented view
  - Diagnosis and Planning
  - Action
* Cognitive view
  - Detecting/Noticing
  - Sensemaking/Understanding
  - Decision Making
  - Action Execution
  - Teamwork (communication/coordination) Macrocognitive functions (NUREG-2114) 12
 
Fundamental Model Naturalistic Decision Making From NUREG-2114, per F.L. Greitzer, et al., Naturalistic decision making for power system operators, International Journal of Human-Computer Interaction, 26(2-3), 278-291, 2010. doi:10.1080/10447310903499070 13
 
Fundamental Model How Things Can Fail Real-world contextual elements and PIFs* can include:
* Specific conditions (e.g., problematic components, mixed crews)
* Scenario dynamics (e.g., shift changes, multiple system shocks)
* Economic concerns
* Social behaviors and relationships
*Usually referred to as Performance Shaping Factors (PSFs) 14
 
Fundamental Model Fundamental Probabilistic Model
* Human Error Probability (HEP)
  - Quantifies aleatory uncertainty
  - Is subject to epistemic uncertainties
  - Is a function of the task, the scenario context leading up to the task, and the relevant PIFs
        =  ,  ,
* Underlying assertion: human actions are predictable (in a probabilistic sense)
  - Performance of specific tasks, often with specific procedures and training
  - Bounded rationality: operators/staff are trying to do the right thing
* Note: HEP functional behavior on PIFs is usually assumed to be multiplicative, but other data might support additivity 15
 
Methods HRA Approaches Holistic Analysis                            Decomposition-Based Analysis (ATHEANA, MERMOS)                             (THERP, SPAR-H, CBDT, etc.)
Tasks + Context (plant situation, scenario,                        HFE and crew factors)
Task 1          Task 2          Task 3 HFE PIFs             Subtasks / Task steps
* Analyze context and develop operational
* Decompose HFE into tasks, possibly story / narrative                                subtasks / steps
* Identify situations deviating from the
* Analyze PIFs for the lowest base story that lead to undesired actions        decomposition level
* Estimate the HEPs of the deviations
* Calculate HEP of every part, combine HEPs for the event Strengths - Preserves context; uses expert        Strengths - Transparency; consistency ability to integrate complex information          Limitations - Formulaic; loss of context, Limitations - Level of effort; subjectivity and  interactions, non-linearities variability                                                                              16
 
Methods Technique for Human Error Rate Prediction (THERP)
* Widely-used HRA method, based on research started in 1976
* Task-oriented, focus on rule-based behavior (but also includes a time-reliability correlation for diagnosis)
* Task successes and failures represented with HRA event tree
* Tables used to quantify task success/failure probabilities
  - Some empirical basis                NUREG/CR-1278
  - Considerable expert judgment
* Provides modifiers for dependent actions 17
 
Methods Human Cognitive Reliability/Operator Reliability Experiment (HCR/ORE)
* Extension of HCR method (which was based on skill/rule/knowledge base categorization of actions)
* Focused on probability of non-response
  - Non-response = failure to diagnose OR failure to initiate response in a timely manner
  - Normalized correlations for groups of HFEs (human interactions) categorized by cue-response characteristics.
  - Analyst estimates median response time and time window; model provides non-response probability.
  - Has no floor for very large time margins
* Included in EPRI HRA Calculator 18
 
Methods Cause-Based Decision Tree (CBDT)
* Originally a supplement to HCR/ORE, now a standalone method in the EPRI HRA Calculator
* Eight decision trees used to develop non-response probabilities, considering multiple PIFs (e.g., training quality, procedures, human-machine interface)
: 1)  Relevant data not available    5)  Procedure steps missed
: 2)  Data not attended to          6)  Misinterpretation of instructions
: 3)  Data errors                    7)  Errors in interpreting logic
: 4)  Data misleading                8)  Deliberate violations
* Initial non-response probabilities modified by a time-based recovery factor and added to the probabilities of execution failure 19
 
Methods Standardized Plant Analysis Risk - HRA (SPAR-H)
* Developed to support SPAR models, event and condition assessments
* Derived from THERP, multiple PIFs (PSFs) aggregated into eight groups based on information processing model 2-7                                            2-5,7 1-4, 6-7 2-7
: 1)  Available time          5)  Procedures (including job aids)
: 2) Stress and stressors    6)  Ergonomics and human-machine interface
: 3)  Complexity              7)  Fitness for duty
: 4)  Experience and training  8)  Work processes                      20
 
Methods SPAR-H Worksheets 21
 
Methods A Technique for Human Event Analysis (ATHEANA)
* Development started in support of low power and shutdown PRA (different conditions from at-power);
evolved into general method
* Focuses on HFE context, identification of error-forcing conditions (EFCs)
* Does not use pre-established list of PIFs (PSFs)
* Holistic quantification via expert judgment; emphasizes involvement of knowledgeable plant staff (operations and training) 22
 
Methods Integrated Human Event Analysis System (IDHEAS)
* Staff response to Commission direction to evaluate the different human reliability models in an effort to propose a single model for the agency to use or guidance on which model(s) should be used in specific circumstances
* General methodology + application modules
  - At-power
  - Event and condition assessment
* Decomposition-based, cognitive focus
* Supported by extensive review of human cognition literature (psychology, cognition, behavioral science, human factors) to identify relevant functions, mechanisms, and factors 23
 
Methods IDHEAS At-Power Qualitative analysis PRA scenario            Total Manual          E-0 to   Enter Decide F&B Implement Xfr FR-H1 F&B FR-H1 LOFW Rx Trip          ES-01    FR-H1    Step 10  Steps 10-13 1      2              3      4              5          6                1 OK Define HFE: fail F&B                  45 min R          R              R 8          9 2 Fail: execution HFE                Task analysis                                                                         Fail: no decision Feasible?                                                                                          3 to establish F&B 7
Fail: no entry to FR-H1 4
and no F&B HEP quantification Tasks                  Failure Modes                                    Context Character a Context Character b Context Character m HEP 1 HEP 2 Enter FR-H1 Critical Task 1            Failure Mode 1 HEP 3 HEP 4 Critical Task 2            Data Misleading Failure Mode 2                                                                           
 
Critical Task K            Failure Mode N
 
Methods IDHEAS-G Tasks are accomplished through the performance of various cognitive activities. These cognitive activities exercise general macrocognitive functions.
Example Task:
Identify Ruptured SG (as part of an action to isolate the ruptured SG)
Cognitive Activities:
* Detect any one of:
* unexpected rise in any SG NR level
* high radiation level from any SG sample
* high radiation from any SG steamline
* high radiation from any SG blowdown
* Understand that any one signal provides indication of the faulted SG. Note:
* The HRA-specified context includes successful reactor and turbine trip, energization of all AC buses, SI actuated, AFW available.
* The specified context does not explicitly address the possibility of confounding signals and demands (e.g., alarms from unrelated SSCs not modeled in the PRA but demanding operator response.)
Macrocognitive Functions:
* Detection
* Understanding 25
 
Methods IDHEAS-G Macrocognitive functions are accomplished through a Example                                set of cognitive processes (elements) and cognitive Macrocognitive Function:
processes are accomplished by cognitive mechanisms.
* Detection                                          Performance influencing factors affect how well the Cognitive Process Elements:                         cognitive mechanisms are executed by challenging
* Establish mental model                            capacity limits for these mechanisms.
* Select, identify, attend to information sources
* Perceive, recognize, classify information
* Verify, modify detection outcomes
* Retain, document/record, communicate outcomes Cognitive Mechanisms:
* Sensing
* Perception of sensing stimuli
* Vigilance maintenance
* Capacity Limits:
* Mismatch between sensory system and signal
* Weak signal
* Reduced vigilance due to sustained cognitive activities Performance Influencing Factors:
* Human-system interface
* Environmental factors
* Stress, time pressure, and anxiety
* Mental fatigue
* 26
 
Methods IDHEAS-G Task failure can be caused by failure of any single cognitive mechanism (which propagates through the Example                          cognitive process/macrocognitive function/cognitive Task:
activity causality chain).* Each potential failure of a Identify Ruptured SG (as part of an action to isolate the ruptured SG) cognitive process is a potential proximate cause for macrocognitive function failure.
Macrocognitive Function:
* Detection
* Understanding Proximate Causes:
* Failure to perceive information
* Failure to attend to source of information
* Cognitive Mechanisms:
* Sensing
* Perception of sensing stimuli
* Vigilance maintenance
* Performance Influencing Factors:
* Human-system interface
* Environmental factors
* Stress, time pressure, and anxiety
* Mental fatigue
*
*Note: from a systems point of view, a task is modeled as a series system with a very large number of potential single-point failures.
27
 
Methods HRA Guidance
* Many methods and viewpoints, but general agreement on high-level model and good practices
* NUREG-1792: high-level guidance, e.g.,
  -  Perform field observations and discussions
  -  Use screening values during initial quantification
  -  Account for dependencies among HEPs
  -  Evaluate the reasonableness of the HEPs
* NUREG-1842: evaluation of several methods against these good practices
* Various documents for specific applications, e.g., NUREG-1921 (fire HRA) and NUREG-1921 Supplement 1 (fire HRA, main control room abandonment) 28
 
Validation Ispra Benchmark Exercises (1986-1988)
* European Commission Joint Research Centre
* Comparison of methods and modeling
* 15 teams, multiple methods                            Validation
* Test and maintenance
  - Failure to detect check valve failure, failure to restore system
  - Good agreement on qualitative characterization (key human error interactions and failure mechanisms), divergence on modeling and quantification
  - Some variance reduction when using a common model
* Complicated transient
  - LOOP, 2/4 EDGs fail to start, partial CCF of EFW valves
  - Differences in modeling (scope of analysis, aggregation) and quantification
  - Large method-to-method and team-to-team differences 29
 
Validation International HRA Empirical Studies
* OECD/NEA Halden Reactor Project
* Comparisons of analysis results with data from HAMMLAB simulator to identify strengths and weaknesses
* 14 operator crews, 13 HRA teams, blind study
* Operational transients:
  - Steam generator tube rupture (SGTR), loss of feedwater (LOFW)
  - Base case and complex, multiple HFEs with varying difficulty
* Findings include:
  - Large variations in how crews followed procedures
  - Large variations in HEPs; many rankings dont reflect difficulty
  - Some analyses dont strongly differentiate across HFEs
  - Methods that emphasize mechanisms and contextual factors provide richer (and often predictive) narratives, but not necessarily better HEPs 30
 
Validation Study Process Challenges Include:
* Differences between HAMMLAB simulator and home plant
* Characterizing crew behaviors (e.g., drivers for performance)
* Statistically small sample
* Defining failure for intermediate HFEs NUREG-2127 31
 
Validation US HRA Empirical Studies
* Similar to international study but using a US PWR (simulator and crews).
Also addressed concerns regarding
  - Lack of testing of team-to-team variability in using the same method
  - Inability of analysis teams to visit simulator, interview crews
* 4 crews, 9 HRA teams
* Operational transients:
  - LOFW followed by SGTR
  - Loss of component cooling water and RCP seal water
  - SGTR
* Findings include:
  - Less variability vs. HAMMLAB study and Ispra: HRA team learning? Better practiced with US crews? Plant visit?
  - Qualitative analyses can be improved
  - HRA improvements should focus on aiding analysts finding and characterizing contextual factors and mechanisms causing cognitive failures 32
 
Validation Comparing Predictions with Performance NUREG-2127              NUREG-2156 33
 
Challenges Technical Challenges
* Complicating factors
  - Specific conditions (e.g., pre-accident conditions including problematic components; specific crew on shift including makeup crews)
  - Scenario dynamics (e.g., mindset established by specific evolution, shift changes, multiple system shocks, changes in local environment, external directions)
  - Additional crew concerns (e.g., economic impact of action, offsite environment)
  - Social behaviors and relationships (e.g., trust within crew, between organizational elements, group behavior) 34
 
Challenges Technical Challenges (cont.)
* Data from actual incidents
  - Statistically sparse, arguably unique characteristics for each event
  - Extremely rich qualitative information for a few events
* Data from other simulator exercises: transferability to HRA/PRA
  - Design and operational differences
  - Data collection protocols
* Technology advances affecting human performance
  - Advanced control rooms
  - Smart/distributed technology
  - Remote operations
  -
35
 
Challenges Socio-Organizational Challenges
* Multiple technical disciplines with varying goals, views on the meaningfulness of a PRA-oriented HRA, views on needed rigor
* Interdisciplinary trust
  - HRA developers: academic/professional reward system =>
proliferation of HRA methods
  - PRA analysts: need for now answers => development of good enough methods, resistance to change
  - PRA users: discomfort with large uncertainties =>
dismissal/discounting of results and insights
  - Science critics: weaknesses in current methods/models => house of cards view on PRA and RIDM affecting willingness to help 36
 
Challenges Grand Challenge - Incorporating Organizational Factors
* Long-recognized as an important influence
  - Culture and climate
  - Resources
  - Direct involvement in events
* Scope >> current PRA scope
  - Time
  - Organizations (functions and structure)
  - Space
  - Technical disciplines
* Data
  - Availability
  - Quality
* Non-monotonic effects 37
 
Challenges Non-Monotonic Effects: Examples
* Good safety culture can reduce worker risk but increase plant-level risk
  - Pre-emptive reactor trip on loss of communications with diver
  - Reluctance to send workers to hazardous areas
* Forceful leadership can overcome organizational inertia but can also stifle important views 38}}

Revision as of 07:54, 20 October 2019

Lecture 5-2 HRA 2019-01-18
ML19011A431
Person / Time
Issue date: 01/16/2019
From:
Office of Nuclear Regulatory Research
To:
Nathan Siu 415-0744
Shared Package
ML19011A416 List:
References
Download: ML19011A431 (38)


Text

Human Reliability Analysis Lecture 5-2 1

Overview Key Topics

  • General description
  • Fundamental model
  • Methods
  • Validation
  • Challenges 2

Overview Resources

  • A. Kolaczkowski, et al., Good Practices for Implementing Human Reliability Analysis, NUREG-1792, April 2005.
  • J. Forester, et al., Evaluation of Human Reliability Analysis Methods Against Good Practices, NUREG-1842, September 2006.
  • J. Forester, et al., The International HRA Empirical Study: Lessons Learned from Comparing HRA Methods Predictions to HAMMLAB Simulator Data, NUREG-2127, August 2014.
  • J. Forester, et al., The U.S. HRA Empirical Study: Assessment of HRA Method Predictions against Operating Crew Performance on a U.S.

Nuclear Plant Simulator, NUREG-2156, June 2016.

  • A.M. Whaley, et al., Cognitive Basis for Human Reliability Analysis, NUREG-2114, January 2016.

3

Overview Other References

  • A. Poucet, Human Factors Reliability Benchmark Exercise: Synthesis Report, EUR 1222 EN, Ispra Joint Research Centre, Commission of European Communities, August 1989.
  • E. Lois, et al., International HRA Empirical Study - Phase 1 Report: Description of Overall Approach and Pilot Phase Results from Comparing HRA Methods to Simulator Performance Data, NUREG/IA-0216, Vol. 1, November 2009.
  • A. Bye, et al., International HRA Empirical Study - Phase 2 Report: Results from Comparing HRA Method Predictions to Simulator Data from SGTR Scenarios, NUREG/IA-0216, Vol. 2, August 2011.
  • V.N. Dang, et al., International HRA Empirical Study - Phase 3 Report: Results from Comparing HRA Methods Predictions to HAMMLAB Simulator Data on LOFW Scenarios, NUREG/IA-0216, Vol. 3, December 2014.
  • H. Blackman, N. Siu, and A. Mosleh, Human Reliability Models: Theoretical and Practical Challenges, Center for Reliability Engineering, University of Maryland, College Park, MD, 1998.

4

Overview Other References

  • A. D. Swain and H.E. Guttmann, Handbook of Human Reliability Analysis with Emphasis on Nuclear Power Plant Applications: Final Report, NUREG/CR-1278, August 1983.
  • P. Moieni, et al., A PC-based human reliability analysis (HRA) software, Proceedings ANS International Topical Meeting on Probabilistic Safety Assessment (PSA 93), Clearwater Beach, FL, January 26-29, 1993.
  • D. Gertman, et al., The SPAR-H Human Reliability Analysis Method, NUREG/CR-6883, August 2005.
  • U.S. Nuclear Regulatory Commission, Technical Basis and Implementation Guidelines for A Technique for Human Event Analysis (ATHEANA), NUREG-1624, Rev. 1, May 2000.
  • J. Xing, et al., An Integrated Human Event Analysis System (IDHEAS) for Nuclear Power Plant Internal Events At-Power Application, NUREG-2199, Vol. 1, March 2017.
  • Y.J. Chang and J. Xing, The general methodology of an Integrated Human Event Analysis System (IDHEAS) for human reliability analysis method development, Proceedings International Conference on Probabilistic Safety Assessment and Management (PSAM 13), Seoul, Korea, October 2-7, 2016.
  • S. Lewis and S. Cooper, EPRI/NRC-RES Fire Human Reliability Analysis Guidelines: Final Report, EPRI 1023001/NUREG-1921, July 2012.
  • A. Lindeman and S. Cooper, EPRI/NRC-RES Fire Human Reliability Analysis Guidelines -

Qualitative Analysis for Main Control Room Abandonment Scenarios, Supplement 1, EPRI 3002009215/NUREG-1921, Supplement 1, August 2017.

5

HRA Importance Human Actions and NPP PRA Operational decisions and actions have played an important role in every major NPP accident and incident

  • Occurrence and progression
  • Successes and failures PRAs that dont account for human contributions are not useful (for most applications) 6

HRA Importance Example Events

  • Browns Ferry 1 & 2 cable fire (1975)

- Worker ignites polyurethane foam, starts cable fire

- Fire suppression delayed 7+ hours (reluctant to use water)

- Operators achieve safe shutdown using non-safety system

- Operator error causes loss of feedwater

- Multiple malfunctions => feed and bleed cooling directed by procedures, would have major economic consequences

- Shift supervisor chooses to wait for recovery of AFW (which is successful)

- Operators close isolation condenser (little effect given accident conditions)

- Operators perform numerous non-proceduralized actions (e.g., scavenge car batteries to supply power) in attempts to save plant

- Ex-control room actions hampered by site conditions (tsunami alerts, aftershocks, damage, dark, radiation, )

7

General Description What is HRA?

  • In the context of NPP PRA: A structured approach used to identify potential human failure events and to systematically estimate the probability of those events using data, models, or expert judgment (NUREG-2122)
  • Human Failure Event: interface with rest of PRA model:

- Terminology used to emphasize connection with NPP PRA model (basic events), avoid connotation of blame (e.g., when time available is insufficient)

- Includes errors of omission, errors of commission

- Can be included at scenario level (event trees) or system level (fault trees) 8

General Description HRA General Process

  • Activities

- Qualitative analysis

- Modeling

- Quantification

  • Supports overall model construction

- Initiating event identification

- Accident scenario modeling

- Systems modeling

  • Not just a quantification activity 9

General Description HRA Dimensions and Descriptors

  • Time
  • Implicit

- Pre-initiator - Actions addressed by other PRA

- Initiator model elements (e.g., initiating

- Post-initiator event frequencies, loss of offsite power recovery, common cause

  • Space failure probabilities)

- Within control room - Pre-initiator decisions affecting

- Outside control room fundamental plant design (e.g.,

flood barrier height) and

  • Organization operations (e.g., resources for

- Control room crew training)

- Field operators

- Emergency response

- Sabotage organization

- Terrorism 10

General Description Typical HFE Level of Detail

  • Macro-level crew actions, e.g.,

- Isolate faulted steam generator

- Initiate bleed and feed cooling

- Recover a failed pump

  • Micro-level modeling (e.g., put control switch X in pull-to-lock position) can support HFE; need to consider micro-level recoveries as well as failures 11

Fundamental Model How Things Work

  • Task-oriented view

- Diagnosis and Planning

- Action

  • Cognitive view

- Detecting/Noticing

- Sensemaking/Understanding

- Decision Making

- Action Execution

- Teamwork (communication/coordination) Macrocognitive functions (NUREG-2114) 12

Fundamental Model Naturalistic Decision Making From NUREG-2114, per F.L. Greitzer, et al., Naturalistic decision making for power system operators, International Journal of Human-Computer Interaction, 26(2-3), 278-291, 2010. doi:10.1080/10447310903499070 13

Fundamental Model How Things Can Fail Real-world contextual elements and PIFs* can include:

  • Specific conditions (e.g., problematic components, mixed crews)
  • Scenario dynamics (e.g., shift changes, multiple system shocks)
  • Economic concerns
  • Social behaviors and relationships
  • Usually referred to as Performance Shaping Factors (PSFs) 14

Fundamental Model Fundamental Probabilistic Model

  • Human Error Probability (HEP)

- Quantifies aleatory uncertainty

- Is subject to epistemic uncertainties

- Is a function of the task, the scenario context leading up to the task, and the relevant PIFs

= , ,

  • Underlying assertion: human actions are predictable (in a probabilistic sense)

- Performance of specific tasks, often with specific procedures and training

- Bounded rationality: operators/staff are trying to do the right thing

  • Note: HEP functional behavior on PIFs is usually assumed to be multiplicative, but other data might support additivity 15

Methods HRA Approaches Holistic Analysis Decomposition-Based Analysis (ATHEANA, MERMOS) (THERP, SPAR-H, CBDT, etc.)

Tasks + Context (plant situation, scenario, HFE and crew factors)

Task 1 Task 2 Task 3 HFE PIFs Subtasks / Task steps

  • Analyze context and develop operational
  • Decompose HFE into tasks, possibly story / narrative subtasks / steps
  • Identify situations deviating from the
  • Analyze PIFs for the lowest base story that lead to undesired actions decomposition level
  • Estimate the HEPs of the deviations
  • Calculate HEP of every part, combine HEPs for the event Strengths - Preserves context; uses expert Strengths - Transparency; consistency ability to integrate complex information Limitations - Formulaic; loss of context, Limitations - Level of effort; subjectivity and interactions, non-linearities variability 16

Methods Technique for Human Error Rate Prediction (THERP)

  • Widely-used HRA method, based on research started in 1976
  • Task-oriented, focus on rule-based behavior (but also includes a time-reliability correlation for diagnosis)
  • Task successes and failures represented with HRA event tree
  • Tables used to quantify task success/failure probabilities

- Some empirical basis NUREG/CR-1278

- Considerable expert judgment

  • Provides modifiers for dependent actions 17

Methods Human Cognitive Reliability/Operator Reliability Experiment (HCR/ORE)

  • Extension of HCR method (which was based on skill/rule/knowledge base categorization of actions)
  • Focused on probability of non-response

- Non-response = failure to diagnose OR failure to initiate response in a timely manner

- Normalized correlations for groups of HFEs (human interactions) categorized by cue-response characteristics.

- Analyst estimates median response time and time window; model provides non-response probability.

- Has no floor for very large time margins

Methods Cause-Based Decision Tree (CBDT)

  • Originally a supplement to HCR/ORE, now a standalone method in the EPRI HRA Calculator
  • Eight decision trees used to develop non-response probabilities, considering multiple PIFs (e.g., training quality, procedures, human-machine interface)
1) Relevant data not available 5) Procedure steps missed
2) Data not attended to 6) Misinterpretation of instructions
3) Data errors 7) Errors in interpreting logic
4) Data misleading 8) Deliberate violations
  • Initial non-response probabilities modified by a time-based recovery factor and added to the probabilities of execution failure 19

Methods Standardized Plant Analysis Risk - HRA (SPAR-H)

  • Developed to support SPAR models, event and condition assessments
  • Derived from THERP, multiple PIFs (PSFs) aggregated into eight groups based on information processing model 2-7 2-5,7 1-4, 6-7 2-7
1) Available time 5) Procedures (including job aids)
2) Stress and stressors 6) Ergonomics and human-machine interface
3) Complexity 7) Fitness for duty
4) Experience and training 8) Work processes 20

Methods SPAR-H Worksheets 21

Methods A Technique for Human Event Analysis (ATHEANA)

  • Development started in support of low power and shutdown PRA (different conditions from at-power);

evolved into general method

  • Focuses on HFE context, identification of error-forcing conditions (EFCs)
  • Does not use pre-established list of PIFs (PSFs)
  • Holistic quantification via expert judgment; emphasizes involvement of knowledgeable plant staff (operations and training) 22

Methods Integrated Human Event Analysis System (IDHEAS)

  • Staff response to Commission direction to evaluate the different human reliability models in an effort to propose a single model for the agency to use or guidance on which model(s) should be used in specific circumstances
  • General methodology + application modules

- At-power

- Event and condition assessment

  • Decomposition-based, cognitive focus
  • Supported by extensive review of human cognition literature (psychology, cognition, behavioral science, human factors) to identify relevant functions, mechanisms, and factors 23

Methods IDHEAS At-Power Qualitative analysis PRA scenario Total Manual E-0 to Enter Decide F&B Implement Xfr FR-H1 F&B FR-H1 LOFW Rx Trip ES-01 FR-H1 Step 10 Steps 10-13 1 2 3 4 5 6 1 OK Define HFE: fail F&B 45 min R R R 8 9 2 Fail: execution HFE Task analysis Fail: no decision Feasible? 3 to establish F&B 7

Fail: no entry to FR-H1 4

and no F&B HEP quantification Tasks Failure Modes Context Character a Context Character b Context Character m HEP 1 HEP 2 Enter FR-H1 Critical Task 1 Failure Mode 1 HEP 3 HEP 4 Critical Task 2 Data Misleading Failure Mode 2

Critical Task K Failure Mode N

Methods IDHEAS-G Tasks are accomplished through the performance of various cognitive activities. These cognitive activities exercise general macrocognitive functions.

Example Task:

Identify Ruptured SG (as part of an action to isolate the ruptured SG)

Cognitive Activities:

  • Detect any one of:
  • unexpected rise in any SG NR level
  • high radiation level from any SG sample
  • high radiation from any SG steamline
  • high radiation from any SG blowdown
  • Understand that any one signal provides indication of the faulted SG. Note:
  • The HRA-specified context includes successful reactor and turbine trip, energization of all AC buses, SI actuated, AFW available.
  • The specified context does not explicitly address the possibility of confounding signals and demands (e.g., alarms from unrelated SSCs not modeled in the PRA but demanding operator response.)

Macrocognitive Functions:

  • Detection
  • Understanding 25

Methods IDHEAS-G Macrocognitive functions are accomplished through a Example set of cognitive processes (elements) and cognitive Macrocognitive Function:

processes are accomplished by cognitive mechanisms.

  • Detection Performance influencing factors affect how well the Cognitive Process Elements: cognitive mechanisms are executed by challenging
  • Establish mental model capacity limits for these mechanisms.
  • Select, identify, attend to information sources
  • Perceive, recognize, classify information
  • Verify, modify detection outcomes
  • Retain, document/record, communicate outcomes Cognitive Mechanisms:
  • Sensing
  • Perception of sensing stimuli
  • Vigilance maintenance
  • Capacity Limits:
  • Mismatch between sensory system and signal
  • Weak signal
  • Reduced vigilance due to sustained cognitive activities Performance Influencing Factors:
  • Human-system interface
  • Environmental factors
  • Stress, time pressure, and anxiety
  • Mental fatigue
  • 26

Methods IDHEAS-G Task failure can be caused by failure of any single cognitive mechanism (which propagates through the Example cognitive process/macrocognitive function/cognitive Task:

activity causality chain).* Each potential failure of a Identify Ruptured SG (as part of an action to isolate the ruptured SG) cognitive process is a potential proximate cause for macrocognitive function failure.

Macrocognitive Function:

  • Detection
  • Understanding Proximate Causes:
  • Failure to perceive information
  • Failure to attend to source of information
  • Cognitive Mechanisms:
  • Sensing
  • Perception of sensing stimuli
  • Vigilance maintenance
  • Performance Influencing Factors:
  • Human-system interface
  • Environmental factors
  • Stress, time pressure, and anxiety
  • Mental fatigue
  • Note: from a systems point of view, a task is modeled as a series system with a very large number of potential single-point failures.

27

Methods HRA Guidance

  • Many methods and viewpoints, but general agreement on high-level model and good practices
  • NUREG-1792: high-level guidance, e.g.,

- Perform field observations and discussions

- Use screening values during initial quantification

- Account for dependencies among HEPs

- Evaluate the reasonableness of the HEPs

  • NUREG-1842: evaluation of several methods against these good practices
  • Various documents for specific applications, e.g., NUREG-1921 (fire HRA) and NUREG-1921 Supplement 1 (fire HRA, main control room abandonment) 28

Validation Ispra Benchmark Exercises (1986-1988)

  • European Commission Joint Research Centre
  • Comparison of methods and modeling
  • 15 teams, multiple methods Validation
  • Test and maintenance

- Failure to detect check valve failure, failure to restore system

- Good agreement on qualitative characterization (key human error interactions and failure mechanisms), divergence on modeling and quantification

- Some variance reduction when using a common model

- LOOP, 2/4 EDGs fail to start, partial CCF of EFW valves

- Differences in modeling (scope of analysis, aggregation) and quantification

- Large method-to-method and team-to-team differences 29

Validation International HRA Empirical Studies

  • OECD/NEA Halden Reactor Project
  • Comparisons of analysis results with data from HAMMLAB simulator to identify strengths and weaknesses
  • 14 operator crews, 13 HRA teams, blind study

- Steam generator tube rupture (SGTR), loss of feedwater (LOFW)

- Base case and complex, multiple HFEs with varying difficulty

  • Findings include:

- Large variations in how crews followed procedures

- Large variations in HEPs; many rankings dont reflect difficulty

- Some analyses dont strongly differentiate across HFEs

- Methods that emphasize mechanisms and contextual factors provide richer (and often predictive) narratives, but not necessarily better HEPs 30

Validation Study Process Challenges Include:

  • Differences between HAMMLAB simulator and home plant
  • Characterizing crew behaviors (e.g., drivers for performance)
  • Statistically small sample

Validation US HRA Empirical Studies

  • Similar to international study but using a US PWR (simulator and crews).

Also addressed concerns regarding

- Lack of testing of team-to-team variability in using the same method

- Inability of analysis teams to visit simulator, interview crews

  • 4 crews, 9 HRA teams

- LOFW followed by SGTR

- Loss of component cooling water and RCP seal water

- SGTR

  • Findings include:

- Less variability vs. HAMMLAB study and Ispra: HRA team learning? Better practiced with US crews? Plant visit?

- Qualitative analyses can be improved

- HRA improvements should focus on aiding analysts finding and characterizing contextual factors and mechanisms causing cognitive failures 32

Validation Comparing Predictions with Performance NUREG-2127 NUREG-2156 33

Challenges Technical Challenges

  • Complicating factors

- Specific conditions (e.g., pre-accident conditions including problematic components; specific crew on shift including makeup crews)

- Scenario dynamics (e.g., mindset established by specific evolution, shift changes, multiple system shocks, changes in local environment, external directions)

- Additional crew concerns (e.g., economic impact of action, offsite environment)

- Social behaviors and relationships (e.g., trust within crew, between organizational elements, group behavior) 34

Challenges Technical Challenges (cont.)

  • Data from actual incidents

- Statistically sparse, arguably unique characteristics for each event

- Extremely rich qualitative information for a few events

  • Data from other simulator exercises: transferability to HRA/PRA

- Design and operational differences

- Data collection protocols

  • Technology advances affecting human performance

- Advanced control rooms

- Smart/distributed technology

- Remote operations

-

35

Challenges Socio-Organizational Challenges

  • Multiple technical disciplines with varying goals, views on the meaningfulness of a PRA-oriented HRA, views on needed rigor
  • Interdisciplinary trust

- HRA developers: academic/professional reward system =>

proliferation of HRA methods

- PRA analysts: need for now answers => development of good enough methods, resistance to change

- PRA users: discomfort with large uncertainties =>

dismissal/discounting of results and insights

- Science critics: weaknesses in current methods/models => house of cards view on PRA and RIDM affecting willingness to help 36

Challenges Grand Challenge - Incorporating Organizational Factors

  • Long-recognized as an important influence

- Culture and climate

- Resources

- Direct involvement in events

  • Scope >> current PRA scope

- Time

- Organizations (functions and structure)

- Space

- Technical disciplines

  • Data

- Availability

- Quality

  • Non-monotonic effects 37

Challenges Non-Monotonic Effects: Examples

  • Good safety culture can reduce worker risk but increase plant-level risk

- Pre-emptive reactor trip on loss of communications with diver

- Reluctance to send workers to hazardous areas

  • Forceful leadership can overcome organizational inertia but can also stifle important views 38