ML24302A087
| ML24302A087 | |
| Person / Time | |
|---|---|
| Issue date: | 10/06/2024 |
| From: | Chang Y NRC/RES/DRA/HFRB |
| To: | |
| References | |
| Download: ML24302A087 (1) | |
Text
HRA Data R&D in the NRC Yung Hsien James Chang, Ph.D.
Senior Reliability and Risk Analyst U.S. Nuclear Regulatory Commission Presented at HRA Data Workshop Sendai, Japan October 6, 2024
2 Presentation Outline
- Overview of recent NRC HRA R&D activities
- Near-term focuses on HRA data
- An example of data generation and integration
- Opportunities for collaborations
3 Overview of Recent NRC HRA R&D Activities
- Complete IDHEAS Suite Development Function Products High-Level HRA guidance IDHEAS-General Methodology (NUREG-2198)
HEP quantification IDHEAS-ECA (NUREG-2256); IDHEAS-AT Power (NUREG-2199);
IDHEAS-Dependency (RIL 2021-14); Error Recovery (TBD);
Time Uncertainty (TBD); and HEP Uncertainty (TBD);
Expert elicitation (NUREG-2255, draft)
Data basis IDHEAS-DATA (in process) and SACADA, Guidance on data generalization and integration.
Software tool IDHEAS-ECA tool (present status: rev. 1.2)
Quality control IDHEAS-ECA desktop guide (in development); independent reviews IDHEAS-ECA Evaluation of SPAR HFEs (in process) - also demonstrate documentation.
NUREGs are available in https://www.nrc.gov/reading-rm/doc-collections/nuregs/staff/index.html.
Research Information letters (RIL) are available in https://www.nrc.gov/reading-rm/doc-collections/research-info-letters/index.html.
IDHEAS Suite
4 Recent HFRBs HRA R&D Products
- HRA R&D five-year plan
- RIL 2024-03 Integrated Human Event Analysis System (IDHEAS) -
Time and Time Uncertainty
- RIL 2024-04 Human Reliability Analysis for Calculating Effects of Performance Influencing Factors: A Review of Historical Approaches
- RIL 2024-05 Simulator Data Analysis to Inform Digitalized Environment Impacts on Human Reliability
- RIL 2024-11 Use of Expert Elicitation Guidance to Inform a Small-Scale Knowledge Judgment Application
- NUREG-2255 Expert Elicitation Guidance (Coming soon)
- IDHEAS-ECA tool, rev. 1.2 Research Information Letters (RIL) are available on https://www.nrc.gov/reading-rm/doc-collections/research-info-letters/2024/index.html
5 Near-Term Focuses on HRA Data
- Publish IDHEAS-DATA reports
- IDHEAS-DATA NUREG
- PNNL review of the draft IDHEAS-DATA (A RIL)
- Develop IDHEAS-ECA desktop guide
- To improve the reliability and consistency of applying IDHEAS-ECA method
- Enhance guidance on the selections of Cognitive Failure Modes (CFMs), performance influencing factor attributes (PIFAs), and error recoveries
- Discuss how the original data of IDHEAS-DATA were generated to support analysts judgments in applying IDHEAS-ECA when needed
- Add and update data
- Continue to explore opportunities to collect HRA data
Data Generalization and Integration in IDHEAS-DATA 6
7 An Example of Data Generalization and Integration
- Performance Influencing Factor Attribute (PIFA) SF3 Scenario is unfamiliar
- Applicability
- Scenario is unfamiliar
- A bias or presence for wrong strategies exists
- Personnel are unfamiliar with system failure modes
- Personnel are unfamiliar with worksite for manual actions
- Plans, policies, and procedures to address the situation are lacking
- Degrees of unfamiliarity - current guidance
- SF3.1: Scenario is trained on but infrequently performed
- SF3.2: Scenario is unfamiliar and rarely performed, e.g.,
Notice adverse indicators that are not part of the task at hand Notice incorrect status that is not a part of the routine tasks
- SF3.3: Scenario is extremely rarely performed, e.g.,
Lack of plans, policies and procedures to address the situation No existing mental model for the situation Rare and unexpected events
8 Selection Detection HEP Understanding HEP Decisionmaking HEP Action Execution 1
1.0E-03 1.0E-02 1.0E-02 1.0E-03 2
3.8E-03 3.3E-02 3.3E-02 9.0E-03 3
6.5E-03 5.5E-02 5.5E-02 1.7E-02 4
9.3E-03 7.8E-02 7.8E-02 2.5E-02 5
1.2E-02 1.0E-01 1.0E-01 3.3E-02 6
1.6E-02 1.4E-01 1.4E-01 9.6E-02 7
2.0E-02 1.8E-01 1.8E-01 1.6E-01 8
2.5E-02 2.2E-01 2.2E-01 2.2E-01 9
2.9E-02 2.6E-01 2.6E-01 2.9E-01 10 3.3E-02 3.0E-01 3.0E-01 3.5E-01
SF 3 Data Sources (1/2)
- In IDHEAS-DATA 9
SF 3 Data Sources (2/2)
- In IDHEAS-DATA 10
[26] SACADA - Scenario Structure and Context Characterization 11
[26] SACADA Situational Factors Specific to Diagnosis/Understanding 12 Diagnosis Basis:
o Skill o
Procedure o
Knowledge Based Familiarity:
o Standard: Crew has previously trained on this challenge.
o Novel: This involves a change to the way the challenge is addressed, such as a new procedure, scenario, or role.
o Anomaly: Standard training must be adapted to fit an anomalous situation (e.g., the procedures do not cover the circumstances.)
Outcome:
o Procedure: The diagnosis results in performing a procedure step or transitioning within or external to the procedure currently being performed. (Specify) Procedure ID:_____ Step:______
o Skill: Diagnosis results in the performance of a skill-based activity.
o Knowledge Based: Diagnosis results in the performance of a knowledge-based activity.
Information Integration
Timing of Information: Includes slow information feed or delayed information.
Ambiguous Information: Information provided by system is vague, unclear, or does not point to the nature of the problem.
Integration Required: Integration of multiple pieces of information is required for diagnosis.
Information Specificity:
o Specific: Alarm/alarm pattern/indication(s) point to the specific system problem.
o Not Specific: The alarm(s) and/or indication(s) do not directly point to the specific system problem, which requires operator cognitive effort to integrate the information and identify the specific system problem.
o Not Applicable.
Information Quality:
Missing Information: include masked information
Misleading Information: Information points to an incorrect diagnosis
Conflicting Information: Information points to more than one possible diagnoses, or conflicts with other alarms or indications.
Others (specify):____________________
[26] SACADA Situational Factors Specific to Decisionmaking 13 Situational Factors and Optional Statuses Decision Basis:
o Procedure: The decision is driven by procedures or other guidance.
o Skill: Skill-driven decision: without procedure, operator can make decision from memory.
o Knowledge: No procedure applicable; crew relies on engineering or technical knowledge and operating experience.
Familiarity:
o Standard: Crew has previously trained on this challenge.
o Novel: This involves a change to the way the challenge is addressed, such as a new procedure, scenario, or role.
o Anomaly: Standard training must be adapted to fit an anomalous situation (e.g., the procedures do not cover the circumstances.)
Uncertainty:
o Clear: No uncertainty or competing goals - clear decision criteria.
o Uncertain: Lack of information or ambiguous decision criteria.
o Competing Priorities: Multiple competing goals, foreseeable severe consequences.
o Conflicting Guidance: Policies, practices and procedure have conflicting guidance.
Other (Specify):__________________
[26] Diagnosis-Specific Context Characterization 14
[26] Overarching Context Characterization 15
[26] Performance Statistics 16
[26] Causal Analysis 17
[26] Use SACADA Data to Inform Desktop Guide Development 18
[4, 5] are Two RESS Papers (Part I and II) 19
[4, 5] Example Data 20
[4, 5] Decisionmaking/Understanding Data Points
- in IDHEAS-DATA and IDHEAS-ECA 21 Varies, [4] Table 7 (Decisionmaking)
EOCs due to cognitive errors in identifying or defining the task.
- 1.1E-3 (1/888) Frequently performed task
- Task remembered incorrectly.
- No additional error forcing factor.
- 2.3E-2 (4/173) Rarely performed task
- Task remembered incorrectly.
- No additional error forcing factor.
- 1.4E-1 (3/22) Rarely performed task
- Task remembered incorrectly.
- Additional error forcing factors present.
SF3: Infrequently performed scenarios Basic HEP 1.00E-03 Selection Decisionmaking HEP 1
1.0E-02 2
3.3E-02 3
5.5E-02 4
7.8E-02 5
1.0E-01 6
1.4E-01 7
1.8E-01 8
2.2E-01 9
2.6E-01 10 3.0E-01 In IDHEAS-DATA Implemented in IDHEAS-ECA
[4, 5] Action Execution Data Points
- In IDHEAS-DATA and IDHEAS-ECA 22 Varies, Table 9 [5] (Action Execution) (This table replaces [4] Table 8).
- EOOs, Carrying out a sequence of tasks
- Memorized task step not remembered.
- 1.0E-3 (3/3067) Frequently performed, no error forcing factor.
- 2.1E-2 (1/48) Rarely performed, no error forcing factor.
- 1.6E-2 (3/185) Rarely performed, moderately high level of stress.
- 4.9E-2 (2/41) Rarely performed, moderately high level of stress, ergonomically deficient work environment.
- 7.8E-2 (1/13) Rarely performed test procedure consisting of many sub-steps, task is part of professional knowledge and has to be remembered, high task load.
- 1.4E-1 (1/7) Rarely performed, moderately high level of stress, error prone PSFs and dynamic work environment.
- 3.3E-1 (1/3) Extremely rarely performed, no error promoting factors 6.6E-5 (1/15200) #58
- Pulling and replacing a simulation pin on an electronic module front cover in a control cabinet
- Replunging omitted.
- Highly trained task, not part of a written procedure but part of professional knowledge, favorable ergonomic design.
SF3: Infrequently performed scenarios Basic HEP 1.00E-04 Selection Action Execution HEP 1
1.0E-03 2
9.0E-03 3
1.7E-02 4
2.5E-02 5
3.3E-02 6
9.6E-02 7
1.6E-01 8
2.2E-01 9
2.9E-01 10 3.5E-01 In IDHEAS-DATA Implemented in IDHEAS-ECA
[4, 5] Data Generalization and Integration (1/2)
- IDHEAS-DATA vs. Original Data 23 Varies, Table 9 [5] (Action Execution) (This table replaces [4] Table 8).
- EOOs, Carrying out a sequence of tasks
- Memorized task step not remembered.
- 1.0E-3 (3/3067) Frequently performed, no error forcing factor. (#30, #59, and #64)
- 2.1E-2 (1/48) Rarely performed, no error forcing factor. (#35)
- 1.6E-2 (3/185) Rarely performed, moderately high level of stress. (#27, #34, and #65)
- 4.9E-2 (2/41) Rarely performed, moderately high level of stress, ergonomically deficient work environment. (#31 and #60)
- - 1.4E-1 (1/7) Rarely performed, moderately high level of stress, error prone PSFs and dynamic work environment. (#28)
- 3.3E-1 (1/3) Extremely rarely performed, no error promoting factors. (#66) 6.8E-4 (1/1470) #30
- Manually operating a local valve
- Valve not operated, step in a sequence of different steps not remembered.
- Frequently performed task.
7.42E-4 (1/1347) #59
- Operating a manual control at a MCR control panel.
- Task not remembered.
- Frequently performed task, part of professional knowledge, position of indicator lamp ergonomically unfavorable designed.
4.0E-3 (1/250) #64 (Action Execution)
- Disassembly of a circuit breaker at a local switch gear cabinet.
- Cable not disconnected, sub-step in a task consisting of four sub-steps not remembered.
- Memory based activity, very slow variation of process parameters, high task load.
2.1E-2 (1/48) #35 (Action Execution)
- Removing a ground connection from a switch cabinet.
- Ground connection not removed, memorized step not remembered.
- Rarely performed task, step not described in written procedure.
In IDHEAS-DATA Original Data
[4, 5] Data Generalization and Integration (2/2)
- Original Data 24 2.0E-2 (1/50) #27 (Action Execution)
- Repeated discontinuous manual control od pump discharge pressure.
- Task not remembered.
- Moderately high level of stress.
5.0E-2 (1/20) #34 (Action Execution)
- Open a valve by MCR panel control.
- Failed to open, memorized task step not remembered.
- Rarely performed task sequence, moderately high level of stress.
8.7E-3 (1/115) #65 (Action Execution)
- Discontinuous check of a process parameter on a MCR control panel.
- Check not performed, task not remembered.
- Memory based activity, very slow variation of process parameter, high task load.
3.6E-2 (1/28) #31 (Action Execution)
- Open a key control.
- Control not operated, step not remembered.
- Rarely performed task, enhanced level of stress due to time constraints, unfavorable ergonomic design.
7.8E-2 (1/13) #60 (Action Execution)
- Operating a circuit breaker at a local switchgear cabinet.
- Task not remembered.
- Rarely performed test procedure consisting of many sub-steps, task is part of professional knowledge and has to be remembered, high task load.
1.4E-1 (1/7) #28 (Action Execution)
- Carrying out a sequence of tasks.
- Memorized task step not remembered.
- Rarely performed, moderately high level of stress, error prone PSFs and dynamic work environment.
3.3E-1 (1/3) #66 (Action Execution)
- Disabling an automation in order to perform a test.
- Automatic operation not disabled, step not remembered.
- High demand on memory performance, rarely performed task.
25 Considerations in data generalization and Integration on [26] and [4, 5]
are the tasks performed by a team. The data includes individual and team error recoveries.
- The error mode in [4, 5] is memorized task step is not memorized. EOPs provide step-by-step instructions that makes the error mode less likely.
- Inform the analysts about the considerations
26 IDHEAS-ECA Desktop Guide
- Provide more details of the original data than documented in IDHEAS-DATA to support analysts applying IDHEAS-ECA method
- Plan to incorporate the desktop guide into IDHEAS-ECA software tool
27 Opportunities for Collaborations
- The NRC welcomes collaborations in HRA method, data, and tool
- The NRC can provide free applications, source codes, and technical support of IDHEAS-ECA and SACADA
- Collaborations required to go through the NRC approval process (e.g.,
establish a memorandum of understanding (MOU))
- NRCs HRA data needs
- Multiple sites: Existing data is from a nuclear site
- Field performance data: Existing data only contains control room actions. Field data, e.g., job performance measures, would expand the coverage
- Timing data: to assess time uncertainty and the drivers