ML17164A077
| ML17164A077 | |
| Person / Time | |
|---|---|
| Issue date: | 06/13/2017 |
| From: | Chang Y NRC/RES/DRA/HFRB |
| To: | |
| Shared Package | |
| ML17080A072 | List: |
| References | |
| Download: ML17164A077 (48) | |
Text
SACADA A Data Collection Tool for Simulator Training Yung Hsien James Chang U.S. Nuclear Regulatory Commission James.Chang@nrc.gov 1
Cicada (from Google images)
Presentation Outline
- Brief history
- SACADA*
- Benefits for operator simulator training
- Framework for partnering with the U.S.
Nuclear Regulatory Commission (NRC) on SACADA
- SACADA: Scenario Authoring, Characterization, and Debriefing Application 2
Brief History
- NRC desires access to operator simulator performance data to inform its human reliability analysis methods
- NRC, partnered with the South Texas Project Nuclear Operating Company (STP), developed the SACADA software to:
- Provide a debriefing tool that facilitates improved operator simulator training
- Capture real simulator performance data to inform human reliability analysis (HRA)
- NRC promotes partnering on the use of SACADA 3
Some of the Developers 4
Current Collaborations
- STP (since 2011)
- Advanced test reactor (since 2016)
- Taiwan Power Company (since 2016, piloting)
- Palo Verde (4/2016)**
- Callaway (2016)***
- Halden Reactor Project (since 2014)
- KAERI(Korea) (since 2013*)
- ÚJV ez, a. s. (Czech Republic)(since 2015*)
- Reviewing taxonomy but not sharing data
- SACADA training is to be delivered on 4/25&26/2017, MOU is process
- SACADA training schedule is in discussion. MOU in process.
5
Collaboration Process
- Upon request, NRC provides an onsite or web meeting to present the SACADA system and demonstrate the software for initial evaluation
- The potential partners can request to pilot the system for evaluation
- NRC provides free training (1.5 days per session),
software, and tech support to support the piloting
- A memorandum of understanding (MOU) will be developed to formalize the partnership
- Implement a formal collaboration based on the MOU 6
SACADA Software Main Functions
- Design simulation scenarios
- Characterize performance challenges
- Debrief performance results
- Output data (Reports) 7
Simulation Scenario Structure 15 45 75 80 90 129 Time Initial Condition Training Objective Elements Loss of ECW 1A POSITION EXPECTED RESPONSE CREW Recognize loss of EW flow to A train.
Crew Secure ECW pump 1A SM Manually trip Diesel Generator prior to any of Diesel Generator trips Crew Ensure CCP 1A is in service Crew Verifies Natural Circulation SM Determines need to cooldown SM Declare an Alert HA1/EAL2 due to damage to EW structure or notify ED that escalation is appropriate.
Malfunctions/Conditions 8
Authoring -
Design Simulation Scenarios 9
Authoring -
Edit Malfunction/Situation 10
Authoring -
Edit Elements 11
Authoring -
Four Levels of Element Importance 1.
Red: Critical tasks directly affect core safety and radioactivity release E.g., manually trip reactor and emergency depressurization 2.
Orange: affect the availability and reliability of the equipment related to reactor safety or other items important to safety E.g., enter correct AOPs/EOPs, EAL classification, and training emphases 3.
Yellow: Professional tasks, not affecting reactor safety TS LCO and three-way communication 4.
White: general tasks 12 EAL: Emergency Action Level; TS LCO: Technical Specification Limiting Condition for Operation
Authoring -
Summary
- Four levels of element importance
- Semi-automatic time data collection
- Develop new scenarios based on old scenarios to save effort
- Import/export functions to share information between plants
- Lock scenario to prevent from editing
- Store scenario package
- Group discussion corners: instructors & crews 13
Characterization -
Task Cognitive Type 14
Characterization -
Cognition Specific Factors 15
Characterization -
Overarching Factors 16
Characterization -
Summary
- From a human-centered, system-neutral perspective to characterize the task and challenges
- Information is important for data analysis to inform HRA
- Extract performance information from multiple plants 17
Timing data
- Initially, SACADA was not designed to input data DURING the scenario
- A need arose to capture the TIMING of specific tasks
- Combined with a mobile platform, SACADA was leveraged to capture the timing of predetermined tasks 18
Debriefing -
Collect Time Data
- The only information that needs to be collected while the simulation is running
- Tablet app available to collect time data 19
Tablet App for Mobility
- Collect
- Time
- Disposition
- Comments
- Download/Upload data from/to server
- Windows platform 20
Debriefing -
How did we do?
21
Debriefing -
Performance Rating
- Default rating: SAT 22
Debriefing -
Deficiency Analysis
- Importance driven debriefing facilitates wise use of debriefing time
- Is mandatory when a red or orange element is SAT or UNSAT
- Is optional when a yellow or white element is SAT or UNSAT
- Systematic analysis
- Class of the performance deficiency
- INPO five operator fundamentals
- Macrocognitive functions
- Specifics of the performance deficiency
- Causes of the performance deficiency
- Recovery of the deficiency
- Effects on the scenario
- Remediation
- Stimulate thinking and discussion in debriefing 23
Debriefing
- Link to INPO Operator Fundamentals 24
Debriefing -
- 1. Macrocognition Based Deficiency Classification Seven classes Monitoring/Detection Diagnosis/Understanding Procedure Following/Decision Making Manipulation Supervision Teamwork Communication Stimulate crew to think Can choose more than one class 25
Debriefing -
- 2. Deficiency Specifics - Diagnosis
- Classify the type of diagnosis deficiency
- Stimulate crew to think what went wrong
- Other: for other specific behaviors
- Comments 26
Debriefing -
- 3. Deficiency Causes - Diagnosis
- Identify the reasons contributing to the diagnosis deficiency
- Stimulate the crew to think why it happened 27
Debriefing -
- 2. Deficiency Specifics - Supervision
- Classify the supervision deficiency
- Stimulate crew to think what went wrong 28
Debriefing -
- 3. Deficiency Causes - Supervision
- Identify the reasons contributing to the supervision deficiency
- Stimulate the crew to think why it happened 29
Debriefing -
- 4. Deficiency Recovery
- Explicitly identify the deficiency recovery mechanism and timing 30
Debriefing -
- 5. Scenario Impacts
- Explicitly identify the impacts on the scenario 31
Debriefing -
- 6. Remediation
- Specify the remediation approaches and the responsible individuals
- Export to crew notebook to save data entry effort 32
Debriefing -
Importance Driven Debriefing Show all elements
- Chorology driven debriefing Show only red and orange
- Importance driven debriefing 33
Summary Table
- Summarize the entered information 34
Debriefing Summary
- Facilitate the debriefing based on task importance rather than the chronology
- Minimize eyes away from monitoring crew behavior to collect time data
- A systematic comprehensive taxonomy to stimulate discussion on performance deficiencies to improve training effectiveness
- Quick data collection to meet limited debriefing time 35
Reports
- Various types of reports available
- Debriefing, authoring, end of cycle, remediation, and all data (for statistical analysis)
- Can be exported to other formats including PDF, MS Word, and Excel
- Flexible to add more report types 36
Debriefing Reports Exportable formats: PDF, MS Word, and Excel 37
Neat Features
- Automatically Email Remediation Report To training managers and operations managers immediately after saving the debriefing file when there is SAT or UNSAT in Red or Orange elements.
38
Neat Features
- Crew Composition and Experience
- Three levels of experience
- Less than 2 years
- Between 2 and 5 years
- Greater than 5 years
- Balances information need and privacy 39
Neat Features
- User Control & Exam Security
- Can be installed on a personal computer or a server computer
- Install in a dedicated computer for exam security
- Install in a server for convenience
- User control in the server computer
- Data accessibility control based on the login identification
- Software lock to control edits 40
Benefits for Simulator Training
- STP Experience (1)
- Improved debriefing quality
- Provides consistency between crews in the debriefs
- Ensure crew addresses an issue
- The issues cannot be minimized - drill down to the root cause
- Can be used to track individual crew performance as well as overall department performance
- End of cycle report
- Used for planning the next training cycle
- Enables rapidly addressing the identified performance issues
- Real time comparison against actual performance 41
Benefits for Simulator Training
- STP Experience (2)
ACAD Objective 3.2 - Managers are engaged in training activities through monitoring and oversight to provide feedback and direction
- Automatic reports
- Included in the end-of-cycle roll-up reports along with other cycle observations to get better picture of overall training performance
- Used to identify department issues to make as a focus area for future training cycles
- Allows management to promptly address performance issues 42
Benefits for Simulator Training
- STP Experience (3)
ACAD Objective 3.3 - Personnel performance and feedback during training are used to evaluate and modify training programs
- Tracks and trends simulator performance issues
- Captures good performance and identifies weaknesses
- Feedback issues identified export to crew notebooks
- Helps evaluate and group performance deficiencies by operator fundamentals 43
STP Overall Experience
- Overall very positive experience
- Has helped in identifying issues and ensuring they are closed out properly
- Has helped improve training by identifying gaps that need to be focused on in later training cycles
- A learning curve in the beginning with crews and instructors on the system and data 44
Partnering with NRC
- NRC welcomes partnerships on SACADA
- Collaboration Principals
- NRC provides free software licenses, training and technical support
- The partners share the data with NRC for HRA research purposes
- For NRC licensees, the NRC will not perform regulatory actions against the licensees based on the provided SACADA data 45
Share Data with NRC
- Each partner has its data stored locally
- At the end of each training cycle, the partner exports the cycle data to the SACADA master database (currently operated and maintained by the Idaho National Laboratory)
- The remediation data will not be exported
- The exported data contains no individual identifications
- NRC hosts but does not own the data. The data are proprietary to the data providers.
46
Sharing Data Insights
- NRC plans to analyze the master databases data focusing on generic instead of plant-specific human performance insights
- Plan to analyze the data on a regular basis.
The frequency is tentatively to be annually
- NRC will share the insights with the SACADA partners 47
Contact Information Yung Hsien J. Chang, Ph.D.
James.Chang@nrc.gov Phone: + 1 301-415-2378 SACADA Technical Lead and Project Manager Sean E. Peters Sean.Peters@nrc.gov Phone: +1 301-415-2293 Branch Chief 48