ML21113A290

Summary of the Public Meeting on IDHEAS for Concurrence

ML21113A290
Person / Time
Issue date:	04/26/2021
From:	Jing Xing NRC/RES/DRA/HFRB
To:	Sean Peters Office of Nuclear Reactor Regulation
Xing, Jing - 301 415 2410
References
Download: ML21113A290 (7)

Text

April 26, 2021 MEMORANDUM TO: Sean Peters, Branch Chief Human Factors and Reliability Branch Division of Risk Analysis Office of Nuclear Regulatory Research Signed by Xin FROM: Jing Xing, Sr. Human Performance Engineer on 04/26/21 Human Factors and Reliability Branch Division of Risk Analysis Office of Nuclear Regulatory Research

SUMMARY

OF PUBLIC MEETING ON INTEGRATED HUMAN EVENT ANALYSIS SYSTEM - PUBLIC FEEDBACK SESSION On April 8, 2021, the NRC staff, Y. James Chang, Human Reliability Engineer, Jing Xing, Sr. Human Performance Engineer, and Sean Peters, Chief of Human Factors and Reliability Branch, Division of Risk Analysis, Office of Nuclear Regulatory Research, organized this public meeting, presented the NRCs new suite of human reliability analysis (HRA) methods, the Integrated Human Event Analysis System (IDHEAS),

to the public, and answered questions from the attendees. The NRC staff also discussed future actions on improving IDHEAS with representatives from the Electric Power Research Institute (EPRI). Enclosure 1 of this summary presents a list of questions provided by the industry. Enclosure 2 contains the NRCs planned activities and desired support.

Meeting Purpose The purpose of this public meeting was to collect comments on the new suite of IDHEAS human reliability analysis (HRA) methods built by the NRC to handle all nuclear HRA applications. These applications include in-control room, ex-control room, Diverse and Flexible Coping Strategies (FLEX), fuel cycle, medical, etc. Members of the public will have the opportunity to provide comments on the IDHEAS General Methodology (IDHEAS-G), IDHEAS method for Events and Conditions Assessment (IDHEAS-ECA), human

Enclosures:

1 - List of questions from the public 2 - The NRC staffs planned activities on IDHEAS improvement and desired support from EPRI CONTACT: Jing Xing, RES/DRA/HFRB 301-415-2410

error date generalized to quantify human error probabilities (IDHEAS-DATA), IDHEAS dependency model, and Scenario Authorization, Categorization, and Debriefing Application (SACADA) database for collecting operator performance data in simulator training. The staff also announced that the NRC issued a Federal Registration Notice, NRC-2021-0089, Integrated Human Event Analysis System for Event and Condition Assessment Method and Software, for the public to comment on the IDHEAS-ECA method and software until July 30, 2021.

Public participation Public participation at the meeting included representatives from Ameren Corporation, BCP Engineers &

Consultants, Callaway Energy Center, Certrec Corporation, Curtiss-Wright Corporation, Dominion Energy, Exelon Corporation, Electrical Power Research Institute, Grand Gulf Nuclear Station, Jensen and Hughes Inc., Nuclear Energy Institute (NEI), and the Pressurized Water Reactor Owners Group.

Meeting outcomes The NRC staff presented the following topics:

IDHEAS Program Overview IDHEAS General Methodology (IDHEAS-G)

IDHEAS-DATA - Human error data supporting the quantification of human error probabilities IDHEAS-ECA and Software Tool Demo IDHEAS- Dependency Model and guidance SACADA - The operator simulator training performance database Future of the IDHEAS Program The presentation slides were made available to the public prior to the meeting through the NRCs ADAMS (ML21096A176). The following related documents were also available to the public:

• NUREG-2198 - The General Methodology of the Integrated Human Event Analysis System (IDHEAS) (ML20329A428, in publication)

• RIL 2020-02, Integrated Human Event Analysis System for Event and Condition Assessment (IDHEAS-ECA) (ML20016A481)

• The SACADA database for human reliability and human performance, (https://www.sciencedirect.com/science/article/pii/S095183201300224X)

Meeting Achievements The meeting purpose was achieved. The NEI and EPRI representatives provided the NRC staff a list of questions and comments. Those included clarifications on the IDHEAS suite and constructive suggestions on improving IDHEAS. The NRC staff discussed the comments and planned future activities on IDHEAS improvement.

Action Items/next steps None.

SUMMARY

OF PUBLIC MEETING ON INTEGRATED HUMAN EVENT ANALYSIS SYSTEM - PUBLIC FEEDBACK SESSION DATED: APRIL 26, 2021 DISTRIBUTION:

RES/rf RidsNroDlse RidsOgcMailCenter RidsAcrsAcnwMailCenter Araguas, Christian, RES/DRA Hughey, John, NRR Kichline, Michelle, NRR Grady, Anne-Marie, NRR Grasso, John, RES/DRA Thaggard, Mark, RES/DRA Werkheiser, Dave, RGN I EXTERNAL:

Anderson, Victoria vka@nei.org Apostolakis, George apostola@mit.edu Bergman, Jana jbergman@curtisswright.com Conly, John john.conly@certrec.com Gunter, Katherine KGunter@jensenhughes.com Hiller, Justin W JHiller@ameren.com>

Julius. Jeffrey JJulius@jensenhughes.com Knous Bill wsk@bcpengineers.com Liao, Harry huafei.liao@gmail.com Linthicum, Roy roy.linthicum@exeloncorp.com Margotta, Daniel dcm@bcpengineers.com Nierode, Craig Craig.F.Nierode@dominionenergy.com Presley, Mary mpresley@epri.com ADAMS Accession No: ML21113A290 OFFICE TS: RES/DRA TS: RES/DRA NAME YChang JXing DATE 04/26/2021 04/26/2021 OFFICIAL RECORD ONLY

Enclosure 1:

List of questions from the public Questions on IDHEAS HRA Public Meeting conducted April 8, 2021

1. IDHEAS-ECA - overall, we think there is a lot of promise for IDHEAS-ECA as a general HRA tool and the team has worked hard to try to address some of the known issues with existing HRA. However, we believe there is still work to be done in order for IDHEAS-ECA to be consistently used and serve as a tool for consensus rather than another point of divergence. Is there a process or plan to improve implementation guidance on this method? If so, what is the role of industry engagement?

2. IDHEAS-ECA - questions on specific topics related to modeling.

a. Qualitative analysis and documentation. We have noted that the qualitative analysis forms are quite cumbersome and we have observed that these forms are not consistently used by the analysts (i.e., often a shorthand qualitative analysis is done, or people jump straight to quantification). Also, because the documentation requirements are so long, it is not clear which portions of the qualitative analysis impact the task breakdown and quantification, making it difficult to review in some cases. We recommend streamlining the qualitative analysis and linking it more explicitly to the quantification, perhaps through the software interface.

b. Increased source of variability. We are concerned that, with many more factors and flexibility in the method will come very large analyst-to-analyst variability. Particularly, more work is needed in the following areas:

i. Distinguishing between CFMs [Crew Failure Modes] of Macro-Cognitive Functions. Previous methods have had trouble distinguishing between when an error is caused by Detection, or by issues in Understanding, or by issues related to Decision-Making; and there appears to be overlap in IDHEAS-ECA for these CFMs. How does the guidance and training ensure proper categorization without double or triple-counting sources of error that may impact these Macro-Cognitive Functions?

ii. PIF anchor point: Anchor points need clear guidance. It is unclear if the anchor point guidance is generic or should be application specific (e.g., is the definition of a complex action or adequate training or adequate procedure the same for LPSD [Low-Power Shutdown], level 1, level 1, SFP [Spent Fuel Pool],

etc.?) A set of examples or conventions would be helpful.

1. Should also include guidance on when PIFs [Performance Influencing Factors] can be disregarded (e.g., we noted in the FLEX workshop that there were often several PIFs that can be used to represent the same underlying factors and often the analysts had to exercise judgement to discount a PIF even though it fit the verbatim description.

iii. Recovery: I believe this is already on your list to provide updated guidance, but we have found, in practice, how recovery is credited to be a large source of analyst variability and this would benefit from clear guidance of when recovery is appropriate and how much.

iv. Timing: For cases where time is not expansive, this can be a very important source of variability since small differences in the distribution can lead to large difference in the resultant HEP [Human Error Probability]. Recommend providing some generic guidance if possible, to reduce this. In fact, in the FLEX example workshop, analysts were unable to use the timing module and did not test that capability.

v. Execution. Was the literature survey include Execution errors in its scope, and were there many sources of information on Execution? Or was the literature review primarily conducted on cognitive errors? Stated another way, does the

research conducted as part of the IDHEAS development show execution errors to be a lower contributor than what has previously been modeled, or was the scope of the research such that execution errors were not really included?

3. IDHEAS-ECA - application beyond PRA [Probabilistic Risk Assessment] for Level 1 internal and external events at-power for currently operating plants.

a. Does the NRC consider this method adequate to model HRA in digital environments? Is there any caveat or limitations to the applicability of this method?

b. Does the NRC consider this method adequate to model HRA for Level 2 PRA?

c. Does the NRC consider this method adequate to model HRA for Low Power and Shutdown PRA?

d. Does the NRC consider this method adequate to model HRA for other applications such as medical?

4. IDHEAS-ECA Publication questions.

a. Some organizations have suggested a benchmarking activity to provide feedback on the method, and is this part of the NRCs plans prior to publication as a NUREG?

b. Will there be a public comment period for the NUREG? The HRA Dependency portion has not been previously published and would benefit from a review.

c. The slides mention validation vs standards.

i. Does this mean the published documents demonstrate how IDHEAS-ECA complies with supporting requirements of the PRA Standard, specifically if/how it supports Capability Category II evaluations?

ii. What validation activity will be conducted?

5. IDHEAS-Dependency - it is very nice to see the NRC looking at new ways to approach dependency and we think there is a lot of promise in this approach. We are wondering if there will be a public comment period or a more formal pilot of the completed method?

6. IDHEAS-Data - What was the independent review process for the underlying data that was mentioned in these slides?

7. IDHEAS At-Power - What, if any, role does this method have going forward?

Enclosure 2:

The NRC staffs planned activities on IDHEAS improvement and desired support from EPRI NRC staff responses to Industry Questions on 04-08-2021 IDHEAS HRA Public Meeting Questions from the public NRC staffs planned action Desired EPRI items collaboration IDHEAS-ECA development - 1) The staff is completing the 1) Review and Dependency draft dependency guidance comment on It is very nice to see the NRC and will make it public the guidance looking at new ways to approach through an NRC Research - summer dependency and we think there Information Letter (RIL) for 2021.

is a lot of promise in this public comment. 2) Voluntarily approach. We are wondering if 2) The staff plans to update the use and there will be a public comment guidance based on the validate the period or a more formal pilot of feedback, then incorporate dependency the completed method? the guidance into the guidance in IDHEAS-ECA report and HRA Software tool. applications.

3)

IDHEAS-ECA development - The staff plans to begin modeling 1) Consult with Guidance on modeling HRA recovery in late 2021/early industry recovery 2022. The staff plans the experts on i)

I believe this is already on your following activities: i) reviewing and ii).

list to provide updated guidance, literature and event reports to 2) Work with the but we have found, in practice, understand recovery industry on iii) how recovery is credited to be a mechanisms, ii) search for data and iv).

large source of analyst variability on likelihood of HRA recovery, iii) and this would benefit from clear possibly organize a formal expert guidance of when recovery is elicitation to develop the recovery appropriate and how much. model, iv) pilot the recovery model in HRA applications, and v) incorporate the recovery model in a future update of IDEHAS-ECA.

IDHEAS-ECA development - 1) The staff is currently working 1) Review and Guidance on estimating time with the Pacific Northwest comment on required National Laboratory the guidance.

For cases where time is not documenting data on time 2) Pilot the expansive, this can be a very required as the technical guidance with important source of variability basis. example since small differences in the 2) The staff plans to develop the events.

distribution can lead to large guidance on time estimation difference in the resultant and publish the guidance as a HEP. Recommend providing RIL for public comment.

some generic guidance if 3) The staff plans to update the possible, to reduce this. guidance and incorporate it into IDHEAS-ECA Software.

IDHEAS-ECA - Usability 1) The staff is developing a 1) The staff is Qualitative analysis and more streamlined template for open to documentation. documenting qualitative suggestions on We have noted that the analysis based on experience better ways to qualitative analysis forms are with IDHEAS-ECA so far. document quite cumbersome and we have The staff plans to replace the qualitative observed that these forms are worksheets in the IDHEAS-

not consistently used by the ECA RIL report with the new analysis in analysts (i.e., often a shorthand template in the upcoming IDHEAS-ECA.

qualitative analysis is done, or IDHEAS-ECA NUREG report.

people jump straight to 2) The staff plans to incorporate quantification). Also, because the updated qualitative the documentation requirements analysis template in IDHEAS-are so long, it is not clear which ECA Users Guidance to be portions of the qualitative developed.

analysis impact the task breakdown and quantification, making it difficult to review in some cases. We recommend streamlining the qualitative analysis and linking it more explicitly to the quantification, perhaps through the software interface.

IDHEAS-ECA - Usability 1) The staff plans to develop a The staff is open Increased source of concise IDHEAS-ECA Users to suggestions on variability. We are concerned Guide to highlight the 1) and 2), and that, with many more factors and important guidelines such as will ask EPRI to flexibility in the method will come selecting applicable cognitive review and very large analyst-to-analyst failure modes and modeling comment on the variability. Particularly, more context with the performance draft guidance.

work is needed in the following influencing factors.

areas: i) Distinguishing between 2) The staff is considering CFMs of Macro-Cognitive developing training materials Functions, ii) PIF anchor along with practical point: Anchor points need clear examples.

guidance. 3) The staff plans to link the PIF attributes to the summary description of the data source references to show what the PIF attributes/CFMs mean in real examples.

IDHEAS-ECA - Usability Inspired by EPRIs experience Welcome the Continuous improvement. with EPRI HRA Calculator, the industrys IDHEAS-ECA is a new method. NRC staff plans to start an participation in HRA analysts will need to learn, IDHEAS-ECA User Group for the User Group.

get familiarized with, and collecting user feedback and practice it in real applications. exchanging experience.

Continuously collecting users feedback helps the learning curve.