ML21113A290

Summary of the Public Meeting on IDHEAS for Concurrence

ML21113A290
Person / Time
Issue date:	04/26/2021
From:	Jing Xing NRC/RES/DRA/HFRB
To:	Sean Peters Office of Nuclear Reactor Regulation
Xing, Jing - 301 415 2410
References
Download: ML21113A290 (7)

Text

MEMORANDUM TO:

Sean Peters, Branch Chief Human Factors and Reliability Branch Division of Risk Analysis Office of Nuclear Regulatory Research FROM:

Jing Xing, Sr. Human Performance Engineer Human Factors and Reliability Branch Division of Risk Analysis Office of Nuclear Regulatory Research

SUMMARY

OF PUBLIC MEETING ON INTEGRATED HUMAN EVENT ANALYSIS SYSTEM - PUBLIC FEEDBACK SESSION On April 8, 2021, the NRC staff, Y. James Chang, Human Reliability Engineer, Jing Xing, Sr. Human Performance Engineer, and Sean Peters, Chief of Human Factors and Reliability Branch, Division of Risk Analysis, Office of Nuclear Regulatory Research, organized this public meeting, presented the NRCs new suite of human reliability analysis (HRA) methods, the Integrated Human Event Analysis System (IDHEAS),

to the public, and answered questions from the attendees. The NRC staff also discussed future actions on improving IDHEAS with representatives from the Electric Power Research Institute (EPRI). Enclosure 1 of this summary presents a list of questions provided by the industry. Enclosure 2 contains the NRCs planned activities and desired support.

Meeting Purpose The purpose of this public meeting was to collect comments on the new suite of IDHEAS human reliability analysis (HRA) methods built by the NRC to handle all nuclear HRA applications. These applications include in-control room, ex-control room, Diverse and Flexible Coping Strategies (FLEX), fuel cycle, medical, etc. Members of the public will have the opportunity to provide comments on the IDHEAS General Methodology (IDHEAS-G), IDHEAS method for Events and Conditions Assessment (IDHEAS-ECA), human

Enclosures:

1 - List of questions from the public 2 - The NRC staffs planned activities on IDHEAS improvement and desired support from EPRI CONTACT:

Jing Xing, RES/DRA/HFRB 301-415-2410 April 26, 2021 Signed by Xin on 04/26/21 error date generalized to quantify human error probabilities (IDHEAS-DATA), IDHEAS dependency model, and Scenario Authorization, Categorization, and Debriefing Application (SACADA) database for collecting operator performance data in simulator training. The staff also announced that the NRC issued a Federal Registration Notice, NRC-2021-0089, Integrated Human Event Analysis System for Event and Condition Assessment Method and Software, for the public to comment on the IDHEAS-ECA method and software until July 30, 2021.

Public participation Public participation at the meeting included representatives from Ameren Corporation, BCP Engineers &

Consultants, Callaway Energy Center, Certrec Corporation, Curtiss-Wright Corporation, Dominion Energy, Exelon Corporation, Electrical Power Research Institute, Grand Gulf Nuclear Station, Jensen and Hughes Inc., Nuclear Energy Institute (NEI), and the Pressurized Water Reactor Owners Group.

Meeting outcomes The NRC staff presented the following topics:

IDHEAS Program Overview

IDHEAS General Methodology (IDHEAS-G)

IDHEAS-DATA - Human error data supporting the quantification of human error probabilities

IDHEAS-ECA and Software Tool Demo

IDHEAS-Dependency Model and guidance

SACADA - The operator simulator training performance database

Future of the IDHEAS Program The presentation slides were made available to the public prior to the meeting through the NRCs ADAMS (ML21096A176). The following related documents were also available to the public:

NUREG-2198 - The General Methodology of the Integrated Human Event Analysis System (IDHEAS) (ML20329A428, in publication)

RIL 2020-02, Integrated Human Event Analysis System for Event and Condition Assessment (IDHEAS-ECA) (ML20016A481)

The SACADA database for human reliability and human performance, (https://www.sciencedirect.com/science/article/pii/S095183201300224X)

Meeting Achievements The meeting purpose was achieved. The NEI and EPRI representatives provided the NRC staff a list of questions and comments. Those included clarifications on the IDHEAS suite and constructive suggestions on improving IDHEAS. The NRC staff discussed the comments and planned future activities on IDHEAS improvement.

Action Items/next steps None.

SUMMARY

OF PUBLIC MEETING ON INTEGRATED HUMAN EVENT ANALYSIS SYSTEM - PUBLIC FEEDBACK SESSION DATED: APRIL 26, 2021 DISTRIBUTION:

RES/rf RidsNroDlse RidsOgcMailCenter RidsAcrsAcnwMailCenter Araguas, Christian, RES/DRA Hughey, John, NRR Kichline, Michelle, NRR Grady, Anne-Marie, NRR Grasso, John, RES/DRA Thaggard, Mark, RES/DRA Werkheiser, Dave, RGN I EXTERNAL:

Anderson, Victoria vka@nei.org Apostolakis, George apostola@mit.edu Bergman, Jana jbergman@curtisswright.com Conly, John john.conly@certrec.com Gunter, Katherine KGunter@jensenhughes.com Hiller, Justin W JHiller@ameren.com>

Julius. Jeffrey JJulius@jensenhughes.com Knous Bill wsk@bcpengineers.com Liao, Harry huafei.liao@gmail.com Linthicum, Roy roy.linthicum@exeloncorp.com Margotta, Daniel dcm@bcpengineers.com Nierode, Craig Craig.F.Nierode@dominionenergy.com Presley, Mary mpresley@epri.com ADAMS Accession No: ML21113A290 OFFICE TS: RES/DRA TS: RES/DRA NAME YChang JXing DATE 04/26/2021 04/26/2021 OFFICIAL RECORD ONLY List of questions from the public Questions on IDHEAS HRA Public Meeting conducted April 8, 2021 1.

IDHEAS-ECA - overall, we think there is a lot of promise for IDHEAS-ECA as a general HRA tool and the team has worked hard to try to address some of the known issues with existing HRA. However, we believe there is still work to be done in order for IDHEAS-ECA to be consistently used and serve as a tool for consensus rather than another point of divergence. Is there a process or plan to improve implementation guidance on this method? If so, what is the role of industry engagement?

2.

IDHEAS-ECA - questions on specific topics related to modeling.

a.

Qualitative analysis and documentation. We have noted that the qualitative analysis forms are quite cumbersome and we have observed that these forms are not consistently used by the analysts (i.e., often a shorthand qualitative analysis is done, or people jump straight to quantification). Also, because the documentation requirements are so long, it is not clear which portions of the qualitative analysis impact the task breakdown and quantification, making it difficult to review in some cases. We recommend streamlining the qualitative analysis and linking it more explicitly to the quantification, perhaps through the software interface.

b.

Increased source of variability. We are concerned that, with many more factors and flexibility in the method will come very large analyst-to-analyst variability. Particularly, more work is needed in the following areas:

i.

Distinguishing between CFMs [Crew Failure Modes] of Macro-Cognitive Functions. Previous methods have had trouble distinguishing between when an error is caused by Detection, or by issues in Understanding, or by issues related to Decision-Making; and there appears to be overlap in IDHEAS-ECA for these CFMs. How does the guidance and training ensure proper categorization without double or triple-counting sources of error that may impact these Macro-Cognitive Functions?

ii.

PIF anchor point: Anchor points need clear guidance. It is unclear if the anchor point guidance is generic or should be application specific (e.g., is the definition of a complex action or adequate training or adequate procedure the same for LPSD [Low-Power Shutdown], level 1, level 1, SFP [Spent Fuel Pool],

etc.?) A set of examples or conventions would be helpful.

1.

Should also include guidance on when PIFs [Performance Influencing Factors] can be disregarded (e.g., we noted in the FLEX workshop that there were often several PIFs that can be used to represent the same underlying factors and often the analysts had to exercise judgement to discount a PIF even though it fit the verbatim description.

iii.

Recovery: I believe this is already on your list to provide updated guidance, but we have found, in practice, how recovery is credited to be a large source of analyst variability and this would benefit from clear guidance of when recovery is appropriate and how much.

iv.

Timing: For cases where time is not expansive, this can be a very important source of variability since small differences in the distribution can lead to large difference in the resultant HEP [Human Error Probability]. Recommend providing some generic guidance if possible, to reduce this. In fact, in the FLEX example workshop, analysts were unable to use the timing module and did not test that capability.

v.

Execution. Was the literature survey include Execution errors in its scope, and were there many sources of information on Execution? Or was the literature review primarily conducted on cognitive errors? Stated another way, does the

research conducted as part of the IDHEAS development show execution errors to be a lower contributor than what has previously been modeled, or was the scope of the research such that execution errors were not really included?

3.

IDHEAS-ECA - application beyond PRA [Probabilistic Risk Assessment] for Level 1 internal and external events at-power for currently operating plants.

a.

Does the NRC consider this method adequate to model HRA in digital environments? Is there any caveat or limitations to the applicability of this method?

b.

Does the NRC consider this method adequate to model HRA for Level 2 PRA?

c.

Does the NRC consider this method adequate to model HRA for Low Power and Shutdown PRA?

d.

Does the NRC consider this method adequate to model HRA for other applications such as medical?

4.

IDHEAS-ECA Publication questions.

a.

Some organizations have suggested a benchmarking activity to provide feedback on the method, and is this part of the NRCs plans prior to publication as a NUREG?

b.

Will there be a public comment period for the NUREG? The HRA Dependency portion has not been previously published and would benefit from a review.

c.

The slides mention validation vs standards.

i.

Does this mean the published documents demonstrate how IDHEAS-ECA complies with supporting requirements of the PRA Standard, specifically if/how it supports Capability Category II evaluations?

ii.

What validation activity will be conducted?

5.

IDHEAS-Dependency - it is very nice to see the NRC looking at new ways to approach dependency and we think there is a lot of promise in this approach. We are wondering if there will be a public comment period or a more formal pilot of the completed method?

6.

IDHEAS-Data - What was the independent review process for the underlying data that was mentioned in these slides?

7.

IDHEAS At-Power - What, if any, role does this method have going forward?

The NRC staffs planned activities on IDHEAS improvement and desired support from EPRI NRC staff responses to Industry Questions on 04-08-2021 IDHEAS HRA Public Meeting Questions from the public NRC staffs planned action items Desired EPRI collaboration IDHEAS-ECA development -

Dependency It is very nice to see the NRC looking at new ways to approach dependency and we think there is a lot of promise in this approach. We are wondering if there will be a public comment period or a more formal pilot of the completed method?

1) The staff is completing the draft dependency guidance and will make it public through an NRC Research Information Letter (RIL) for public comment.

2) The staff plans to update the guidance based on the feedback, then incorporate the guidance into the IDHEAS-ECA report and Software tool.

3)

1) Review and comment on the guidance

- summer 2021.

2) Voluntarily use and validate the dependency guidance in HRA applications.

IDHEAS-ECA development -

Guidance on modeling recovery I believe this is already on your list to provide updated guidance, but we have found, in practice, how recovery is credited to be a large source of analyst variability and this would benefit from clear guidance of when recovery is appropriate and how much.

The staff plans to begin modeling HRA recovery in late 2021/early 2022. The staff plans the following activities: i) reviewing literature and event reports to understand recovery mechanisms, ii) search for data on likelihood of HRA recovery, iii) possibly organize a formal expert elicitation to develop the recovery model, iv) pilot the recovery model in HRA applications, and v) incorporate the recovery model in a future update of IDEHAS-ECA.

1) Consult with industry experts on i) and ii).

2) Work with the industry on iii) and iv).

IDHEAS-ECA development -

Guidance on estimating time required For cases where time is not expansive, this can be a very important source of variability since small differences in the distribution can lead to large difference in the resultant HEP. Recommend providing some generic guidance if possible, to reduce this.

1) The staff is currently working with the Pacific Northwest National Laboratory documenting data on time required as the technical basis.

2) The staff plans to develop the guidance on time estimation and publish the guidance as a RIL for public comment.

3) The staff plans to update the guidance and incorporate it into IDHEAS-ECA Software.

1) Review and comment on the guidance.

2) Pilot the guidance with example events.

IDHEAS-ECA - Usability Qualitative analysis and documentation.

We have noted that the qualitative analysis forms are quite cumbersome and we have observed that these forms are

1) The staff is developing a more streamlined template for documenting qualitative analysis based on experience with IDHEAS-ECA so far.

The staff plans to replace the worksheets in the IDHEAS-

1) The staff is open to suggestions on better ways to document qualitative

not consistently used by the analysts (i.e., often a shorthand qualitative analysis is done, or people jump straight to quantification). Also, because the documentation requirements are so long, it is not clear which portions of the qualitative analysis impact the task breakdown and quantification, making it difficult to review in some cases. We recommend streamlining the qualitative analysis and linking it more explicitly to the quantification, perhaps through the software interface.

ECA RIL report with the new template in the upcoming IDHEAS-ECA NUREG report.

2) The staff plans to incorporate the updated qualitative analysis template in IDHEAS-ECA Users Guidance to be developed.

analysis in IDHEAS-ECA.

IDHEAS-ECA - Usability Increased source of variability. We are concerned that, with many more factors and flexibility in the method will come very large analyst-to-analyst variability. Particularly, more work is needed in the following areas: i) Distinguishing between CFMs of Macro-Cognitive Functions, ii) PIF anchor point: Anchor points need clear guidance.

1) The staff plans to develop a concise IDHEAS-ECA Users Guide to highlight the important guidelines such as selecting applicable cognitive failure modes and modeling context with the performance influencing factors.

2) The staff is considering developing training materials along with practical examples.

3) The staff plans to link the PIF attributes to the summary description of the data source references to show what the PIF attributes/CFMs mean in real examples.

The staff is open to suggestions on

1) and 2), and will ask EPRI to review and comment on the draft guidance.

IDHEAS-ECA - Usability Continuous improvement.

IDHEAS-ECA is a new method.

HRA analysts will need to learn, get familiarized with, and practice it in real applications.

Continuously collecting users feedback helps the learning curve.

Inspired by EPRIs experience with EPRI HRA Calculator, the NRC staff plans to start an IDHEAS-ECA User Group for collecting user feedback and exchanging experience.

Welcome the industrys participation in the User Group.