ML19309E316
| ML19309E316 | |
| Person / Time | |
|---|---|
| Issue date: | 09/18/2019 |
| From: | Christiana Lui Advisory Committee on Reactor Safeguards |
| To: | |
| Lui, Christiana, ACRS | |
| References | |
| NRC-0572 | |
| Download: ML19309E316 (551) | |
Text
Official Transcript of Proceedings NUCLEAR REGULATORY COMMISSION
Title:
Advisory Committee on Reactor Safeguards Reliability and Probabilistic Risk Assessment Docket Number: (n/a)
Location: Rockville, Maryland Date: Wednesday, September 18, 2019 Work Order No.: NRC-0572 Pages 1-399 NEAL R. GROSS AND CO., INC.
Court Reporters and Transcribers 1323 Rhode Island Avenue, N.W.
Washington, D.C. 20005 (202) 234-4433
1 1
2 3
4 DISCLAIMER 5
6 7 UNITED STATES NUCLEAR REGULATORY COMMISSIONS 8 ADVISORY COMMITTEE ON REACTOR SAFEGUARDS 9
10 11 The contents of this transcript of the 12 proceeding of the United States Nuclear Regulatory 13 Commission Advisory Committee on Reactor Safeguards, 14 as reported herein, is a record of the discussions 15 recorded at the meeting.
16 17 This transcript has not been reviewed, 18 corrected, and edited, and it may contain 19 inaccuracies.
20 21 22 23 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 www.nealrgross.com
1 1 UNITED STATES OF AMERICA 2 NUCLEAR REGULATORY COMMISSION 3 + + + + +
4 ADVISORY COMMITTEE ON REACTOR SAFEGUARDS 5 (ACRS) 6 + + + + +
7 RELIABILITY & PROBABILISTIC RISK ASSESSMENT 8 SUBCOMMITTEE 9 + + + + +
10 WEDNESDAY 11 SEPTEMBER 18, 2019 12 + + + + +
13 ROCKVILLE, MARYLAND 14 + + + + +
15 The Subcommittee met at the Nuclear 16 Regulatory Commission, Two White Flint North, Room 17 T2D10, 11545 Rockville Pike, at 8:30 a.m., Dennis 18 Bley, Chair, presiding.
19 20 21 22 23 24 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
2 1 COMMITTEE MEMBERS:
2 DENNIS BLEY, Chair 3 RONALD G. BALLINGER, Member 4 MICHAEL L. CORRADINI, Member 5 VESNA B. DIMITRIJEVIC, Member 6 WALTER L. KIRCHNER, Member 7 HAROLD B. RAY, Member 8 JOY L. REMPE, Member 9 MATTHEW W. SUNSERI, Member 10 11 DESIGNATED FEDERAL OFFICIAL:
12 CHRISTIANA LUI 13 14 15 16 17 18 19 20 21 22 23 24 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
3 1 C-O-N-T-E-N-T-S 2 Opening Remarks . . . . . . . . . . . . . . . . . 4 3 Introductory Remarks . . . . . . . . . . . . . . 17 4 HRA Research Program Overview/History . . . . . . 18 5 The General Methodology of an Integrated Human Events 6 Analysis System (IDHEAS-G), DRAFT NUREG-2198 7 (ADAMS Accession Number ML19235A159) . . . 28 8 IDHEAS-G: An Integrated Process for Human Reliability 9 Analysis with IDHEAS-G . . . . . . . . . 108 10 IDHEAS-G: Time Uncertainty Analysis; Generalization of 11 Human Error Data for Estimation of Human Error 12 Probabilities . . . . . . . . . . . . . . 149 13 IDHEAS-ECA (Event & Condition Assessment): Differences 14 between IDHEAS-G and IDHEAS-ECA; IDHEAS-ECA Tool 15 and Examples . . . . . . . . . . . . . . 233 16 IDHEAS-ECA Testing and HRA Future Work . . . . 344 17 Public Comments . . . . . . . . . . . . . . . . 379 18 Subcommittee Discussion . . . . . . . . . . . . 388 19 Adjourn . . . . . . . . . . . . . . . . . . . . 399 20 21 22 23 24 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
4 1 P-R-O-C-E-E-D-I-N-G-S 2 8:31 a.m.
3 CHAIR BLEY: Good morning. The meeting 4 will now come to order. This is a meeting of the 5 Reliability and Probabilistic Risk Analysis 6 Subcommittee of the Advisory Committee on Reactor 7 Safeguards.
8 I'm Dennis Bley, Chairman of the 9 Subcommittee meeting. ACRS members in attendance are 10 Ron Ballinger, Joy Rempe, Vesna Dimitrijevic, Walk 11 Kirchner, Harold Ray, and Mike Corradini. Matt 12 Sunseri should join us this afternoon after his flight 13 arrives.
14 Christiana Liu of ACRS staff is the 15 designated federal official for this meeting. And she 16 is, so we can go ahead.
17 This is the first meeting of the 18 Reliability and PRA Subcommittee since the passing of 19 Mary Drouin on September 5th. For many of us she's 20 been a colleague, a friend, and well, once in awhile 21 an antagonist.
22 For all of us in the PRA community she has 23 been a moving force as an analyst, and manager of PRA 24 projects. And later at NRC a leader in the 25 development of many PRA related programs.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
5 1 She lead the earlier HRA work on ATHEANA 2 for awhile. And was instrumental in the development 3 of the PRA standards. And was the leader of the 4 Technology Neutral Framework Project more than ten 5 years ago, which has risen again in the form of the 6 Licensing Modernization Project.
7 She was awarded the Distinguished Service 8 Award from the NRC. And I think we can say, Mary, 9 we'll miss you. We do miss you.
10 The Subcommittee today will hear 11 presentations, and hold discussions with the NRC staff 12 on a draft NUREG report, NUREG 2198, the General 13 Methodology of an Integrated Human Events Analysis 14 System, referred to as IDHEAS-G.
15 The Subcommittee will gather information, 16 analyze relevant issues and facts, and formulate 17 proposed positions and actions as appropriate, for 18 deliberation by the full Committee.
19 Before I get into the rest of the boiler 20 plate, I want to give just a short history of this 21 longstanding project. Some for new members, and some 22 for others who just weren't around during the early 23 part of that history.
24 The history of HRA really began back with 25 WASH-1400 in early '70s, late '60s. Most of what I'm NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
6 1 reporting here took place before many of the folks 2 here were involved. So, there's very little overlap 3 with the slides the staff's going to present.
4 In the '70s and '80s there were very early 5 HRA methods of three different types, decomposition or 6 database techniques. THERP, ASEP, HEART, were the 7 leaders among them. Time reliability approaches like 8 TRC, HCR/ORE, and expert judgment based techniques, 9 simple judgment, direct numerical estimation, paired 10 comparisons, SLIM, and confusion matrices.
11 All of these methodologies addressed 12 questions from the PRA team in the form of, what's the 13 probability that operator fails to do something?
14 That's what the PRA team sent over to some PRA 15 analysts who may or may not have been part of the real 16 PRA team.
17 Some years later, primarily the '90s and 18 early 2000s, there was an effort to try to bring 19 cognitive basis, and understanding of the cognitive 20 basis of human error, and the interdisciplinary nature 21 of human performance.
22 And the set of methods that became known 23 as second generation methods, ATHEANA, CREAM, CAHR, 24 ADS-IDA, and MERMOS, one of the first. And they 25 really took a new view of human error. Not as really NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
7 1 a failure of humans, but as a symptom of trouble 2 deeper inside the system. Not where people go wrong, 3 but where the overall system sets up the people for 4 failure.
5 Given the growing number of available 6 methods, and at one point I counted about 40 of them, 7 depending on what you call methods, the Commission in 8 about 2006 asked the ACRS to work with the staff to 9 propose either a single method or guidance on which 10 models to use when. And the staff will tell us more 11 about that, the SRM.
12 That SRM led to a number of international 13 meetings. And they were coincident with the Halden 14 and US HRA empirical studies, sort of benchmark 15 studies. This Committee has heard from those studies 16 along the way.
17 The general community agreed that a hybrid 18 method, borrowing on qualitative and quantitative 19 approaches for many of the others was the best path, 20 rather than trying to develop something new from 21 scratch. And that concept is what turned into IDHEAS.
22 And IDHEAS-G we're hearing about today.
23 I'm going to depart from my usual approach 24 in these meetings, and offer some preliminary 25 comments. I did it primarily because of my own bits NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
8 1 of frustration as I was reading the report, and things 2 I've heard from other members, in the hope that it may 3 allow us to focus better on technical details once the 4 presentations begin.
5 So, I'm going to take a quick walk through 6 the report. The structure of the report, at least 7 from where I sat, kind of overwhelms the reader with 8 high level philosophy.
9 It kind of makes it hard to believe that 10 there's a usable method hiding inside here. You're 11 likely to lose your readers and potential users before 12 they get to the part that would make them feel better.
13 I think there's an easy way out for that 14 problem. And I think that means expanding Chapter 1, 15 and the beginning of Chapter 2. Perhaps with well 16 constructed graphics to anticipate reader complaints.
17 Stressing why Chapters 2 and 3 are needed.
18 To provide a basis for applications, and promising 19 that the method will continue will come after the 20 preliminaries are done.
21 In Chapter 2 you really, I think, need to 22 explain why some new language and concepts have been 23 chosen. And point to the appendices for additional 24 information.
25 A very brief summary of the scientific NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
9 1 basis in 7.1.2, with a pointer to the appendix section 2 would go a long way to calm any discomfort that people 3 run into in Chapter 2. I think those could help a 4 lot.
5 I'm going to buzz through the chapters 6 kind of quickly, and just give you some high level 7 comments. These are just from me.
8 The Introduction, subject to the caveat I 9 just went through. I found a great improvement. It's 10 organized top down. And I think it tells the story 11 pretty well.
12 Time will tell if it really helps this 13 deliver easy to use, and that's a quote, application 14 specific HRAs. You haven't improved that yet. Or 15 maybe you will this afternoon when we see your first 16 applications.
17 Chapter 2, the cognition model-cognitive 18 basis structure. I think just the title of it, 19 Chapters 2 and 3, either you need a parallel subtitle 20 for Chapter 3, or just call this one the cognitive 21 basis structure. Because it's sitting there trying to 22 figure out why you got sort of half the same name.
23 Because of the chapter organization, first 24 read through this really feels very high level 25 philosophical, rather than a useful categorization NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
10 1 that really is, that will be helpful the rest of the 2 time. It's kind of jargon filled and cryptic.
3 Figure 3, when you come to it, with all 4 the cross connections is really not helpful without 5 some more explanation. These new names pop up for the 6 first time. There are lines that you don't know why 7 they're going where they are.
8 Macro cognitive functions, these are the 9 terms, processors, cognitive mechanisms, and 10 performance influencing factors, PIFs, are introduced 11 without really explaining the choice of language, and 12 labels, and what they mean.
13 In Section 7.1.2 scientific basis, we get 14 to that. But it's really rather late for a reader 15 who's been struggling now for many chapters, if they 16 made it that far, without at least some pointer back 17 to that section, or a brief excerpt from it.
18 In Section 23.12345, for each of the 19 macrocognitive functions I think if you introduced 20 each of those with the figure that comes way at the 21 end, people will see that structure. And then, those 22 next Sections 1, 2 will make sense.
23 And the Section 3, that's really how you 24 disrupt the cognitive mechanisms, would make more 25 sense. At least for me, it took several readings to NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
11 1 kind of figure out what you were doing there.
2 And then maybe Figure 2-3 will kind of 3 make sense if it comes at the end. But I'm not sure 4 if we'll need it or not. Something like it comes up 5 again later in the report.
6 Chapter 3, the cognition model, had a 7 phrase that at first put me off. It was, the PIFs 8 model context. And I was thinking back to when we 9 were applying ATHEANA. And what we found there was 10 when we looked at the human, and the system level 11 context.
12 Once we specified that in a qualitative 13 analysis, the PIFs were kind of fixed from that. And 14 on reflection I think it's the same, we're saying the 15 same thing.
16 You do a good qualitative analysis. Then 17 the PIFs model what you've already found in the 18 context. And you can use that in the quantification.
19 So, I think that it works well. It was just a little 20 awkward. I'm not suggesting you change that.
21 The PIF criteria seemed to me a really 22 desiderata. I am not sure there is any way to achieve 23 orthogonal PIFs. You want to try to. But you 24 probably can't.
25 And it's difficult to have no overlap.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
12 1 Maybe you can. So, it's more of what you want, rather 2 than, this is a criteria. And we won't go forward 3 until we meet them.
4 In any case, in Chapter 3 you need to 5 point to the PIF appendices, which are A and B.
6 They're a little cryptic. But pointing them out will 7 be helpful.
8 Chapter 4, the integrated process. The 9 introduction to Chapter 4 is a little late to tell the 10 reader that the appendices represent the guidance. I 11 think that needs to come a little earlier, with a 12 reference to the back.
13 Because you're searching for that. You've 14 already said there's guidance. But, where is it? I'm 15 looking all around. And when you get to Chapter 4, or 16 you read the appendices you find it.
17 I think it works pretty well. But a real 18 example would help. And maybe that's what you're 19 going to show us this afternoon. I'm not sure if 20 that's true of not.
21 Time uncertainty analysis is probably 22 okay. I might grumble you left out a reference I 23 though was significant there, from 1988.
24 The generalization of human error data, 25 we'll hear, I assume James will be here to talk about NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
13 1 that later? We'll hear about that. I'm not fully 2 onboard. But I don't completely understand it yet.
3 So, I've got to give that one some more time.
4 And finally, general discussion and 5 comments. You talked about you had this reviewed by 6 20 experts. You don't say who they are. I'm not sure 7 why. Maybe you can't. Or whether they're paid or 8 volunteers. And did they all really give you 9 comments?
10 You had one line I think is very cogent.
11 Methodology is intended to be high level guidance, 12 basic principles, not a handbook or manual for HRA 13 practices. So, you make that clear.
14 You make a number of claims, like guidance 15 is capable of producing good transparency and 16 traceability. Some others you soften and say, it 17 should give us a good result.
18 I don't know if you really can make those 19 firm statements that you make, if you have a basis for 20 it. If you do, it would be good to say what the basis 21 is.
22 Section 7.2 is areas needing further 23 research. Two kind of jumped out at me. And one was 24 dependency. And when I read your appendix, I think 25 you're really closer than it seemed that you think.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
14 1 The material in the appendix gives some 2 nice structure and context, which is the key to many 3 of the more severe dependent events in the operating 4 history.
5 I think you're pretty close on that. And 6 you need to give yourself more credit. I don't think 7 there's all that much more work to do. I think it's 8 more organizing.
9 The combined effects of PIFs, you hint at 10 it many times that if you want to use many PIFs there 11 are a lot of practical problems. And there really 12 are. And I think more emphasis on not getting into 13 that situation 14 You want them available. But you need to 15 pick out the most salient ones. Otherwise, and you do 16 say this somewhere in one of those earlier sections.
17 Everything washes out.
18 You got a middle kind of quantitative 19 result. People aren't able to down score the ones 20 that aren't as important, as the ones that are. And 21 you really, it gets very awkward.
22 And finally, in your references, I'm not 23 sure if you just haven't had time to clean them up.
24 There's a lot of cases where you give all the authors 25 of papers. There are many other cases where you just NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
15 1 say, et al. And that's just a personal annoyance.
2 So, I'd get rid of those.
3 Back to the meeting. The ACRS was 4 established by statute, and is governed by the Federal 5 Advisory Committee Act, FACA. The Committee can only 6 speak through its published letter reports.
7 We hold these meetings to gather 8 information, and perform preparatory work that 9 supports our deliberations.
10 The rules for participation in all ACRS 11 meetings were recently updated, and announced in the 12 Federal Register on June 13th of this year.
13 The ACRS section of the NRC public website 14 provides our Charter, By-laws, agendas, letter 15 reports, and transcripts of all open and full 16 Committee meetings, including slides presented there.
17 The meeting notice and agenda for this meeting were 18 posted on that website.
19 As stated in the Federal Register notice, 20 and in the public meeting notice on the NRC website, 21 interested parties who desire to provide written or 22 oral comments may do so, and should contact the 23 designated federal official five days before the 24 meeting, as practicable.
25 We have received a request from Mr. John NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
16 1 Stetkar. The time to make oral statements is when we 2 proceed to the public comments item on today's agenda.
3 In addition, there will be time set aside for spur of 4 the moment comments from other members of the public 5 attending or listening to our meetings.
6 We have a bridge line established. You've 7 heard it beep several times. For interested members, 8 and for the public to listen in. To preclude 9 interruption of today's meeting the phone bridge will 10 be placed in a listen only mode during the 11 presentations and Committee discussions. We will 12 unmute the bridge line when we proceed to the public 13 comments agenda.
14 A transcript of the meeting is being kept, 15 and will be made available on the NRC public website.
16 Therefore, we request that participants in this 17 meeting use the microphones located throughout the 18 meeting room when addressing the Subcommittee.
19 The speaker should first identify 20 themselves, and speak with sufficient clarity and 21 volume, so that they may be readily heard. Make sure, 22 for all of us, that the green light's on when you 23 talk. And turn it off when you're not talking. The 24 button is the nearest point of the microphone to you, 25 where it says push.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
17 1 At this time I request that meeting 2 attendees and participants turn off, silence their 3 cell phones and other electronic devices. We'll now 4 proceed with the meeting. And I'm going to call upon 5 Sean Peters of the NRC staff to begin. Sean.
6 MR. PETERS: Yes. Thank you, Dennis. I 7 would like to personally thank the ACRS Subcommittee 8 for allowing us the opportunity to come and present 9 our work.
10 I do feel a lot of the introductory 11 presentation that I was going to give has been 12 graciously done by Mr. Bley.
13 CHAIR BLEY: Come on. I don't think so.
14 MR. PETERS: But, and also, we have a lot 15 of great comments to consider. So, I look forward to 16 getting the transcripts from the meeting. We, just 17 going through the slides real quick, we're talking 18 about our general methodology. And these are the 19 acronyms that we use later in the slide.
20 And I'm the Chief. I'm Sean Peters, Chief 21 of the Human Factors and Reliability Branch. And this 22 work has been part of my life for I guess about 11 23 years now. And I'll give you a little bit of history 24 of why that's been a part of my life for 11 years.
25 So, why are we here? Dennis discussed NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
18 1 that, with the SRM 061020. And this is really a 2 tasking to the ACRS Subcommittee, to choose a method 3 or set of methods for the Agency to use.
4 I had the pleasure of, thanks to Jonathan 5 DeJesus. He pulled out the meeting transcripts from 6 the 2006 SRM. And I also got to talk to Dr.
7 Apostolakis a few months back, and ask him a little 8 bit about the history of this work.
9 In the methodology, in the Commission 10 hearing they really talked about the NRC staff having 11 three methods, and the industry having one. And that 12 they were looking for some type of consolidation 13 between us and the industry, as far as, can we kind of 14 coalesce around a central theme, or concepts that we 15 find acceptable.
16 That's kind of what they proposed to the 17 Commission at the time. And they were talking about 18 it in the meeting. And then they came back with this 19 SRM, which is, kind of expands on it a little bit 20 that, you know, we need to work with internal, 21 external stakeholders.
22 And we need to at least, if we don't 23 coalesce around particular models, we need to get 24 guidance on particular methods, and when they can be 25 utilized.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
19 1 So, kind of a little bit of a timeframe, 2 a timeline. So, we started a lot of this work before 3 the 2006 SRM, by developing the HRA good practices, 4 and evaluating those methods against the good 5 practices.
6 And we have the reports. And I think we 7 presented to the -- This was right before my time.
8 But I think we presented to the ACRS Subcommittees on 9 that, on those results of the HRA methods versus good 10 practices.
11 I was just reading those slides yesterday.
12 So I, just to refresh my memory. And around that same 13 time, in the 2006 timeframe we started out 14 international HRA empirical study, 2006, 2007 15 timeframe.
16 And coming in that international study, 17 and looking at the best practices, what they were 18 finding was that there really wasn't a methodology, or 19 even a set of methodologies that really scratched the 20 itch for all scenarios.
21 So, most of the methodologies that have 22 been out there and developed were developed for a very 23 specific context. And they simplified the human 24 action so that it could be utilized for that 25 particular instance.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
20 1 And what was a big beef in the Commission 2 meeting was that people were now taking these very 3 simplified methods, and they were applying them 4 outside of their original intent and scope.
5 And so, that's kind of where we were. And 6 coming with an international study, and say in the 7 2008, 2009 timeframe, we were starting to get the 8 preliminary results back from that study.
9 And we determined again that there wasn't 10 just one method that really fit everything. But we 11 found there were a lot of good pieces of all the 12 different methodologies.
13 So, what the staff and our partners, 14 including industry, what we decided at that time was 15 that we would take what we thought were some of the 16 best practices of each of the methodologies. And try 17 to combine them into one method. And that's where we 18 started in that timeframe.
19 And so, you got to look at in that 20 timeframe, in the 2009 timeframe, this is pre 21 Fukushima, the almost entirety of what we used human 22 reliability for was internal events at power 23 conditions. Basically modeling design basis 24 accidents, and how we would proceed forth through 25 those scenarios.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
21 1 So, we started that. And it took a long 2 time to get ideas that were consolidated. Because 3 when you're working with a ton of people, we had a, 4 basically we had a ton of cooks cooking one pot. And 5 eventually we got down to a consensus, and were able 6 to pull that together.
7 But when Fukushima hit in 2011 we had a 8 different concept. We saw a lot of interest being 9 sparked beyond this internal power applications 10 methodology.
11 We learned that, you know, we're going to 12 need a method that can do stuff beyond, outside of the 13 control room, for external actions, for FLEX 14 equipment.
15 And then, when we started thinking about 16 this as a team, we determined that, well, HRA is not 17 just going to be for nuclear power reactors in the 18 industry in the, or in the NRC in the future.
19 We think that we need a methodology that 20 can not just do power reactors, but could do like 21 spent fuel storage transportation. Could help with 22 security perhaps. Could help with medical events.
23 So, we needed something that was a little 24 less reactor focused, but more human centered. And 25 that's where we started down the IDHEAS-G process.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
22 1 So, I'll go to the next slide.
2 These are just the references. So, this 3 is just a repeat here on the bullets for IDHEAS-G.
4 So, we evaluate all the HRA methods. And they had 5 strengths and weaknesses.
6 And we wanted to integrate those strengths 7 and weaknesses. We wanted to improve our application 8 scope. We wanted to improve the scientific basis, 9 with the goal of reducing variability. This is one of 10 the big issues in the 2006 SRM.
11 And we wanted to try to improve the use of 12 data. We wanted to increase the amount of data we 13 utilize, and rely less on expert judgment. That's a 14 long term goal for us.
15 So, the intended uses of IDHEAS-G. Really 16 the concept for IDHEAS-G is to be high level guidance, 17 so we can develop these application specific tools.
18 So, we can do the simplified tools for the particular 19 problem we're solving.
20 And we've had the distinct pleasure over 21 the last few months of being able to do that. We had 22 a very major push from our internal staff, and from 23 industry, that they want to be able credit FLEX 24 equipment in their HRA scenarios and their PRA 25 scenarios.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
23 1 And what we've been able to do is develop 2 an IDHEAS-ECA tool in a matter of a few months, that 3 we will be testing. So, we'll be talking about that 4 in a little bit. But that is a simplified methodology 5 based upon the IDHEAS-G.
6 Unfortunately, the timeline for delivery 7 of that is at the end of September here, early 8 October. So, it's a little bit, just a few weeks late 9 for letting the ACRS Subcommittee take a look at it.
10 But we're still working on that right now.
11 The other intended uses of IDHEAS-G, it 12 can be used as a platform to generalize data from 13 various data sources, so that we can inform our human 14 reliabilities.
15 We can use this, as Dennis was indicating 16 he saw that it might be a little cumbersome to use it 17 as an HRA for nuclear applications. But we have 18 utilized it for those application purposes. And we 19 have some examples of doing that.
20 And you can also use it as a process 21 identification, problem identification resolution 22 tool. So, this would be systematically analyzing 23 human events using that type of format.
24 So, there's multiple uses you can get out 25 of this. But the real intention for this general NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
24 1 methodology was to provide something that is all 2 encompassing and human centered, so that we can then 3 apply it to specific problems.
4 There's just a nice picture. This is, you 5 guys have seen this in the NUREG itself. But this is 6 just a framework. So, we developed a cognitive basis, 7 along with literature, operational experience, and 8 existing HRA methodologies and data.
9 We've formed this general methodology.
10 And out of that general methodology, if you down 11 select there are simplified application specific 12 methods that come out of it.
13 So, that is the extent of my introduction.
14 And I would like to pass it over to Dr. Jing if there 15 are no questions.
16 CHAIR BLEY: On your last slide --
17 MR. PETERS: Yes.
18 CHAIR BLEY: Are there any other 19 applications on your table right now that you're --
20 MR. PETERS: There are --
21 CHAIR BLEY: -- maybe going to pick up?
22 MR. PETERS: There are none at the moment.
23 I actually have a kind of belief that once we have 24 IDHEAS' ECA out, the Event Condition Assessment out, 25 that will be utilized quite extensively for most NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
25 1 applications that we use.
2 CHAIR BLEY: Okay. Thanks.
3 MR. PETERS: But nothing, there's no push 4 right now to develop a new scenario. But you never 5 know in HRA. I never knew we would be doing this 6 beyond design basis HRA stuff back before 2011. So --
7 MEMBER REMPE: So, some of these documents 8 were co-authored, or whatever, collaborative efforts 9 with industry, right?
10 MR. PETERS: Yes. So, the idea is 11 internal events had power application. So, NUREG 2199 12 was our original intent and scope of the IDHEAS 13 project. And that was fully collaborated with 14 industry.
15 MEMBER REMPE: But this G was not. And 16 that was industry's decision, or --
17 MR. PETERS: Yes. Industry decided that 18 it didn't need a general methodology. It already had 19 an HRA toolbox and, or HRA calculator. And they 20 thought that they had what they needed for that time.
21 The interesting part is, now within the 22 last year or so they realize they didn't have all the 23 pieces they needed. And they ended up developing 24 their own approaches to handling things like FLEX 25 equipment, external events. But --
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
26 1 MEMBER REMPE: So, this IDHEAS-ECA would 2 show up under your other specific HRA applications.
3 But it's not collaborative with industry? They did 4 their own?
5 MR. PETERS: We are collaborating right 6 now with industry. So, they didn't collaborate in the 7 initial development, because this is a subset of 8 IDHEAS-G.
9 But what they are collaborating with, and 10 we'll talk a little bit more about it, is the actual 11 testing and implementation. And they're providing 12 feedback.
13 So, I'm missing some of my key HRA people 14 here today. Because they're out at the Peach Bottom 15 Site right now, with industry teams, trying to model 16 FLEX scenarios. So, developing. And we'll talk more 17 about that later today.
18 MEMBER REMPE: But they're not going, 19 where will this IDHEAS-ECA, it's a separate document, 20 right?
21 MR. PETERS: So, this will be a separate 22 document. And there's also a computerized tool that 23 we developed, that we'll show you guys later.
24 MEMBER REMPE: So, it will be a NUREG?
25 MR. PETERS: It will be a NUREG plus the NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
27 1 tool. So, an actual --
2 CHAIR BLEY: We have sessions on that this 3 afternoon.
4 MR. PETERS: Yes, we do. We do.
5 MEMBER REMPE: Yes. I just was like 6 looking at your framework, and wondering --
7 MR. PETERS: So, it's --
8 MEMBER REMPE: -- why I don't see it up 9 there.
10 MR. PETERS: It's actually the middle box 11 on the very bottom. So, IDHEAS is for event and 12 condition assessment.
13 MEMBER REMPE: Oh that's, okay. You're 14 right. It's there.
15 MR. PETERS: Yes.
16 MEMBER REMPE: And --
17 MR. PETERS: But it's really small print.
18 So --
19 MEMBER REMPE: Okay. Yes, it's, well, I 20 just was expecting to see that. But it's not 21 collaborative with industry. It will be, or when you 22 finally issue it, it will be --
23 MR. PETERS: Well, the development wasn't.
24 But the actual, the feedback from industry groups 25 about its usability and function, we will be taking NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
28 1 them into account, to update. So it's, I would call 2 it semi collaborative with industry.
3 MEMBER REMPE: Okay.
4 MR. PETERS: Because they are on the 5 testing teams --
6 MEMBER REMPE: Okay.
7 MR. PETERS: -- for it.
8 MEMBER REMPE: Thanks.
9 MR. PETERS: Jing.
10 DR. XING: Okay. So, are we going to 11 switch?
12 MR. PETERS: Yes. We can switch if you 13 like.
14 DR. XING: Okay. Okay. Good morning, 15 everyone. I'm Jing Xing, a senior human performance 16 engineer, working for Sean. And this, today is my 17 12th time presenting ideas to ACRS committee over the 18 last seven years.
19 And I really appreciate all the input we, 20 our team have been received from ACRS over this long 21 time period. And I appreciate Dr. Bley's comment on 22 our report up front. And promise that we will address 23 those comments, as we've been doing in the past.
24 So, for this to present ideas we are going 25 to have a three part presentation. So, the first NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
29 1 part, much like this morning, we'll present you the 2 general methodology of IDHEAS, the IDHEAS-G.
3 Then followed by that I will talk about 4 the IDHEAS-G application that Sean already mentioned, 5 later a bit. And at the end of the day, and I'm sure 6 we will have time, we will give you a demonstration of 7 the IDHEAS-G application, which is IDHEAS-ECA software 8 that we've been developing over the last three months.
9 So, I'll go, and the first part will be 10 presented by me and Jonathan. But I like to thank for 11 Jonathan made all the slides in the first part. That 12 was a lot of work.
13 Okay. Just a quick view of IDHEAS-G. So, 14 when people ask me, what is IDHEAS-G? And I will 15 answer, IDHEAS-G has two part. It has, IDHEAS-G 16 include a cognition model that model human performance 17 aspect, and HRA process that implement the cognition 18 model.
19 So, cognition model has two part, a 20 cognitive basis structure that models the internal 21 process of human cognition, and a PIF structure that 22 model the context, which has a condition set with a 23 factor human performance.
24 Then we implement the basis of cognition 25 model in four stages, which Jonathan will give you the NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
30 1 idea. And the little box in between is indicating the 2 cognition model allows us to generalize the human 3 error data from a variety of sources.
4 And we use that to support HEP 5 quantification. So, that part I will talk in the 6 afternoon.
7 So, start with the overview of the 8 cognition model. And thanks for Dr. Bley's 9 introduction. I guess I don't need to justify why we 10 started this project about the developing the 11 cognition model.
12 Mainly we want to understand the how and 13 the why human success and failure from task. So, I'll 14 talk the first part is cognitive basis structure.
15 So, this is a diagram of the cognitive 16 basis structure. HRA models important human actions, 17 which we call the human failure event in PRA models.
18 A human action typically consists of one or several 19 displays or tasks. So, sometimes actions too vague, 20 we break down into tasks for easy analysis.
21 So, every human task can be modeled 22 There's five macrocognitive functions, the red boxes 23 you are seeing here. Macrocognitive functions are 24 hard to come up with definition. They are the high 25 level brain functions that accomplish a task.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
31 1 So, this slide might help better what are 2 the function. So, these five functions, just to think 3 about you do any task. So, first function is 4 detection, which is noticing the cues and gathering 5 information in your working environment.
6 Then with that information you would come 7 to the function understanding, which is the 8 integration of pieces of the information in the 9 working environment, with the person's mental model, 10 your previous understanding of this matter. So, to 11 make sense of the actual scenario or situation.
12 Then it's the decisionmaking. The 13 decisionmaking is easy, just making decision, which 14 would include activities like selecting strategies, 15 planning, adapting a plan, or evaluating options, 16 making judgment on qualitative or quantitative 17 information.
18 And with all this you come to action 19 execution, which is the implementation of your 20 decision or plan, typically this involve make a change 21 in some physical component or system.
22 And we also model teamwork. I would like 23 to emphasize that the teamwork here is specifically to 24 address the interaction and the collaboration among 25 many teams, like in a complicated event such as NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
32 1 prioritize or allocating resources, oversight all the 2 communication between different parties. So, that's 3 the teamwork part.
4 CHAIR BLEY: Jing, before you leave this, 5 these macrocognitive functions look and smell a lot 6 like what used to be called, at least one form of an 7 information processing model. Is this somehow 8 different? And if it is, what are we focusing on that 9 makes it different?
10 DR. XING: Yes. The information process 11 models in the cognition science allows the study to 12 talk information process model. And you see lots of 13 terminology kind of same, like detection, or 14 understanding.
15 But the main difference from the 16 microcognitive function for information process, 17 information process model typically focus on the 18 macroprocess. The microprocess, which is a much 19 detailed level. In fact, my next slides will probably 20 show you what are the focus of the information 21 process.
22 CHAIR BLEY: Okay.
23 DR. XING: And this slides are just to Dr.
24 Bley's early comment. It's very instructive if I just 25 talk. This is a, this slide. This shows what is the NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
33 1 cognitive basis structure look like.
2 But if we go through the bigger later on 3 in the report it will help. So, I'll just go this.
4 So basically, this shows our, how our cognitive basis 5 structure look like.
6 For every macrocognitive functions, which 7 on the left most box, we delineated a set of, it was 8 shaped by a set of information processors. Yes, 9 that's what you asked. And so --
10 CHAIR BLEY: Yes. It's a new term for 11 this set of things. And --
12 DR. XING: Yes. I try, yes. We use for 13 all our previous versions, for those of you who have 14 been with us for this, yes. We used to call this as 15 proximate causes, which means it immediately cause to 16 the failure of the function.
17 And in 2017 we had, our internal staff 18 reviewed our version of report that we never presented 19 to ACRS. And the internal staff had a lot of the 20 concern with the use of that term.
21 And say it's conflicting, or have a 22 different, kind of causing confusion with the staff, 23 some similar term for different meaning PRA. So, we 24 tried to figure out a different term. It's the same 25 thing as we've been talking proximate cause for the NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
34 1 last, for the first five years.
2 CHAIR BLEY: So, these are processes that 3 you're --
4 DR. XING: Yes, these are --
5 CHAIR BLEY: -- listing here.
6 DR. XING: They actually are --
7 CHAIR BLEY: So, it's probably a good 8 term. But it's new for many of us here.
9 DR. XING: Yes. So, regardless of what we 10 call that, what would be the best term for this, it 11 means macrocognitive function is achieved through a 12 series of these processes.
13 So, taking basis of detection as a 14 example. Say one way to do detection information 15 process model would just say, okay, I, you show 16 something on the computer screen, or in my 17 environment. I look. I see. That's it.
18 But in term of macrocognitive function, 19 that to perform the task in complicated environment, 20 you started with the, you start with initiating the 21 detection, which means you establish a mental model 22 for what information you're going to see.
23 Like, in the control room there are 24 thousand alarms. You have to know which ones you are 25 going to pay attention. And then you would select NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
35 1 identify, and attend to the source of your 2 information.
3 Again, a simple information process model 4 may only have one or two source of information there.
5 A real working environment can have thousand of 6 pieces, source of information.
7 And then, these three is very much like 8 information process model. Models that you will 9 perceive the information, recognize what it is. And 10 you most likely, you will categorize or classify 11 what's this information.
12 This is alarm. But what type of alarm?
13 And that's not the end. Again, think of in a control 14 room environment. You take that, perceive that 15 information.
16 You would verify the information. And you 17 will probably modify the outcome you're detection.
18 That verification can done by yourself, and by your 19 peer checking or supervision.
20 So, when also you found information is 21 confirmed, you want use that information. You either 22 retain on yourself, I remember I'm going to use that 23 for my decision making on next process.
24 Or you communicate with other members of 25 your crew. Or you may record it somehow for later NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
36 1 use. So, this whole process from D1 to D5 2 accomplishes the detection task.
3 CHAIR BLEY: So, the detection cognitive 4 mechanisms are the, between the error stuff that 5 either works or doesn't work. And for each of these 6 you give us a selective list of things that might 7 disrupt these mechanisms, and make it a source of 8 error, if you will.
9 DR. XING: Yes. So, we call this the 10 processors, and then the brain, the neuro, the neuro 11 side of behavior subscription. They have this, what 12 later are called mechanisms that can make this 13 processors work, and to make sure they work reliably.
14 So, these mechanisms have their, only 15 works under certain conditions. The brain is not 16 infinitely capable. Beyond those conditions they were 17 less effective, or not effective at all. That's when 18 people can make a mistake, make errors in the 19 processes.
20 CHAIR BLEY: Somebody on the phone line 21 isn't muted. Please mute your phone. Go ahead.
22 DR. XING: Okay. So, we can look at why 23 it's some of the mechanisms. I think we probably, 24 most are comfortable with attention. So, attention is 25 a brain mechanism that make the resource focus on what NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
37 1 you need to.
2 Without fully attention you still can see 3 some, can see things. But the reliability for you 4 detecting information is dramatically reduced. You, 5 often you will not pay attention, like somebody give 6 you a distraction.
7 Or you were doing two tasks 8 simultaneously. And you can, when you come back to 9 your primary task you may forgot where you were.
10 That's when you can make a mistake. So --
11 CHAIR BLEY: Sorry to interrupt you.
12 You're getting very soft for in here. So, either 13 speak more into the microphone, or speak a little 14 louder.
15 DR. XING: Oh, okay.
16 CHAIR BLEY: Thank you.
17 DR. XING: Is this better?
18 CHAIR BLEY: Yes.
19 DR. XING: Okay. Thank you. Every time 20 it's our presentation I always need to remind myself 21 speak louder.
22 CHAIR BLEY: They need to get you on the 23 record. And the folks behind me need to be able to 24 hear you.
25 DR. XING: Thank you. Okay, remind me if NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
38 1 my voice goes soft again. And see, my attention was 2 on the content, but not the voice. So, that's how the 3 system works. So, cognitive mechanism, given the 4 explanation why you can fail some task.
5 And on the other hand, as we state later, 6 it will also connect to the performance influencing 7 factors, some performance influencing factor. Kind of 8 like, I was tired. So, I wasn't paying attention. Or 9 I was distracted.
10 So, this structure allows us to have, 11 understand in detail why a human can, how they can 12 fail. You can fail in any of these processes if you 13 fail detection. And why you fail. And what makes you 14 fail. So --
15 MEMBER KIRCHNER: May I ask a question?
16 How do you deal with the first category there on the 17 right, the mental model? When I think of say being in 18 a control room, because of training and such you have, 19 let's pick one of the more difficult scenarios, an 20 ATWS.
21 And we've watched now on plant visits how 22 the team, the operators respond. So, obviously 23 there's a mental model that's been trained or 24 ingrained in --
25 DR. XING: Yes.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
39 1 MEMBER KIRCHNER: -- the operators, that 2 when they get this set of cues, this is telling me I 3 have an ATWS situation. And then they start working 4 through the scenario.
5 But it seems to me very difficult to 6 generalize. I mean, the mental model, what I'm 7 talking about is actually much like constructing the 8 PRA. I mean --
9 DR. XING: Yes.
10 MEMBER KIRCHNER: But it's, you know, it's 11 burned in, in short term memory for the operator. I'm 12 having, the intention, you picked as an example 13 attention. And I can see how one can grade that, you 14 know, in a qualitative or quantitative sense.
15 The mental model strikes me as extremely 16 difficult to build, and very complex. So, how do you, 17 you know, perception of sensory information, that 18 could be, you know, visual and so on.
19 So, some of them, now pattern recognition 20 strikes me to be very similar to sorting through cues.
21 So, could you elaborate on what you mean there by the 22 mental model? Because I'm having difficulty imaging 23 constructing that --
24 DR. XING: Okay.
25 MEMBER KIRCHNER: -- given the infinite NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
40 1 variety of scenarios that could be presented to an 2 operator. And when you get out of the control room, 3 where things aren't controlled, then it's even much 4 more complicated. So --
5 DR. XING: Yes. Thank you. But I think 6 that your question already had the answer there. But 7 I'll elaborate on that perspective. So, very much as 8 you said, mental model is your understanding, what you 9 learned from training, procedure, instructions --
10 MEMBER KIRCHNER: Right.
11 DR. XING: So, how this is, and how you're 12 going. In term of the mental model for detection, one 13 part you already mentioned, pattern recognition.
14 Like, typically control room operators told me they 15 typically don't recognize a single alarm. They 16 recognize a pattern.
17 MEMBER KIRCHNER: Yes.
18 DR. XING: How they recognize pattern, 19 that's in their mental model that they, I say baseline 20 of alarms. I know what that means. That's a part of 21 mental model.
22 Other parts of mental model is within 23 like, whenever you are doing some tasks, let's say you 24 are perform a steam generator tube rupture procedure, 25 you are not just waiting passively, waiting some NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
41 1 indication alarm to show up. You have anticipation 2 what's coming up. Again, that's from your training, 3 your experience.
4 MEMBER KIRCHNER: Right.
5 DR. XING: And the look, on purpose 6 looking for that. So, the reliability for detect 7 information is much higher if you anticipating the 8 information.
9 Of course, the down side for this is, you 10 can have mental model introduced to your bias. You 11 are not anticipate something. Something is not 12 important, is not in your mental model. Therefore, 13 you can likely just not see it.
14 And the other part of mental model is, 15 when you detect something, to perform a detection test 16 you have to have your understanding of the criteria, 17 for example, procedure when detection tasks in the 18 procedure tell you.
19 Take, it's a pressurize. The pressure has 20 been, is greater than 2550. And it has been steadily 21 increasing. The first part is easy. You recognize, 22 oh, it's 2260. That's, you can say that's greater than 23 2250.
24 However, the second part, what do you mean 25 by stabily or steadily increasing? You have to have NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
42 1 some judgment here, to say this. Because it's 2 vibrating. It's fluctuating. So, that's also part of 3 the mental model.
4 MEMBER CORRADINI: So, can I ask a general 5 question? This is much more in detail than I'm used 6 to. Well, so just from a benchmarking standpoint, 7 what does Naval Reactors do for human factors?
8 DR. XING: I'm sorry. I didn't get the 9 question.
10 MEMBER CORRADINI: What does Naval 11 Reactors program do for human factors training, or 12 failure analysis?
13 MR. PETERS: So, we've, I'll step in here.
14 We've only had very limited interactions with Naval 15 Reactors. We've had a recent program where we helped 16 them initiate human reliability analysis in some of 17 their surface fleet.
18 And what they were doing was, they were 19 taking one of our old models, SPAR-H model, and 20 starting applying it to do a failure analysis. But 21 before, as I understand it, the Naval Fleet has their 22 human factors guidance.
23 When you're, so I guess I need to make a 24 distinction. There's human factors, and then there's 25 human reliability analysis. And you in fact --
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
43 1 MEMBER CORRADINI: Which one are we 2 talking about today?
3 MR. PETERS: We're talking about human 4 reliability analysis.
5 MEMBER CORRADINI: Okay.
6 MR. PETERS: And so, there has not been a 7 lot done in the Naval Fleet with human reliability 8 analysis until recent times. And this is based upon 9 the two recent accidents that they've had in the Navy.
10 They've instituted a new program to start looking at 11 the --
12 MEMBER CORRADINI: I see.
13 MR. PETERS: -- reliability of their 14 design systems. So, human factors tells you how to 15 design a system. HRA tells you when, you know, what 16 are the probabilities of success or failure --
17 MEMBER CORRADINI: Yes.
18 MR. PETERS: -- with that system. And so, 19 the Navy has had human factors guidance too. But as 20 I understand, there have been some challenges with 21 implementing that, as evidenced by some of the changes 22 that they've done in their surface fleet.
23 MEMBER CORRADINI: So, to say it in a 24 different way, just so I understand. So, when you say 25 human factors, there is a training program which they NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
44 1 effectively have implemented.
2 MR. PETERS: Yes.
3 MEMBER CORRADINI: And they rely on a 4 training program.
5 MR. PETERS: That's the --
6 MEMBER CORRADINI: The HRA, and that's 7 human factors.
8 MR. PETERS: Yes.
9 MEMBER CORRADINI: The HRA is, given all 10 of that, what's the chance it would go awry?
11 MR. PETERS: Yes.
12 MEMBER CORRADINI: And they historically 13 have not done that?
14 MR. PETERS: Yes. There has not been a 15 human --
16 MEMBER CORRADINI: Oh.
17 MR. PETERS: -- reliability program --
18 MEMBER CORRADINI: Okay.
19 MR. PETERS: -- in the Navy until just the 20 last year or so.
21 MEMBER CORRADINI: So, this is more of a 22 digression. The Chairman will tell me to be quiet.
23 Is this under the assumption their training is good 24 enough?
25 CHAIR BLEY: I think we shouldn't pursue NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
45 1 this one.
2 MEMBER CORRADINI: Okay. Fine.
3 MR. PETERS: Okay. Yes.
4 MEMBER CORRADINI: That's fine. First, 5 they don't share a lot.
6 MR. PETERS: Yes. You know, that's a good 7 point. I'm only speaking from what I know.
8 MEMBER CORRADINI: That's fine. And then, 9 so one other one, if you'll allow me. Airlines?
10 Training for pilots? Similar sort of breakdown? Has 11 that been looked at in terms of a benchmark?
12 MR. PETERS: We have done some pilot 13 studies. So, when we're looking at human error 14 probabilities the airline industry is very similar to 15 our industry.
16 MEMBER CORRADINI: Oh, okay.
17 MR. PETERS: Highly trained operators.
18 Highly trained, highly proceduralized in how they 19 train scenarios and systems. And as we've gotten a 20 lot of human reliability data from the airline 21 industry.
22 MEMBER CORRADINI: Okay.
23 MR. PETERS: And so, it's, I'm not overly 24 familiar with what type of human reliability analysis 25 they do. But --
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
46 1 MEMBER CORRADINI: I was just trying to 2 understand similarities. Because --
3 MR. PETERS: There are.
4 MEMBER CORRADINI: As you get into the 5 details it kind of loses me. But that's okay. Sorry.
6 MEMBER DIMITRIJEVIC: Yes. This is what 7 I wanted us to comment. You know, when Dennis gave 8 you the, his overview, he said that these first two 9 chapters confused the people how, this is very nice, 10 you know, detection, understanding, decision making.
11 They all show how humans do something.
12 But how does this help us to get the human 13 error probabilities? And I will, it actually it is 14 not clear at all. You have break up to, you know, D1 15 to D5, and the DA to D. And this is all true.
16 But what does that have to do with 17 anything? That's basically -- So, if you can connect 18 these models to us. Is this only used, like model, 19 mental model of cues do?
20 For every human action we have to evaluate 21 what is their mental model of cues? Or do you use 22 these to transfer to these performance indicators?
23 What is the point of these break up?
24 DR. XING: Yes. The point is, just your 25 last sentence. This last layer, the cognitive NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
47 1 mechanisms directly tied to performance influencing 2 factors.
3 In fact, there had been lots of studies, 4 experimental data, and data from other field, like 5 aviation data, has been done on how severe is factors 6 different to performance influencing factors? With a 7 factor there's cognitive mechanisms. Therefore, lead 8 to the errors in the processors.
9 So, without legend, without these two 10 layers of the processors and the cognitive mechanisms, 11 that's where we were like seven years ago. We just 12 say, okay, workload a heavy impact on these task 13 performance. High workload they are more likely 14 failure.
15 But we don't speak in detail what aspect 16 of workload, to what kind of, what aspect of a task 17 make the failure. Therefore, our modeling, the 18 estimation for the failure probability can be largely 19 subjective.
20 So, I when I was think of high workload I 21 was thinking, oh, just too much work to do. I'm 22 stressed. And another person would think, oh, high 23 workload. You gave me too much, and too many. I need 24 to do concurrent task.
25 So, these two will make a very different NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
48 1 impact in term of human error probability. So, with 2 this structure we can model to the specific aspect of 3 the --
4 MEMBER DIMITRIJEVIC: If you will say, 5 like, workload influences these attention --
6 DR. XING: Yes.
7 MEMBER DIMITRIJEVIC: -- or something.
8 Whatever --
9 DR. XING: Right.
10 MEMBER DIMITRIJEVIC: -- how many factors.
11 All right. Okay. Well, this is what you want to say.
12 But, you know, if you can correct to us how the, these 13 factor, which maybe you can even move faster than 14 through those. It's one thing we know, how that fits 15 in the big picture.
16 DR. XING: Okay. Yes. I think that 17 that's --
18 (Off microphone comment) 19 MEMBER DIMITRIJEVIC: Right. It's that 20 you hear me.
21 (Simultaneous speaking) 22 CHAIR BLEY: It's interfering with --
23 MEMBER REMPE: But it would be good for it 24 to be on the record.
25 CHAIR BLEY: The record may be having NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
49 1 trouble.
2 MEMBER DIMITRIJEVIC: Okay.
3 DR. XING: Okay. Yes. Thanks for the 4 question. And so, I will not go so into detail for 5 everything --
6 MEMBER REMPE: Well, okay. So, I'd like 7 to share Vesna's comment. But I don't have her 8 background. But this thing about, don't see how it 9 connects, I really struggled with it.
10 But the reason why I struggled with it, I 11 saw a lot of examples that didn't make sense to me, to 12 try and emphasize your point.
13 For example, under understanding. Assess 14 the status based on indirect information. Assessing 15 system status typically involves integration 16 processing and inference from many pieces of 17 information to interpret the situation.
18 For example, assessment of the nuclear 19 power plant core damage involves many aspects of the 20 plant's status, such as whether the core debris is 21 relocated, whether the reactor pressure vessel is 22 breached, and whether the containment has uncontrolled 23 leakage.
24 These plant conditions don't have 25 instruments to provide a direct indication. And the NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
50 1 crew must integrate multiple pieces of information to 2 determine the status.
3 It was three years after the Three Mile 4 Island accident happened before they recognized half 5 the core was gone. The crew never had that 6 information. I mean, there was a hydrogen pressure 7 spike.
8 But it's like, there were examples that I 9 just couldn't see how you would estimate a failure 10 probability, and use it in HRA. I mean, I ended up --
11 There were other things too where I think there's just 12 typos.
13 Like, you talk about a steam generator 14 tube rupture. And later then you talk about a 15 ruptured steam generator. And I don't think those 16 things are the same.
17 And so, I think, like, I don't have the 18 perspective that Dennis and Vesna have. But boy, it 19 would make it easier for someone who doesn't 20 understand all the nuances, to understand where you're 21 going.
22 But anyhow, I have other examples. I'll 23 bring them. I have other examples I can bring up 24 later that I'd like some clarification on too.
25 DR. XING: Thank you. So, we'll look into NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
51 1 that to say, to import in the report. Okay.
2 CHAIR BLEY: Don't go past teamwork.
3 Don't go past teamwork. This one smells different 4 than the others. And --
5 DR. XING: Okay.
6 CHAIR BLEY: I want to ask you about a 7 couple of things. You say that this deals with inter-8 team teamwork. And you talk about what that is. And 9 then you say, the within team interactions are part of 10 the other four macrocognitive functions.
11 I looked back, and they weren't directly 12 discussed under the other four. You might want to 13 bring out, what I'm thinking, I saw some cognitive 14 mechanisms that could be internal teamwork related.
15 But you might want to bring that out in the other 16 four. Because it's not --
17 DR. XING: Okay.
18 CHAIR BLEY: When you say it --
19 DR. XING: Yes.
20 CHAIR BLEY: -- here, and the reader goes 21 back and tries to figure out, well, where is it? I 22 don't think you find it. At least, you don't find it 23 directly. And just a minor comment.
24 MEMBER DIMITRIJEVIC: Especially because 25 this actually can improve. All your performance NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
52 1 indicators are negative impact on the HRA, based on 2 your formulas you use to calculate.
3 So, team can actually improve performance.
4 Because you have a multiple inputs to the decision 5 making. So --
6 MEMBER KIRCHNER: And also can disimprove 7 --
8 MEMBER DIMITRIJEVIC: Okay. It also can 9 do both.
10 MEMBER KIRCHNER: -- at times too, which 11 --
12 MEMBER DIMITRIJEVIC: Yes.
13 MEMBER KIRCHNER: -- is the worst --
14 MEMBER DIMITRIJEVIC: Yes.
15 MEMBER KIRCHNER: Hopefully, in general it 16 does improve the decision making and other factors 17 that you identified. But the teams could work at 18 cross purposes.
19 MEMBER DIMITRIJEVIC: That's true. So, 20 it's unclear how does this counts.
21 DR. XING: Yes. One point about the 22 teamwork in the individual functions. It's like 23 earlier when we talked detection. The last two 24 processors.
25 After you perceived the information, it's NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
53 1 one, probably one person perceived the information.
2 You need to go through a verification or explanation.
3 That's highly rely on teamwork. And there 4 has been lots of studies show that if one person, this 5 peer checking, without peer checking, versus with peer 6 checking. And the versus as supervisor feedback.
7 So, human error rate are dramatically 8 different in those situations. That's how we model 9 the teamwork aspect in individual functions.
10 CHAIR BLEY: Yes. And I think you 11 identified mechanisms that either help teamwork or, do 12 a good job or not. But then you give ways in which 13 that can fail. And I think that's kind of the key.
14 This is kind of similar to Joy's comment.
15 It, you have sort of an example. And you talk about, 16 gee they're, when you model this for nuclear power 17 plant control room, it always consists of three or 18 more operators working together to perform their 19 tasks. Well, that's normal. That's not always.
20 DR. XING: Yes.
21 CHAIR BLEY: There are, Robinson Fire.
22 There was a fire at a couple of other plants where you 23 ended up with really two people on the team who were 24 doing that work. And everybody else was doing other 25 stuff. So, that's a normal situation, and not a NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
54 1 guaranteed situation. Anyway.
2 DR. XING: I saw that they have --
3 CHAIR BLEY: Careful.
4 (Simultaneous speaking) 5 CHAIR BLEY: Well, the rule is they got to 6 be there. But if they get tied up doing something 7 different --
8 DR. XING: Okay.
9 CHAIR BLEY: -- they're not participating.
10 Go back and read the Robinson Fire story again.
11 You'll see.
12 MEMBER DIMITRIJEVIC: And so, if you 13 hadn't mentioned, I was told the power internally 14 rise. And you into Level 2, then you have even 15 different sensors making decisions. This becomes much 16 more complicated.
17 MR. DeJESUS: And that's where we were 18 hoping to have some of this teamwork factors help us 19 calculate, and help us understand for these external 20 teams that interact with the decision making 21 processes.
22 MEMBER REMPE: I have a curiosity question 23 too. Like, Figure 210 has this whole thing about the 24 field operator detects something, and reports back to 25 the folks in the, I guess the technical support NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
55 1 center. He reports back to the supervisor.
2 Has anyone ever modeled this in a PRA in 3 the past? And do they, I mean, it's more related to 4 Vesna's question. And how this is really going to 5 help doing an HRA analysis in a PRA. And do people 6 really do that? Do we have data for it?
7 I mean, it goes into this nice figure, and 8 this whole process. But has anyone used this? And 9 what would you do with it if you had that ability to 10 model it?
11 MEMBER DIMITRIJEVIC: Well, I mean, there 12 was -- Yes. People make attempts to model that in the 13 Level 2, you know. But everybody has a different 14 solution. There was a couple of papers in conferences 15 on those subjects. But, you know --
16 MEMBER REMPE: Because again, I don't look 17 at this stuff. But people actually do this? And try 18 and get data to simulate this? Because it just seems 19 like it --
20 MEMBER DIMITRIJEVIC: Well, there is no 21 data. There is no data. This is all based on the 22 models.
23 MEMBER REMPE: So, you might put a 50 24 percent probability for failure, with plus or minus a 25 factor of whatever on it?
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
56 1 MEMBER DIMITRIJEVIC: I estimate those in 2 performance indicators, you know, what is impacting 3 his performance.
4 MEMBER REMPE: Okay.
5 MEMBER DIMITRIJEVIC: How much time he 6 has, and things like that.
7 MEMBER REMPE: Okay. So, this is a valid 8 thing to be --
9 (Simultaneous speaking) 10 CHAIR BLEY: -- qualitatively this way 11 lets you see where there might be problems.
12 MEMBER REMPE: Okay. So --
13 CHAIR BLEY: And then maybe you have to 14 use judgment. And maybe it's not so funny if it's a 15 .5, you know. Maybe it's something different because 16 there's a lot of confidence that it ought to work.
17 But yes. At least you can understand what's going on, 18 and where the problems could crop up. And that's what 19 they're trying to do here, I think.
20 DR. XING: Yes.
21 MEMBER REMPE: Okay. So, I just was 22 surprised. Because I just have not seen that kind of 23 stuff. Again, this is not --
24 CHAIR BLEY: Well, no --
25 MEMBER REMPE: -- my area.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
57 1 CHAIR BLEY: And nobody would lay out a 2 simulation of this with sticking numbers on all those 3 little pieces.
4 MEMBER REMPE: Okay.
5 CHAIR BLEY: You just can't. You're 6 right. There's no basis.
7 (Simultaneous speaking) 8 MEMBER DIMITRIJEVIC: -- to sound really 9 like steam generators, you know, which you want to 10 use, all right. But when you come to the mouth, you 11 know, you want to scrub.
12 So, you have a contradictory Level 1 and 13 Level 2 action. It's an interesting area. But nobody 14 ever did the really good job in --
15 MEMBER REMPE: Okay.
16 MR. CHEOK: So, this is Mike Cheok. I 17 want to make a comment on what Vesna said earlier, and 18 what Dennis said. I think Dennis said it correctly.
19 So, one reason why we have broken up all these 20 different cognitive factors into smaller chunks, so to 21 speak, is somewhat for us to be able to evaluate human 22 actions. For example, in specific actions.
23 So, a lot of the useful HRA nowadays is in 24 event condition analysis, ECAs. And in that sense I 25 think a lot of the criticism of the models we have now NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
58 1 is that, you know, the, because of the somewhat 2 limited set of performance shaping factors. And the 3 fact that we have very set functions for the 4 probabilities, for the different success levels for 5 these performance shaping factors.
6 The argument for more refined numbers 7 always was, you know, we didn't have enough factors to 8 talk about. As Dennis had said earlier. I think this 9 allows us to at least in a qualitative sense bring up 10 the different factors that would influence. So, in 11 performance influencing factors.
12 And then that will actually provide us 13 with a platform for discussion, as to what certain 14 factors should be, in terms of what a probability of 15 a failure of a certain event would be.
16 DR. XING: Thanks. Okay. I think we 17 already discussed this. Like, why we like this 18 cognitive basic structure. How it help us to 19 understand HRA.
20 The main purpose for this structure is to 21 help us, give us a systematic way to understand the 22 failure of the human action. So, follow this process.
23 So, a failure of any macrocognitive 24 function will lead to the failure of the task of human 25 action. The next level we understand, failure of the NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
59 1 function resulted from errors in one or more 2 processors.
3 And the error in the processor may occur 4 if one or more associated cognitive mechanisms do not 5 work properly or reliably. And because PIFs affect 6 the capacity limit of those mechanism, makes it not 7 work reliably.
8 So, this give us a systematic 9 interpretation, and a more concrete, concise way to 10 model a human failure. So, we, that we move closer to 11 what we want our HRA method. It's all, it's 12 transparency. We can clearly see what, well, what 13 went wrong, why, and how.
14 Okay. So next, any question, please? I 15 will move to the PIF structure. So again, thanks for 16 Dr. Bley introduce the context concept. So, context 17 are the conditions that affect the human performance 18 of an action. They can either have a positive or a 19 negative context.
20 And that they are more, they are general 21 descripting. Performance influencing factors, I like 22 to say, we say performance influencing factors it's 23 the same as the performance shaping factors in some 24 HRA method. So, we just call them PIF.
25 So, performance influencing factor is NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
60 1 actually a model, to model the context. So, Dr. Bley 2 commented on this.
3 So, as with, one lesson we learned from 4 the International HRA Benchmark Study, one important 5 lesson learned was the PIFs are kind of vague or 6 confusing, say which PIF be like -- Should I use this 7 PIF or that one? What exactly does PIF mean?
8 So, one direction from that study is, we 9 do need a better PIF model. So, as we started this 10 project we were thinking, what we mean by better? So, 11 we develop this set of criteria. So, okay. That's 12 what we would think. If we can achieve this, that 13 would be great.
14 So, first of all, initially we put the 15 word completeness there. Then we said, okay, that's 16 never possible. So, it was, okay, the PIFs need to be 17 pertinent to human error.
18 And they need to be comprehensive enough 19 to address our current knowledge of human performance 20 issues. So, and the way I, we hope they can be 21 independent from each other.
22 I heard what, Dr. Bley's comment early.
23 But we are still try to make close to this. And they 24 need to be specific enough, so to know what you're 25 reading into this one.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
61 1 And additionally we have these three, what 2 we decided. They should be explainable. So, if we 3 should be able to explain this PIF. How they affect 4 human success or failure.
5 And they should be assessable. If you 6 have a PIF there, like we used to have workload in 7 many HRA methods. You have that. People can be at 8 least 100 different interpretation what we mean by 9 workload. So, it should be objective. You can assess 10 it. And --
11 CHAIR BLEY: IDHEAS at-power kept 12 workload. But you gave it a structure that I thought 13 was very useful at that point. I forgot to go back 14 and compare that structure with the current set of 15 performance influencing factors. You kept essentially 16 all of them, but gave them new names, or something?
17 They aren't called --
18 DR. XING: Yes.
19 CHAIR BLEY: -- workload anymore.
20 DR. XING: Okay. You want me just talk 21 about that?
22 CHAIR BLEY: I want you to talk about 23 that, yes.
24 DR. XING: Okay.
25 CHAIR BLEY: Thank you.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
62 1 DR. XING: So, first example, in IDHEAS 2 at-power application actually program wise we develop 3 the at-power and IDHEAS-G almost in a parallel way.
4 And earlier we had performance shaping factors there 5 called the workload.
6 Then, when we come to expert elicitation 7 to estimate how the bad, or not, for workload, a 8 normal workload, how much that related to the human 9 error probability.
10 The entire expert group spend like about 11 half a day debating what is workload? Then the 12 researchers keep giving example. Then the expert, 13 like trainers, say to supervisor, no, no. That's not 14 workload to us at all. Yes, it's more work. But 15 doesn't mean we're going to make more error.
16 So, we actually did a small study, 17 combined with what research, the comments related 18 should talk about workload. Then I gave those, that 19 matrix to the expert.
20 So, now you tell me, what, which one of 21 this you think is workload, which will really be to 22 human errors. So, through that interaction we 23 actually develop a set of matrix actually. Here we 24 call it attribute of workload. Those are --
25 CHAIR BLEY: Here you call what?
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
63 1 DR. XING: In this report they are more 2 like attribute, PIF attribute, yes. That's when the 3 very beginning we had this concept. Back then we just 4 called it indicators for workload. Those are the 5 seven ones I remember.
6 The first one, according to the expert, 7 it's a scenario familiarity. If you come to a 8 scenario you're not familiar with, you never trained 9 on, you are doomed to fail. That's the number one, 10 they think, that one. So, that's high workload. And 11 whenever I come to a scenario I'm familiar, that's 12 high workload.
13 The second one was multi tasking, okay.
14 So, I don't care how much each individual tasks, how 15 many steps I have to push, push, push. But if you 16 want me do multi tasking, I'm going to fail.
17 And others like the complexity of the 18 test, tank pressure. And they even mentioned that is, 19 you know, at control room, physical demand, and 20 special requirement on your physical manipulations.
21 So, in IDHEAS at-power we actually break 22 down the workload into many of those indicators. So, 23 that probably explain this, the criteria with which 24 more objective, assessed for, and quantified for.
25 And this is our goal, try to make the PIF NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
64 1 structure to what this means. And we probably never 2 achieve the ideal status. But we are moving toward 3 that direction.
4 MEMBER DIMITRIJEVIC: So, what do you mean 5 by quantifiable?
6 DR. XING: Quantifiable, you, because 7 essentially all the HRA method pretty much depend on 8 you assess the PIF. And then either use some kind of 9 data, or expert judgment. Say, okay, this, again use 10 the workload as example. A high workload will make 11 the HIP as a .1, or would increase the HEP three 12 times.
13 MEMBER DIMITRIJEVIC: I know. But that's 14 not, quantifiable means that you have some method to 15 quantify it. It's a sort of expert judgment. It's a 16 guess.
17 DR. XING: Yes. So --
18 MEMBER DIMITRIJEVIC: It's a educated 19 guess.
20 DR. XING: Yes. So, when we mean 21 quantifiable we hoped the PIF should connect to what 22 we know the human performance measures or human error 23 data.
24 MEMBER DIMITRIJEVIC: Okay.
25 DR. XING: That way it was, PIF is the NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
65 1 direct link to those information. Then we can 2 quantify. We can, we're in a better position to 3 quantify.
4 Okay. A starting point for us to develop 5 this structure. First of all, we reviewed all the 6 PIFs in existing HRA method, make sure we don't 7 missing the ones already there.
8 And beyond that we also reviewed a number 9 of human events outside the control room. So, make 10 sure we capture the human performance issues, or PIFs, 11 in those domains, like in severe accident, or extreme 12 operating conditions, like a beyond the design basis 13 event.
14 So, we expand the PIF scope from what we 15 had before. And we also considered the PRA standard, 16 which have a list of PIFs that you should consider 17 this, in the PRA. And we made sure we covered all 18 those ones.
19 And we also develop new PIFs to address 20 the criteria we discussed in the previous slide. I 21 can give the example, in term of a specific 22 specificity.
23 Again, so like, we know from the cognitive 24 literature that, actually lots of study, when human 25 perform a concurrent task that will, the human error NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
66 1 rate will increase a lot. Before the HRA analyst we 2 observed, they used different. Because there isn't 3 such a PIF explicitly.
4 So, they use different ways, and to 5 stretch the existing PIF to model this. Some analyst 6 modeled and stress, and say, oh, if you have me do 7 multi task I'm stressed.
8 And other analyst modeled it as a 9 complexity. There's a PIF there. However, in our 10 SPAR-H model, and in some other HRA method, stress 11 probably they give you a increase of your HEP like 12 twice.
13 MEMBER DIMITRIJEVIC: Okay. Thank you.
14 DR. XING: Yes. Complexity can increase 15 up to 50 times higher. So, they assess the same 16 context model in different PIF. Ended up very 17 different HEP.
18 So, that's the situation we see. We have 19 data. We have a component basis, know the component 20 mechanisms. We make a separate new PIF with 21 concurrent tasks. You're performing concurrent 22 multiple tasks.
23 (Off microphone comment) 24 CHAIR BLEY: Microphone.
25 MEMBER DIMITRIJEVIC: Oh, sorry. How did NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
67 1 you assess that one?
2 DR. XING: Yes. Okay. In order to assess 3 that, that's actually a PIF we call the concurrent 4 tasks. And you found interruption or distraction.
5 So, they're different when you perform a concurrent 6 task.
7 If you perform two tasks simultaneously, 8 and both are your primary tasks, you have to do base 9 entry that. And that's the highest impact. Some 10 later data show that when you perform a concurrent 11 task, in fact human errors can increase ten times 12 higher if you're doing the model of detect 13 information.
14 But if you are doing the diagnosis of 15 problem you are simultaneous at diagnosis to cite a 16 problem. Research data, some experiment shows the 17 error rate can increase 30, up to 30.
18 One data show this error rate increase 37 19 times, because you're doing a concurrent diagnosis.
20 Both have some intermingle of the prompters in 21 between. That's the highest level of concurrent task.
22 The first is from other situation. You 23 have a primary task to do. But you, there's some 24 interruption, had take me one minute or two minute for 25 something else. We can imagine that can happen a lot NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
68 1 in that situation.
2 Or there's lots of study on that kind of 3 multi tasking, which we define as interruption. In 4 hospitals, doctor, nurses work. That's another 5 attribute, effect it has.
6 And, a lower level on that is a 7 distraction. So, I'm doing my job, just lots of 8 distraction. Like we say -- You have a question?
9 CHAIR BLEY: I have a question and a 10 suggestion. If I do a quantitative analysis of 11 something I'm interested in for the PRA, some human 12 action, my qualitative analysis ought to identify the 13 potential macrocognitive function areas that could be 14 troublesome.
15 They ought to be able to identify the 16 processors that could be troublesome. Maybe they'll 17 identify the cognitive mechanisms. With this really 18 massive list of PIFs in Chapter 3, I probably can't 19 easily connect that.
20 When I go back to your Appendix B I see 21 the connections. But they're complex. I can't judge 22 right now if you're complete there, if you've done a 23 good job or not. But it sort of looks like it. It 24 passes the quick test.
25 Two points. One, I think it would help NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
69 1 anybody trying to use this, before they get to Chapter 2 4, to have some kind of statement. Maybe it's Chapter 3 1, something in the introduction saying, you know, the 4 way you do this analysis is, you do a qualitative task 5 analysis, and identify all the key things. Those will 6 show up in Chapters 2 and 3, and Appendix B.
7 If I really try to develop a method that 8 uses the plethora of PIFs, and those linkages that I 9 see in Appendix B, you pretty much have to have a 10 computer program that will remind you, if this is your 11 cognitive mechanism, here's the set of PIFs you ought 12 to be considering, from which you could then winnow it 13 down to the ones that are important for your 14 qualitative analysis.
15 So, I think, I kind of think you need 16 something up front to say, the real starting place is 17 at qualitative analysis. Then you apply these things.
18 That will get you down to maybe the cognitive 19 mechanism level.
20 And then you, I guess this afternoon we'll 21 see how you're trying to apply it. And you talk about 22 a computer program. I'm hoping that takes me from an 23 identified macrocognitive function processor, and 24 cognitive mechanism, to asking me about the relevant 25 PIFs.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
70 1 Is, first question is, is that true?
2 Because I don't know how you do it without a computer 3 program. I mean, I can't lay all these tables out and 4 --
5 DR. XING: Okay. I --
6 CHAIR BLEY: And I certainly can't 7 remember them all.
8 DR. XING: I cannot direct say true or not 9 true. Because there are two parts there.
10 CHAIR BLEY: Go ahead.
11 DR. XING: The first part is not true. I 12 think that's our fault. We not saying this clearly in 13 the introduction. The bottom layer, the cognitive 14 mechanisms, is the way for us to understand the 15 method.
16 But we never intended for HRA analyst go 17 through that. So, HRA analyst can stop at the high 18 level. They identify what macrocognitive functions.
19 CHAIR BLEY: Okay. I can do that.
20 DR. XING: Yes.
21 CHAIR BLEY: I agree with you.
22 DR. XING: What macrocognitive --
23 CHAIR BLEY: I can do that.
24 DR. XING: -- function this task. Then 25 they go to, from there they can direct go to assess NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
71 1 the PIF attribute, which task --
2 CHAIR BLEY: How do I get from the 3 macrocognitive function to the relevant PIFs?
4 DR. XING: Okay. You can, they can just 5 result of thinking about the link, which one link to 6 which one. You can just, you assess the PIF, based on 7 your context. So, I can have this PIF --
8 CHAIR BLEY: I can do that when I've got 9 six or 12 PIFs.
10 DR. XING: Yes. Normally you don't get 11 more than that. You were saying this after seeing 12 James' software. Even we have probably 20 PIF, but 13 for each, and not --
14 CHAIR BLEY: And each of which has many 15 attributes. Right?
16 PARTICIPANT: Yes.
17 CHAIR BLEY: Yes.
18 DR. XING: Well, we, that's we worked in 19 the IDHEAS-ECA, which there are only, we only receive 20 one requirement in ECA from our user. Make it easy to 21 use. Almost as simple as, no more complicated than 22 SPAR-H.
23 CHAIR BLEY: Well --
24 MEMBER DIMITRIJEVIC: Are we, will you 25 show this later, how he selects --
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
72 1 DR. XING: Yes.
2 MEMBER DIMITRIJEVIC: -- which PIF?
3 CHAIR BLEY: Okay.
4 MR. PETERS: And you also --
5 DR. XING: Yes.
6 MR. PETERS: -- need to understand the 7 difference between the general methodology and an 8 application --
9 DR. XING: And the specific --
10 MR. PETERS: -- specific methodology, is 11 that with specific methodologies we try to initially 12 whittle out the non important factors, the factors 13 that wouldn't really apply in the situations this 14 methodology supposed to be used. So, it whittled that 15 set of performance shaping factors down to a more 16 manageable number.
17 CHAIR BLEY: If we whittle it down to five 18 or six --
19 MR. PETERS: That's --
20 CHAIR BLEY: I can readily deal with that.
21 MR. PETERS: Yes, yes. But then again, 22 that's probably too few for the granularity you want.
23 But there is, we do have a way to do that in our ECA 24 tool. And James will --
25 CHAIR BLEY: Okay.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
73 1 MR. PETERS: -- present that.
2 CHAIR BLEY: So, we'll see it then?
3 MR. PETERS: Yes.
4 CHAIR BLEY: And now we need, the reader 5 needs some hint that they'll get help through this --
6 DR. XING: Okay.
7 CHAIR BLEY: -- massive detail. And the, 8 you know, like I said, the linkages are hid in 9 Appendix B. But when I start reading Chapter 3 I say, 10 well, how do I connect these? And you do it in the 11 appendix. But I hope you did it well.
12 DR. XING: Yes. Well, we had a massive 13 set of reviewers that reviewed the early versions of 14 report. Several reviewers suggested, can you just 15 eliminate all your discussion of cognitive mechanisms?
16 Because the PRA analysts don't need to know that to do 17 their job.
18 MEMBER DIMITRIJEVIC: Yes. That's because 19 well, that is so true. Because for me this is, I 20 have, I'm a new member. So, I will just --
21 You are all been around. And you think 22 that's a useful matter here. That's what you wanted 23 all the time. And you reading there. Because then 24 you just sort of go with fate. That's what I decide.
25 I'm going to believe that it is a useful matter here.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
74 1 But I couldn't prove that to myself.
2 DR. XING: Yes. And that's so true.
3 That's, I think is a, that's the set of, you are be 4 the first set here. And we also had a same comment, 5 similar comment on the same issue but just the 6 opposite.
7 Say, oh, the company can. Great. And now 8 I understand why they are previously, when I assess 9 the PIF I have always not sure if it's really have a 10 impact on this action. But now, by knowing the 11 mechanism, I know.
12 So, we've been struggling in this last 13 version, in this recent re-writing. So, we still 14 decide to keep them. But because this is a general 15 methodology.
16 But in the, in application specific 17 method, like IDHEAS-ECA, you will not see this. This 18 terminology not even appeared in our ECA before the 19 analysts -- That's just the substrate to help us 20 establish the connection.
21 CHAIR BLEY: I, this is personal opinion.
22 I think the connections you make in Chapter 2 across 23 the processors to the mechanisms are pretty clear, and 24 make sense.
25 And I think it is a help for people to NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
75 1 understand. And if you want to use any HRA method, 2 you really ought to understand what's behind it, and 3 what's in peoples' heads, you know, that you're trying 4 to model.
5 The Chapter 3 discussion of the PIFs has, 6 yes, makes sense. But maybe you need, maybe you have 7 the list. Once we get to the tables -- Yes, I don't 8 -- The link from the text to the tables, I'm not sure 9 how good that is.
10 Maybe I just wasn't reading carefully 11 enough. Maybe it links it. But, you know, the tables 12 have loads of information. And they're good to go 13 through.
14 But then you wonder, oh, how am I going to 15 use this? Because any one of these could potentially 16 affect any of the key attributes, the processors, and 17 mechanisms, and macrocognitive functions.
18 So, I think you just need some hints for 19 people, of what it's for. But in any case, I wouldn't 20 get rid of --
21 DR. XING: Okay.
22 CHAIR BLEY: -- the concepts in Chapter 2.
23 I think they're pretty important. And after I start 24 using --
25 DR. XING: Okay.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
76 1 CHAIR BLEY: -- this I'll go back and 2 study that some more. My worry about HRA methods is, 3 we want something that's fast, easy to use, and you 4 don't have to think. And that leads you into really 5 bad analysis. Go ahead.
6 DR. XING: Okay. So, we have discussed 7 the most part of what we want to say on PIF structure.
8 So basically, PIF structure we have these four layers.
9 On the top are the four where you categorize the 10 context in four categories.
11 So, the context can be addressing 12 environment and situation, systems, your working base, 13 personnel, who does the job, and the tasks, what 14 actually you're doing.
15 So, for each category we have a set of PIF 16 describe them. Then for each PIF we, again to make 17 the objective specific assessable, we develop a set of 18 what we call the PIF attributes, or characterization 19 of this PIF.
20 And this PIF attribute actually, as Dr.
21 Bley just said, they direct link to the cognitive 22 mechanisms, which showed in Appendix B.
23 So, we are like look at the, break down 24 the models of context in progressively two levels.
25 The first is the high level PIF, then the PIF NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
77 1 attributes. So, if we want to take a look of the PIFs 2 we have. So, these are the --
3 CHAIR BLEY: Can I --
4 DR. XING: Yes.
5 CHAIR BLEY: -- toss in another idea? The 6 table you just had up there, which is Figure 3-2 in 7 the document. If you somehow, in the headings to the 8 tables carried over that, which of those contextual 9 elements that PIF table is about, I think it would 10 help. So, you know, the first four or five are 11 environmental and situational. And that's there. Go 12 ahead.
13 DR. XING: And then for the, I realize I 14 should practice the requirement, which you need a 15 caption. And for that paper, diagram, for me to 16 explain what each item means. We will do that. Thank 17 you.
18 Okay. So, I will quickly go through this 19 PIF. This, unless you want me explicitly explain 20 somewhat. So, for the environment and situation 21 context we have five PIFs to model.
22 So, the accessibility or habitability of 23 a workplace, including the travel path, workplace 24 visibility, and the noise in the workplace, and the 25 communication pathway, like in the, if you use several NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
78 1 communication.
2 And cold, heat, or humidity, and the 3 resistance to physical movement, such as the rain, 4 water on the, still water also same. So, based on 5 that --
6 CHAIR BLEY: You know, I'm sorry to 7 interrupt you again. But even at this level and these 8 little tables you made, are kind of nice. And maybe, 9 well, you kind of have that in paragraphs in the 10 report.
11 But, you know, if I'm a user, right here 12 I can look through the environment and situation 13 context, given my qualitative analysis, and I can say, 14 workplace visibility's no problem. There's no noise 15 problem here. There's no resistance to physical 16 movement.
17 You know, I might have one of these I 18 shall keep. I might not even have any, depending on 19 my qualitative analysis. So then, it clarifies much 20 more quickly which ones you ought to be using. I 21 think some tools, the bigger tables have so much 22 detail it's kind of hard to see the forest through the 23 trees.
24 DR. XING: Yes. That's exactly the way we 25 hope user to use. Like, IDHEAS-ECA --
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
79 1 CHAIR BLEY: Yes.
2 DR. XING: We have the, you do the up 3 front qualitative analysis, assess the context, and 4 you quickly narrow down a set of PIF that you want 5 model this, or take a detailed log, a detail. Okay.
6 Everything happen in the control room, not 7 this going to use. You will not consider this again.
8 But then you say, okay, test the complexity. That's 9 something I need to -- I'm not sure it has impact or 10 not. But I want to know the detail that --
11 CHAIR BLEY: I'm going to say again, I 12 really like this structure of linking the PIFs to 13 elements of context. And back when we did ATHEANA we 14 saw this sort of thing, that once we specified the 15 context, and we saw we were, you know, that locked in 16 the PIFs.
17 Why do we even need the PIFs? We've, but 18 you need those for thinking about quantification.
19 I'll give you that. So, this idea that, you know, 20 they are an image of the context is really a good one.
21 Go ahead.
22 DR. XING: Thank you. And as you can see 23 the structure is highly influenced from ATHEANA. I 24 read ATHEANA report many times, to keep our team try 25 to. Honestly, I know more ATHEANA lines than the NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
80 1 original author did.
2 But it really, we think this is a great 3 concept look at the overall context. That's what 4 really determine the performance. But --
5 CHAIR BLEY: And that's what you're 6 getting from, you ought to be getting from the 7 qualitative analysis.
8 DR. XING: Yes. But the one HRA benchmark 9 study, last one was, ATHEANA is the best, you know, 10 help guiding people analyze the context. The problem 11 was how to model the context for quantification.
12 That's why we try to do it here, using this PIF 13 structure.
14 In the system context, the system is the 15 optics of the actions. So, which the work may take.
16 So, the three PIFs are system and instrument and 17 control transparency to personnel. PRA models 18 reliability of the system and the I&C.
19 But for the transparency to human 20 operators it's not clearly modeled. And the 21 traditional PIF is a human system interface. And 22 equipment and the tools, which can be more impacting 23 in, at the control side, control room actions, then 24 control room actions.
25 And the PIFs for Personnel context.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
81 1 Personnel are the people who do the job, like this, 2 including individuals, teams, and organizations. So, 3 five PIFs are staffing, Procedure, Training, Teamwork 4 and organization factors, and work process.
5 So, the last category is task context, 6 which, as I said, many of these factors previously 7 were lump into one grandfather factor, which called 8 workload.
9 Now we model them separately, which 10 including information availability and reliability, 11 and scenario familiarity, multitasking, complexity, 12 mental fatigue at the time you perform the task, and 13 time pressure and stress, and physical demand.
14 Yes. So, we can look at in a -- Then for 15 each of these PIFs, as we say, we actually, still is 16 hard to judge, to quantify them. How you quantify a 17 good versus a poor human system interface?
18 That's, we went down first in there to 19 characterize each of these PIF. Use what we called 20 attribute. So, I think it's, we can just look at this 21 example, see what attribute is --
22 CHAIR BLEY: Can I reinforce my last 23 comment? I think in your tables in the report, and in 24 these examples, if you included kind of a heading here 25 of the contextual element that this PIF is modeling, NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
82 1 if that's your word, I think it would help.
2 DR. XING: Okay. Thank you.
3 CHAIR BLEY: And I think it would help --
4 DR. XING: Is that your comment?
5 Otherwise, Jonathan is right behind --
6 PARTICIPANT: Well, when we get on the --
7 DR. XING: Oh, we have transcript.
8 PARTICIPANT: When we get on the --
9 DR. XING: Yes.
10 PARTICIPANT: Eventually we'll get it.
11 DR. XING: Okay. Thank you.
12 CHAIR BLEY: But who knows what it will 13 say, but you'll remember.
14 MEMBER DIMITRIJEVIC: You said that you 15 had the, actually I read some of the, you had plenty.
16 CHAIR BLEY: Microphone, please. So, the 17 guys --
18 MEMBER DIMITRIJEVIC: I press it and 19 nothing happens.
20 CHAIR BLEY: We'll have a training session 21 at PNP.
22 (Simultaneous speaking) 23 MEMBER DIMITRIJEVIC: You, I noticed 24 actually, and I read in your slide there is 20 PIFs, 25 right?
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
83 1 DR. XING: Yes.
2 MEMBER DIMITRIJEVIC: But you actually 3 only have tables for 17, right? And now, in the 4 report. And I notice, actually when you talk about 5 environment that you talk about two.
6 But actual it's five. But there's only 7 two in those detailed tables. So, there is some 8 discrepancy of the numbers you mention, and what I was 9 --
10 DR. XING: Okay.
11 MEMBER DIMITRIJEVIC: -- you know, 12 following.
13 DR. XING: Yes. I can explain the fact we 14 have 17 tables. Because they're, two is repeats. I 15 think those are about, those PIFs are the 16 environmental one.
17 MEMBER DIMITRIJEVIC: Right.
18 DR. XING: Each of them only have two or 19 three attributes. So, try to save some papers I just 20 lump them in that table.
21 CHAIR BLEY: Bad idea.
22 DR. XING: Okay.
23 MEMBER DIMITRIJEVIC: Okay. I notice that 24 you have two instead of five. So, I asked if those 25 are three missing. And --
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
84 1 DR. XING: Yes. I just, okay, let's save 2 some pages, papers. So, Jonathan, we, that was a bad 3 idea we did.
4 PARTICIPANT: I, yes.
5 MEMBER DIMITRIJEVIC: I think is a good 6 idea to save the pages.
7 DR. XING: Paper. But it causing a lot 8 more confusion with the missing three. And again, 9 that's actually a good example of the interface. You 10 presenting the confused, the information, not 11 presented consistently.
12 MEMBER DIMITRIJEVIC: Well, the quality of 13 procedures, which is not one of your PIFs, right, 14 quality of procedures? Is that the --
15 DR. XING: That's one of the, it's not one 16 of the attributes. And, but we have other indicators.
17 So, instead of just say quality, because we can't say 18 this one is a quality five, that one's a quality ten.
19 We break down several --
20 CHAIR BLEY: Wait a minute.
21 DR. XING: -- for quality.
22 CHAIR BLEY: Quality's maybe a tough word.
23 You have the procedures under personnel context. But 24 always remember, when I go to this, I've done my 25 qualitative analysis.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
85 1 And the question isn't, is this a good 2 procedure or a bad one. Given the scenario of 3 interest, and the contact of that scenario, does this 4 procedure work well? And that's a place we get into 5 trouble.
6 The procedure works well almost all the 7 time, except under this unusual condition. Damn, it 8 doesn't quite work right anymore. And that's a really 9 great thing to learn.
10 And if you own the plan, and you own the 11 procedures, you probably want to fix that, unless it's 12 an extremely rare situation. Then you just want to be 13 aware of it. But you can't call procedures good or 14 bad, absent the context.
15 DR. XING: Yes. That's the, actually you 16 said the same thing as what our user said. She said, 17 for something like a procedure you can't pre-eliminate 18 it based on your analysis of context.
19 Because normally we would think procedures 20 are good. For a particular situation it may not fit.
21 Therefore, you always need to look at as, actually be 22 able to look at every attribute to make sure, okay.
23 Because each attribute is perhaps one bad 24 way. So, you look through the attribute, and make 25 sure, okay, there's no problem with this.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
86 1 CHAIR BLEY: Could you explain this 2 though? These attributes are all negatives.
3 DR. XING: Yes. Attributes --
4 CHAIR BLEY: So --
5 DR. XING: -- are all negatives. We 6 describe PIF in the neutral manner. But all the 7 attributes are negatives. Each attribute represent 8 one way this PIF can negatively impact the 9 performance.
10 MEMBER DIMITRIJEVIC: Now, previously we 11 discuss that there can be negative and positive. So, 12 what happens with positive?
13 CHAIR BLEY: Yes.
14 DR. XING: That's a touch question. We 15 had lots of discussion on that.
16 MEMBER DIMITRIJEVIC: This is smaller than 17 what?
18 CHAIR BLEY: Well, you know --
19 DR. XING: Yes.
20 CHAIR BLEY: I'd just say, sometimes you 21 introduce new technologies to reduce these problems, 22 not to increase the number of problems.
23 DR. XING: So earlier, one thing we been 24 struggled, because, you know, our current SPAR-H 25 method, it does have positive PIF. So, we have NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
87 1 extremely good training, which mitigate a lot of 2 problem.
3 So, we took a set of example where people 4 can say that we have positive PIF. Ended up every, 5 because each PIF we already assume is good enough 6 status, I wouldn't say ideal, but is good enough.
7 It's an honest average.
8 CHAIR BLEY: So, average?
9 DR. XING: Yes. Average, good enough. If 10 what you mean you are better than good enough, we look 11 at those. Typically when you have a positive PIF you 12 are in reduce negative attribute in another PIF. Say, 13 this is a, I have a complex task.
14 However, my procedure really clear, gave 15 me step by step guidance on how to go through this 16 complex diagnosis. So, we wouldn't give extra credit 17 to procedure, other than it's already good.
18 But we would give credit when you assess 19 PIF, the complexity. You want to even base the 20 appearance as a complex task. But this procedure, 21 this current procedure is not complex. So, that's 22 how the positive PIF.
23 MEMBER DIMITRIJEVIC: I mean, there is no 24 reason for you, I mean, negative PIF means just 25 smaller than one, right? Because only a PIF is -- So, NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
88 1 why didn't you include the, you know, the beneficial 2 PIFs, not to call them positive? Because it's not 3 plus, minus. Is, why didn't you include them?
4 Especially for the things like teamwork or --
5 DR. XING: Yes. Like, for the things like 6 teamwork, with teamwork it's already included in the 7 PIF. Assume that you already have the base assumption 8 when you have PIF. Teamwork you would assume they 9 have, you have peer checking. And you have cross 10 verification.
11 Those are actually the PIFs, the 12 attributes in the teamwork, which means, if you don't, 13 you have base, means you have good teamwork. If you 14 don't have it your performance would be drop. So --
15 MEMBER DIMITRIJEVIC: Well, but that's 16 assume that your average one you're starting for these 17 then for optimum, and not average conditions. And 18 that's mean they have, everything is superb. And 19 then, you know, you're measuring. So, it should be 20 done for average, not for the optimum --
21 DR. XING: Yes. That's --
22 MEMBER DIMITRIJEVIC: -- conditions. So, 23 that will imply that you could have a, you know, the 24 smaller than one piece. And I don't see why would be 25 the reason not to have this.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
89 1 DR. XING: Yes. Well, the --
2 MEMBER DIMITRIJEVIC: And I like how 3 extremely low stress situations, especially, I mean, 4 you may have them.
5 DR. XING: Yes. And another part --
6 CHAIR BLEY: I kind of get that, 7 especially if you're doing a formulaic kind of 8 quantification with the --
9 MEMBER DIMITRIJEVIC: Right.
10 CHAIR BLEY: -- PIFs. If you're using the 11 PIFs to essentially evaluate the context, then if your 12 context doesn't challenge any of the PIFs, you're 13 good. I mean, you -- I can see both approaches. But 14 I --
15 DR. XING: Yes.
16 CHAIR BLEY: I think if I'm --
17 DR. XING: Well --
18 CHAIR BLEY: If I'm developing a 19 qualitative analysis and a context, then I use the 20 PIFs to say, is this context going to challenge the 21 operators? And then you could say, could it make the 22 operators better? Is there something about this 23 context that makes it less likely for failure? I'm, 24 I think it's okay they way they've done it.
25 MEMBER DIMITRIJEVIC: Well, from NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
90 1 qualitative evaluation I have no issue. And then you 2 have, you know, one plus with duplication of PIFs, 3 which multiplies your average.
4 CHAIR BLEY: I don't know how they're 5 going to handle the PIFs. We'll get, I'll wait to see 6 that.
7 DR. XING: Yes. James can comment on 8 this.
9 (Off microphone comment) 10 MR. CHANG: James Chang, Human Reliability 11 Engineer in Office of Research. I think this, if the, 12 in the, we are talking about the optimum context for, 13 about for our peer performance. In this thing that 14 Jing, even the situation is optimal.
15 Humans still make mistake. That was we 16 call the residual human error probability. That 17 thinks somehow that you couldn't talk to at that 18 level. That's one thing.
19 The second thing that you mention about 20 the teamwork, this thing that make reduce the entire 21 human error probability lower. In IDHEAS-ECA 22 implementation it has a separate factor with don't 23 cause a PIF. But we cause recovery.
24 Recovery, that means that you have a 25 outside overseer or the independent checker that NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
91 1 stands, able to spot you make mistake, and correct it.
2 That's what we model in recovery, not in the PIF.
3 DR. XING: Yes. Thanks, James. Actually, 4 the recovery factor, lots of the factors we credit 5 recovery, such as team. So, the teamwork through the 6 work process, and even plenty of time, have more than 7 extra time you need. Those are all the things we 8 credit in recovery, when we come to quantification.
9 Thanks, James. Yes.
10 CHAIR BLEY: You have half a dozen more 11 slides before we finish Chapter 3. I'd like to take 12 a break, unless you really want to push ahead to get 13 that.
14 DR. XING: Yes. To your choice. To keep 15 our schedule I think all this discussion pretty much 16 covers rest of the slides.
17 CHAIR BLEY: Do you? Okay.
18 DR. XING: We don't have to --
19 CHAIR BLEY: Well, let's see how that 20 goes.
21 DR. XING: Otherwise, we can back. I'll 22 keep talking after the break.
23 CHAIR BLEY: So, these are just PIFs for 24 other, other specific PIFs? Yes. You want to jump to 25 conclusions?
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
92 1 MR. PETERS: Scroll through your slides.
2 PARTICIPANT: Can you just show another 3 one, or two?
4 DR. XING: Okay. So, this explain --
5 CHAIR BLEY: And example.
6 DR. XING: -- how the PIF work. We 7 already talk how the attributes work. We already 8 talk. And well, we may want talk this. This what, 9 then you are here to how we model the PIF effect on 10 the HEPs. So, maybe we talk this when we come back.
11 CHAIR BLEY: Okay. We're going to recess 12 for 15 minutes. Be back at 25 until 11:00 a.m.
13 (Whereupon, the above-entitled matter went 14 off the record at 10:20 a.m. and resumed at 10:36 15 a.m.)
16 DR. XING: -- How we look at PIF affect 17 HEPs. That's what we really wanted. Okay. So again, 18 the lesson learned from HRA defense models is basis 19 error for improvement.
20 Because the quantitative relationship 21 between PIF and the HEPs has been ambiguous. So, we 22 started to address this. We said, okay, then we need 23 to look at data to explain these three aspect 24 regarding quantification.
25 How you, assessment of a PIF attributes.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
93 1 Now you we have attributes we are in a much better 2 position before, than just assess the PIF, however.
3 And the quantification of the change in the HEP, even 4 the attribute change.
5 And the most headache one is a combination 6 of multiple PIFs on HEP. Somehow we have to address 7 it. So what, with data we performed the metadata 8 analysis from the literature.
9 I would say, I never count how many 10 literature we went through. I would give a minimum, 11 maybe 1,000 research articles. Each of these article 12 actually have a explicit human error rate in the 13 report.
14 So we, to gain insight on these three 15 aspect -- Oh, by the way, that metadata analysis, 16 that's, Sean will talk at the end of day. We plan to 17 document all those we did, in the next two years.
18 CHAIR BLEY: That would be grand. We've 19 talked about this for a long, long, long, long time.
20 DR. XING: I always say, well, I finish 21 that. You see, I will move on the documentation of 22 that. Now we finally, I think we come to the end of 23 the turn there.
24 And so, talk about assessment of PIF 25 attribute. Some HRA method use discrete levels, like NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
94 1 low, medium, high, to model the state of PIF. For 2 example, we can say complexity can be low, medium, 3 high.
4 I think two person can argue forever what 5 is medium versus a high level complexity. So, in the 6 cognitive studies, actually when we look at the data 7 we had, they don't say complexity low or high.
8 They, the experiment to perform specific 9 PIF attribute, like how many items you need to 10 integrate them together to get an answer. That's what 11 they look.
12 And quantification of the HEP should be 13 based on the PIF. So, from there we draw the insight.
14 Okay. Quantification of HEP should be based on PIF 15 attributes, and how they change from a baseline where 16 there's no impact on HEP.
17 So, this one, this is how we approach to 18 quantification. So, we try to, quantifications are 19 changing in HEP when the PIF attribute change. So, 20 the cognitive studies of data we have, they measure 21 human error rates when you systematically change an 22 attribute.
23 You know, you can change one or more 24 attribute, or change around the attribute you want 25 more PIFs. Then the measurement typically reported is NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
95 1 no impacts, or lower impact versus high impact on the 2 error rate.
3 Say, okay, we have no more. All the 4 information needed are available. Then experiment and 5 manipulation of some operation data will show, with 6 some information missing, or even misleading. How 7 much change in the reported or observed error.
8 So, if we use this measurement we call the 9 PIF weight, or PIF weight factor, which is a change in 10 the error rate between the impact state, between the 11 impact versus no impact state. So, that's --
12 CHAIR BLEY: What you're doing here is a 13 lot similar, I think, to what Jerry Williams and the 14 later folks did with HEART, CAHR, and NARA.
15 DR. XING: Very much the same.
16 CHAIR BLEY: Although, they never showed 17 us their data either.
18 DR. XING: Okay. I --
19 (Simultaneous speaking) 20 CHAIR BLEY: But it is similar. Okay. Go 21 ahead.
22 DR. XING: Actually, a lot of the data 23 already in IDHEAS-ECA. So, IDHEAS-ECA has generalized 24 data, which I will talk on this afternoon. Does not 25 have all the original data. But we documented those NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
96 1 on computer.
2 MEMBER KIRCHNER: Let me, can I test you 3 on this? So, I'm still thinking back to the one PIF 4 you showed us, where among, it was the human system 5 interface.
6 If I go through this example mentally, 7 with the mislabeled, Dennis and I were talking about 8 this during the break, a mislabeled control, a button.
9 You think you're pushing this button to reduce the 10 level. And it actually is doing something else.
11 So that, I mean, the error, it's one. I 12 don't know what rate. I mean, by demand the error is 13 one. So, how, versus the base. I'm a little -- So, 14 you have this PIF with that in there.
15 DR. XING: Yes.
16 MEMBER KIRCHNER: I'm just trying to 17 mentally work through an exercise. And I don't know 18 what the base is. The base is --
19 DR. XING: Okay, yes.
20 MEMBER KIRCHNER: If it's mislabeled it's 21 wrong. So, you would have an error to begin with in 22 the baseline of the PIF. So, I'm having trouble 23 working this formula to any practical use, in terms of 24 weighting.
25 DR. XING: Okay.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
97 1 MEMBER KIRCHNER: Can you give me, can you 2 elaborate --
3 DR. XING: Yes. That's a very --
4 MEMBER KIRCHNER: -- and give me an 5 example.
6 DR. XING: -- good example we can talk 7 about. First of all, think about if the label is 8 really clear. And if the label's really clear, and 9 all your test is just push this button. So, that's a 10 very simple task. No other bad PIF.
11 MEMBER KIRCHNER: Yes.
12 DR. XING: You would get, you still not 13 get a zero HEP. And actually early, in later '60s 14 Army did lots of research on this, try to, over 10,000 15 peoples study on this if how much, the chance of 16 errors you can make just push a button.
17 So, they have this baseline study. Okay, 18 if they just push, press a button, the error rate I 19 think is 0.9998, which there's chance of you still 20 have three-tenths of one out of 10,000 to try out.
21 That's a baseline error.
22 And then they have manipulation. They 23 didn't say it's entirely mislabeled. But they tested.
24 The label is ambiguous. It's unclear. So, the -- And 25 in that case I cannot exactly recall the error rate.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
98 1 But it was something like 0.95, something like that.
2 So, which is like, you can have five times 3 wrong out of 100 time. So, if they put it an entirely 4 mislabeled one in the experiment section, you're 5 right. You probably get a error rate as one.
6 However, we have, when we look at our 7 data, especially they have lost such data in aviation.
8 And this, there's sometimes there are mislabel in the, 9 in cockpit. Still, pilot doesn't get 100 percent of 10 error. Because they have other source of the 11 information help them verify that.
12 So, one study I saw, they make like 20, 30 13 percent of, like over population. Twenty to 30 14 percent of people would make a mistake.
15 MEMBER KIRCHNER: Yes.
16 DR. XING: But not entirely. So, but that 17 were case to case dependent. If you don't have any 18 other information, or if you don't, the system doesn't 19 give you feedback.
20 MEMBER KIRCHNER: Yes.
21 DR. XING: Tell you your data wrong. You 22 press the wrong button. Then you entirely wrong. But 23 because it have these other things to help you avoid 24 a completely failure, that's, we come to the 25 probability domain.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
99 1 MEMBER DIMITRIJEVIC: Well, I have to 2 correct what we discussed before. Because the way how 3 you define the weighting factor you did it having 4 negative. And you would have a negative, positive.
5 Because if there is no impact you are 6 zero. Your weighting factor is zero. And if that is, 7 you double your rate, then is one. So, you said is 8 two. So, the different --
9 DR. XING: Yes.
10 MEMBER DIMITRIJEVIC: -- there is no 11 smaller than one. It's smaller than zero.
12 DR. XING: Yes.
13 MEMBER DIMITRIJEVIC: So, how I define 14 before. However, what I wanted brought -- well first, 15 what is the rate? Where you said rate instead of 16 probability, you meaning what --
17 DR. XING: Percent.
18 MEMBER DIMITRIJEVIC: -- percentage of it?
19 DR. XING: Percent of, correct, yes.
20 MEMBER DIMITRIJEVIC: So, let's say that 21 you have 100 percent rate --
22 DR. XING: Yes.
23 MEMBER DIMITRIJEVIC: And your expectation 24 was ten percent, right? I mean, your base case is ten 25 percent. What would your base case --
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
100 1 DR. XING: Yes. My base case is 0.001 2 percent, yes.
3 MEMBER DIMITRIJEVIC: Is it all --
4 DR. XING: That isn't perfect.
5 MEMBER DIMITRIJEVIC: Well, let's say ten 6 percent, just for --
7 DR. XING: Okay.
8 MEMBER DIMITRIJEVIC: -- simplicity of the 9 math.
10 DR. XING: Yes. Okay.
11 MEMBER DIMITRIJEVIC: So, you have a 90 12 minus ten, which is 80 percent, divided by ten 13 percent, which would be 80. So, that would be like 14 their weight factor, or something.
15 DR. XING: Yes. Say, in my base case --
16 MEMBER DIMITRIJEVIC: Okay.
17 DR. XING: Our base is ten, or .1. And 18 you are have a bad PIF, the indicator unclear, it's a 19 .3 then or better. So, weight would be --
20 MEMBER DIMITRIJEVIC: Well, this rate is 21 much more difficult to understand than other 22 performance factors, which are just multiply HEP.
23 Because that tell you exactly how many times you 24 expect this to increase. This is a little more --
25 DR. XING: Yes.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
101 1 MEMBER DIMITRIJEVIC: -- convoluted, this 2 is.
3 DR. XING: That's a very good point 4 actually. We did try that. And of course, the two is 5 is just the dividers. The other way is the minus one.
6 We had a reason for minus one. Because it make the 7 formula to calculate each easier. Much less --
8 MEMBER DIMITRIJEVIC: Well, that's, and 9 I'm not sure. But okay. I mean, my opinion had the 10 one weakened. All right.
11 DR. XING: Okay. Well, we think about 12 that later on, yes. We might get same feedback from 13 the ECA test team on this. See if much more easier 14 just think of dividing than --
15 MEMBER DIMITRIJEVIC: Right. Yes.
16 DR. XING: -- the platform baseline. I 17 agree with you on that.
18 CHAIR BLEY: Jing, we're losing your voice 19 again.
20 DR. XING: Okay. I will raise my voice.
21 Thank you. Keep reminding me that. Yes. Okay. I 22 wish I can avoid the, talking about this slide.
23 Combine the multiple PIFs.
24 So, in the later trend, the existing HRA 25 method we can, while we can keep or try to avoid it by NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
102 1 doing a holistic approach, without thinking about each 2 individual PIF as a factor.
3 I will just make a combination. Okay.
4 Given the fact that you have a high workload, bad 5 interface, and poor training, what is the probability 6 for people fail this task?
7 So, that's actually how we did our expert 8 elicitation in IDHEAS-at power. So, that way we avoid 9 a thing called individual factor. However, the 10 problem with that approach is when you have many PIFs, 11 many possible combination.
12 All the combination you didn't have in 13 your expert elicitation, you would have no way to 14 approach to this, to better -- You can't afford it 15 instantly overnight to have an expert elicitation 16 vetting your number.
17 So, then the other approach, which often 18 used in HRA method, is a combination of individual 19 factors. So, we estimate the individual factor, like 20 weight, defined earlier, then combine them together.
21 For this one we look into data. So, we 22 want to search the studies which showed people 23 actually measured this error rate of reporting, when 24 a individual factor tend to, versus they have data of 25 several, two or more factor change at the same time.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
103 1 So, we, under the preliminary result we 2 find there was a factor for combination marked for 3 PIFs can be roughly estimated by adding the effect of 4 the individual weight. So, if one factor, let's say 5 if workload, let's say if --
6 CHAIR BLEY: Without seeing the source of 7 your data though, that leaves me wondering what 8 happens when there's a higher degree of dependence 9 between these two PIFs, and where, in experiment you 10 were citing.
11 DR. XING: Yes, true. So, in the data I 12 used for the, actually we published the data on this 13 in 2015, PSA conference. And when we looked for the 14 data to support our analysis I specifically looked 15 into the dependency between the factors.
16 So, I tried to preliminarily rely on the 17 lab studies which people really tried to isolate the 18 factors. I was being very cautious in using the 19 operational data, because of dependency.
20 In fact, I had a conversation at the 21 career people, HRA folks. He said, oh, we see the 22 factors are really multiplied in here, not adding 23 together.
24 I said, okay, let's look at your data.
25 And it turned out that the two factors that they NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
104 1 analyzed are highly dependent on each other.
2 So, but nevertheless, this is a area 3 really we would like to do future research, including 4 more extensive metadata analysis, to establish a good 5 cognitive basis of how to combine them. And even now 6 we have more data than we had time to analyze them 7 all. So --
8 CHAIR BLEY: So, if you, I'm just 9 wondering if you might have a user need. Or if this 10 is more in making the distinction that Sean did 11 earlier in human factors, or HRA. Are you going to be 12 able to pursue this as future research? Do you have 13 any idea? And if so, under which of those approaches?
14 DR. XING: Yes. We would have to do this 15 anyway. But we would love to have a user need to do 16 this.
17 MR. CHEOK: So, I think, this is Mike 18 Cheok again. This is, the staff would have in our, 19 longer to implant in the medium, to implants that we 20 would do this.
21 But then we would like to get support from 22 our user offices. And we'll ask for that. But if the 23 ACRS would like to weigh in on that, it would also be, 24 we would also take that advice into how we go forward.
25 CHAIR BLEY: Thank you, Mike.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
105 1 DR. XING: Thank you. So, having said 2 that, concluding the PIF structure. On this slide you 3 can mainly look at the last bullet, which we feel in 4 this PIF structure we tried to make the improvement 5 according to the criterias we set forth earlier.
6 And, but big takeaway from this, we think 7 about sharing the PIF. Because right now every HRA 8 method have their own PIF, set of PIF. And even for 9 the same PIF they different definitions and different 10 way.
11 So, we think a shared PIF structure in the 12 overall HRA community should increase the consistency 13 of different HRA methods, and allow the comparison of 14 HRA quantification results from different methods.
15 CHAIR BLEY: I want to go back to your 16 multiple PIFs. Is that something you think 17 documenting your many sources of data could let you 18 address? Or is that something you need to sponsor 19 experiments? Or have you thought about how you would 20 actually try to improve that situation, get better 21 understanding?
22 DR. XING: Yes. Two ways. So, currently, 23 as I said amount over 1,000 paper, research paper I 24 collected. There are probably several hundred of 25 studies. People studies more than one factor, and NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
106 1 show the effect.
2 I only analyzed like 30 of them.
3 Basically because it's very time consuming. You 4 really need to read through the paper to address what 5 Dr. Bley said, whether there's a dependency between 6 those factors, if those, how the attribute 7 manipulating the factor.
8 So, we could, first we continue to go 9 through more data. And also, we've been closely 10 working with the Halden Reactor Project. And earlier 11 we recommend them use our PIF structure in their 12 experiment design.
13 So, in their experiment design they can 14 actually say, we are testing factor A, B, and the 15 combination of them. They already, they the ones 16 started like that on information.
17 CHAIR BLEY: Have the published anything?
18 DR. XING: This is preliminary report 19 ready. I will check with Andreas for the final 20 report. I think for, yes. So, you must --
21 CHAIR BLEY: Yes. I'd like to see it.
22 DR. XING: -- study base concept. They 23 tested the, for people in the SBO scenarios. They 24 need shut down the reactor. They have, information is 25 incomplete. Some information indicators missing.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
107 1 And indicator misleading versus both. So, 2 that's a very good data point for us to look at the 3 combined factor. So, we are planning, so, the short 4 answer to your question is, we try both existing data, 5 and more experiment data.
6 CHAIR BLEY: Well, the Halden idea's a 7 good one. I'll, be interesting to see if they -- Have 8 you had a chance to read that preliminary report, or 9 --
10 DR. XING: Yes.
11 CHAIR BLEY: Is there good information in 12 this area in it?
13 DR. XING: Yes.
14 CHAIR BLEY: Okay. Thanks.
15 DR. XING: In that study, because I 16 practically participate in the baseline. Okay. So, 17 if no more question I would like to give to Jonathon.
18 We finish the combination model. Jonathan will talk 19 about implementing the combination model in HRA 20 process.
21 CHAIR BLEY: Great. Jonathan. I think 22 this is your first time to visit with us. Is that 23 right?
24 DR. XING: Okay, Jonathan.
25 MR. DeJESUS: Good morning. Yes. This NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
108 1 would be my first time on this topic. Actually, this 2 is my second time. I did it a few years ago. I think 3 you were there, in the field cycle oversight process, 4 when we were trying to --
5 CHAIR BLEY: Oh, yes.
6 MR. DeJESUS: -- revise it back in 2011.
7 So, yes. I was there.
8 CHAIR BLEY: Well, we probably asked you 9 at the time. But I'm going to ask you again. A 10 little voir dire, you know, what's your history, and 11 how did you end up coming to this role?
12 MR. DeJESUS: Okay. So, I'm Jonathan 13 DeJesus. I've been with the, I'm a liability and risk 14 analyst. And I've been with the NRC since 2006. I 15 started with research, moved to NMSS data, and year 16 plus in NRR. And I got into the Grow-Your-Own PRA 17 Analyst Program.
18 So, and I, what got me to research was 19 actually the graduate fellowship program. I did the 20 graduate fellowship program for NRC at the University 21 of Maryland. And that's something I'm still working 22 on, on my own time.
23 So, I was assigned to the Human Factors 24 Reliability branch. And when I came back, then I got 25 assigned to this project. And --
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
109 1 CHAIR BLEY: Yes. Very good.
2 MR. DeJESUS: And here's my reward for 3 helping the, what we --
4 (Laughter) 5 CHAIR BLEY: Congratulations.
6 DR. XING: And we couldn't get the report 7 down without Jonathan. Because he's new to the 8 project he brought a fresh view of what we're doing.
9 So, that really helps with reliability and usability.
10 If you see the improvement of this version's report 11 compared to our previous versions, a big credit goes 12 to Jonathan.
13 MR. DeJESUS: And if I may quickly respond 14 to your comment earlier about the references. It's 15 the, this personal preference of yours of addressing 16 all authors, and one author, et al. The thing is, 17 that was, I did it with software. So, a software 18 manager that helped us manage those 130 references.
19 CHAIR BLEY: So, if there's greater than 20 two authors it --
21 MR. DeJESUS: If there's greater than 22 three authors --
23 CHAIR BLEY: Three authors.
24 MR. DeJESUS: -- then it adds the et al.
25 So, that's why. It wasn't intentional that we were NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
110 1 trying to left out anything and anybody. But anyway 2 --
3 CHAIR BLEY: A bunch of enemies with 4 authors four through ten.
5 MR. DeJESUS: But it can be fixed.
6 DR. XING: Okay, Jonathon, you want to 7 start from here?
8 MR. DeJESUS: I got the mouse. I think 9 I'm good to go.
10 DR. XING: Okay.
11 MR. DeJESUS: All right. So, moving onto 12 --
13 CHAIR BLEY: Are you leaving, Jing? Are 14 you leaving?
15 DR. XING: I'm sitting here.
16 CHAIR BLEY: Okay.
17 MR. DeJESUS: So, for those people on the 18 phone, we are Slide number 40. So, hopefully this, I 19 guess --
20 DR. XING: Yes, right.
21 MR. DeJESUS: So, in here the, I was 22 expecting some animations. But they're not there.
23 Anyway, so in, my intent of this presentation is to 24 give you, put the whole process together.
25 And essentially this Slide number 40 gives NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
111 1 more detail on what Jing showed on Slide 11, regarding 2 the stages of the HRA, of the IDHEAS-G HRA process.
3 So, starting with Stage 1, the purpose of 4 that stage is to understand the scenario, and collect 5 information that would help in the quantification of 6 the human error probability.
7 So, that includes developing the 8 operational narrative, identifying the scenario 9 context, and identifying what we call the important 10 human actions.
11 And the reason we use that terminology is 12 because of, important human action, is because IDHEAS-13 G is a general methodology, applicable to multiple 14 nuclear applications. So, if you think about it from 15 a PRA perspective, it's the same as a human failure 16 event, or HFE. So, they're the same.
17 So, on Stage 2 of the process, the purpose 18 of that stage is to model the important human actions 19 for structure and analysis, for a structure analysis, 20 and for HEP quantification.
21 And as shown in the slide it includes the 22 identifying and analyzing the critical tasks, 23 identifying the applicable cognitive failure modes, 24 and assessing the PIFs.
25 And hopefully in this part we'll be able NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
112 1 to answer your questions regarding the PIFs, on how to 2 assess those.
3 And then moving on to Stage 3, it's the 4 HEP quantification stage. And essentially its name 5 gives away its purpose. And IDHEAS-G does the HEP in 6 two parts.
7 There's the error probability attributed 8 to the uncertainty in time available and time 9 required. So, that's what we call PT. And the error 10 probability attributed to the cognitive failure modes, 11 which we call PC.
12 And the, Stage 4 essentially includes the, 13 it's the integrative analysis. And essentially what 14 it does is document the uncertainties throughout the 15 analysis, and assess the dependencies between the 16 human failure event.
17 CHAIR BLEY: Jonathan?
18 MR. DeJESUS: Yes, sir.
19 CHAIR BLEY: I'm not sure, but I don't 20 think this figure is in the report.
21 MR. DeJESUS: It is. It's I think --
22 CHAIR BLEY: Or somewhere --
23 MR. DeJESUS: -- Figure 4 --
24 CHAIR BLEY: That's all right.
25 MR. DeJESUS: It's at the end.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
113 1 CHAIR BLEY: That's all right.
2 MR. DeJESUS: Figure 4-9.
3 CHAIR BLEY: Oh, it's toward the end.
4 Okay.
5 MR. DeJESUS: 4-9.
6 CHAIR BLEY: But what I wanted to ask you, 7 we now go from macrocognitive functions to crew 8 failure modes.
9 MR. DeJESUS: Cognitive failure modes.
10 CHAIR BLEY: Cognitive failure modes. And 11 those aren't the processors, or the cognitive 12 mechanisms, or the PIFs, or their attributes. They're 13 something different?
14 MR. DeJESUS: They are related to the 15 macrocognitive functions. They're related to the 16 processors, and the cognitive mechanisms. And I'll 17 talk about it when I talk in more detail --
18 CHAIR BLEY: Okay.
19 MR. DeJESUS: -- about Stage 2.
20 CHAIR BLEY: Thank you.
21 MR. DeJESUS: So, that's an overview of 22 the entire process. So, going to Stage 1, it talks, 23 the objective of the operational narrative is to 24 develop an in depth understanding on how the scenario 25 progresses from the initiating event to the end state.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
114 1 The baseline scenario describes the 2 expected scenario progression, and if available, it is 3 recommended to include operating experience into 4 developing that baseline scenario.
5 The operational narrative has two parts.
6 It's, as shown in the slide, the scenario narrative, 7 and the timeline. So, the timeline is a story telling 8 style documentation that provides an overview of the 9 event, the initiating event, the initial conditions, 10 the boundary conditions, and the end state.
11 So, essentially a PRA model provides this 12 information. If you look at the, if you're modeling 13 an HFE through the, using the event trees in the PRA, 14 you should be able to get that information.
15 So, the scenario progression should 16 emphasize the, involving important human actions, and 17 describe the expected responses from personnel and the 18 systems. So --
19 CHAIR BLEY: Can I jump ahead a little?
20 Maybe it's ahead or behind. When, this is the 21 qualitative analysis --
22 MR. DeJESUS: Right.
23 CHAIR BLEY: -- essentially, the 24 operational narrative. It's a good idea. For an 25 event, and the same beginning status, progression NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
115 1 could change.
2 There could be many different contexts.
3 Are you going to get to when you include the context 4 or multiple contexts, and how you come up with those?
5 MR. DeJESUS: That has to do with how the 6 PIF attribute changes throughout the context. And 7 that, initially the baseline scenario is developed.
8 And if something changes from that, then what we have 9 in the slide, showing like additional scenarios.
10 There are deviations from that baseline.
11 CHAIR BLEY: Okay. That's what I want to 12 hear about. And that could include multiple contexts?
13 MR. DeJESUS: Correct. Certainly.
14 CHAIR BLEY: Okay. Go ahead.
15 MR. DeJESUS: So, and the timeline 16 documents the important human actions, and the 17 responses in chronological order. So, we're back in 18 talking about the baseline scenario.
19 And as I mentioned, well, we just 20 discussed, the additional scenario can be identified 21 asking what if questions. For example, the change in 22 the context.
23 So, that's the first part of the Stage 1.
24 Moving on to the identification of the scenario 25 context. And what that means is, the search for NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
116 1 conditions that challenge or facilitate the human 2 performance in the scenario.
3 So, as mentioned before, the context 4 provides a basis for estimating the HEPs, and is 5 represented by the PIF attributes. And that search 6 process focuses on the macrocognitive functions.
7 And it is documented in the four context 8 criteria shown in the slide, which are the same, if 9 you look at the PIF structure, the top level, the 10 environmental --
11 CHAIR BLEY: I think we're on a different 12 slide.
13 (Off microphone comments) 14 MR. DeJESUS: Yes, sorry. Yes, Slide 42 15 for those on the phone. That's what I was talking 16 about. Forgot to click the slide.
17 So, there are the four environmental, the 18 four PIF context categories. The same as we show 19 before, environment and situation, systems, personnel, 20 and task.
21 And the environmental conditions can 22 affect human performance. And examples of that 23 include fire, smoke, earthquake, extreme temperatures, 24 and radiation.
25 So, the environmental conditions can NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
117 1 change throughout the scenario. And they may vary in 2 different locations where the actions are performed.
3 The systems, well we, that term, it's a 4 broad term used to refer to structures, systems, and 5 components, sensors, equipment, and the human system 6 interface.
7 The identification of the system context 8 focuses on conditions that create conflicting 9 priorities, confusions, and distractions. For 10 example, operators may be concerned with the equipment 11 that is not directly related to the scenario. Or they 12 may turn off automatic systems based on well 13 intention, but on incorrect belief.
14 The personnel context considers the 15 framework in which personnel work, that is, 16 individuals, teams, and organizations. Examples of 17 considerations in the personnel context include the 18 availability of personnel, lines of communication, and 19 chain of command.
20 Finally, the task context includes the 21 conditions that affect how the tasks are performed, 22 which are specified by the characterization of the 23 human system interface, and the conduct of operations.
24 The conditions in the task context should 25 be evaluated against the five macrocognitive functions NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
118 1 to determine if any of them are challenged.
2 MEMBER KIRCHNER: So, how do you narrow 3 down the scenarios that you're actually going to 4 examine? Do you use the PRAs? Say you're looking at 5 a plant, just to make it simple, just the full power.
6 Do you go through the PRA as the, a means 7 to pick the scenarios that are of interest? Or, how 8 do you bound this problem starting?
9 It seems like a rather daunting task to 10 search through all the scenarios that would challenge, 11 or facilitate human performance. I mean, how do you 12 begin selecting scenarios? Do you use the PRA? Or --
13 MR. DeJESUS: That would be the starting 14 point, yes.
15 MEMBER KIRCHNER: Yes.
16 MR. DeJESUS: Starting with what actual 17 HEP human failure event that the PRA is trying to 18 model.
19 MEMBER KIRCHNER: So then, you would go 20 through the PRA, and look for those junctures where a 21 human interaction could change significantly the 22 course of the outcome? Or, I'm just trying to get a 23 feeling for how you do a completeness test in 24 identifying the scenarios.
25 MR. CHEOK: So, this is Mike Cheok. I NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
119 1 think what we would do, as you say it's through the 2 PRA. The event trees in the PRA would define the 3 event sequences that are important, given different 4 initiating events.
5 Once we have those event sequences we will 6 have fault trees for those events that the different 7 branch points for the event trees. In those fault 8 trees you would have equipment --
9 MEMBER KIRCHNER: Right.
10 MR. CHEOK: -- reliability, and human 11 reliability. And so, in places where an event 12 sequence is asked for, we would depend on operator 13 action. That's where we would model these events that 14 Jonathan --
15 MEMBER KIRCHNER: Okay.
16 MR. CHEOK: -- called the HFEs, the human 17 failure events. That's how they would be defined.
18 CHAIR BLEY: I want to hit on that a 19 little bit. I'm not sure, and even though I've read 20 this a couple of times I might have missed things.
21 I hate to keep bringing up ATHEANA. But 22 for the last 20 years there's been a kind of consensus 23 among people who do this kind of work in PRAs, that 24 you really shouldn't have a bunch of HRA analysts 25 sitting in a room and, you know, give them events to NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
120 1 analyze.
2 But they are to be integrated with the PRA 3 team, and experts on the plant, and operators. And 4 together, through the PRA, the PRA would have 5 identified those places in the procedures where human 6 action is required.
7 But also, there are techniques for 8 looking, and saying do the gray areas here, if you get 9 to a point where there's a decision to be made, but 10 maybe the scenario hasn't caught up to the procedure, 11 could this come up later, you know, identifying more 12 places?
13 I'm not sure you have laid out that kind 14 of a search for these things in this document. Maybe 15 that comes in the specific application documents.
16 I have one comment though on Table 4-1.
17 You've introduced a term that's anathema to many 18 people doing PRAs. And that's credible. What are the 19 credible cues? What are the credible information?
20 It's not a word that goes with probabilistic analysis.
21 Come up with something else.
22 MR. DeJESUS: Okay.
23 MEMBER REMPE: So, while we're doing nits.
24 On Page, it's 100 out of 334. But you talk about, and 25 you had this actually in an appendix too. The NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
121 1 Fukushima event provides multiple examples of 2 performance delays and performance failures, resulting 3 from miscommunications, lack of clear understanding of 4 roles and responsibility, and complex chains of 5 command and control.
6 Could you elaborate what you mean? And 7 then probably the next thing I'm going to ask is, 8 what's your reference for that?
9 MR. DeJESUS: So, what page are you 10 quoting?
11 MEMBER KIRCHNER: At the bottom of the 12 page it will give you the real page.
13 MEMBER REMPE: Well then, I have to use my 14 touchscreen. Hold on. Is the problem here. Hold on.
15 If you're in the report it is Page 411.
16 MR. DeJESUS: I got that.
17 MEMBER REMPE: That's the only thing you 18 say. And you say it twice, the exact same words. But 19 I don't know what examples you're citing. Could you 20 tell me?
21 MR. DeJESUS: The bottom of 411?
22 MEMBER REMPE: It's in the middle of 411.
23 MR. DeJESUS: Okay. Here it is, second, 24 I guess --
25 MEMBER REMPE: Yes. End of the --
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
122 1 MR. DeJESUS: Third paragraph after --
2 MEMBER REMPE: -- first full paragraph, 3 after the bullets.
4 DR. XING: Okay. So, for your --
5 CHAIR BLEY: You've failed now, the 6 interprobability of getting --
7 DR. XING: I don't know which button I 8 should press --
9 (Simultaneous speaking) 10 CHAIR BLEY: Push.
11 MR. DeJESUS: It says push. Green light 12 is your cue.
13 CHAIR BLEY: I think it's that end of the 14 table.
15 DR. XING: Yes.
16 MEMBER REMPE: It's catching.
17 DR. XING: So, your question is asking the 18 references?
19 MEMBER REMPE: No. I want to know what is 20 it, when those failures. Tell me what happened at 21 Fukushima that substantiates that statement. Because 22 you've said it twice in this report.
23 DR. XING: Okay. We did recover, it's 24 important, this last sentence. Do you have any 25 preference which one you would like we talk?
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
123 1 MEMBER REMPE: No. Tell me what happened 2 at Fukushima. I can read the whole paragraph to you.
3 But that doesn't tell me what happened at Fukushima to 4 say that it provides multiple examples of performance 5 -- And I can read that sentence again.
6 DR. XING: The performance, yes.
7 MEMBER REMPE: But what happened? Give me 8 some examples.
9 DR. XING: Okay.
10 MEMBER REMPE: Where did the operators --
11 DR. XING: Yes.
12 MEMBER REMPE: -- do that?
13 (Off microphone comment) 14 MEMBER REMPE: 411.
15 DR. XING: Yes. Well, in some part I can 16 talk as, in Fukushima, when the isolation condenser 17 was actually not working, it took the peoples long 18 time to figure out it's not working.
19 MEMBER REMPE: And how is that due to lack 20 of communication, and those things --
21 DR. XING: Yes.
22 MEMBER REMPE: -- that you say there?
23 DR. XING: When they finally figure out 24 it's not working they gave that information to their 25 emergency, the command center, the tech.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
124 1 MEMBER REMPE: Okay. So --
2 DR. XING: However, it took, I don't 3 remember exact number for hours. Nathan may know. I 4 think my recollection is took about ten hours for the 5 people in the command center --
6 MEMBER REMPE: Okay. So --
7 DR. XING: -- realize, oh, it's not 8 working. For that entire ten hours they had been 9 under the assumption of the isolation condenser was 10 working. So, they made the decisions based on that 11 assumption.
12 MEMBER REMPE: So, I suggest you go look 13 at some more recent evaluations by TEPCO. Because 14 they spent a lot of time looking at the isolation 15 condenser.
16 DR. XING: Yes.
17 MEMBER REMPE: When the earthquake 18 happened they followed procedures and turned it off.
19 DR. XING: Yes.
20 MEMBER REMPE: Because they didn't want to 21 worry about over cooling. They didn't expect, and 22 this was a procedure that came from --
23 DR. XING: Yes.
24 MEMBER REMPE: -- General Electric input, 25 okay. But anyway, they didn't expect the tsunami was NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
125 1 going to happen. And they wouldn't be able to restart 2 it. They didn't have instrumentation. I don't know.
3 And then they actually, because they were 4 concerned. They went and had some people outside.
5 But look at, make sure you really have good references 6 that are up to date of what happened.
7 DR. XING: Yes.
8 MEMBER REMPE: There was a lot of 9 misinformation that came out.
10 DR. XING: Yes. I appreciate that part.
11 Actually, when I observed the Fukushima report of the, 12 I have documentation on all the human performance 13 related Fukushima.
14 And that is, I did see, as you said, 15 people had the conclusion on that, or the suspicion on 16 that has been changed. Initially they thought it 17 because this. Later on they find it's a different 18 reason why it wasn't it working.
19 For the particular example, I was talking 20 in term of related delayed action, and the 21 miscommunication. I was talking, regardless whatever 22 reason, it wasn't working.
23 From the moment that the site people was, 24 confirmed it wasn't working, to the time that the 25 people in the emergency operating center realized, NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
126 1 that is -- Okay.
2 There probably information, just stayed in 3 their room, or the person heard of that. But they 4 didn't thought that important. That part I haven't 5 see clear analysis why it's, for ten hours they 6 didn't.
7 MEMBER REMPE: I looked at some, there's 8 a whole unresolved, unconfirmed issues report that 9 TEPCO has issued in the last several years. And they 10 have like five volumes. I suggest you go back and 11 look --
12 DR. XING: Yes. I'll --
13 MEMBER REMPE: -- at more recent things --
14 DR. XING: -- definitely look.
15 MEMBER REMPE: -- to make sure you really 16 can support this statement. Because I think maybe 17 it's, early on a lot of people did say things. But I 18 think that they've tried to spend a lot of time 19 looking into --
20 DR. XING: Okay.
21 MEMBER REMPE: -- what really occurred.
22 DR. XING: Yes.
23 MEMBER REMPE: And, yes, okay, it's from 24 TEPCO. But I think was an honest effort, in my 25 evaluation. And there's been folks who have looked, NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
127 1 and reviewed those reports. And they believe it's an 2 honest evaluation too. And let's make sure it's 3 factual, okay?
4 CHAIR BLEY: There's --
5 DR. XING: Thank you. I appreciate that.
6 CHAIR BLEY: There's a associated problem 7 there. And Joy's touched on it. And that is, the 8 truth may be really hard to dig out. And I, you know, 9 this is such a controversial event. The examples form 10 there are probably going to get you into trouble, no 11 matter --
12 MEMBER REMPE: Yes.
13 DR. XING: Yes. Okay.
14 CHAIR BLEY: -- what you say. There were 15 extensive reports, and even a book came out, of people 16 who were actually hands on there. And they have a 17 different view than engineers who analyze later.
18 And I just, it's a touchy area. And I'm 19 not completely sure, you know. The truth about 20 physically what happened, and when it happened, is 21 probably, not probably, is much clarified.
22 But the hows and whys, and what was going 23 on for the people, by now that's probably even lost, 24 except for the people who wrote it down at the time.
25 MEMBER DIMITRIJEVIC: Yes.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
128 1 DR. XING: Thanks. I --
2 MEMBER DIMITRIJEVIC: I would not 3 reference --
4 DR. XING: Yes.
5 MEMBER DIMITRIJEVIC: -- that.
6 DR. XING: I definitely see --
7 MEMBER DIMITRIJEVIC: You know, not 8 because the P factors are out of the range, you know, 9 stress, the worry, you know, about the lack of 10 instrumention, you know, after the DC batteries 11 failed.
12 I mean, only listed in positive way, when 13 they have to be inventive and bring batteries from the 14 car to start to get instrumentation. There is 15 absolutely no need to use those examples I think. I 16 would --
17 DR. XING: Yes. Very, I agree with you.
18 We'll work on, we have other good examples to replace 19 this one uncertainty. Thank you.
20 MR. DeJESUS: So, and I guess the takeaway 21 from this slide, Slide 42, is that these four context 22 categories are not intended to be an exhaustive 23 classification system. Sorry.
24 And so, the four context categories are 25 not intended to be an exhaustive classification NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
129 1 system. Rather, they guide the search for the 2 context.
3 And as mentioned before, if the action 4 takes place in the control room, then there may not be 5 a need to consider the environment and situation 6 context.
7 Next in Stage 1 is the identification of 8 important human actions. So, an important human 9 action is the unit of analysis of an HRA.
10 In general important human actions are 11 identified as actions that are required in the 12 scenario progression to achieve the goal of the event.
13 For example, that we provide on the slide, they 14 achieve a safe and stable state.
15 CHAIR BLEY: Now, that would be true for 16 our so called areas of omission.
17 MR. DeJESUS: Correct.
18 CHAIR BLEY: Which is really a figment of 19 the analyst's imagination. The so called errors of 20 commission don't quite fit that definition.
21 MR. DeJESUS: I agree. So, the search for 22 the baseline, in the baseline and deviation scenarios 23 should consider the interaction of humans with mission 24 critical and non critical systems. And identify 25 errors of commission that impact mission critical NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
130 1 systems if they are errors of commission.
2 The important human actions are defined at 3 a level that describes the human failure of the 4 action, and linking it to the affected systems. The 5 definition should include the success criteria of the 6 action, the consequence of the failure, the cues and 7 indications and their timing, the relevant procedure 8 guidance, and the time available for the action.
9 MEMBER DIMITRIJEVIC: Well, how did you 10 define, to be begin with importantly, human actions?
11 Every human action, everything will always 12 have important action associated with. So defining 13 important human action doesn't reduce a number of the 14 human actions that you have to look in.
15 Because for every human action there is a 16 task associated with it. So there is always action to 17 accomplish the task, right?
18 MR. DEJESUS: Yes. And I think in a 19 couple of slides I make that differentiation between 20 human action and important human action. Yes, in the 21 next couple of slides, if you --
22 MEMBER DIMITRIJEVIC: All right. Okay.
23 MR. DEJESUS: Yes.
24 MEMBER KIRCHNER: Where is it defined? Or 25 is it defined there by default?
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
131 1 I mean, obviously you're looking through 2 your scenarios. By default, I agree with Vesna, it's 3 just, those things to realize the interface of the 4 human with that scenario that you defined. And then 5 they're all important. Or I'm missing something.
6 MEMBER DIMITRIJEVIC: Right. That's how 7 I see it.
8 MEMBER KIRCHNER: Yes.
9 DR. XING: Yes, we have, I've got the 10 appendix number. One of the appendix identification 11 of important human actions.
12 MEMBER DIMITRIJEVIC: Right. He said he's 13 going to talk --
14 DR. XING: Yes. Okay.
15 MEMBER DIMITRIJEVIC: Are you going to 16 have examples?
17 MR. DEJESUS: Well, at least let me get 18 there and you can ask again.
19 So, the identification -- and the next 20 step is to identify and analyze critical task. And 21 we'll go back to the top level in a minute.
22 So, the objective of that part of the 23 process is to identify the critical task in an 24 important human action. Oh, Slide 44 everybody.
25 So, the objective is to identify the NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
132 1 critical task and characterize them. And a critical 2 task is an essential, it's essential to a success 3 criteria of the important human action and is the 4 basic unit for the HEP quantification.
5 And the failure of any critical task is 6 the same as, will give the failure of the important 7 human action. So I think about it like a series 8 system, if you will.
9 So, the next slide, 45, to provide more 10 insights of what we mean by important human action and 11 critical task, the figure shown on Slide 45 provides 12 a little bit more detail then what was shown on Slide 13 13. Essentially a human event or a scenario has one 14 or more human actions.
15 So, the important human actions, I don't 16 know, are divided into task and critical tasks. So 17 the, with respect to the important human action, it's 18 what the HRA, that's what it models. There may be 19 other actions.
20 I don't know if you can help me out here, 21 Jing, on differentiating the important human action 22 and the human action. There may be things like 23 confirming that something, that a parameter reach a 24 certain state, that may not be a important so it 25 doesn't get modeled into the HRA.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
133 1 DR. XING: Yes. I think technically we 2 started this important human actions from a PRA model.
3 A PRA model already identified those human failure 4 event. That's our starting point.
5 And in many PRA models, a lot of like 6 operator actions, now safety critical system are not 7 included in the PRA model. However, in some 8 particular scenarios, those actions can be important.
9 The failure of those action on the non-10 safety critical system can have a direct impact on the 11 safety stability of the reactor.
12 So, those what we also call the important 13 human actions into the LOCA. And at some part it's in 14 our agency, NRC staff's, STP analysis for Robinson 15 event.
16 For Robinson fire event, there is already 17 PRA model there. So they analyze the HFEs that was 18 already defined in the PRA model. I believe there was 19 eight HFEs, I could be wrong.
20 But again, the staff also identified two 21 or three additional human actions that was not 22 included in the PRA model. However, they were 23 critical to the success or failure in that event.
24 But I don't remember what other, those 25 three additional actions.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
134 1 MR. DEJESUS: Thank you, Jing.
2 CHAIR BLEY: I'm just a little confused.
3 You coined a new term with the important human 4 actions, which seems a common everyday term.
5 But in at least some design 6 certifications, maybe all, in the human factors 7 engineering, the same words are used to mean something 8 a little bit different. Just so you know that.
9 DR. XING: Yes.
10 CHAIR BLEY: You've defined what you mean 11 by them here.
12 DR. XING: Yes. For that we had actually, 13 we had several staff meeting based on HRA, PRA folks 14 on that. We would initially, earlier, would just call 15 them HFEs, to be consistent with our PRA model.
16 However, we realized as say IDHEAS-G is 17 for all HRA are risking from the nuclear applications.
18 In some application areas, such as dry 19 cask handling, nuclear waste material handling, there 20 isn't a PRA model, clear PRA model exists. But they 21 want to use this analysis, that's why we say 22 regardless --
23 CHAIR BLEY: I wouldn't really see a 24 problem with using HFE over there. I'm just worried 25 you'll get some confusion because important --
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
135 1 DR. XING: Yes.
2 CHAIR BLEY: -- human actions are already 3 defined. You might never run into those people who 4 are doing that work, but you might.
5 DR. XING: Yes. We had a sentence up 6 front in the --
7 CHAIR BLEY: HFEs is pretty --
8 DR. XING: -- however, I think we should 9 clearly cut out or reconsidered this in the report.
10 Thanks for the comment.
11 MEMBER KIRCHNER: Let me just, I'm looking 12 at the team work box and I'm thinking, normally what 13 you would hope, say the control room scenarios, the 14 team work actually helps understand, helps decision-15 making, helps execute.
16 So, isn't that an attribute almost or all 17 those other four boxes rather than, I'm just trying to 18 think how you would, if you're going to go and 19 quantify this, how having a separate box, is it a 20 positive factor or is it a negative factor?
21 Because team work, you know, it could work 22 at cross purposes or there could be confusion, which 23 would result in a delay in decision-making, et cetera, 24 et cetera. There's all kinds of complexity in that 25 box.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
136 1 So, if you're trying to get to something 2 that's quantifiable, I'm just trying to think about 3 the math of how that is included with the other four 4 boxes there for critical task.
5 MR. DEJESUS: So, if I understand your 6 question correctly, I don't know if this was mentioned 7 before, but the first four macrocognitive functions, 8 it can be used for an individual or a single team.
9 MEMBER KIRCHNER: All right. That wasn't 10 clear because --
11 MR. DEJESUS: First four.
12 MEMBER KIRCHNER: -- you had this box 13 called team work and I'm thinking --
14 CHAIR BLEY: But they specified that that 15 box is for teams interaction of separate teams.
16 MEMBER KIRCHNER: Yes. And there's that 17 dimension too. No, I understand, the control room may 18 know what it's doing, but someone may be out in the 19 plant doing something else at the same time that the 20 event is happening and that could cause all kinds of 21 complexity.
22 So that part, yes, I did understand. But, 23 Jing, while I think of the teamwork as, okay. Okay.
24 MEMBER DIMITRIJEVIC: So, there is some 25 actual events from the previous discussion, they NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
137 1 assume optimum situation. Everybody is in control 2 room, they work as a team, and that's your base case.
3 Now, if anything preserve that, some of 4 these outside, there is that, they only add the 5 negative. So optimum case, yes.
6 DR. XING: Thank you.
7 MR. DEJESUS: Okay. Okay. So, in the 8 third level of this figure on Slide 45, there is the 9 distinction between a critical task and a task.
10 So, why, the reason we define it that way 11 because task may compete for resources or interfere 12 with critical task. Therefore they should be 13 identified as part of the characterization of the 14 critical task.
15 They're not necessarily analyzed at the 16 macrocognitive function level, but they are, they're 17 there. So it's just part of getting the entire 18 context, if you will.
19 So, and then the critical task consists of 20 the cognitive activities with our, achieved through 21 the macrocognitive function. And the failure of a 22 critical task is represented by the cognitive, by the 23 applicable cognitive failure modes.
24 Next in Stage 2 is the identification of 25 the analysis of a critical task. And the NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
138 1 identification is a, the identification of the 2 critical task is usually done, or it's essentially 3 done using a task analysis.
4 And the, an essential element of a task 5 analysis is a task diagram. And what a task diagram 6 does is help identify the critical task, their 7 relations, cues and time information.
8 And they show the success path and order 9 of the task. And what I'm showing here on Slide 46 is 10 an example of a task diagram.
11 Moving on.
12 MEMBER DIMITRIJEVIC: So, in this task 13 diagram, are those analyzed as a separate task or is 14 that one task?
15 MR. DEJESUS: Well, each of those would be 16 one critical task on the, what, the rectangles.
17 MEMBER DIMITRIJEVIC: And each one of 18 those tasks will have their own HFEs associated with 19 those?
20 MR. DEJESUS: Correct.
21 MEMBER DIMITRIJEVIC: So this task is to, 22 you know, the RCP, the shield failure assumes so.
23 MR. DEJESUS: Yes.
24 MEMBER DIMITRIJEVIC: It's actually broken 25 now into three tasks. And each one of those tasks it NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
139 1 has it's own analysis of cognitive --
2 MR. DEJESUS: Activities.
3 MEMBER DIMITRIJEVIC: Man.
4 DR. XING: Yes. Yes, just to, I'd like to 5 make a comment on that.
6 So, once, when performing an HRA, analysts 7 have been troubled with, to what level of detail you 8 break the action or HFE into tasks.
9 MR. DEJESUS: That's the next slide.
10 DR. XING: What I heard from the HRA 11 analyst is you cannot specify a universal applicable 12 rule to what level of detail breakdown the action into 13 tasks. But when we come there we know where we should 14 have stopped.
15 But for us, we gave the general guidance 16 breakdown task. It's breakdown the critical task.
17 You breakdown when you need it. When you need it is 18 because you're going to assess the PIF.
19 If the PIF change between these two tasks, 20 then you should break it down.
21 MEMBER DIMITRIJEVIC: But every task has 22 detection and a procedure as a pathway.
23 DR. XING: Yes.
24 MEMBER DIMITRIJEVIC: So, what I conduct, 25 no, because these are part of the tasks.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
140 1 DR. XING: Yes. So, you can actually if 2 you break, in the past, in our exercise, we saw 3 people, some people would take, oh, I'm taking the 4 detection as a separate task because you have to 5 collect all this information then we'll decide whether 6 you're going to into E0 or E3.
7 MEMBER DIMITRIJEVIC: Okay.
8 DR. XING: I bet people will say, I will 9 just take this as a whole task.
10 MEMBER DIMITRIJEVIC: I know. But then 11 when you come to opening the valve, or whatever, or 12 the tripping the pump, you don't have detection 13 anymore in there.
14 DR. XING: Yes, you don't have --
15 MEMBER DIMITRIJEVIC: Because you separate 16 it.
17 DR. XING: Yes. So --
18 MEMBER DIMITRIJEVIC: And you don't have 19 a procedure and you don't have training and you have 20 taken all of these cognitive parts because they came 21 from the, in the other tasks.
22 DR. XING: You probably still have 23 procedure. I mean, even you just to --
24 MEMBER DIMITRIJEVIC: Protection.
25 DR. XING: -- open a valve, you need to NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
141 1 check a bunch of parameters. The procedure will tell 2 you which parameters you need to check.
3 MEMBER DIMITRIJEVIC: This will be part of 4 the, AOP training. Okay.
5 DR. XING: Yes.
6 MEMBER DIMITRIJEVIC: All right, let's see 7 how it works. We cannot wait for examples.
8 DR. XING: Thank you.
9 MR. DEJESUS: And the next slide we sort 10 of touch on it a little bit. It's the how, the 11 guidance we provide on breaking down the HFE into 12 critical task.
13 And we understand that this is an 14 important aspect of the process, so that's why we 15 included the guidance. And I'm on slide 47.
16 The first guideline is to use as few 17 critical tasks as possible. And then further 18 breakdown the HFE, only with the PIF attributes, vary 19 for different portions of the action.
20 And next, the important human action 21 should be broken down into critical tasks at a level 22 that retains the context. And that can be represented 23 by the macrocognitive functions.
24 And then last, the breakdown of the action 25 stops at the level where there is performance NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
142 1 indications or data available to support the 2 quantification of the human error probability.
3 So it's not, we're not telling the 4 analyst, yes, go crazy and, yes. We understand this 5 tedious process.
6 MEMBER DIMITRIJEVIC: There was several 7 methods because you went to a procedure break every 8 task and everything, I mean --
9 DR. XING: Yes.
10 MR. DEJESUS: Next is the character, the 11 first part was to identify the critical task, and them 12 breaking them down. And now we're in the 13 characterization of those critical tasks that are 14 identified.
15 So, the characterization, what it does is 16 defines, refines the scenario context in the PIFs.
17 And I'll show in the next slide what I mean by that.
18 And also identifies the cognitive 19 activities involved in the critical task using a 20 taxonomy of cognitive activities, which you have seen 21 before. It's those first boxes in the, each of the 22 macrocognitive function elements in Chapter 2.
23 So, once you have the cognitive activities 24 required by the critical task, you determine which 25 macrocognitive function and processor require that NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
143 1 critical task. And if the characterization is the 2 basis for identifying the cognitive failure modes that 3 are applicable to the task.
4 So, on the next slide, this table, it's in 5 the report. So the, specifying the task goal, the 6 specific requirements, the cues and supporting 7 information, procedures, personnel, task support, that 8 has to do with the context of the action.
9 And one of the characteristics of the task 10 is the cognitive activities, as I mentioned before.
11 And one of the other aspects is concurrent task and 12 teamwork considerations.
13 MEMBER DIMITRIJEVIC: Well, example of 14 what you had before about this reactor coolant pump, 15 when you came to the last step, let's say trip the 16 pump instead of restoring cooling, there is not 17 anymore cognitive activities. All cognitive 18 activities in the procedures has to perform physical 19 action.
20 MR. DEJESUS: Well, that executing the 21 action, we consider that a macrocognitive function.
22 We consider that, at least a part of cognition, if you 23 will.
24 MEMBER DIMITRIJEVIC: All right.
25 MR. DEJESUS: So it's sort of like a NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
144 1 different way of seeing it.
2 MEMBER DIMITRIJEVIC: Okay.
3 DR. XING: Yes.
4 MR. DEJESUS: Okay, moving on, on Stage 2, 5 Slide 50, for those on the phone. It's the 6 identification of the applicable cognitive failure 7 modes.
8 And in here I want to step back a little 9 bit and bring, I guess, a philosophical view on how 10 the cognitive failure modes were developed. And so, 11 we developed several criteria for the cognitive 12 failure modes in HRA.
13 And with respect to completeness, the 14 cognitive failure modes should adequately represent 15 the ways in which a task might fail. The non-16 overlapping, again, this is a, I guess a criteria we 17 intend to do that, I'm not sure if it's possible.
18 The scope of the individual cognitive 19 failure mode should not overlap with the, should not 20 overlap with each other. So that is a human failure 21 represented by one cognitive failure mode, it's not 22 represented by other cognitive failure modes.
23 The specificity on sensitivity, that 24 refers to the cognitive failure modes should be 25 specific enough to differentiate the failures caused NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
145 1 by different context. That is, cognitive failure mode 2 should specifically link to a limited set of PIF 3 attributes. And be sensitive to changes in those 4 attributes.
5 And the observability is to estimate the 6 human error probability of a cognitive failure modes.
7 It should be done using available data or evidence.
8 Or evidence. And the cognitive failure mode should be 9 observable. And related to data.
10 And since IDHEAS-G is a general 11 methodology, the cognitive failure modes that are 12 provided in the IDHEAS-G should be independent of HRA 13 application.
14 MEMBER DIMITRIJEVIC: How are the 15 cognitive failure modes connected to cognitive 16 macrocognitive tasks we've discussed before? Are they 17 direct?
18 MR. DEJESUS: That's the next slide.
19 Thank you for the segue.
20 So the, I didn't find the applicable 21 failure modes. IDHEAS-G has this basic set of 22 cognitive failure modes, and there is three levels.
23 There is the high level cognitive failure 24 modes. The high level, it's the failure of the 25 macrocognitive function.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
146 1 Then there is the middle level, which is 2 the failure of the processor. Of the macrocognitive 3 function.
4 And then below that is the detailed 5 cognitive failure mode, which is the behaviorally 6 observable failure of the processor. So there is 7 three levels.
8 And we developed a reference set of the 9 detail cognitive failure modes. They're provided in 10 Chapter 4.
11 And so, for a specific HRA applications, 12 the developers may develop its own set of the detailed 13 cognitive failure modes or from the middle cognitive 14 failure modes or adapt what we provided as the 15 reference set.
16 Moving on, Slide 52. The assessment of 17 the performance influencing factors. The PIF have a 18 baseline. I think we discussed that before, where 19 there is no or low impact to the human error rates.
20 So they are determined based on the 21 context in the scenario analysis, which is from Stage 22 1, the development of the operational narrative, and 23 the task characterization, which I discussed a few 24 slides ago.
25 And when the context, having this baseline NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
147 1 where there is no impact, and I think this goes back 2 to the discussion before, having a PIF attributes 3 being all negative, so when the context challenges the 4 task performance, it maps to the PIF attributes, which 5 was discussed before.
6 But the key point I want to make here, I 7 guess add to that, I guess it's when the context 8 facilitates the task performance, it moves PIF 9 attribute to the baseline. So that's, I guess, the 10 positive impact how IDHEAS-G models that.
11 MEMBER DIMITRIJEVIC: Zero baseline.
12 (Off microphone comment.)
13 MR. DEJESUS: Base, I wouldn't zero, it's 14 a minimum. I don't know what that minimum HEP would 15 be.
16 MEMBER DIMITRIJEVIC: Not HEP, but PIF is 17 zero.
18 MR. DEJESUS: Oh, the WI, yes, the impact 19 --
20 MEMBER DIMITRIJEVIC: Yes.
21 MR. DEJESUS: -- yes.
22 MEMBER DIMITRIJEVIC: Okay.
23 MR. DEJESUS: So, moving on to Stage 3, 24 which is the quantification of the HEP, on Slide 53.
25 So, the estimation of the HEP is based on the modeling NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
148 1 of the important human actions in Stage 2.
2 And the HEP has two parts, as I mentioned 3 before. The Pt, the error attributed to the 4 uncertainty in the time available and time required.
5 And Pc, which is the error probability attributed to 6 the cognitive failure modes of the critical task.
7 The overall HEP is calculated as the 8 probabilistic sum of Pt MPC and is the equations shown 9 in the slide. And as you can tell, this calculation 10 assumes that Pt and MPC are independent. That's an 11 assumption.
12 Pt is calculated as the convolution of the 13 probability distributions of time available and time 14 required. The equation is shown in the slide.
15 And Pc is the probabilistic sum of the 16 error probability of each critical task. Which is the 17 equation I'm showing in the bottom.
18 And then the probability of the error of 19 the critical task is calculated as also the 20 probabilistic sum of the error probability of the 21 cognitive failure modes.
22 MEMBER DIMITRIJEVIC: And if the, because 23 the time Pt is zero?
24 MR. DEJESUS: I'm sorry?
25 MEMBER DIMITRIJEVIC: For the reconsider NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
149 1 Pts cross section within to distribution, if they 2 don't cross that, it's zero.
3 MR. DEJESUS: So, Pt doesn't make a 4 contribution to the overall human error probability.
5 MEMBER DIMITRIJEVIC: So, if that's a 6 matter, does he had a four to six hours, that doesn't 7 have an impact on anything?
8 MR. DEJESUS: Yes. It would not have a 9 contribution, yes.
10 MEMBER DIMITRIJEVIC: No, it has to have 11 contribution survey, at least very low stress or 12 something? You have plenty available time, that 13 doesn't have a contribution?
14 MR. DEJESUS: Jing.
15 DR. XING: To plenty of time would be 16 modeled, will be considered in recovery. So you have, 17 we have a good collection of experimental data shown, 18 which is a little bit against our previous blame.
19 So, when you have more than adequate time 20 you needed, extra time doesn't really reduce your 21 chance of error. Because you still make error due to 22 those cognitive failure mode.
23 However, the extra time would reduce the 24 overall chance of error because that give you more 25 opportunity to recover. Provided if the error was NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
150 1 recoverable. So that's it.
2 CHAIR BLEY: Yes, but you're built into 3 the cognitive failure mode probability. If there 4 isn't much time, and I know I don't have much time, 5 then that effects my performance separate from the 6 time calculation.
7 That's just a real world, this is --
8 DR. XING: Yes.
9 CHAIR BLEY: -- you can either get it done 10 or you can't.
11 DR. XING: We have a PIF there from --
12 MR. DEJESUS: Oh, there is a PIF about 13 time pressure --
14 CHAIR BLEY: Right.
15 MR. DEJESUS: -- in the task. I think 16 it's in the task context category.
17 CHAIR BLEY: Yes. Not only is there, if 18 you know you have time pressure. Yes.
19 MR. DEJESUS: Yes.
20 MEMBER DIMITRIJEVIC: And also, if you 21 have too much time then you get bored and you may 22 forget.
23 CHAIR BLEY: That's true too.
24 (Laughter.)
25 MEMBER DIMITRIJEVIC: Do you have a PIF NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
151 1 for that?
2 DR. XING: Yes. Yes, we actually have a 3 PIF for that for execution. If you have too much time 4 and you don't need to immediately perform the 5 exclusion, then that increases the change of error.
6 MR. DEJESUS: Okay. So let's talk about, 7 on Slide 54, start on Pc. Get that, explain that 8 component.
9 So, the probability of a cognitive failure 10 mode can be estimated in one or a combination of 11 following three ways. The database estimation is 12 essentially calculating the error, the error as a 13 ratio of the number of cognitive failure modes to the 14 number of times the task --
15 MEMBER DIMITRIJEVIC: Can I get you back 16 to the previous slide --
17 MR. DEJESUS: Sure.
18 MEMBER DIMITRIJEVIC: -- because I have a 19 question that I missed to ask.
20 MR. DEJESUS: Slide 53.
21 MEMBER DIMITRIJEVIC: You release 22 different PIF for every CFM. Is there some importance 23 of that, that you said go to aid in 17 then 317 then 24 710?
25 MR. DEJESUS: Oh, thank you for bringing NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
152 1 me back to that. Yes.
2 The probability of a cognitive failure 3 mode is essentially, I think we have touched on it but 4 I guess I'll put it in concise terms. It's a function 5 of the PIF attributes. That's what it's meant by the 6 different CFM 1, 2, 3, 4 shown in the slide.
7 So it's a function of, for example, Pcfm 8 1 is a function of the attribute of PIF2 and PIF8, et 9 cetera. Whatever, and we'll get into how we quantify 10 that in a moment. That's CFM.
11 MEMBER DIMITRIJEVIC: So, CFM, you already 12 have idea what that CFM is going to be critical 13 failure mode, you already have in mind which one is 14 that?
15 Okay, that's --
16 MR. DEJESUS: Well, that's not in the 17 qualitative part of the analysis identified, which are 18 the cognitive failure modes. And we'll talk about 19 this when we talk, in the IDHEAS-ECA.
20 In that specific application, the 21 cognitive failure modes are, we left that at the high 22 level. Failure of the macrocognitive function.
23 MEMBER DIMITRIJEVIC: Right. But how do 24 you know which PIFs will be applicable because that's 25 a selection of analyst, whatever he's going to define NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
153 1 as a critical failure mode.
2 MR. DEJESUS: That's part of the task 3 characterization. When the critical task is 4 characterized, correct me if I'm wrong, Jing --
5 DR. XING: Yes.
6 MR. DEJESUS: -- there's this scenario 7 analysis where, for example, where the action is 8 taking place. That's part of the PIF and the 9 environmental situation context.
10 And then there is the attributes of, with 11 respect, what tools are needed, essentially what the 12 table shown on, if I get it correctly, Slide 49.
13 That's how, when that's done as part of the 14 qualitative analysis, that's where those failure modes 15 and critical task are identified.
16 So, the cognitive failure modes are 17 identified based on the task characterization.
18 MEMBER DIMITRIJEVIC: Okay.
19 MR. DEJESUS: Hope that helps. Okay. So, 20 Slide 54.
21 We were talking about the probability of 22 the cognitive failure modes being estimated when one, 23 or a combination of the three ways, shown in this 24 slide. The database estimation is made, determined or 25 estimated as the ratio of the number of cognitive NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
154 1 failure modes to the number of times the task 2 involving the cognitive failure modes is performed for 3 a given set of PIF attributes.
4 And this provided that there are adequate 5 human error data. However, human, as we all know I 6 think, human error data are scarce, so IDHEAS-G 7 provides a framework to generalize and integrate 8 various sources of human error data, which will be, 9 Jing will discuss later.
10 Then there is the expert judgment. Given 11 the scarcity of data, then there is the use of 12 experts. The experts, what they provide is a 13 distribution of the HEP.
14 And it represents their best knowledge.
15 However, using expert judgment usually requires a lot 16 of resources.
17 Then that leads us to the HEP 18 quantification model, which is a, I would say a trade 19 off between, okay, not having enough data and the 20 substantial resources that would be required for a --
21 CHAIR BLEY: And this is what Jing showed 22 us before with the weight functions?
23 MR. DEJESUS: It is there.
24 CHAIR BLEY: Okay.
25 MEMBER DIMITRIJEVIC: Well, no, but you NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
155 1 still haven't made the base case. So, what is 2 actually HEP quantification?
3 DR. XING: Yes.
4 MR. DEJESUS: That's in two slides from 5 now.
6 DR. XING: That's what I will talk in the 7 afternoon, the data. We needed this base data to, we 8 need the data to establish the base case.
9 MEMBER DIMITRIJEVIC: This equation, but 10 this equation there is many, I mean, just mentioned, 11 but there is not any inputs to --
12 DR. XING: Yes. So we use the existing 13 data as the base.
14 MEMBER DIMITRIJEVIC: Because you don't 15 want to tell us you're using all of the existing HEP 16 model by the base, to quantify base --
17 DR. XING: Yes.
18 MEMBER DIMITRIJEVIC: -- then that will be 19 all right.
20 DR. XING: In this case we tried to not 21 use the base HEP existing HRA model because we already 22 collected a generalized substantial cite of real 23 performance data. So we used those.
24 But I still looked at based numbers.
25 Other HRA measured, trying to get a sense of where we NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
156 1 are compared to other ones.
2 MEMBER DIMITRIJEVIC: Okay.
3 DR. XING: Yes. So we didn't use them, we 4 used them for verification. Go ahead, Jonathan.
5 MR. DEJESUS: And this HEP quantification 6 model was based on the cognitive literature research 7 that Jing performed.
8 So, going to the next Slide, 55. So this 9 HEP quantification model has two assumptions. This 10 slide describes the first one.
11 Is, the first assumption is the use of 12 base performance influencing factors and their 13 respective base human error probabilities.
14 A review of the cognitive literature 15 performed by Jing found that three of those 20 PIFs 16 have the ability to change the HEP from a minimum 17 value, whichever that is, to one. Which is 18 represented in the slide by the blue curve.
19 So, those three PIFs are information 20 availability and reliability, task complexity and 21 scenario familiarity.
22 Then the remaining PIFs are shown, the 23 remaining 17 PIFs are what we call in the report the 24 modification PIFs. So, how to show this in the slide 25 is essentially, you are in a, and that's where the PIF NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
157 1 weight factor enters.
2 And what it does essentially is moves the 3 blue curve to the right or to the left, depending on 4 what, well, in this case would be to the left. Which, 5 if you take for example this, the very line in the 6 middle of this slide, you're in the blue curve and 7 that's your HEP from the base PIFs.
8 So, if you include the modification PIFs, 9 what it does is it moves that curve to the right, to 10 the left in this case, which increases the HEP. So 11 that's represented by the orange and red curves in 12 this slide. And --
13 CHAIR BLEY: Explain your x-axis, the 14 attribute of base case scenarios.
15 MR. DEJESUS: Oh. The x-axis, it's the 16 attributes of the base PIFs. And --
17 CHAIR BLEY: That's a label, that's not an 18 explanation.
19 (Laughter.)
20 MR. DEJESUS: I know.
21 DR. XING: The three show there 22 information, reliability, types of complexity and the 23 scenario familiarity.
24 MEMBER DIMITRIJEVIC: Those are you're 25 PIFs?
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
158 1 DR. XING: Yes, those are --
2 MEMBER DIMITRIJEVIC: But this is, I think 3 this is just there substantive to the solution for 4 base case.
5 DR. XING: Yes. So, let's --
6 CHAIR BLEY: Is that what this is?
7 DR. XING: Yes, that is. Let's take an 8 example. Supposed we, out of the three, we only have 9 an issue, information reliability in the SPO, some 10 indicators is not reliable, stuck there. So what it 11 shows is, was a half hour ago. So that is giving 12 misleading information.
13 And the third were site map base failure 14 rate. Let's say .1. I have a ten percent chance I 15 will fail if I get this misleading information.
16 CHAIR BLEY: Okay.
17 DR. XING: So, I will gather this, what we 18 called a base HEP because of this factor. Then, in 19 performing the task, if I have some other PIFs 20 attribute to appear to like, I didn't have peer 21 checking or I have concurrent task distraction.
22 Those are the two, PIF1, PIF2, move me 23 from .1 versus up, maybe multiple by five or multiple 24 by whatever number. That's what --
25 MEMBER DIMITRIJEVIC: It's actually some NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
159 1 --
2 DR. XING: Yes. Right.
3 MEMBER DIMITRIJEVIC: So, if you have a 4 nine multiplier, two and one multiplier or three, 5 you're actually getting four, four times --
6 DR. XING: Right. That's what we did.
7 That's based on all the data we looked at. Bases 8 reached, have really pushed the full range of the 9 probability.
10 Example we had earlier, the HSI is human 11 system interference. We can say that it's modified.
12 But what if the example you gave me 13 earlier, like if the indicator is mislabeled, that 14 will set you to one. Yes, that is a case, but in the 15 data we have, that's a very rare case because, one, 16 when we perform a PRA we are in the assumption, base 17 HFE is physical.
18 If it's completely, entirely mislabeled, 19 there is no way you recognize it's mislabeled. You 20 already determined this is a physical action, we don't 21 even go to quantify it.
22 MEMBER DIMITRIJEVIC: Is this line in the 23 middle --
24 DR. XING: I will say the baseline.
25 MEMBER DIMITRIJEVIC: Do you have that NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
160 1 graph in the report?
2 CHAIR BLEY: Yes.
3 MR. DEJESUS: Figure 4-8.
4 MEMBER DIMITRIJEVIC: I mean, this is 5 total lack of the definition you guys should remember.
6 I mean, what it is. I mean, you know, you have to 7 give it the good title and then to define what is the, 8 what type of decision, this line, things like that.
9 DR. XING: We need the right figure. We 10 need the right figure captions, that's our --
11 CHAIR BLEY: Right now, at least to me, 12 that section in the report, and looking at these 13 slides, is not transparent. Your blue curve is 14 accounting for the three most important PIFs, I think.
15 But are they at their worst condition, are they at 16 the, I don't know how to use this curve and I don't 17 understand how to carry this out.
18 Maybe this is a good time to break for 19 lunch and after lunch you can be refreshed, we'll all 20 be refreshed, and you can explain to me how you use 21 this thing.
22 MEMBER DIMITRIJEVIC: You have, up to the 23 Slide 60, right, before you enter the new --
24 CHAIR BLEY: Yes. Just a couple more, but 25 --
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
161 1 MEMBER DIMITRIJEVIC: Yes. You want to 2 finished up to 60 then? I say let's go to 60 so we 3 can have more problems to --
4 CHAIR BLEY: Yes, go ahead. But I don't 5 --
6 MR. DEJESUS: Okay.
7 CHAIR BLEY: -- the picture is pretty. I 8 see one curve is higher than the other. I don't see 9 anything that's relating how you do this 10 quantification to the status of the PIFs.
11 It might be along that baseline but I 12 don't know what that is, except something that says, 13 attribute to the base PIFs. Are they in a base 14 condition?
15 DR. XING: Yes.
16 CHAIR BLEY: I don't know quite what to do 17 with this.
18 DR. XING: I just realized, it was my 19 fault, I used the same word for two different meaning.
20 When you say, earlier when we talked about the PIFs, 21 we said each PIF start from the baseline attribute 22 wherein to the bad situations.
23 Now, the base here have an entirely 24 different meaning. We just classified the PIFs into 25 these two types.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
162 1 One type we call it base PIF, the other 2 type we call it modification PIFs. So, I think if 3 your mental model now, if you replace the word, base 4 PIF, with something like dominating PIF or important 5 PIF, that might work better.
6 MEMBER KIRCHNER: And that's determined by 7 human judgement or which base PIFs?
8 DR. XING: Yes. So, prime example would 9 better in this case. Previously when we use the SPAR-10 H model, primarily it's not all, most of the HRA we 11 used it for control room actions using EOP, which are 12 well proceduralized.
13 In SPAR-H, there is minimum HEP value.
14 For execution, it's e minus 3 I think. And for 15 understanding, or a combination for understanding a 16 decision making is e minus 2.
17 So, that's an assumption made. Assuming 18 you have all the information you need. And the 19 scenarios you are familiar with because already in 20 your training. So, SPAR-H does not model those two 21 factors adding more.
22 But they do model complexity. But that's 23 assumption without complexities that all these three 24 factors are good, your HEP is generally minimal e 25 minus 3.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
163 1 However, when we move to action, control 2 room action, for example, in 2016 NEI's guideline, HRA 3 for crediting flags, they didn't have much information 4 on how to give a detailed model. But at that time 5 they thought, okay, because this action, control room 6 action, they are saying things like scenarios not 7 familiar, you may not have all the information.
8 Let's just assess the base HET as Code 1.
9 So, all the actual control room action, if no other 10 thing is bad, the HET will be .1. If anything else 11 comes up, then you will multiple that .1.
12 CHAIR BLEY: So, if I just look at the 13 blue curve --
14 MR. PETERS: Dennis, can I do my layman's 15 interpretation since I'm not an HRA expert?
16 If I want to look at this graph, the blue 17 curve would be a distribution function of, let's 18 pretend the blue curve or modeling tasks complexity.
19 So, in a low task complexity we would look 20 at that blue curve, when there's very low complexity, 21 our error rates would be around e to the minus 4. As 22 complexity increases, it would move up that curve all 23 the way up to, trending towards the value of one.
24 So that's what we would see for those 25 three tasks that you are modeling for.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
164 1 CHAIR BLEY: So this isn't a curve one 2 uses for quantification, it's just an illustration 3 that as the PIFs get worse, the probability of failure 4 gets higher?
5 MR. PETERS: Absolutely correct.
6 CHAIR BLEY: If that's how it showing, 7 okay.
8 MR. PETERS: Yes.
9 CHAIR BLEY: It looks like it's trying to 10 show me something quantitative, how I could use this 11 to pick off a number.
12 MR. PETERS: No, no. We're not picking 13 numbers off this curve.
14 DR. XING: Sorry, I never realized that's 15 --
16 CHAIR BLEY: Okay.
17 DR. XING: I apologize for that.
18 MR. PETERS: The only thing it's trying to 19 illustrate is for those three curves. That's how the 20 distribution function will work.
21 And if you have the 17 other curves, it 22 would move your curve left and right --
23 CHAIR BLEY: Okay.
24 MR. PETERS: -- based upon that infancy 25 factor. Yes.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
165 1 DR. XING: Yes.
2 MEMBER DIMITRIJEVIC: Well, okay, well, 3 that's different what your slide puts. Because it 4 just says, 17 factors move your, I mean, if this 5 presents just PIF and not basic HEP then the factors, 6 they'll have to be parallel, they all can relate and 7 go on the different level, they can be identical.
8 There will not be 17 curves.
9 DR. XING: Yes, there will not be 17 10 curves. Because you combine them. That's early we 11 measure --
12 MEMBER DIMITRIJEVIC: Because, in my 13 understanding, blue curve is your HEP, your base case 14 HEP multiplied, that gets multiplied, or some with the 15 different PIF.
16 This is very confusing and you should 17 definitely --
18 MR. PETERS: Yes, so the blue curve is 19 just representing the human error probabilities for 20 those three --
21 MEMBER DIMITRIJEVIC: Yes.
22 MR. PETERS: -- performance implementing 23 factors. And you would move it, so they calculate it 24 in a different way. This is just trying to illustrate 25 how those function.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
166 1 CHAIR BLEY: Maybe if we go to the next 2 couple of slides.
3 MEMBER DIMITRIJEVIC: But it's not 10 to 4 minus 4 performance in the --
5 CHAIR BLEY: I think they're just trying 6 to show us that --
7 MEMBER DIMITRIJEVIC: Performance 8 indicators --
9 CHAIR BLEY: -- the HEP goes up as the 10 PIFs get worse. If that's all you're trying to show 11 me, I didn't need the curve.
12 (Laughter.)
13 MR. DEJESUS: Got it.
14 MEMBER DIMITRIJEVIC: Well, it's showing 15 that there is parallels, but because they're all going 16 to come to one, they're little --
17 MR. DEJESUS: Yes, the point of this slide 18 is show that first assumption of the, I guess the 19 differentiation of the different PIFs.
20 MR. PETERS: Let's move as quickly as we 21 can to the next slide.
22 MR. DEJESUS: Yes, that's the point.
23 (Laughter.)
24 MR. DEJESUS: So, here is the second 25 assumption. And here you have Slide 56.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
167 1 The second assumption is that the linear 2 combination of the PIF effects. So, this is the 3 equation we use to calculation the probability of the 4 cognitive failure modes, and that first term, the Pcfm 5 base, I guess we'll change that term, base, it's based 6 on, it's related to the three PIFs.
7 The information reliability, 8 unavailability, task complexity and scenario 9 familiarity. So those three go there in that first 10 term.
11 The second, the one plus the summation, 12 that's where the other 17 PIFs come in to modify the 13 probability, the --
14 MEMBER DIMITRIJEVIC: Wait, wait, wait.
15 MR. DEJESUS: -- error probability.
16 MEMBER DIMITRIJEVIC: Where does three go?
17 MR. DEJESUS: The first three, in the 18 first term. The Pcfm base. They go there.
19 MEMBER DIMITRIJEVIC: Why?
20 MR. DEJESUS: Because that's, according to 21 the cognitive literature search, that's what can 22 change the HEP, in this case, the probability of a 23 cognitive failure mode from the minimum value to one.
24 That's how, I guess that first assumption 25 is implemented into a quantification model. Which is NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
168 1 what we're showing here on Slide 56.
2 MEMBER DIMITRIJEVIC: So your base case 3 already has the three PIFs in, that's what you're 4 saying?
5 DR. XING: Yes. So, put it in base way.
6 If a task is not complex at all, it's very simple, 7 it's the same as just press a button.
8 And the HEP for that would be e minus 4.
9 So very unlikely you will get it wrong. Therefore, 10 even you have a bad procedure, that's a bad procedure, 11 even the procedure is not good, the training is not 12 good. So you have some W there.
13 You will have, you're over or the HEP will 14 still be very low. Your HEP will be e minus 4 15 multiplied probably 1.1 or 1.2.
16 So that, however, if you have very 17 complex, the task, if you have this task, let's see, 18 still using example, you don't have the right 19 information because your indicator gave you the wrong 20 information. That alone will set your HEP failure 21 rate at .3.
22 So, in this case, even if you have the 23 ideal procedure, you would still end up in the error 24 probably, .3. So that's why we said, these three 25 factor based on the data, the metadata analysis, NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
169 1 there's really dominate which range of HEP you are.
2 CHAIR BLEY: You know, that paper that 3 won't be here for two years I guess is the key to 4 this.
5 DR. XING: Yes. Okay.
6 MR. DEJESUS: Okay.
7 DR. XING: Yes, go ahead.
8 MR. DEJESUS: So, the third factor, the C 9 factor, is the PIF interaction factor. And it's set 10 to one, with the linear combination of PIF effects.
11 And then there is one over the recovery 12 factor, which is set to one, unless there is data 13 suggesting otherwise.
14 And the reason we choose to do the 15 division is to have that recovery factor be an integer 16 rather than be like, you can say, oh, recovery factor 17 of ten. So instead of recovery factor of only one.
18 So sort of like, that's why there is a division there.
19 And with respect to the, I just recall 20 that what Jing talked about before, the PIF weight 21 factor, which is given by the equation in the slide.
22 CHAIR BLEY: Okay. If you go back to 23 Slide 54.
24 MR. DEJESUS: No, I'm not that far. Okay, 25 one more.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
170 1 CHAIR BLEY: You have three different ways 2 to quantify. Which one are we looking at here?
3 MR. DEJESUS: HEP quantification model.
4 CHAIR BLEY: Okay. And this is the, Jing, 5 where you start?
6 MR. DEJESUS: Correct.
7 CHAIR BLEY: Are you going to talk about 8 the other two?
9 MR. DEJESUS: Well, the data-base 10 estimation, I think Jing will talk about with respect 11 to generalizing data from other domains into HRA --
12 CHAIR BLEY: That's in the data?
13 MR. DEJESUS: Yes, that's in the data 14 part. The expert judgment is, you probably know this 15 better than me, getting experts and getting into a 16 distribution of HEPs.
17 CHAIR BLEY: Is that right, okay.
18 MR. DEJESUS: So that, I think the point 19 in this slide, 54, was to how we got to the HEP 20 quantification model.
21 CHAIR BLEY: Okay.
22 MR. DEJESUS: So given that there is not 23 much data, no expert judgment and --
24 CHAIR BLEY: Yes. So --
25 MR. DEJESUS: -- and so on.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
171 1 CHAIR BLEY: -- if we believe this stuff, 2 we might be comfortable using HEP quantification 3 model. But getting comfortable with that and this 4 linear combination of those weights, really requires 5 getting comfortable with the database, so we can't 6 really do that now.
7 DR. XING: Yes.
8 CHAIR BLEY: And maybe you're comfortable 9 with it. I think you've been comfortable with it for 10 several years, but it's still, there's nothing there 11 yet for us to see.
12 Okay, so maybe the equation works great.
13 And we've got the data to show that this is a 14 reasonable model. Why don't you go on to the next 15 slide.
16 MR. DEJESUS: Okay. So next we go into 17 Pt, or the time uncertainty analysis. And this model 18 was developed in response to ACRS comments.
19 And actual events have, so the PET, as I 20 mentioned before, is composed of time available and 21 time required. So, in actual events there are 22 inherent uncertainties in those quantities.
23 And the best, the mathematical way, I'm 24 not sure, to show that, is through a probability 25 distribution. And I apologize that the slide is not NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
172 1 showing the time available distribution. Maybe the 2 color I chose was gray, so, you can see it on my 3 slide.
4 MEMBER DIMITRIJEVIC: Yes, we can see it 5 too.
6 DR. XING: There is another line, you just 7 go like this.
8 CHAIR BLEY: It's in the book.
9 MR. DEJESUS: I don't know if this is a 10 PDF version of the slides, but yes.
11 DR. XING: No, it's on the computer screen 12 that's okay, it's just --
13 MR. DEJESUS: Oh, okay, not on the screen 14 --
15 DR. XING: -- not on the projector screen.
16 MR. DEJESUS: -- okay, sorry. So, because 17 of those inherent uncertainties in what goes into time 18 available and time required, the best way to represent 19 it mathematically is through a probability 20 distribution.
21 So it's, the Pt is the convolution of the 22 time available and time required distributions and is 23 the probability that the time required is greater than 24 the time. Time required is greater than the time 25 available.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
173 1 And Pt is proportional to the area where 2 the two tails of the probability distributions 3 overlap.
4 MEMBER DIMITRIJEVIC: So, I assume the 5 available time comes from the success rate failure, 6 thermal hydraulic analysis?
7 MR. DEJESUS: Yes.
8 MEMBER DIMITRIJEVIC: And where does 9 required time come from?
10 MR. DEJESUS: Required, that's my next 11 couple of slides.
12 MEMBER DIMITRIJEVIC: All right.
13 MEMBER BALLINGER: Is this just a calcium 14 distribution?
15 MR. DEJESUS: In the example shown --
16 CHAIR BLEY: No.
17 MEMBER BALLINGER: It can't be. Okay.
18 CHAIR BLEY: Well, go ahead, make your 19 bid.
20 MR. DEJESUS: In the example shown, yes.
21 MEMBER BALLINGER: Okay.
22 MR. DEJESUS: But that's for illustration 23 purposes.
24 MEMBER BALLINGER: Okay.
25 CHAIR BLEY: In most cases, Ron, this NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
174 1 would be from an expert elicitation essentially.
2 MEMBER KIRCHNER: Well, there's certain 3 scenarios, I'll throw up the hypothetical, ATWS in the 4 BWR again, there is some distribution on the 5 calculations where oscillations would, the onset of 6 oscillations would start. And you could put, I guess, 7 some spread on the time required.
8 CHAIR BLEY: That would be taking thermal 9 hydraulic calculations and applying expert judgment to 10 a distribution on hand. Unless you do a whole bunch 11 of cases.
12 MEMBER KIRCHNER: Yes. Well, often they 13 do.
14 CHAIR BLEY: Yes.
15 MEMBER KIRCHNER: So you have an 16 uncertainty --
17 CHAIR BLEY: So you could.
18 MEMBER KIRCHNER: -- kind of analysis on 19 the --
20 CHAIR BLEY: So you base it on as much 21 quantification, calculation as you can. But the time 22 required is really a judgment call. It's not a 23 stopwatch thing, you can go out and do a --
24 MEMBER KIRCHNER: No, no, not at all.
25 CHAIR BLEY: -- acknowledge that other NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
175 1 things will make them take a little longer or a little 2 less.
3 MEMBER KIRCHNER: Well, if they don't drop 4 the level then they're going to be in serious trouble, 5 so --
6 CHAIR BLEY: On that particular scenario.
7 MEMBER KIRCHNER: -- on that particular 8 scenario, yes.
9 MR. DEJESUS: Okay. So, for the time 10 available, there is the quantification model for Pt 11 does not credit situations where there is excessive 12 time available, and that's because there are studies 13 that show that it doesn't have an impact on the error 14 rates in the task performance.
15 However, having the extra time available 16 makes recovery possible but does not guarantee it.
17 And if I recall correctly, the IDHEAS at-power 18 application, having extra time, it's one of the 19 factors contributing to making a decision that 20 recovery is possible.
21 The factors that affect the distribution 22 of time available, as mentioned, as touched before, 23 the thermal hydraulic calculations, the system time 24 window and time delay for the cue, and I took HRA 25 training recently and I asked, hey, do those thermal NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
176 1 hydraulic calculations give a distribution and I was 2 told no. So --
3 (Laughter.)
4 MR. DEJESUS: And there were people 5 started talking about, oh yes, but you got this, you 6 got that. This could change, this could, that. Well, 7 exactly, you're making my point, it should be a 8 distribution.
9 But we take what we get and try to make it 10 better. And another --
11 MEMBER KIRCHNER: If they're best estimate 12 calculations, and with uncertainty, then they will 13 have a distribution. Yes. The answer is yes.
14 MR. DEJESUS: Okay. And the other factor 15 that effect the distribution of time available, it's 16 the crew-to-crew variability in performing the action.
17 And that is because the, one crew may take 18 a little bit more time in one action and that effects 19 the time available for subsequent actions. So that's 20 another factor that effects the variability.
21 And this is my last slide I believe. So, 22 this is the, for time required or needed. So, it 23 assumes that the action is performed at a normal work 24 pace.
25 And includes the time to attend the cue, NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
177 1 understand the situation, diagnose the problem.
2 Essentially the, going through the macrocognitive 3 functions, the time that it takes to go through that 4 process.
5 And also includes the time to travel to 6 the location if it's something outside the control 7 room. It's there in the time required.
8 And as we mentioned before, the time 9 pressure is accounted for in Pc. And the factors 10 affecting the distribution of time required, I 11 summarize it as the PIFs.
12 If you look at, I think it's Table 5-2 of 13 the report, essentially talks about the different 14 facts. So, environmental factors, plant condition, 15 work site accessibility, information availability.
16 So, that's I guess my summary bullet.
17 And also, we talk about the crew-to-crew 18 variability affecting the distribution of time 19 required.
20 MEMBER DIMITRIJEVIC: So, how did you get 21 distributions for your examples?
22 MR. DEJESUS: We're going to talk about 23 that.
24 DR. XING: We actually recently had some 25 exercise with our users at NRR. We talked how to do NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
178 1 this estimation.
2 So, the best way, if you have all personal 3 data, have loss of the simulator role, so you can get 4 a distribution for the action.
5 In the case that, like our user say, well, 6 we know those data exist but we don't have access, so 7 still, we encourage them to estimate this. I can 8 estimate a single number but I cannot estimate 9 distribution.
10 But we did some exercise to show, when you 11 estimate a single number of time needed, you already 12 mentally estimated distribution. So you are 13 estimating what's worst case, the maximum time within 14 30 minutes.
15 Ninety-five percent, if I have a crew of 16 100 people, 95 of them will definitely complete this 17 task within 30 minutes. Or, you also estimate as a 18 lower bound.
19 Like, if you gave me 15 minutes, I am 20 confident I will have no more than five member can 21 complete this task. So you gradually approach to the 22 most likely by, actually, you say a model, a Jim is 23 going to show you, we recommend them.
24 You can use this five point estimation.
25 The best and worst case approach to the middle, then NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
179 1 you get a distribution.
2 MEMBER DIMITRIJEVIC: I can give a very 3 good distribution time required to run Boston 4 marathon.
5 (Laughter.)
6 CHAIR BLEY: All right.
7 MEMBER DIMITRIJEVIC: A thousand people 8 running, you can feed them and all. But I don't 9 really know time required to the ease of a steam 10 generator.
11 CHAIR BLEY: If you run, I'll come watch.
12 (Laughter.)
13 MR. DEJESUS: I want to get there some 14 day.
15 CHAIR BLEY: When we come back from lunch, 16 the slide show, we're moving into applications. You 17 haven't quite talked about Chapter 6 and 7. Will they 18 come up as part of the applications?
19 DR. XING: Yes. Chapter 6 actually 20 belonged to application. The reason we add that 21 chapter to the report is because our staff feel that's 22 a very important aspect of this methods ability of 23 generalized data, used data being performed HEP.
24 So we're going to talk about that in the 25 afternoon, in Part III.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
180 1 CHAIR BLEY: Okay, that's good.
2 MEMBER DIMITRIJEVIC: And 7 is uncertainty 3 and we will definitely have many questions about that.
4 CHAIR BLEY: 7 is general discussion.
5 MEMBER DIMITRIJEVIC: Oh. 5 was 6 uncertainty.
7 CHAIR BLEY: Okay. We're running behind.
8 We'll hope the applications go faster. We're going to 9 recess for lunch, and let's be back at 1:30. See you 10 then.
11 (Whereupon, the above-entitled matter went 12 off the record at 12:25 p.m. and resumed at 1:31 p.m.)
13 14 15 16 17 18 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
181 1 A-F-T-E-R-N-O-O-N S-E-S-S-I-O-N 2 (1:31 p.m.)
3 CHAIR BLEY: We're back. Jing.
4 DR. XING: Yes.
5 CHAIR BLEY: Back in session.
6 DR. XING: Okay, thanks. I just realized 7 over the lunch that this was the first time we 8 actually stayed on scheduled.
9 Because, for those of you on the phone, in 10 the agenda we said afternoon part would begin with the 11 gentleman's continued talk about the last part of 12 IDHEAS-G, the time uncertainty analysis. He actually 13 finished that before lunch. So, we are --
14 CHAIR BLEY: Caught us up, that's good.
15 DR. XING: Yes. So, we will start Slide 16 Number 60, the IDHEAS-G applications.
17 So, just a little bit of background on the 18 application. We finished most part of IDHEAS, the 19 cognitive matter and the IDHEAS-G, HRA process in 20 2016.
21 And for the last two years we've been 22 working on improving the report. We made a lot of 23 changes in the report from one version to another.
24 And also, at the meantime, we tried 25 different applications of using IDHEAS-G to the areas NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
182 1 that we won't intended to use that Sean talked this 2 morning.
3 So, for the next how many minutes I'm 4 going to talk about the three applications that we've 5 had so far.
6 MEMBER REMPE: Jing? Jing?
7 DR. XING: Yes.
8 MEMBER REMPE: This slide brought to mind 9 a question that I forgot to mention earlier. At the 10 introduction of the report you had something similar 11 to these three bullets that talks about the IDHEAS as 12 a method. And then it says it's a methodology for 13 developing guidance later.
14 And is it a method or is guidance for 15 developing tools? What is it?
16 I mean, sometimes it goes back and forth 17 throughout that report --
18 DR. XING: Yes.
19 MEMBER REMPE: -- and what was the intent 20 and what is it nowadays?
21 DR. XING: Okay. James is smiling because 22 we, our team has been struggling on this many times.
23 So, the answer is, it's false. By itself 24 it is HRA. It can be used as a HRA method for event 25 analysis.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
183 1 However, as we already seen this morning, 2 it's too, you can use it but it's too complicated.
3 Not user friendly for people to use.
4 So, it also can be used as a general 5 methodology from which you can develop applications, 6 specific method. That's what I'm going to talk in --
7 MEMBER REMPE: So, then --
8 DR. XING: -- Bullet 1 to 3.
9 MEMBER REMPE: I have follow-on questions 10 then. I don't know if you saw, truly someone saw our 11 biennial research review from, I don't know, 2017, 12 2018. But in there the Committee had put, we look 13 forward to a coherent articulation of the general 14 methodology that provides a unifying concept for HRA 15 and meets the need of the SRM.
16 And we struggle when we put that sentence 17 in, but did you ever see that and do you think you've 18 met this or been addressing this recommendation and 19 the work that's been, and maybe this is a question for 20 Sean. Did you understand what we were trying to 21 convey there?
22 MR. PETERS: Yes, I think we understand 23 what you're trying to convey. And when we think we 24 are meeting the intent of a statement, the general 25 methodology we're looking at as that kind of unifying NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
184 1 context for how we understand HRA.
2 And I think some of the intent was based 3 upon previous versions of the report were less 4 comprehensible then this version right now. So we're 5 making strives to make it more, at least 6 understandable to the users.
7 And I think once you'll see, when we have 8 derived the applications out with the IDHEAS-ECA, you 9 will see a user friendly methodology for implementing 10 IDHEAS.
11 MEMBER REMPE: So, I might get the 12 guidance and the need, I get the, again, I'm not a HRA 13 person but from a distance, my understanding is I can 14 understand that you need to start delving down to the 15 weeds and have a consistent way, forcing people to 16 document what they're doing by asking questions. But 17 that's a methodology and guidance that could be used 18 to other, with other applications.
19 But when you start talking about, you have 20 the HRA method and it's so complicated, difficult for 21 people to use, I'm not sure that really helps meet 22 what we wanted when we wrote that recommendation.
23 MR. PETERS: Yes.
24 MEMBER REMPE: And, again, this is from 25 afar, you can disagree with me. But I still struggle NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
185 1 with this.
2 MR. PETERS: What I think you'll see is 3 the easy to use version is IDHEAS-ECA. The challenge 4 we had with IDHEAS-G is we also had to capture all the 5 scientific basis behind the methodologies in general.
6 And so, you can make something very simply 7 if you kind of leave that scientific basis information 8 out and the basis and the technology behind it. But 9 if you keep it in there, then it becomes a little bit 10 more cumbersome and hard to use as a standalone 11 guidance document.
12 And so, we're kind of making that 13 balancing act here, that we have this 14 comprehensiveness in IDHEAS-G. But we have the 15 simplified methods that come out in the other 16 methodologies. So, that's kind of how we're trying to 17 juggle it with this report.
18 MEMBER REMPE: But if you're trying to 19 help reduce variability with predictions for HRA, it 20 seems that the guidance is the most effective way to 21 go there more than the tools, since the tool takes 22 some intelligence and, it takes a lot of user training 23 to make users and like to use it, so it seems like the 24 guidance is more important.
25 MR. PETERS: Yes, yes. And what you will NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
186 1 see, hopefully the next meeting we're having with the 2 HRS we'll be able to share the IDHEAS-ECA methodology 3 with you.
4 What you're seeing is that's a simplified 5 implementation of the IDHEAS-G guidance. And you'll 6 be able to look through that. It will be a step-by-7 step process.
8 And so, if the user of that has questions 9 about the technical basis or what was truly met by one 10 step or another, this IDHEAS-G document is kind of 11 this reference library to go back and look at, to 12 evaluate versus that.
13 MEMBER REMPE: Okay.
14 MR. PETERS: So, our goal is to make it 15 more scientifically robust, but yet more user 16 friendly. And so, we can get that repeatability.
17 So it's a tough balancing act because I 18 feel this whole project here has been, we want 19 everything. We want it to be scientifically robust, 20 we want it to have data, we want to have improvements 21 in X, Y and X, in the methodologies, oh yes, but we 22 also need to make it user friendly so our users will 23 want to do it.
24 Oh, and by the way, we want you to work 25 with industry to come up with some kind of NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
187 1 consolidated mind set around it. Oh, and by the way, 2 industry does HRA differently than you do, industry, 3 90 percent of the time they do perspective HRA whereas 4 90 percent of the time we do retrospective HRA.
5 So, it's been a big challenge in this 6 program to try to consolidate all of those concepts 7 and IDHEAS into one report. And so, what we're, I 8 think we've gotten to somewhere relative good with 9 respect to where we are with a general framework.
10 And I hear what the Committee is telling 11 us that there is still some confusion about, when 12 reading through the report. And I think we can make 13 that a little bit better.
14 MEMBER REMPE: Well, again --
15 MR. PETERS: Yes.
16 MEMBER REMPE: -- I don't think we're 17 trying, it wasn't in my vision, and I'm just one 18 member of the Committee, that we're not only trying to 19 tell you to correct the words in the report so it's 20 easy to understand, we're trying to say, get somewhere 21 where you've met the intent of the SRM, which was to 22 reduce variability.
23 MR. PETERS: Yes. And so, we've done 24 some, and this will be more at the at-power 25 methodology. So the SRM was written for the at-power NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
188 1 methodology, right?
2 It was written prior to Fukushima in 2011.
3 And we went through that process, and we actually 4 tested that at-power methodology and its ability to 5 reduce variability.
6 We saw some reduction in variability with 7 that at-power methodology. And so, that was kind of 8 that, one of the intents of the SRM. And we think we 9 tried to drive down variability a much as we could 10 through that process.
11 We're trying to keep that intent as we 12 expand upon the SRM into realms outside of the, 13 outside at-power applications.
14 MEMBER REMPE: Okay. The other comment I 15 wanted to follow-up with before lunch was, early, 16 again, in the introduction of the report, I believe 17 you said you had applied this methodology to the 18 evaluations at Fukushima.
19 And as you heard earlier this morning, I 20 had some concerns about that statement, about 21 Fukushima. Is that document where you applied it?
22 Because, I mean, that's what's in your 23 report. It says, and I can draw you to the sentence 24 if I need to and look it up, you've applied it to 25 Fukushima. Is that a document you've issued or NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
189 1 published?
2 MR. PETERS: Jing, do you want to --
3 DR. XING: Yes, I can answer that 4 question. The answer is, we documented but not 5 published.
6 MEMBER REMPE: Okay. So now your NUREG 7 says, we've done this, somebody can ask. I'd like to 8 see that. And before you --
9 DR. XING: I actually, over the last two 10 years, I have sent it to many people who want to see 11 it. And what started was, initially I, I don't think 12 we explicitly say evaluated, I saw we analyzed and 13 documented Fukushima events.
14 Initially I made a collection of all the 15 reports and talks I can find on Fukushima related to 16 human performance. Quickly, I got a folder over 17 300,000 papers from the patients.
18 Then I tried to consolidate all of this 19 information into one place organized them. I first 20 tried to use some human factors taxonomy to 21 consolidate those information. It wasn't going to 22 work well, so I was, okay, let me try IDHEAS-G because 23 IDHEAS-G actually give you a layered out structure.
24 So I ended up, I documented all the 25 information from those various reports in this one NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
190 1 single report in the format of IDHEAS-G.
2 MEMBER REMPE: So it is a report? Because 3 I mean, your last bullet says, the NRC Staff has used 4 IDHEAS-G to analyze the documents, including the 5 Fukushima Daiichi Nuclear Power Plant accident in 2011 6 to test this methodology.
7 DR. XING: So, the whole entire, I 8 wouldn't call it a report, the manuscripts. The 9 entire manuscripts is in my computer, I can email you 10 at any time.
11 MEMBER REMPE: If it's every going to be, 12 again, I'm not particularly interested in seeing this, 13 I'm more concerned that if you document it as the last 14 bullet then the NRC should be ready to issue this.
15 And if you're going to issue it, then I would like to 16 see it before it comes out. Okay?
17 DR. XING: Okay.
18 MEMBER REMPE: And so, that's where I'm 19 at. Either take the bullet out or let's make this 20 public is where I'm going, okay?
21 DR. XING: Okay.
22 MR. PETERS: Yes. And I think Jing also 23 has some of that. If you want to continue, you have 24 a shortened version that you --
25 DR. XING: Yes, I have a shorter version NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
191 1 published in a conference paper.
2 MEMBER REMPE: Okay.
3 MR. PETERS: So she has published some of 4 this --
5 MEMBER REMPE: Okay. So, yes, then you 6 should cite that reference.
7 MR. PETERS: Okay.
8 MEMBER REMPE: But I would be interested 9 in seeing that reference. Maybe not what's in your 10 computer that's not --
11 (Simultaneously speaking.)
12 MEMBER REMPE: -- if the public were to 13 ask for this, I would like to see what your going to 14 put in and make some comments on it.
15 MR. PETERS: Absolutely.
16 MEMBER REMPE: Okay.
17 MR. PETERS: And what you'll see when I am 18 doing the wrap-up slides is we have a lot of, what I 19 call post-cleanup work. Getting all the technical 20 basis documents and all these analyses of the staff 21 done.
22 We have to get all that published. But 23 it's our secondary priority, given our primary 24 priority is to get a tool right now for our staff and 25 industry to use for document FLEX actions.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
192 1 So, it's one of those things that we will 2 be getting done, it's just not done prior to the 3 publication of --
4 MEMBER REMPE: Okay.
5 MR. PETERS: -- the initiation reports.
6 MEMBER REMPE: Okay.
7 MR. PETERS: Yes.
8 MEMBER REMPE: Thank you for your 9 tolerance of my back up questions. Go ahead.
10 DR. XING: Yes. And a little bit, go 11 back, adding to Sean's comment on your earlier 12 question, on the 2017 ACRS recommendation, I think in 13 the 2017, I believe it was May 1st, that ACRS meeting 14 we discussed how we going to use IDHEAS-G.
15 I think the recommendation were not rated, 16 but just the comment from ACRS committee was. When we 17 developed application specific method, which resulted 18 in hours of qualitative analysis guidance in IDHEAS-G.
19 Because that's really important for analyst to truly 20 understand the operation narrative document as a 21 context.
22 And the simplification will come to the 23 quantification part as far as the estimated HEP. It 24 could be really tedious for an HR analyst if we use 25 the detailed failure mode. It could be really tedious NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
193 1 when you go through all the failure modes.
2 So, at some point it would be in IDHEAS 3 at-power, we had 14, we used the 14 detail failure 4 mode instead of the whole set that was about 15. So, 5 I think to address your question, even any application 6 specific method, we resolve all the up front 7 qualitative analysis guidance.
8 MEMBER REMPE: Again, I really want to 9 emphasize that in my opinion I was looking for 10 something that would reduce variability. And I can 11 get that documenting how, if there's a disagreement 12 between industry and the NRC on what should be 13 credited, having that documentation helps reduce 14 variability --
15 DR. XING: Yes.
16 MEMBER REMPE: -- in the guidance is 17 important.
18 It's just, when you produce a tool that's 19 so complicated most people don't want to use it, I'm 20 not sure if that helps.
21 DR. XING: Yes. Actually when, in the 22 later slide, even your IDHEAS-ECA, we had two part.
23 The first part we did, we laid out five worksheets 24 which asks our analysts to document all their analysis 25 from the operation of their team to task analysis to NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
194 1 context. Document all those things in the 2 standardized format. That's not a part of the total.
3 After you document all these analysis you 4 can take the outcome of your analysis as the input to 5 the total. Get an HEP.
6 If you get two different HEP, which is 7 most likely, you can go back to your documentation 8 analysis say, oh, my assumption, in my context I 9 assume they have all the information you assume in 10 this scenario. They can have indication failure.
11 MEMBER REMPE: That's why I think the 12 guidance might be more important than the tool. But 13 that's just from a distance.
14 DR. XING: Yes.
15 MR. PETERS: And one of the things that 16 we're finding, by trying to, go in all these years 17 trying to reduce variability is, the one variability 18 that's tough for us to tackle is the variability of 19 what an analyst perceives as an important human 20 actions based upon on how they interviewed and how 21 they analyzed a scenario versus another analyst.
22 If they have competing mental models of 23 where they think scenarios are going, you're never 24 going to get agreement until they get down to those, 25 that base assumptions of what they're trying to model.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
195 1 And I think that's why it's important to have that 2 documentation step.
3 And that's what our benchmarking study 4 show, that if you can truly document what you were 5 modeling, that's where the different analyst can go 6 back and then compare notes and say, okay, we'll, 7 let's come to some common agreement about what we're 8 modeling. And hence, that reduces the variability.
9 MEMBER REMPE: I think we're all in 10 agreement.
11 MR. PETERS: Yes.
12 MEMBER REMPE: The guidance, the 13 documentation is important.
14 MR. PETERS: Yes.
15 MEMBER REMPE: It's just the other parts 16 may be, we'll have to, that's now on Dennis to decide, 17 okay?
18 MR. PETERS: Absolutely. Absolutely. No 19 problem.
20 MEMBER DIMITRIJEVIC: Well, on that note 21 actually, when you say simplify model, right, there is 22 a tiered and there is a simplified model. And you 23 just mentioned something which had 35 failure modes, 24 cognitive failure modes, and then we'll simplify to, 25 I don't know, two or three, right?
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
196 1 Well, if 35 in a row, failure modes will 2 produce complete different number. I mean, is that a 3 simplified model?
4 Simplified model should produce the same 5 similar closed model --
6 DR. XING: Yes.
7 MEMBER DIMITRIJEVIC: -- to the full 8 model, right?
9 DR. XING: Yes.
10 MEMBER DIMITRIJEVIC: It's not just 11 different analyst, it's --
12 DR. XING: Right.
13 MEMBER DIMITRIJEVIC: -- just 14 simplification, right?
15 MR. PETERS: Yes. I think you guys are 16 going to see a practical application of the 17 simplification in the later slides. And it will show 18 you how a lot of these shaping factors are removed 19 through the process.
20 And I think that would be relatively 21 repeatable based upon the context that they're 22 analyzing. Because the contexts that they're 23 analyzing won't change much, so those same shaping 24 factors should come out of each analysis.
25 DR. XING: Yes. In line of that, where NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
197 1 I'm coming from, the Subcommittee, our 2015 or '16 2 version of the report. And that Dr. Bley asks us, 3 explicitly stated, when you use a simplified subset of 4 the failure mode, even if you think this is good 5 enough, let's say for control room at-power action, 6 there may be particular scenario.
7 Some failure mode you eliminated become 8 important here. So you should always visit as a full 9 set of failure mode, if needed, to bring back to your 10 simple model.
11 MR. PETERS: And I think, just to beat a 12 dead horse, I have one other thought that I'm not 13 certain variability and analysis is a bad thing.
14 If I have two analyst that come to a same 15 situation and analyze and come up and say, hey, I 16 think human errors in this fashion or these context 17 are important and another analyst goes and analysis 18 the same thing and says, well no, I think these are 19 the important actions and context that are associated 20 with, I think what you'll find is a richer 21 understanding of the possibilities of the scenario.
22 So, there's, if we start centralizing 23 everything around, and this is one of my big 24 disagreements with the SRM, if we start centralizing 25 everything around one model, you know, models are not NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
198 1 reality, right, they're a simplification, they're a 2 wave course to try to comprehend as close to reality 3 as we can.
4 But if we all centralize around just one 5 model and one method, they're might be missing context 6 that the model and method aren't good with.
7 MEMBER REMPE: If you have good guidance 8 though, wouldn't that take care of this?
9 So, you might not take care of it for that 10 example, but you look at the differences and you say, 11 oh, the guidance should be expanded, right?
12 CHAIR BLEY: Or, you look at them and you 13 figure out what they're different and you revise the 14 analysis. You know, maybe there were two context 15 hiding there and maybe when you find those two you 16 look and you find another one too.
17 So you ought to use that to expand and 18 refine the analysis. Not just as good that we got 19 different results.
20 MR. PETERS: Yes, I mean, I agree. It 21 gives you that insight that you have missed with one 22 type of modeling technic.
23 MEMBER DIMITRIJEVIC: The main problem, 24 and I completely agree with you, that it is a value.
25 But how do you express it?
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
199 1 How do you, when you give somebody number, 2 you know, 40 minus two, how do you express your 3 variability in that number because it doesn't come 4 through the uncertainty analysis, it never comes 5 through any uncertainty analysis.
6 So when we have these huge variability, 7 how does that reflect the numbers to provide to the 8 users? That's what you cannot do.
9 DR. XING: Yes. Thanks for this 10 discussion. I actually like deviated a little bit 11 from the slides to talk about an example related to 12 this.
13 In 2008, NRC had a expert elicitation for 14 FLEX HRA, estimated HEP for several FLEX human 15 actions. EPRI also had a parallel activity. Not 16 expert elicitation but use their current HRA measure 17 to stretch it to model FLEX.
18 We've got a huge difference in the HEP.
19 For example, we had a, our expert elicitation forward, 20 declare ELAP other use FLEX equipment.
21 We had a HEP .2 from our expert 22 elicitation. EPRI had HEP E minus 4. So that's a 23 huge difference.
24 So, I compared our documentation in the 25 EPRIs report. So, the assumption or the boundary NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
200 1 condition that we asked people to document are 2 performed.
3 We assumed, so the guideline was, so the 4 instruction for declare ELAP is you need to declare 5 ELAP, within six minutes you lost power and you are 6 constant the power is not coming back in the next four 7 years.
8 However, our experts said, in that pace we 9 would have first perform the, to use our generated 10 diagnosis if we can recover it. Except these 11 generated diagnosis procedure takes 90 minutes. So 12 it's impossible.
13 We have the information we needed to make 14 that decision by 60 minutes. That's why they credit, 15 that's the reason they were higher HEP.
16 Look at EPRI's document on the same human 17 action. They say, okay, this is a simply human 18 action. EPRI there is no, they're cut to perform 19 this.
20 And this action, all it needs is to judge 21 the time within 60 minutes. There is a clock on the 22 wall.
23 They can, either you are in the SBO or 24 LOOP saturation, you also have, you always can see the 25 clock is low on battery. You look at the clock before NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
201 1 60 minutes you declare ELAP.
2 So, in that context, E minus 4 is not bad.
3 Same in our context. We have justification. This is 4 not, I mean, I don't know how we can have one HR 5 method completely reduce this kind of variability.
6 But IDHEAS-G have the capability to 7 document it all your justification, all the way like 8 from up front, your boundary condition, your contest, 9 your PIFs, all the way while you get this number. We 10 say our assumption was, assumptions in the two 11 analysis are dramatically different.
12 I think that's the way we consider, we 13 wouldn't reduce the variability, but we make the 14 variability truthful and transparent.
15 MEMBER REMPE: So, the current 16 applications, of whether this would be used and what's 17 important, you may have different numbers. But if you 18 document it and you have appropriate guidance for 19 documenting it, that would still be important.
20 Maybe we shouldn't, do we really need a 21 new method at this point or would it not have been 22 appropriate to have just said, make sure you have good 23 documentation for what you're doing and have some 24 guidance?
25 Again, the ship has already sailed. We've NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
202 1 been doing this for a long time.
2 MR. PETERS: Yes, we've been doing it for 3 a long time. But I personally think we do need a new 4 method because of the scope creep over the last 20 or 5 30 years, that the methods have been built for 6 particular application purposes and they're starting 7 to stretch them beyond what they're originally 8 intended and using them outside the context.
9 And even the Commission was worried about 10 that back in 2006. So, I think that improving that 11 technical basis beyond the methods and allowing that 12 scope to be encompassed by the method is important.
13 And so, I just wanted to finish something 14 that Jing was just saying there, that it sounds like 15 the variability may not be able to change, is our two 16 analyst's opinion about if something is an easy thing 17 or a hard thing.
18 But what we think we've done is if the 19 analyst agreed to the context and agree if a task is 20 easy or hard, we have repeatability in our HRA numbers 21 with this methodology. So, if you can get analysts, 22 analysts to agree on the context, then we can repeat 23 those numbers very well.
24 But just getting one analyst to use a 25 methodology and describe if this is difficult, this NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
203 1 medium is hard, it's hard to get different analysts to 2 agree to those particular things.
3 MEMBER REMPE: But then the staff has to 4 go with an owner/operator, making an assumption in 5 their particular PRA, if the staff can understand why 6 they did it.
7 MR. PETERS: Yes.
8 MEMBER REMPE: -- guidance.
9 MR. PETERS: Absolutely. Yes, we think we 10 have that down relatively well. Just the 11 interpretation of the context.
12 DR. XING: Okay. So, I will only very 13 briefly talk about the first application. Actually, 14 we already talked.
15 So, when we see bases being used as a HRA 16 method for human event analysis, because it can 17 provide the outcome. If we figure human event into 18 the IDHEAS-G process, we can have the output 19 systematically documented event context. Which are 20 the conditions, challenge of facility performance.
21 And we also got human, important human 22 actions, the critical test. This tells us what are 23 the potential failures and the consequence with the 24 test or diagram.
25 And that by going through, say which NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
204 1 failure mode applicable to the test, and what are the 2 PIFs, which PIF attribute applicable are relevant.
3 So, base tells us how the human action may fail and 4 what contribute to the failure. So it's like a 5 systematic root cause analysis.
6 So, we did several, I call it piloting 7 study, as we caught some of them not officially 8 documented yet. So, we said earlier, I tried to use 9 this to document all the Fukushima accident reports 10 relevant to human performance. That was the short 11 version of it published in your conference paper.
12 And early this year, Mason should lead a 13 group of research staff locate a set of seismic event.
14 I was looking at the human performance aspect, so I 15 used the IDHEAS-G culminating functions and the 16 performance shaping factors to went through all, I 17 think seven events. Seven seismic events.
18 That was also published in the conference 19 paper.
20 The third one, we worked with our data 21 instrumental control staff. Located, together located 22 several data instruments in the control event.
23 In their reviews they find a lot of the 24 event has been reviewed. At the end, come to, it's a 25 human performance issue, not just simply an instrument NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
205 1 or control failure.
2 So it's, okay, you did your instrument 3 control part there. We use IDHEAS-G to go through the 4 analysis, to document as a failure mode and the 5 contributing factors.
6 That one is not in any official 7 documentation. It was just a piloting study. We want 8 to see how it's applicable.
9 And the last one was interesting is that 10 pilot study we did this year. This actually as had a 11 conference paper published.
12 Halden performed the last of the human 13 performance experiments for 30 years. That's a very 14 valuable resource for our research.
15 And we tried to consolidate those 16 information in one place that our staff can easily 17 find the information, use it. So we use the IDHEAS-G 18 to document it as they are experimental.
19 What are the initiating event as a 20 boundary condition. Everything like we talked this 21 morning.
22 And by the end we show, okay, this is our 23 analysis, if you want, we can actually give you a 24 prediction. Not an HEP number, but which is more 25 likely fair and to compare our analysis with actual NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
206 1 experimental result.
2 So, and Halden and also the conference 3 audience, were very interested in this trial and the 4 way we're going to do more work on the base line.
5 So, that's a summary of what we did use 6 HRA, use IDHEAS-HRA method. The result give you a HEP 7 number. Everything before the HEP number.
8 So, normally I would like spend this part 9 of the talk focused, using IDHEAS-G as a platform to 10 generalize and integrate a human error data. So, HEP, 11 human error probability, normally we do it, it's 12 probability of failure and the specific context.
13 Theoretically it's important number for 14 errors you made divided by the number of occurrence 15 you performing this task. And the identical, same 16 context.
17 Therefore, in the ideal world, if I want 18 HR data for HRA, I would want the same task for in a 19 failure model. If repeated 1,000 times with the same 20 people and the same context, that will allow me to get 21 to a very low number of HEPs.
22 And to do this and all the combinations of 23 possible for context. That's the ideal world, I hope 24 for.
25 And when we look at reality, what kind of NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
207 1 a human error data we have in the real world. The 2 good news is, that there's a whole lot of human error 3 data in human error analysis, operational database and 4 the culminating human factor studies. So, it's not we 5 don't have data.
6 And some not so good news is, across all 7 these sorts of data, in some cases the failure mode is 8 not analyzed at all. Coming with greater variety. Or 9 the context we talked earlier, were not documented or 10 not repeated.
11 So we see this number but we don't know if 12 it was performed and what kind of context. And the 13 limited coverage, we don't have sufficient data for 14 every failure mode and every context.
15 The worst problem is this data sources are 16 not talking to each other. I think this effort 17 started, at least what I know, started ten years ago.
18 There was a workshop on HRA data, people part of 19 different sources, are a common, different formats, 20 different purpose, different domains, how you can use 21 them together.
22 So that's, we try to address that using 23 IDHEAS-G.
24 So, our approach is like this. We have 25 different source of information. They talk in NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
208 1 different terminologies, different level of detail.
2 But if we can interpret a data source in 3 the common taxonomy, locate whether tasks in the 4 context, embedded data source, and analyze what are 5 the failure modes in the PIFs. Basically not a 6 regional situation to our taxonomy of failure mode in 7 the PIF.
8 We do this to every data source. Then we 9 will have the data for failure mode. And as a given 10 combination or a big single PIF. That's our approach.
11 Then we think IDHEAS-G is suited for this 12 purpose. Because for this taxonomy, it has to be 13 generic enough if you only talk about changing the 14 failure mode arch.
15 It starts as a pump, align something or 16 connect something, it may not talk to another source 17 of data. So you have to use a generic term.
18 And also, the data can come at a different 19 level of details. So, the generic taxonomy should be 20 able to address the different taxonomy.
21 So, in Jonathan's presentation you have, 22 we have the failure mode defined at three levels of 23 detail. And the PIFs, we also define in multiple 24 levels of detail.
25 So, in that way we basically use our NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
209 1 IDHEAS-G process to document the data. So this 2 diagram shows the IDHEAS-G process.
3 So up front you document as an event. In 4 the middle we model the failure as a failure mode in 5 the PIF. So we use the same weight due for the data.
6 So when we say we're generalizing data, we 7 take three steps. The first step, we evaluate the 8 data source, which one criteria, I use this and we 9 have to use original paper, at least three times 10 before you see anything bothered or truly understand 11 what they were doing, what was the task, any dependent 12 state between the factors that they tested. What was 13 the uncertainties.
14 So, we identified the human actions, a 15 task, and the context there.
16 Then, the next stage we call the interpret 17 and the represented data. That's what we said, you 18 have a task there where you say in the task you have 19 people perform what was the failure, culminating 20 failure mode you got there and what was the PIFs.
21 Once we're done with this interpretation, 22 the third stage is consolidated data into three, we 23 call the human error data tables.
24 These three tables, the first table is the 25 HEP table. So it gave us the error rate. Sometimes NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
210 1 it's probably, we can find the very baseline or 2 minimal HEP.
3 For example, we have this data source from 4 NASA in command center, which is a highly controlled 5 everything. They claim that they make sure all the 6 PIFs are positive. So ideal situation. That's what 7 gave us the minimal number of error rate as probably 8 representable.
9 The best HEP, the best human performance 10 that you could possibly get. So we document that line 11 of information.
12 And we also document the PIF have a second 13 table, it's a PIF weight table, which we talked this 14 morning. We calculated it.
15 If a study or the presentator showed us 16 the human error rate in the good or not impact status 17 versus you have some bad PIF attribute, we can 18 calculate it as a weight. There's a difference 19 between the human errors.
20 And the third paper we also discussed it 21 this morning. We try to capture the inter, called the 22 PIF interaction table.
23 This study gave the data about the human 24 error. And the individual PIF. They also gave the 25 data performed by the same set of people but when two NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
211 1 more PIF were presented at same time. So, we also 2 have this table documenting that data.
3 CHAIR BLEY: So, in the first one on the 4 right, the HEP table, you have failure rates for 5 similar activities under the same PIFs?
6 DR. XING: Yes and no. Same activities, 7 which means same failure mode. But PIF sometimes the 8 same, sometimes different. We document those. This 9 is error --
10 CHAIR BLEY: But that's in the table. So 11 if I want to find all the cases of particular failure 12 mode with particular PIFs, I can pull --
13 DR. XING: Then you go to the --
14 CHAIR BLEY: -- those out of the table?
15 DR. XING: -- second table.
16 CHAIR BLEY: What?
17 DR. XING: Then you go to the second 18 table, PIF weight. Yes.
19 The first table mostly documents the data 20 that you people did not change the status of a PIF.
21 Like in the NASA command center data, they said, we 22 have all the PIFs perfect.
23 CHAIR BLEY: Was that true for all of the 24 data in that in table or is it mixed?
25 DR. XING: It's a mix. But we were NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
212 1 clearly documented. This way it's a PIF reported, 2 these three PIF.
3 This one we have the error rate, but the 4 PIF wasn't reported. It doesn't mean that they are 5 good. So we try to document all this. Also, in the 6 table, we documented uncertainties.
7 MEMBER DIMITRIJEVIC: Those tables?
8 DR. XING: I think in Chapter 6 I gave an 9 example for these tables. Yes, we actually already 10 talked about the first tables. HEP tables.
11 It will have the information for each data 12 point. We have the information about what failure 13 mode, the CFM. And human error rate, or HEP.
14 In the table, whenever we see it's HEP, 15 most likely that was obtained by expert judgement.
16 And otherwise, it's just experimental data or 17 operations data we're reporting human error rate.
18 Which means the percent of error of the data.
19 And the PIF attribute applicable to the 20 data point. Time and information, whether the task 21 goals perform, these are results of time-constraints.
22 And we put a brief narrative of the task 23 and the types of human failures in the original data 24 source. And finally, we got asked to document the 25 uncertainties in the data source. Not just in the NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
213 1 data source, also in our interpretation.
2 Again, using the example of NASA command 3 center data, they reported error rate of astronaut, 4 trainer astronaut, error in diagnosis task. Which was 5 very low. Like, between E minus 3, E minus 4.
6 I never saw diagnosis error like that low.
7 So I read through their report. There was, in the 8 appendix, there was a discussion about the task 9 performed.
10 Really, once this is diagnosed, it wasn't 11 diagnosed at all. They don't need to relate multiple 12 pieces of information together. It's more like a 13 pattern of recognition.
14 So, if your data, if the saturation match 15 this parameter, you go to A. Your conclusion is A.
16 If they match another set of parameter, you go to 17 conclusion B.
18 So, the real challenge in that task is 19 really, is that they tag team the information and they 20 gather information rather than diagnosis. So, in the 21 uncertainty part I made a note, just as I said, we 22 believe this is detection, not diagnosis. So that's 23 how we worked on this paper.
24 CHAIR BLEY: I'm having a cross connection 25 problem here. Do you have the report with you?
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
214 1 DR. XING: The NASA report?
2 CHAIR BLEY: No, your report. IDHEAS-G.
3 DR. XING: No, I don't have it with me but 4 I probably can recollect.
5 CHAIR BLEY: Table 6-2 is mapping the HEPs 6 and IDHEAS at-power, to IDHEAS-G HEP table.
7 DR. XING: Okay.
8 CHAIR BLEY: Now, I assume the HEP table 9 is the first table we were talking about. But this 10 has different failure rates for different kinds of 11 PIFs. Or is this the --
12 DR. XING: Yes.
13 CHAIR BLEY: -- second kind of table?
14 DR. XING: Yes, this table, the reason I 15 mapped the IDHEAS at-power expert elicitation, expert 16 judgment of HEPs to the HEP table, because in that, 17 so, HEP was obtained with an assumption like 18 holistically.
19 You have these three poor PIFs like 20 documented. It's the first line. You have high 21 distraction, confusing indications, lack of detail in 22 the procedure.
23 CHAIR BLEY: Which gives you a pretty high 24 failure rate.
25 DR. XING: Yes, they give a pretty high NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
215 1 failure rate.
2 CHAIR BLEY: Now, the 3, 4E minus 3, is 3 that --
4 DR. XING: That's, everything, all these 5 three factors are perfect phenomena.
6 CHAIR BLEY: And that's what's in --
7 DR. XING: Yes. So I cannot put this in 8 the PIF table because I cannot differentiate. That 9 high failure rate is 3.6E minus 1.
10 It's hard to tell which factor contribute 11 how much. So I can only give you the HEP for when 12 they, all these three factors together, at this much.
13 CHAIR BLEY: Right.
14 DR. XING: Yes.
15 CHAIR BLEY: In your previous slide, 67, 16 you said I have three different data tables.
17 DR. XING: Yes.
18 CHAIR BLEY: But this one you're looking 19 at in the report, table 6-2, is not one of those 20 three, it's something else.
21 DR. XING: This table, it belongs to the 22 first, when we say the HEP table, it's a crazy long 23 table.
24 CHAIR BLEY: So, this is part of the HEP 25 table?
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
216 1 DR. XING: Yes, this is just a very small 2 portion of the whole table.
3 CHAIR BLEY: So it does include PIFs?
4 DR. XING: Yes, it does have PIF --
5 CHAIR BLEY: When they were there.
6 DR. XING: -- but just not --
7 CHAIR BLEY: I'm sorry, I thought you told 8 me it didn't.
9 DR. XING: Oh, I'm sorry, I didn't 10 explain. We said we have a --
11 CHAIR BLEY: Okay.
12 DR. XING: -- column.
13 CHAIR BLEY: So this set of tables --
14 DR. XING: Yes, you see the third --
15 CHAIR BLEY: -- are extracts from that HEP 16 table, okay. Go ahead. No, they're not.
17 DR. XING: No.
18 MEMBER DIMITRIJEVIC: Like, example, your 19 table 6-3 includes task complexity.
20 DR. XING: Yes.
21 CHAIR BLEY: Vesna, put your mic on 22 because the poor guy behind you can't understand you.
23 MEMBER DIMITRIJEVIC: Well, I want to say, 24 in the 6-3 you state simply that that's the tax 25 complexity where the Table 6-2 you don't say it's with NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
217 1 anything, you just discuss PIFs.
2 DR. XING: Yes. So, test the complexity 3 one is that table was extracted from the very long HEP 4 table. I pick up all the data point that the failure 5 mode is task complexity. I'm sorry, has a task of 6 complexity there.
7 MEMBER DIMITRIJEVIC: And then there is a 8 distraction in 6-4. So what is the 6-2? 6-3 is task 9 complexity and the 6-4 is --
10 DR. XING: 6-4 is not for any specific 11 PIF. It's --
12 MEMBER DIMITRIJEVIC: Don't we make it to 13 those three PIFs, which you involve, in the diagram, 14 which we had a problem?
15 DR. XING: Okay, so we're talking, okay.
16 Oh my, we have to table 6, oh, continue. Okay.
17 So, Table 6.2 is the data, Table 6.2, in 18 the report is a part of the long table of, the HEP 19 table. So, because we met this table there.
20 MEMBER DIMITRIJEVIC: All right.
21 DR. XING: And then the next one, the 22 complexity table, I said, okay, I won't look at, all 23 the data that have the type of complexity PIF there.
24 So, I generalized the, except, this is not 25 all, just several examples. If I want to look at that NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
218 1 particular PIF, I can extract it from the big table.
2 MEMBER DIMITRIJEVIC: So, this is not 3 those PIFs you discuss in these graphs?
4 DR. XING: Yes, it's not. So --
5 MEMBER DIMITRIJEVIC: So it's not, 6-2 is 6 blue line?
7 DR. XING: You are very right. The 6-2 8 actually is the blue line.
9 MEMBER DIMITRIJEVIC: All right. And then 10 --
11 DR. XING: Yes. And then the next table 12 is multi-testing. You can think that's the orange or 13 red line.
14 MEMBER DIMITRIJEVIC: Yes.
15 DR. XING: Thank you. That's really 16 helpful.
17 CHAIR BLEY: So when you did the table, 18 which was probably before you did some of the other 19 word, you were still using PIFs that you no longer 20 use, like workload and stress?
21 DR. XING: I was trying, in this table, I 22 tried to document if there is, if this is from another 23 HRA method, they have a different term for their PIF, 24 I tried to document with them.
25 That was the first, because this, in this NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
219 1 first paper, my main focus was on the HEP values. And 2 the PIF status what the impact column, what I showed 3 in this particular example. I used the original term 4 in the data source. I didn't show --
5 CHAIR BLEY: No, I'm talking about the 6 PIFs. So, if you went to IDHEAS at-power --
7 DR. XING: Yes.
8 CHAIR BLEY: -- application and you picked 9 up one and it had high workload, that analysis, also 10 as I recall, would have said it was high workload 11 because of task, multitasking.
12 DR. XING: Yes.
13 CHAIR BLEY: So you could have then --
14 DR. XING: Yes, I --
15 CHAIR BLEY: -- put in here multitasking 16 so it was not your current --
17 DR. XING: Yes.
18 CHAIR BLEY: -- arrangement.
19 DR. XING: You're right, I --
20 CHAIR BLEY: But you haven't done that --
21 DR. XING: Yes.
22 CHAIR BLEY: -- or you haven't done it.
23 DR. XING: I could have done that. I 24 actually state that. The only reason I want to keep 25 this original term, so we can see the, easy to trace NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
220 1 where it was. That was only, so this table was only 2 for demonstration.
3 CHAIR BLEY: It was for illustration?
4 DR. XING: Yes, for illustration.
5 CHAIR BLEY: But now, somewhere in the 6 world here at NRC, you have a real data table.
7 DR. XING: Yes.
8 CHAIR BLEY: And in that real data table, 9 did you leave it the way it is here or did you put it 10 in the terms of the current PIFs?
11 DR. XING: Yes. That's, on this slide you 12 see Bullet 3 is the PIF attribute. That means we 13 incorporated on that for the IDHEAS-G.
14 But in the second, which is the last 15 bullet, was a brief narrative of the original 16 information. That's more like what you see here.
17 CHAIR BLEY: That's here, okay.
18 DR. XING: Yes.
19 CHAIR BLEY: Just --
20 DR. XING: Yes.
21 CHAIR BLEY: -- clearly for my questions 22 I'm not seeing things --
23 DR. XING: Yes.
24 CHAIR BLEY: -- absolutely clearly, but 25 then I haven't seen --
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
221 1 DR. XING: Yes.
2 CHAIR BLEY: -- your three big tables or 3 what supports it.
4 DR. XING: You see, in the actual table --
5 CHAIR BLEY: Is there, right now, is there 6 really a real table, HEP table and a real PIF weight 7 table and a real PIF interaction table?
8 DR. XING: Yes.
9 CHAIR BLEY: So they're somewhere on 10 somebody's computer, you do have these tables?
11 DR. XING: My computer in the SharePoint.
12 CHAIR BLEY: Okay.
13 MEMBER DIMITRIJEVIC: And they're huge?
14 DR. XING: Not that pretty huge. It's a 15 software of information. Like the original paper 16 document, that was huge. So it was really scared to 17 me early this year when we upgraded our agency --
18 MEMBER DIMITRIJEVIC: So, if they're not 19 huge, why didn't you involve them in the presentation 20 or in the report?
21 DR. XING: In that sense it's huge. For 22 example, the table 6-2, the HEP table, it actually, I 23 think it has nine columns. That's too much of a 24 spread. So I only demonstrated four columns here.
25 The more you, like, the uncertainties, NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
222 1 original data source, the reference ID, all these are 2 not in the table.
3 CHAIR BLEY: Anyway, you have a SharePoint 4 cite where analyst can go and get this information?
5 DR. XING: Yes.
6 CHAIR BLEY: Are you intending to make 7 that a public SharePoint cite so anybody can use this 8 or is it just for internal use?
9 MR. PETERS: She's looking to phone a 10 friend here. So, yes, the point is to make all of our 11 data publicly available. And a lot of this will be 12 documented when we have these last steps of cleanup.
13 So the point is to have all those nine 14 columns established in a particular report. But for, 15 we're already at 320 something pages and the IDHEAS-G, 16 general methodology.
17 And given that this is, we don't go into 18 a lot of detail on quantification and the IDHEAS-G 19 methodology, it doesn't seem like the right place for 20 us to put all this data --
21 CHAIR BLEY: No, I'm not saying to print 22 out all that stuff, I understand. I would hope when 23 you're done you'd have a place on the public website 24 where people can go and have access to those tables.
25 MR. PETERS: Well, our long-term goal was NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
223 1 to have a robust data program where we show where we 2 have weaknesses and strengths in our data tables, and 3 that will help guide us to where to go forward.
4 CHAIR BLEY: Yes.
5 MR. PETERS: And we're also trying to work 6 together on SACADA data available to the general 7 public so that we can get free resources out there, 8 analyzing that data and feeding that out into the 9 general scientific literature.
10 So, we absolutely do want to do what 11 Dennis says. We want to get the data out there and 12 for people to analyze and scrutinize and see its 13 capability.
14 The issue we're having is, we're still 15 trying to build a model here. Because one of the uses 16 we have with, from the NRC perspective of using an HRA 17 tool is, expert elicitation is not a valid way for us 18 to do our SEP announcement analyses, it's a primary 19 tools of what we use at the NRC.
20 We don't have the kind of resources to do 21 that. We need some type of quantification methodology 22 that they can utilize, that we're hoping to improve on 23 the current SPAR-H methodology quantification.
24 And so, this is our attempt to actually 25 incorporate that data from these multiple data NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
224 1 sources. And we do want to get that out there for the 2 public to see so we can create that more simplified 3 quantification capability.
4 DR. XING: Yes. So --
5 CHAIR BLEY: You may not be surprised, but 6 at least one member of this Committee would have 7 trouble endorsing this data approach without having 8 seen the data.
9 MR. PETERS: Yes, absolutely. I totally 10 understand that, yes.
11 DR. XING: So, I think we can walk through 12 one example, we talk of data points. So this is 13 actually a research paper published in Human Factors 14 Journal.
15 So, it's about a human errors in flight 16 simulator study. So, the task is to, for the pilots 17 make a decision on de-icing in flight simulator under 18 icing weather.
19 So, the context was the pilots were 20 provided with either incomplete information or 21 inaccurate information for handling icing and whether 22 information they received was 20 minutes ago.
23 Now, they have three different sources of 24 weather, about weather. There's different weather 25 sources that don't completely agree with each other.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
225 1 Situations like that. And that this is a 2 time critical decision-making, they cannot wait 3 forever to make a decision. So even they were given 4 -- Okay.
5 The failure mode, here we just use a high-6 level failure mode. It's a failure for decision-7 making.
8 They didn't make the right decision for 9 de-icing. Either they didn't do de-icing at all, or 10 de-icing too later or over.
11 So, the PIFs we state in Phase 1 is 12 incomplete or unreliable information, they manipulate 13 in the major 30 percent of information. Or I could be 14 wrong.
15 And the last thing, in my part 16 uncertainty, I wasn't sure this was a time pressure or 17 inadequate time. I couldn't tell from their research 18 paper. Because they said that you have to do this 19 position as quick as you can. But what they did in 20 that position.
21 So, I actually know the author of this 22 report. I talked with her. She couldn't answer 23 either. She was, oh, we don't know, we just tell them 24 do it as quick as they can.
25 So, this is a certainty part in the data.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
226 1 But nonetheless, so, the table is what's actually 2 recorded in the report.
3 So, each column shows one manipulation.
4 The first one is the baseline accurate and the 5 complete information. They have 18 percent of error.
6 So this gave me the sense that probably 7 they were not given enough time. Otherwise the error 8 wouldn't be this high.
9 And the second column, they manipulate, 10 the information is accurate but incomplete.
11 The third situation was inaccurate, but 12 had the complete information. You get all information 13 but 30 percent is incorrect but they don't know which 14 part was incorrect.
15 So, as --
16 MEMBER DIMITRIJEVIC: Is this de-icing, 17 that's not in flight, they usually de-ice on ground?
18 DR. XING: Both. This particular 19 simulation is in-round, it's en route. Yes, 20 supposedly you are flying and you are flying through 21 bad weather and you should make a decision, should I 22 de-ice or not.
23 The transcript says, ground people tell 24 them, you have a line of five miles of weather, they 25 were trying to decide how this five mile weather look NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
227 1 like. So, a lot of information is inaccurate they put 2 in.
3 So the chance of error was actually pretty 4 bad with 30 percent of uncertain, misleading 5 information or inaccurate information. They have 89 6 percent of error.
7 So, that's the way we documented this in 8 the table as a failure mode. It's a failure of 9 decision-making.
10 The PIFs we put in the adequate 11 information plus inadequate time for the first case.
12 But I make a note, it's a experiment, they didn't 13 specify adequate or inadequate time.
14 MEMBER DIMITRIJEVIC: And how many trials 15 they held there?
16 DR. XING: I don't remember exactly. I 17 think these study was a test. They did multiple tests 18 on this over several years of period.
19 The one test they reported was about 40 20 pilots. Each pilots work three different scenarios.
21 And I think they repeated at least once. But between 22 the repeat they didn't place identical scenario, they 23 made changes in scenario.
24 So, that was one data point looked like.
25 And, again, we cannot use this single, one single data NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
228 1 point for our HEP calculation, we're also looking for 2 as many sources that we can to pull them together for 3 the information.
4 So, we already talked the second paper, 5 the PIF impact the table. Again, it's has multiple, 6 more dimensions of information than I presented in the 7 report for simplification purposes.
8 But we can look at the second example 9 showing what goes into the PIF table. This is a study 10 done by VISN, I think it's VISN Military subject.
11 Studies the effect of long working hours, which in our 12 PIF it's mental fatigue.
13 So, the test was a team, have a team make 14 a judgement of threat on Military surveillance task.
15 So, it's a situation assessment.
16 So, the failure mode, we mapped it into 17 incorrect failure of understanding, which the process 18 is a incorrect situation assessment. And several PIFs 19 have been tested here.
20 Long working hours, which goes to mental 21 fatigue. No feedback information, that's one way of 22 the teamwork. So, you made an assessment and your 23 supervisor didn't say a single word.
24 Give you some feedback about cause like or 25 no supervision peer-checking at all. So basically it NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
229 1 was like, you have one single person do the task or 2 you have to people, another person, do the peer-3 checking for you.
4 So basically it's three manipulations.
5 And their conclusion is the result of sleep loss 6 affects assessment, accuracy and the time needed to 7 make assessment.
8 So, if we look at this table, if we only 9 look at the first rule, that's basically four feet 10 back. That means after your dates assessment the 11 supervisor will tell you your assessment was right or 12 wrong. Where you were wrong.
13 And the numbers there shows the percent or 14 error. So, result of sleep loss, the error was 4.2 15 percent. Based on sleep loss, I think that's the 16 second they did for not getting sleep. That 5.5.
17 In fact, a little bit of surprise on third 18 day, they're performance was slightly better than 19 second day. So this is their second data.
20 And then you look at the bottom two rows, 21 that's the difference if you look at it vertically in 22 the first.
23 The bottom two rows, the first, the 24 column, that's the situation, no sleep loss. You will 25 performed by one single person versus two person as a NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
230 1 team. So, the percent for error is the difference was 2 one, is six percent as others, 4.5 percent.
3 So, we're calculating the, with this 4 information we can calculate in the W of PIF with, 5 between those three. Between no fatigue and with 6 mental fatigue, between base peer-check and the result 7 of peer-check. That's another way we, so basically 8 the table you are looking at actually went to the PIF 9 table.
10 And within this table you already see the 11 result also goes to the PIF interaction table.
12 Because you actually have the information, no sleep 13 loss versus sleep loss, combined of a single person or 14 a team. So, we can see the interaction of that.
15 So, that's what we talked about, the 16 interaction table.
17 So, this is the, again, it's a 18 demonstration, a part of the extraction from the low 19 PIF impact table. I look at that long paper, I don't 20 know how many pages, it's probably around 50 pages.
21 Then I extract them by different PIFs.
22 So, all the data I have related to multitasking, I 23 make a sub-table there. So this is a sub-table above 24 the effect of multitasking.
25 So, the columns, again, I only show four NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
231 1 columns, and you stand all the column in the table.
2 The first one is a task and macrocognitive function.
3 The second column are the context in the 4 original data, how they're manipulated. And the third 5 column was the actual error rate reported in the 6 report. The last column is a PIF, which I calculated.
7 So, then we can look at this first rule.
8 Vesna studied on multitasking. The two tasks are 9 intermingled, are kind of related.
10 So, the failure mode, the macrocognitive 11 function required in this test, two activities like 12 you're missing the cue. That's a failure of 13 detection.
14 Or you're missing the changes because you 15 were supposed to look at the cue, you were also 16 supposed to look at the change in the cue. So, that's 17 called a change of detection. So you manipulate a 18 single task versus dual task.
19 So, if we look at the last column, the PIF 20 with, for the true detection failure, one is seven 21 times more increase, other is four times more 22 increase. But for the diagnosis, it was reported as 23 37 times more increase because error rate went from 24 one percent to 37 percent.
25 So, this is what the data shows. In the NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
232 1 uncertainty part I documented I have a question on 2 that one percent. Maybe because, maybe they didn't, 3 they ripped the piece and not big enough.
4 If they repeat it with more subjects you'd 5 probably get a higher diagnosis rate than the one 6 percent. But we don't know, so I documented that as 7 uncertainty. Yes.
8 So, James, you come. So, I showed you the 9 two example, both from the experiment study, but we 10 now have a really good data source, which is SACADA, 11 human operator performance database that James is the 12 evaluator, so I'll have James talk this.
13 MR. CHANG: My name is James Chang, I 14 briefed the committee on all SACADA data twice in the 15 past few years. Jing asked me to talk about SACADA so 16 I'll briefly talk.
17 This we've been studying about a decade 18 ago that every reactor unit has a fire operating crew, 19 one step crew. That each crew, that in one year, they 20 go through five to six weeks, a training week.
21 And within this training week that they 22 would do at least two or three simulator scenario.
23 And that's rush data source for us. But before that 24 time NRC did not systematically to correct this 25 information to inform HRA.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
233 1 So that was the motivation that we are, 2 data that we work with STP, that is a two unit plant 3 down in South Texas that we do work with them to defer 4 a similar that they can use for the routine operator 5 training. For them it's a one, two operating training 6 department, they want to improve their training.
7 But deferring the methodology, we are able 8 to design the system, that corrected information, that 9 help HRA portion. So this portion here that I would 10 like to talk about is the data that we use for HRA.
11 I think we don't need these two slides.
12 Yes, that's a slide --
13 CHAIR BLEY: James, I just want to remind 14 you --
15 MR. CHANG: Yes.
16 CHAIR BLEY: -- we are a public meeting 17 and we might have people listening on the phone. So, 18 I don't know if that makes a difference on anything 19 you might talk about here --
20 MR. CHANG: Okay. So I should not talk 21 about plants identification. We spoke with one plant.
22 Thank you.
23 So, now that it was, we originally were 24 there for the simulator, SACADA simulator training 25 data and now it is decided that we have a plant to NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
234 1 work with, and now it's expanding, trying to correct 2 the emergency preparedness human performance 3 information.
4 This was, back to a few years ago that I 5 presented to a committee. The data we collected at 6 one point, there's a different portion of data. The 7 one is, we called it contextual factors.
8 So this contextual factor, the data was 9 okay. In the simulation, simulator training scenario, 10 start from the beginning that they have initial 11 condition. Either is 100 full power operation or that 12 you are down power, increase power or that you have a 13 certain component that's not available. That's all 14 things initial condition.
15 And given that scenarios start, the 16 instructor will inject malfunction into the scenario.
17 And the crew need to respond to the malfunction.
18 The malfunction could be simple 19 instrumentation failed that need to respond. Or it 20 could be something big like design basis if any tube 21 rupture or LOCA, that they need to respond.
22 So, the scenario to defer this scenario, 23 and then let the crew respond to each malfunction, 24 once the mal, the instructor see that the crew 25 successfully respond to the malfunction and then the NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
235 1 initial, the next malfunction for the crew to respond.
2 So, scenario that, the training scenario 3 for the crews are identical. But the crew performance 4 responds to these scenarios could be different.
5 So that we correct two type of 6 information. One is contextual information that's 7 shown here. That for each, the crew responds here 8 that we watch the context crew responding.
9 The crew responds here. I'll give you an 10 example. For example, like I say, I lost the 11 essential cooling water pump in Train 8. The pump 12 lost, once the training went to see the important 13 operating action respond to that.
14 So, in the scenario training they identify 15 each of these crew responds item, they call it the 16 training objective element. We call it element.
17 The typical element would be, the first 18 thing is detection. You detect the loss of the ACW A 19 pump. And then the second you correct, make a 20 diagnosis, that typically represent that you follow 21 the correct procedure to solve the problem.
22 And then that has a action. Now, what's 23 a physical action that the operator take to manipulate 24 a plant to resolve the issue that we are relying the 25 past.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
236 1 And there is additional activity, like a 2 shift manager, need to declare the emergency action 3 label, or the other thing. So this thing, from the 4 training objective that the trainer want to see crew 5 perform this. And then that's the crew performance 6 evaluator based on the information.
7 And the information, even though that's 8 for training purpose, but we think, well, that's super 9 for the HRA. And the other unit resolution is very 10 similar of the two events we are looking for.
11 And we characterize this information, 12 detection, diagnosis, understanding of decision-making 13 action. And then there is team work. Team work, but 14 it separated lines in the data correction.
15 This is because, when we designed this 16 that, every crew take about one and a half to two 17 hours1.967593e-4 days <br />0.00472 hours <br />2.810847e-5 weeks <br />6.4685e-6 months <br /> to learn the scenario. After that they have 30 18 minutes to debrief their performance.
19 And we want to get this, lots of 20 information within this 30 minutes, as much as 21 possible. It's very possible.
22 So, we define methodology that tool that 23 are needing, okay, that's a practical method they 24 could use in the simulator training environment.
25 So, one of these character here that, the NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
237 1 first column here that this is the fourth, I think 2 it's the fourth diagnosis, type of activity. If the 3 training, unduly these diagnosis and that that 4 software will pop up coming to the action of the 5 trainer input this information.
6 And then the person, okay, response 7 planning, sorry. So that's a, the first action is 8 asking, what's the basis of this response planning is 9 that there we have a three option for them. The 10 skill-based, procedure-base or knowledge-base.
11 Skill-base means there is a table that you 12 don't, it's based on the what, the incident ran that 13 it can use or it can reliably perform.
14 And procedure-base means this test take a 15 little bit longer without procedure assistance. That 16 task cannot be reliable performed.
17 And knowledge-based in the nuclear power 18 plant context, they typically have procedure. But 19 because the way the scenario design. This 20 malfunction, it plays in the very beginning of 21 scenario.
22 And the placing of in the very end of the 23 scenario that before the things that already, as 24 things happen. Now the system already unavailable in 25 the, happen in the earlier malfunction.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
238 1 That operator may need to interpret 2 procedure too based on the context. So there is just 3 some knowledge that is in, even the operator 4 superficial, is following procedure. But there is 5 knowledge, element to interpret the procedure that, 6 according to the context, would implement.
7 So that's a, we asked the trainer, okay, 8 you give that to, look at the entire crew perform this 9 training objective, and what do you think that this 10 response. What type of response, is it skill-based, 11 procedure-based or be knowledge-based.
12 And the second thing are familiarity.
13 More recruit for when familiar with this task. Maybe 14 this just updated procedure, that the crew was 15 training on the updated, renewed, revised procedure.
16 So there is a different familiarity on 17 this. And so, this thing on the right that we have 18 this, asking this question, that workload for example, 19 normal or concurrent or the multiple concurrent task, 20 this language here is the trainer gave a, STP trainer 21 gave, okay, so they used a concurrence of multiple 22 concurrence and they know what they meant. Okay, 23 let's take a --
24 MEMBER DIMITRIJEVIC: Is --
25 MR. CHANG: Yes.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
239 1 MEMBER DIMITRIJEVIC: -- what you're 2 saying is, let's say that you have 2,000 data points, 3 like for example, 1,200 of them have information on 4 that, on these skilled, procedure or knowledge-based.
5 Is that what you are saying?
6 MR. CHANG: Yes. That's, I'm very glad 7 you said that. Okay, here, within that, this was a, 8 within this, this is about five years ago that we have 9 7,000 data points. Given these data 1,000 data 10 points, some of the detection, some are diagnosis, 11 some are response training, some are action.
12 So, given that we spend time that we have 13 1,990 data points is characterized as the response 14 training type.
15 MEMBER DIMITRIJEVIC: So, if every, a data 16 point can only belong in one of those rows?
17 MR. CHANG: Well, actually, it's a 18 combination. Let me explain to you. So, within this 19 1,990 data points in the response training, and then 20 how many of them is characterized, I think you have, 21 okay, procedure-based, is 1,282 data points is the 22 response training, this procedure.
23 And then when we go down more and more, 24 context fracture, so that's, I have response training 25 and then the procedure-based and the standard NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
240 1 uncertainty is clear, there is no uncertainty. And 2 workload is concurrent to the end, and says will come 3 to the end, I have a 201 data points. They have the 4 same context. I have the same context.
5 And given that, given this 2,001 data 6 points, I have a performance, how many data points is 7 graded with respect to performance. We use a four 8 scale and satisfactory. Except that means that is, 9 performance is satisfactory but there is a data, a 10 group, performance to be better.
11 And satisfactory means that performance is 12 normal. And then surpass, that means super. So that 13 we have a grading. Within this 201 data points we 14 have rating.
15 So, this was using, come back to the 16 IDHEAS, the data points here, we have denominator.
17 And then given in this specific context and then 18 what's the ratio that the crew failed a task.
19 MEMBER DIMITRIJEVIC: So, where do you 20 have a number? Well, yes or no? Satisfactory, not 21 satisfactory?
22 So every square has both number --
23 MR. CHANG: Yes.
24 MEMBER DIMITRIJEVIC: -- times 25 satisfactory?
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
241 1 MR. CHANG: Yes. Yes. Okay, back to you.
2 DR. XING: Okay, thanks, James. I don't 3 believe we have any questions. More questions for 4 James? Okay, I consider none. You might just stay 5 here.
6 MEMBER REMPE: Can I interrupt for a 7 second? I just want to make sure I understand 8 because, again, this isn't my area.
9 But in an earlier slide you talk, you 10 have, SACADA is a database for HRA and there's plant 11 specific data as well as generic data. So, when I 12 look at this table, how many plants are involved in 13 this?
14 The underlying reason I'm asking this 15 question, are you being, if you use this data for 16 something, are you being unduly influenced because 17 it's not generic, it's very much affected by one or 18 two plants or three?
19 How many plants are in this database?
20 MR. CHANG: I cannot say.
21 MEMBER REMPE: Just one, okay.
22 MR. CHANG: Correct.
23 MEMBER REMPE: And then, so, I just am 24 wondering is --
25 MR. CHANG: Yes.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
242 1 MEMBER REMPE: -- the database is --
2 MR. CHANG: Yes, exactly. That's the one 3 thing, so we tried to diverse. That's --
4 MEMBER REMPE: But you haven't done it 5 yet?
6 MR. CHANG: Now we have a second plant 7 doing the same thing.
8 MR. PETERS: Well, we have two units and 9 we'll have two more units coming onboard.
10 MEMBER REMPE: So, without talking about 11 which plants are in or out, are they all BWRs, PWRs, 12 are they representative of the fleet, I man, are you 13 getting stuff that's really generic yet?
14 MR. PETERS: We're just two units from one 15 facility, we're not representative of the fleet.
16 MEMBER REMPE: Okay.
17 MR. PETERS: But what this is, is a human 18 centered context. So, what we're trying to find out 19 is, if you put a human in a particular context, how 20 well do they respond to that or not, do they detect 21 the cues, do they perform the functions when they have 22 multiple demands or fewer demands, how different does 23 their performance change.
24 So, we try not to break it down by like, 25 say, how well the plant performs, but how does human NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
243 1 respond to an environment. So, we think from that 2 perspective it's a little more generalizable than 3 saying, okay, this is a boiling water reactor and 4 they're, or this is a pressurized water reactor and 5 they're going through a steam generator tube rupture.
6 We're not kind of related it to the plant 7 context, we're putting it back to the personal 8 cognition context.
9 MEMBER REMPE: But this, again --
10 MR. PETERS: This is --
11 MEMBER REMPE: -- it came from one --
12 MR. CHANG: Yes.
13 MEMBER REMPE: -- place so they've had 14 similar training, they have had --
15 MR. PETERS: Yes.
16 MEMBER REMPE: -- you said there's some 17 maybe undue influences from locations --
18 MR. CHANG: Yes.
19 MR. PETERS: And trust me, we would love 20 to have every plant --
21 MEMBER REMPE: I know.
22 MR. PETERS: -- in the United States 23 onboard here. We were only able to get one to, or one 24 utility onboard. And the reason being is, because 25 there's not a lot of trust of the NRC to scrutinize NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
244 1 all their human performance --
2 MEMBER REMPE: I get all that, but I just 3 want to bring up a point that might be relevant.
4 MR. CHANG: Yes.
5 MEMBER REMPE: Or not relevant --
6 MR. PETERS: No --
7 MEMBER REMPE: -- from what I know on 8 this.
9 MR. PETERS: -- we truly understand. We 10 know that relevance. But where we are on the human 11 performance role, we're just trying to take any data 12 we can get and try and see if it's utilizable.
13 So yes, there is some challenge there that 14 it might not be all the facilities and representative 15 of all the types of crews that are out there in their 16 context.
17 MEMBER REMPE: Representative.
18 MR. PETERS: Yes.
19 MEMBER SUNSERI: But I would add though, 20 my experience with the whole fleet is that all of the 21 plants try to break down their human performance into 22 skill procedure and knowledge-based error modes, or 23 performance modes they call them.
24 I mean, all of the plants had adopted that 25 philosophy, so I would think then that there wouldn't NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
245 1 be too much variation --
2 MR. PETERS: I hope not.
3 MEMBER SUNSERI: -- from a specific plant.
4 MR. PETERS: We have these standard 5 approaches to training that are kind of industry-wide.
6 We hope the utilities have at least a minimum standard 7 that they're setting.
8 But again, we have a very proactive 9 licensee. We may be sampling from the cream of the 10 crop, we don't know. We assume that they're not the 11 worst, but yes, your, these are very valid points.
12 MR. CHANG: Yes, so, one thing that I want 13 to add. A few months ago, that we have evolved, 14 that's more than 10,000 data points and then has all 15 these data that's cast, grouped into the detection, 16 detection subgroup, detection indicator movement. Or 17 the detected alarm. That's a separate, two separate 18 group of data.
19 And then there's a diagnosis response 20 training and action. So, each of these that has 21 information like this, give this specific context, 22 what's the, in a manner, that's how many responses we 23 have. And then how many unsatisfactory performance 24 here.
25 And that was, that we put in the public NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
246 1 domain back a few months ago.
2 DR. XING: As you see in the report, we 3 were able to map in the IDHEAS-G and the SACADA 4 taxonomy to each other. Mainly because they share the 5 same cognitive framework.
6 So we can do this mapping. Not 7 necessarily a one-to-one mapping, but we know how to 8 map them.
9 And like James said, it's the error modes 10 that test take off of SACADA can be generalized for 11 IDHEAS-G HEP table. And the table gathered has a 12 different context factor, like those columns he showed 13 you earlier, can be generalized into the PIF impact 14 table and the PIF interaction table.
15 And in this mapping process we find that 16 the scope of the functions in the current SACADA.
17 Because it was designed specifically for control room 18 actions performed by the licensed operating crew.
19 So, the SACADA error mode and the context 20 factors only map the two, a subset of IDHEAS-G failure 21 mode in the PIFs. But as we said, a SACADA-2 will 22 collect operators, will collect emergency response 23 data that will provide a data in the broad scope.
24 So, for the reason --
25 CHAIR BLEY: SACADA --
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
247 1 DR. XING: -- discussed, because we now 2 only have a very small number of plants contributing 3 to --
4 CHAIR BLEY: What's SACADA-2?
5 MR. CHANG: It's a --
6 CHAIR BLEY: Is this a revision to SACADA 7 or is it --
8 (Simultaneously speaking.)
9 DR. XING: -- methodology, but it's more 10 like in the IT profile to make it easy to use. And 11 then the two subs, some of this software issue that 12 was have there.
13 But then the other thing is that it 14 explains the scope. In SACADA-1 that's all we, we 15 only designed for, collecting the crew performance 16 information in the simulator training.
17 And during the outreach there is a couple 18 companies, include Duke and Entergy, tells us, you're 19 performance for them, that you're performance only 20 track to the crew level. And at that time, when we 21 worked with this particular plant, that crew 22 performance is shift managers --
23 CHAIR BLEY: Right.
24 DR. XING: But the, Duke and Entergy, they 25 want to track the performance to individual operator.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
248 1 Also work, did not design in that way.
2 And the second thing they want too, Duke 3 and Entergy want, they are not only interested in 4 simulator training, they want to have all operator 5 performance in single place. Then when the operator 6 job performance measure in this data space and then a 7 simulator in this data space, we're going to get 8 exempt in this data space.
9 So, they want a single place that putting 10 together able is able to track individual operator 11 performance in all these area. So that was more 12 efficient with, because the limitation, the structure 13 limit in the SACADA-1 would come to the different, 14 instead of modifying it, it's easier to defer new 15 ones.
16 CHAIR BLEY: Is that under development or 17 is it out there now?
18 MR. CHANG: Currently it's deferred for, 19 able to track individual. That's already deferred.
20 That's where it is, yes.
21 And now we are looking for a different 22 area that's emergency training. For this particular 23 plant, because the crew for the, in the control room 24 and emergency planning is a different crew, they 25 prefer to be separate.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
249 1 I think we will borrow a lot of code that 2 we have issued in the second one, component here. But 3 build a separate software for emergency training. But 4 a lot of things are, I think it will be very similar.
5 CHAIR BLEY: Is the original plan to be 6 picking up SACADA-2 or --
7 MR. CHANG: They will use a SACADA-2 for 8 simulator training, but their emergency program that 9 are not interested.
10 CHAIR BLEY: Not interested --
11 (Off microphone comment.)
12 MR. CHANG: Not interested in use, 13 correct.
14 CHAIR BLEY: So that's going to be a 15 problem trying to get to more and more plants, is 16 everybody has got their own needs and desires.
17 MR. CHANG: Back to a few years when I did 18 the outreach, the information I got is, if we put all 19 these crew performance information in one bucket and 20 then single stop able track individual crew for 21 bringing these, and they will go ahead.
22 And the problem now is this plant 23 management changes so fast. When we come to the 24 developing, okay, we think we're ready to start, and 25 sorry, manager changing, new manager is not so NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
250 1 interesting in pursuing. That's a chance that we 2 face.
3 CHAIR BLEY: Wow. Let me ask one other 4 thing. When you're merging SACADA data with the other 5 data, most of the data sources you've collected, Jing, 6 are looking at actual failures.
7 As you said, James, the SACADA data is 8 unsatisfactory performance on a task. It might not 9 equate to failure really. How are you dealing with 10 that?
11 MR. CHANG: Not, as far as the data, are 12 classified unsatisfactory. That means it's typical 13 the procedure has stuck, you need to do this within a 14 certain time, certain point.
15 And failure means that you failed to let, 16 explicitly explain the point.
17 If the crew is just doing the slow 18 sluggish, okay, that was, set data is not unset. So, 19 given the human error, I think that's a way that we 20 define because it express procedure. Even the step, 21 okay.
22 You need to perform this and complete 23 within two minutes, three minutes, or the certain step 24 won't be for a certain setpoint. That, I think, is 25 equivalent to the human error probability we use in NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
251 1 Jing's numbers.
2 CHAIR BLEY: But it's clearer to somebody 3 using the SACADA data that that was the criteria for 4 the unset?
5 MR. CHANG: Yes. Before employees do this 6 we provide training. That we ensure the message is 7 carried over.
8 CHAIR BLEY: No, I understand that. But 9 now you've posted the data on the website or 10 something, so I can go use it to do my HRA.
11 MR. CHANG: Yes.
12 CHAIR BLEY: Is it clear to me what unset 13 meant to the people to the plant?
14 MR. CHANG: I need to check where we 15 provide the explanation --
16 CHAIR BLEY: Okay.
17 MR. CHANG: -- for unset, yes. Yes.
18 MEMBER SUNSERI: I think you're hitting on 19 something there, Dennis, with the SACADA-2. SACADA-2.
20 Because, at least as I'm looking at this, 21 I think you're going to be introducing more 22 uncertainty in the analysis in the SACADA-2 and here 23 is why.
24 I mean, if you look at just SACADA-1, 25 there is a small group of very focused simulator NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
252 1 instructors, probably just a handful at most, looking 2 at all the crews and making all the decisions on what 3 the error actually was, right?
4 But if you go out to like job performance 5 measures, which are usually distributed by the crews 6 to implement out in the field, there could be any 7 number of people trying to make that judgment. And 8 so, Dennis' point becomes more valid then, how are 9 they doing it. And let me give you an example.
10 So, when you're thinking about procedures, 11 right, you can have a procedure error by either 12 failure to implement the procedure correctly or 13 implementing the wrong procedure.
14 Well, you know, if you don't have a lot of 15 experience with that, you could put failure or 16 implementing the wrong procedure in the knowledge 17 bucket versus the procedure bucket, right? And so, I 18 think that could introduce some uncertainty in your 19 analysis for the SACADA-2, that you might want to 20 think about --
21 MR. CHANG: Yes. SACADA-2 is current, 22 that we haven't really do a chapter from the measure 23 because the company withdrawal from one intention.
24 For the emergency preparedness, we'll work 25 with this 2 and to first understand what, how they NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
253 1 conduct the emergency preparedness drill that, what 2 information is current and correct and what's the 3 practical way that we can correct the information for 4 HRA purpose.
5 CHAIR BLEY: Okay.
6 DR. XING: Okay, thanks, James.
7 MEMBER RAY: Just a second, let me ask 8 Dennis a question. Dennis, I had assumed that the 9 data that we're talking about here, actual operating 10 plant crew performance data --
11 CHAIR BLEY: The SACADA data. Yes.
12 MEMBER RAY: -- was, that all of the data 13 that we used to develop and validate models is so we 14 can apply it in the future.
15 It sounded to me like in this last 16 exchanged it was also being, expected to be applied --
17 CHAIR BLEY: I can tell you a little 18 because I was involved with them when they did this.
19 The plants have held on to this kind of data and never 20 shared it.
21 MEMBER RAY: Absolutely.
22 CHAIR BLEY: They got a good relationship 23 with one utility who said, well, it was proposed by 24 James, we can help you develop a computer tool that 25 will you give you help in setting up your scenarios NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
254 1 for drills and evaluating your crews. And we can give 2 you a lot of free work if you share the data with us 3 afterwards.
4 So it's, they spent a lot of time getting 5 the language just right for the training people in the 6 plant so it was good for them. And then they 7 effectively had to translate from one to the other, if 8 you're going to pull out data to use for PRA/HRA kind 9 of work.
10 So, my impression watching that project 11 was, 80 percent, 90 percent of the work was really 12 getting the software into a form that was very useful 13 to the utility so that the training people got what 14 they needed. And as a benefit, and now you're up to 15 50,000 data points or something, I don't know.
16 MR. CHANG: More than 20,000. I didn't 17 keep following --
18 CHAIR BLEY: Okay. But yes, so you're 19 right. It was a --
20 MEMBER RAY: But I still thought that it 21 was a way of developing a tool that we would use for 22 future plants rather than something that would be 23 factored into the ongoing management of an existing 24 plant. And I'm just asking --
25 CHAIR BLEY: I think that was their goal.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
255 1 They were hoping this would show up in lots of plants 2 because there's this gigantic database of simulator 3 drills and exercises that nobody is collecting or 4 using.
5 MEMBER RAY: Well, right, but again, 6 looking at it from the standpoint of the managers that 7 are constantly turning over, he's talking about, I 8 would think participation would be a contribution 9 toward developing a model that's useful in the future 10 rather than helping me manage my crew at the plant.
11 CHAIR BLEY: Well, in an ideal world I 12 think that's right. What they did here was put a lot 13 of effort into making it useful at the plant so that 14 the data could come out.
15 Now, they're going to use the data too 16 because they have PRA work.
17 MR. PETERS: Yes. I think one of the 18 challenges is how plants are managed. There is an 19 operations department and then there is like 20 engineering groups, and they're out of different 21 management structures and they have different 22 purposes.
23 MEMBER RAY: Well, I know, but again, I'm 24 looking at it from a standpoint of the agency and what 25 are we doing.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
256 1 MR. PETERS: Yes.
2 MEMBER RAY: And, again, I thought this 3 was a part of the building the model that we need in 4 order to cite and improve future plants. And this is 5 one compliment of the --
6 MR. PETERS: I would be --
7 MEMBER RAY: -- risk analysis.
8 MR. PETERS: Yes, yes. So we're using 9 this data here at the NRC and we're going to be 10 building these risk analysis tools that we think 11 utilities and plants can go out and utilize.
12 MEMBER RAY: Okay. Well, that's the 13 difference then.
14 MR. PETERS: Yes.
15 MEMBER RAY: I didn't realize we were 16 expecting existing plant to benefit from ultimately 17 what we're doing here. I'm surprised by that as a 18 long time CNO.
19 MR. PETERS: Yes, yes. The main, we've 20 actually had our utility partner out there advertising 21 our work. They thought they were benefitting such 22 much from an operations standpoint.
23 MEMBER RAY: Okay.
24 MR. PETERS: From a SACADA program that 25 they were advertising all the utilities to join along.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
257 1 And that's the primary benefit they see, but from us 2 we see it as this way to build up a more robust tool 3 with real data that utilities and the NRC could 4 utilize in the future for performing PRAs and HRAs.
5 The big challenge right now is I don't 6 think industry does a, they don't do the same amount 7 of work in PRA/HRA that they did when they originally 8 initiated the programs, right? There were these huge 9 massive programs to put in the first PRAs and 10 associated with that, analyzing human actions.
11 MEMBER RAY: Yes, absolutely. I have been 12 there.
13 MR. PETERS: Yes, yes.
14 MEMBER RAY: I know --
15 MR. PETERS: Yes.
16 MEMBER RAY: -- you only do things like 17 this where you can see a benefit that is worth the 18 effort required.
19 MR. PETERS: Absolutely.
20 MR. CHANG: Yes.
21 MR. PETERS: And so, what you will see is 22 --
23 MEMBER RAY: Okay then.
24 MR. PETERS: -- we're representing later 25 today, you'll see that the industry is really gung-ho NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
258 1 right now about our FLEX work and using this new tool 2 to model FLEX applications. So that's where they're 3 seeing the return for the investment.
4 MEMBER RAY: I see.
5 MR. CHANG: Just a supplement point. The 6 training, operating training that work with the NRC on 7 the SACADA, they assume that, they use this to 8 demonstrate to INPO, to get INPO's positive feedback 9 on it. On their training program.
10 MEMBER SUNSERI: At the risk of extending 11 this conversation just a little longer, and building 12 on Harold's kind of thought there, I thought that what 13 we were doing here is developing factual, if I can, 14 maybe information, credible information growing human 15 reliability that could be used in these PRAs so we 16 could build like a NUREG or something, similar to the 17 one on components that said that if I have a valve, 18 this is it's failure rate, if I have a motor this is 19 the failure rate.
20 So if I have a human doing these kind of 21 things, this is their likelihood of having an error, 22 if you will. Is that --
23 CHAIR BLEY: That was the driving force --
24 MEMBER SUNSERI: Okay.
25 CHAIR BLEY: -- from one end.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
259 1 MEMBER SUNSERI: So if you have that then, 2 whether it's a new plant, old plant, whatever, it 3 wouldn't matter, right, it would be just useful if you 4 decide to use it.
5 MR. PETERS: As long as we have the right 6 context.
7 MEMBER SUNSERI: PRA or whatever.
8 MR. PETERS: Yes. Yes. Our hope is they 9 can put a data --
10 MEMBER SUNSERI: Okay.
11 MR. PETERS: -- NUREG out there.
12 Something similar to a data NUREG that would show the 13 error rate with these particular context.
14 MEMBER SUNSERI: Okay, thanks.
15 CHAIR BLEY: I'm looking for a good place 16 to take a break.
17 DR. XING: This is --
18 CHAIR BLEY: This is like, two more 19 slides?
20 MR. CHANG: This is a perfect place --
21 CHAIR BLEY: You're sure?
22 MR. CHANG: My last slide is pretty much 23 we just discussed. SACADA data and we are able to 24 generalize them when we have more data from more 25 plants.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
260 1 The only thing I want to mention in these 2 slides, as we all realize, the data generalization is 3 the ongoing continued effort. So we should, in the 4 down role, like periodically, look at what data we 5 have and update them, generalize them, integrate them 6 so the update has HEP numbers. That's down the road.
7 MR. PETERS: So this is a good place.
8 CHAIR BLEY: We're going to recess for 15 9 minutes. Be back at 25 after please.
10 (Whereupon, the above-entitled matter went 11 off the record at 3:11 p.m. and resumed at 3:25 p.m.)
12 CHAIR BLEY: We are back in session.
13 Jing's still up, or who's up now? Jing?
14 DR. XING: Yes, interface. Okay, so we 15 come to talk the third IDHEAS-G application which 16 we've been looking forward to for this whole day, 17 IDHEAS-ECA. So I will first talk about the philosophy 18 in developing IDHEAS-ECA which luckily, early this 19 afternoon, we already pretty much discussed 20 everything. And then James will give you a demo of 21 the software.
22 So we already talked about everything in 23 this slide which talk why we want to develop 24 application specific HRA method from IDHEAS-G. That's 25 on the top, the three reasons. And on the lower NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
261 1 portion, the table, which speaking at a very high 2 level, what's the delta between IDHEAS-G and an 3 application specific method?
4 So look at the right column. From 5 specific application method, of course we want it to 6 be more specific for the application. And we want it 7 be concise and easy to use.
8 The most important is the last item.
9 IDHEAS-G we can reference to human error data tables, 10 get a sense what is likely the human error where the 11 probability probably is. In the application-specific 12 method, mostly I think that the people who want this 13 method, it wants a way to get HEP of the human 14 actions.
15 CHAIR BLEY: Jing, what makes IDHEAS-G 16 specific to nuclear applications?
17 DR. XING: Honestly, not much.
18 MR. PETERS: I don't think, you know, 19 it's just specific to nuclear applications. What we 20 were trying to do is make it specific to all things 21 that we would do at the NRC. And so that was our 22 entire context. But almost all things we do at the 23 NRC are applicable in the real world also.
24 DR. XING: Yes.
25 CHAIR BLEY: Yes, I agree with you. But NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
262 1 it does cover all of our applications to --
2 MR. PETERS: But we never went through 3 that delta analysis to go look at all possible 4 applications of HRA out in the real world. And so 5 we're sticking with what we know. We know we were 6 encompassing what's in the nuclear domain.
7 DR. XING: Yes. It's more of speaking to 8 the public, a way data work for where our paycheck 9 comes from, just kidding.
10 Back to two years ago, I talked with my 11 previous co-worker at the Federal Aviation, FAA, and 12 say are you doing anything HRA? Maybe you can share 13 your human error data with us. And FAA's project 14 manager answer was, oh, we rely on NRC to develop the 15 method. Then we just take it to use.
16 MEMBER DIMITRIJEVIC: But this division 17 here actually answered to one of Joy's comments which 18 was is this guidance a methodology? Because this 19 IDHEAS-G is methodology, and then application to be 20 presented is a guidance. And maybe they can be either 21 separated as appendixes --
22 DR. XING: Yes.
23 MEMBER DIMITRIJEVIC: -- in a main report 24 or something. Because here ---
25 DR. XING: I think that's a much better NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
263 1 way. Otherwise, we are playing this word game of 2 methodology versus method. It can be many different 3 interpretations. Thank you.
4 So the general approach with development 5 application-specific guidance, IDHEAS guidance, we 6 first define the scope of the application 7 requirements, typically from the people who want to 8 use it, and available sources of the intended use.
9 And this comes to where we discussed 10 earlier this morning and back to the ACRS 11 recommendation in 2016. We want to keep all the 12 qualitative analysis the same as in IDHEAS-G. The 13 guidelines should be the same. And we developed 14 application-specific sets of failure mode PIFs and HEP 15 model, so we preserve all the --
16 So the table in the bottom actually 17 explains this, spells this out. So left column's for 18 IDHEAS-G, right column is for an application-specific 19 guidance. So all the guidance for scenario analysis 20 is going to be the same. But you can make some 21 specifications, for example, to identify deviation 22 scenario to identify context.
23 IDHEAS-G provided some general questions 24 to help you search for context. You may adapt to 25 those questions be more specific for your application, NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
264 1 same for human actions and identify HFEs and the task 2 analysis. Use the same guidance and you may give some 3 specifications on how to develop a test diagram.
4 For example, in our IDHEAS as power 5 guideline, we actually had a very detailed guideline 6 on how to develop accrual response diagram for task 7 analysis. Because presumably, the tasks are all 8 performed based on procedures. And procedure allows 9 you to go line by line to delineate how they perform 10 a task. What are the possible paths, what are the 11 recovery possibilities if you have that? So that's a 12 specification. But the general guideline's the same.
13 Okay, the text in blue really shows the 14 difference. So we come to this specific application.
15 We probably only want to use a set of the failure 16 modes instead of everything. And we probably use a 17 subset, or the full set with some adaptive set of PIF 18 and the attribute. And we should provide some base 19 HEP values and PIF weights for calculating HEP. And 20 the time and certainty analysis stays the same.
21 So come to IDHEAS-ECA, so to define what 22 we want in this application by our NRR users, the 23 scope is they wanted this ECA tool or ECA method for 24 all the nuclear HRA applications. In this sense, 25 there's no fundamental difference from IDHEAS-G.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
265 1 Specifically, they want to use it for FLEX-HRA. But 2 they also want to use it for any other control room 3 action involved, a lower power shutdown, which action 4 lasts for days.
5 And the requirement was really simple.
6 When we got one requirement, easy to use, as simple as 7 SPAR-H. And the resources we have available at that 8 time back to a couple of months ago, we have some 9 IDHEAS-G HEP tables, well sorry, not just HEP tables, 10 IDHEAS-G human error data tables, the three tables.
11 And we also have some HEP numbers from our 12 2018 FLEX-HRA expert elicitation. And the users want 13 a way use those numbers, those HEP numbers, at least 14 as a benchmark of validating the HEP calculation in 15 this method.
16 So the bottom portion specifically listed 17 what's the delta between IDHEAS-G and the IDHEAS-ECA.
18 We already said that the qualitative analyses up front 19 are the same, no change. So it's only in the 20 quantification part.
21 IDHEAS-G provides a basic set of failure 22 mode in three levels details. When we come to ECA 23 because we want to model a broad set of defense, and 24 the user wants to keep it simple, so the 25 compromisation is that we will only use the five high NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
266 1 level failure modes, which is those short-worded, the 2 failure of detection, failure of understanding, 3 failure of decision making. So we only quantify them 4 at this level.
5 CHAIR BLEY: So the macrocognitive 6 function?
7 DR. XING: Yes, macrocognitive functions.
8 That's where we also have data for. And IDHEAS-G has 9 20 PIFs. Each PIF has an extensive set of attributes.
10 We preserved all the 20 PIFs, but we have a compressed 11 set of PIF attributes. And that compression is based 12 on the human error data available.
13 See, we see several attributes. Their 14 impacts, the W, the PIF width, on the same failure 15 mode are the same. So there's no reason we have them 16 separate here. And also, user wants a relatively 17 shorter list, not an extensively long list, quite 18 interesting --
19 CHAIR BLEY: Do you remember how many 20 attributes there are altogether? There's a lot of 21 them.
22 DR. XING: Okay, I would say the shortest 23 attribute in the environmental PIF, they probably 24 only, like, around anywhere from three to five. The 25 longest one is a test complexity, which we didn't get NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
267 1 really many of them. Because each one modeled a 2 different aspect.
3 So we have about totally, for past 4 complexity, we actually really break down the specific 5 attribute for each macro company to function. Each 6 macro company detection function has anywhere from, 7 like, eight to ten attributes.
8 CHAIR BLEY: I thought there were seven, 9 based on my count. So I think it was --
10 DR. XING: Oh, okay. That's probably --
11 CHAIR BLEY: I'm trying to keep track of 12 those --
13 DR. XING: Oh, I actually put some 14 examples here. Again, that's for demo, not the whole 15 set. Otherwise, you would have to read a very small 16 sized font.
17 And from our human error data table, we 18 actually integrated the numbers for the base HEPs we 19 need and the PIF width. So this will allow the HEP 20 calculation for any given failure mode and PIF 21 attribute.
22 Because it was very clear we got from our 23 users. We couldn't afford of keep doing the expert 24 elicitation. Each expert elicitation would take, 25 like, the one with data, they did it in six months.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
268 1 They said, well, that was too fast.
2 MEMBER DIMITRIJEVIC: So how many data 3 failure mechanisms and how many PIFs do you have in 4 the data?
5 DR. XING: Failure, how many? For this 6 level, as we said this morning, the analyst, when they 7 look at this IDHEAS-ECA user guide on that report, 8 they don't even say the word accommodating mechanisms.
9 Because they don't want to go to that level of detail.
10 But we just tell them if you want really 11 understand why this PIF attribute would impact the 12 detection but not understanding, you can go to IDHEAS-13 G report where we explain there.
14 MEMBER DIMITRIJEVIC: So basically, you 15 have a five times twenty, 100 inputs?
16 DR. XING: Oh, that's --
17 MEMBER DIMITRIJEVIC: Some are empty, I 18 assume, I'm not sure.
19 DR. XING: Yes. I think at one point, 20 see, we need to get something around, okay, let's say, 21 we have 20 PIF. If each PIF on the average you have 22 10 attributes, that's 20 times 10. And then you have 23 five failure modes. So it's 20 times 10, times 5 to 24 solve that.
25 MEMBER DIMITRIJEVIC: Yes.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
269 1 DR. XING: So that's the numbers we 2 struggled with in the whole summer to get those 3 numbers. But luckily, we probably only need a half of 4 that many numbers. Because some PIF attribute only 5 affect one or two macro companies to function but not 6 the other ones.
7 MEMBER DIMITRIJEVIC: So you do micro-8 cognitive. I just heard you discussing that you only 9 do it on macro level, but you do it actually on micro 10 level.
11 DR. XING: We do micro-cognitive functions 12 at the highest level, but we don't do it at the detail 13 level in this method.
14 MEMBER DIMITRIJEVIC: So you do more than 15 five. For every five you have a ten micro ones.
16 DR. XING: Oh.
17 MEMBER DIMITRIJEVIC: In addition to D, U, 18 DM, you know, the --
19 DR. XING: Yes. We do at least five.
20 MEMBER DIMITRIJEVIC: Five, and then under 21 that you go further?
22 DR. XING: We don't do the end, but for 23 each one we needed the --
24 (Simultaneous speaking.)
25 MEMBER DIMITRIJEVIC: PF.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
270 1 DR. XING: Yes.
2 MEMBER DIMITRIJEVIC: Okay. All right.
3 Well, once when we see some example I will understand 4 too.
5 DR. XING: Yes. Because if we go one 6 layer down, the number we need to get would be 7 tremendous, ten times the increase than we have now.
8 Okay. And again, this morning IDHEAS-G 9 introduced us with three different ways of the 10 combinations of quantify HEP. In this particular 11 method, we use HEP quantification models that Jonathan 12 talked this morning.
13 And this is example you said you want to 14 see. Again, this is probably the full table of, the 15 HEP table for information, availability, and the 16 reliability. As we said, for this PIF, we get the 17 number, they write the HEP number, not the weight.
18 So this one actually only you can say have 19 two attribute. The first attribute, INF1 is for 20 information incomplete or information availability.
21 The second one is for information reliability. But 22 for each one, for the top one we have these three 23 anchors which show the different level of this 24 attribute.
25 So the lightest level is the information NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
271 1 is temporarily incomplete or not readily available 2 which means that you probably can recover the data 3 later on or you have the memory from early data can 4 make some compensation.
5 And the second case is somewhere like in 6 the middle. Yes, you do miss some information.
7 And third benchmark will be the worst 8 case. The information is largely incomplete, say the 9 key information is the mask or indicator of missing, 10 like, for example, in the talk this morning. The 11 indicator label is wrong. So that was the three 12 situations.
13 Because we define this PIF specifically, 14 say it's information, you need it for understanding 15 the situation and making decisions, therefore in the 16 five columns there's no number we have not applicable 17 for the three functions, the detection, execution, and 18 team interaction. We only have the number for 19 understanding and decision making.
20 So each individual number represented the 21 data point from multiple sources. I would say basic 22 generalization but, like, I have five research papers 23 of all the data sources can fit into one of these box.
24 And these five numbers are not necessarily 25 the same. If I have enough numbers, they will bring NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
272 1 me a good distribution, the most likely, the middle 2 point, and the wrench which eventually, when do HRA, 3 we won't get the uncertainty. So if there's 4 uncertainty wrench in the data we were in. But for 5 this project specifically, because right now we only 6 present the most likely wrench.
7 MEMBER DIMITRIJEVIC: Okay. I forget that 8 you going too many details, I don't understand it on 9 the high level. Why you are only looking in 10 understanding and decision making and not in 11 detection?
12 DR. XING: That is in the definition of 13 this PIF. The detection means that you give, 14 assumption is the information is there. What is the 15 failure probability you didn't detect them or you 16 detect them incorrectly? So it was already in the 17 assumption for detection.
18 MEMBER DIMITRIJEVIC: But it says that, 19 okay. All right, let's look in the stuff which you 20 show on the shutting the pump down which you broke on 21 detecting problem, entering right procedure and 22 shutting the pump down 23 DR. XING: Yes.
24 MEMBER DIMITRIJEVIC: So let's look, I 25 want to calculate that decision making to shut the NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
273 1 pump down. So what do I do? How do I go about that?
2 DR. XING: James, can you elaborate on 3 that?
4 MR. CHANG: I would defer this question to 5 later. It's simple how we do this. It's we look at, 6 in short, looking at the task you were describing that 7 what's the macro-cognitive function involved there, 8 and then that's a good, in the ECA, looking at the 9 macro-cognitive function say that your diagnosis 10 doesn't show anything. That information in diagnosis 11 situation.
12 And then look into that these are PIF 13 affecting diagnosis. And then each PIF has these PIF 14 attributes. What attribute applies to this diagnosis 15 situation. That's one. And then based on the ---
16 MEMBER DIMITRIJEVIC: Well, so give me 17 some example, I mean, than just I'm going to trip the 18 pump. What attributes? Was my information complete?
19 Yes, let's say. I mean, I don't know what we want to 20 ask. If you ask me, I will know what's happened in 21 the plant, and I can answer to you. But I don't know 22 what attributes you want to look in.
23 MR. CHANG: Okay. So in this case, that 24 when we say talking about detection, that's a mixed --
25 .
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
274 1 MEMBER DIMITRIJEVIC: Yes.
2 MR. CHANG: Yes. And now this, if I say 3 that, given this situation, I should follow procedure 4 ---
5 MEMBER DIMITRIJEVIC: Right.
6 MR. CHANG: -- whatever procedure to 7 there. And then the procedure will ---
8 (Simultaneous speaking.)
9 MR. CHANG: -- guide me to trip the pump.
10 MEMBER DIMITRIJEVIC: Got it, okay.
11 MR. CHANG: The detection portion says 12 that's all the key permitted I have detect, assume 13 that is correct.
14 MEMBER DIMITRIJEVIC: Right.
15 MR. CHANG: Now I follow this procedure.
16 Is there anything that I still need to diagnose, 17 integrate information that's in that aspect, that will 18 lead me to go to a direct action stat to trip the 19 pump? And then we are looking at that diagnosis, that 20 type of condition, and looking at the PIF and 21 attributes. What are the attributes applying to that 22 operator working at that task in that condition?
23 MEMBER DIMITRIJEVIC: What are advantages?
24 So we already diagnosis ---
25 MR. CHANG: Yes.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
275 1 MEMBER DIMITRIJEVIC: -- where he has a 2 problem.
3 MR. CHANG: Oh. Okay, sorry, I 4 misunderstood. So that question is the prior 5 diagnosis that now the problem.
6 MEMBER DIMITRIJEVIC: Right.
7 MR. CHANG: And now it's following the 8 procedure, that procedure, if that's only one way that 9 operator would lead into trip the pump.
10 MEMBER DIMITRIJEVIC: Right.
11 MR. CHANG: The decision portion, there's 12 no alternative of how we'll make the decision --
13 MEMBER DIMITRIJEVIC: Or is there anything 14 from this table he will use? That's what I'm asking.
15 So I don't understand how this table works.
16 MR. PETERS: I think some of the confusion 17 comes from this table is only from the HEP for 18 information availability and reliability.
19 DR. XING: I actually have ---
20 MR. PETERS: So there are other tables 21 that are ---
22 (Simultaneous speaking.)
23 MR. PETERS: -- are for other types of 24 human error probabilities.
25 MEMBER DIMITRIJEVIC: So what I mean, this NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
276 1 could be in case the information is not how available 2 or something. Maybe, you know, breaker for the pump 3 is not well, you know, identified, things like that.
4 I mean, what does it mean HEP for information 5 availability? Every HEP needs ---
6 MR. PETERS: So detection would be there's 7 an alarm that goes off in the plant, right?
8 MEMBER DIMITRIJEVIC: Right.
9 MR. PETERS: So information availability 10 doesn't matter, because it's the alarm or not the 11 alarm. So from an information availability it would 12 be, say your procedures say you have a particular set 13 point that, by that set point, you need to trip the 14 pump.
15 So you already got the alarm. Now you're 16 through the procedure, and you're looking for that 17 particular set point. If that set point's not 18 displayed on your panels or you can't get that 19 information, that means that's not really available.
20 I don't know if I'm coming close to that parameter.
21 MEMBER DIMITRIJEVIC: All right. Okay, so 22 if the, let's say data is displayed on the panel, 23 okay.
24 MR. PETERS: Yes.
25 MEMBER DIMITRIJEVIC: Everything is fine.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
277 1 DR. XING: Yes.
2 MEMBER DIMITRIJEVIC: Set point displayed.
3 Then what do I use from this table?
4 MR. PETERS: So let's see if I can read 5 this far.
6 DR. XING: I actually have an example of 7 ---
8 MEMBER DIMITRIJEVIC: Yes. I would love 9 to see that, some example. Because for me this is 10 just tables. I don't know what to do. I don't know 11 why don't you have a case when everything is all 12 right. I mean, how do numbers fit?
13 DR. XING: So in the U.S. benchmark study, 14 four crews performed the three scenarios in one of the 15 U.S. plants. And then a bunch of HRA analysts used a 16 different HRA method to analyze those scenarios. So 17 the three scenarios, one is a simple, standard steam 18 generator tube rupture?
19 MEMBER DIMITRIJEVIC: Excellent. That's 20 good.
21 DR. XING: Yes, assuming you have 22 everything, have good procedure and have all the 23 information you needed. And also, of course, state 24 data correctly success. And all the HRA analyst team, 25 even if they had variabilities, they all get the HEP NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
278 1 very low for that one.
2 And then they also have a comparison 3 scenario which is also steam generated tube rupture.
4 But they purposely introduced many complications. One 5 complication is make the information misleading. They 6 manipulate it.
7 I think that was a small, that was 8 actually two deep. Therefore, you only say that, 9 people only say the information think that's a big 10 deal, because they didn't realize that actually two 11 deep are the same indication.
12 So that would affect, when I go through 13 this table, I would go to that category. Key 14 information is masked. You should have the indication 15 for the leaking, but the information about the leaking 16 for the second, for the LOCA was masked in the 17 information for ---
18 MEMBER DIMITRIJEVIC: So you would go to 19 the third one?
20 DR. XING: Yes. I would take the key 21 information is masked.
22 MEMBER DIMITRIJEVIC: Okay, then the ---
23 DR. XING: In that case, base HEP is 24 supposed, for understanding, for the diagnosis, the 25 two cases, If everything else perfect, the HEP is NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
279 1 already pretty high. Like, they would, that's Point 2 2 which means you have a good 20 percent chance of 3 failure. But that scenario, there are several other 4 complications. The results, I think, is three.
5 MEMBER DIMITRIJEVIC: There is a, I mean, 6 like in the steam plant, the tube rupture, there will 7 be multiple indication. He does not rely only on one, 8 I mean --
9 DR. XING: Yes. So at the end of all the 10 four crews' failure, not the crew identified that 11 LOCA, that masked by the steam generator table --
12 MEMBER DIMITRIJEVIC: Are you today going 13 to present to us something you take from the 14 beginning, and then you come with HEP in that?
15 MR. CHANG: Yes.
16 MEMBER DIMITRIJEVIC: You will present 17 something for us, what is here, what does the, you 18 know, the cognitive failure mode, how did he select 19 task? What cognitive modes are related? What PC 20 selected? Then what was HEP on the them, and where 21 did he get the data to answer that?
22 DR. XING: Yes. James had prepared an 23 example.
24 MR. CHANG: One example.
25 DR. XING: So that means --
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
280 1 (Simultaneous speaking.)
2 DR. XING: Yes. I will finish my part, 3 talk quickly. I think you all looking forward to hear 4 what he's going to show.
5 MEMBER DIMITRIJEVIC: Okay.
6 DR. XING: Okay. This is another demo for 7 the PIF weight tables, but let's skip this unless you 8 want to see the detail.
9 So this is summarized. This is a diagram 10 we are going to give to the analyst. This is from the 11 IDHEAS-ECA report that we are in progress writing. So 12 Jonathan prepared this diagram and laid out so each 13 box is a step of activity you need to do to use the 14 IDHEAS-ECA.
15 We don't need to go through every box 16 here, because he already went through this morning.
17 But look at the three, we call that orange color, 18 that's the difference between IDHEAS-G and ECA.
19 IDHEAS-G can be all those white boxes up front, 20 qualitative analysis.
21 Then IDHEAS-ECA, this is the software. But 22 without software, you have to do it manually by hand 23 which I already heard analysts say they don't want to 24 do math, even if it's a simple math.
25 So IDHEAS-ECA provided you the way to NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
281 1 calculate in the PC, calculate in PT, and calculate in 2 the overall HEP. And in the IDHEAS-ECA ---
3 MEMBER KIRCHNER: Can I make an 4 observation? It's seeming to me that that figure you 5 just showed, the previous view graph, we didn't really 6 see that earlier. We saw it in words and multiple 7 view graphs. But this figure simplified, like, you 8 might remember, I asked where are you getting this 9 narrative from? So you show here that you're taking 10 the PRA model as the input to start developing the 11 narratives.
12 Some stylized version of this much earlier 13 would be much useful, because now we're just getting 14 to the point of putting all the pieces together in 15 some coherent manner that suggests you can actually 16 use this. I think it almost goes back to Dennis' 17 opening comments, that at the end of going through all 18 this, there's some utility to what you're presenting, 19 just an observation.
20 DR. XING: Thank you. We'll definitely 21 take that.
22 CHAIR BLEY: I mean, when you say here 23 it's the same process as IDHEAS-G, that's great. But 24 it would have made sense then to have this at least in 25 Chapter 4, or some simplified version of it maybe way NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
282 1 up front.
2 MEMBER REMPE: How does this differ from 3 other methods. I mean, you come down and say, well, 4 you really just have the three orange boxes. Is it 5 because you documented the scenario narrative and all 6 that? How does this differ from other methods that 7 existed before IDHEAS-G?
8 DR. XING: I would say it depends on each 9 different method. Pretty much every HRA method 10 explicitly or implicitly asks you to analyze a 11 scenario, identify the context, do a task analysis.
12 But as I said, some methods doing this 13 implicit. For example, you know, NRC's method, 14 SPAR-H, a lot of people just read the SPAR-H report 15 would think, oh, you started from only this end part, 16 identify the PIFs, then get a HEP number.
17 But our NRC staff, when they use SPAR-H, 18 they always do a lot of similar up front analysis.
19 But the main difference in IDHEAS-G, compare others, 20 it will make this very explicit.
21 So other than, because SPAH-H does not 22 have this guideline, so it very much depends on 23 individual analysts, what I want to analyze up front 24 and what I want documented. So in the HRA benchmark 25 study, two teams used the SPAR-H. So I was in the NRC NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
283 1 SPAR-H team.
2 Our reports were very different. So our 3 report, NRC staff's report using SPAR-H, we documented 4 a lot of upfront information. And the SPAR-H 5 performed by a different team, they barely documented 6 any information. It's just a, oh, here's my selection 7 of PIF. That's not HEP --
8 CHAIR BLEY: Jing, I disagree with you.
9 Before ATHEANA, the only methodology that laid out a 10 process for doing HRA was the EPRI thing, SHARP1. And 11 then ATHEANA did. Most of the other methods left this 12 up to the analyst to figure out what to do.
13 And many of them, the analysts didn't 14 figure out, they just went to tables of performance 15 shaping factors, and grabbed numbers off, and used 16 them. So, you know, SHARP1, ATHEANA, and then this, 17 have a process. And I think that's pretty important.
18 Now, if you ask me how much different is 19 this from some of the others, I don't know, one thing 20 I'd say is the approach here has documented the 21 cognitive basis probably more thoroughly than anybody 22 else has done. And I won't try to think beyond that.
23 Because my brain's not working great.
24 MR. PETERS: So just to --
25 DR. XING: Thank you, Dr. Bley.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
284 1 MR. PETERS: -- just to elaborate a little 2 bit, I just wanted to make one correction. So our 3 process is the entire sheet, so the white part and the 4 orange part. The orange part is just going to be the 5 tool that James is presenting.
6 CHAIR BLEY: And all the methods had some 7 kind of orange part to them. And in fact, mostly 8 that's all they were, were quantification set of 9 tables or some such thing.
10 DR. XING: Okay. Thank you, Dr. Bley. I 11 completely agree with your disagreement. And that 12 give us more credit than I would dare to claim, ha, 13 ha. I don't want to criticize other methods for not 14 having it. They will immediately say, oh, we don't 15 have it written, but we will do it. Okay, so ---
16 CHAIR BLEY: I think you're right. The 17 empirical study showed that, yes, maybe they do. But 18 mostly they don't.
19 DR. XING: Yes. They don't, or they do it 20 very differently. So it's hard to compare between 21 them. Because they documented this information not in 22 any systematic way.
23 MR. PETERS: And that makes it hard to 24 repeat. And that affects the variability, so ---
25 DR. XING: Yes. So one thing, in 2016 we NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
285 1 did a test IDHEAS at-power which is a qualitative 2 part, pretty much the same. And the two outcomes, one 3 was the testers complain, oh, this takes too much more 4 time than it did before, because I have to spend so 5 much time documenting analysis.
6 Then at the end, they say the advantage 7 was when we compare the differences between the 8 different teams, it was crystal clear to see where 9 they make a difference. Because they said, oh, we are 10 in that box.
11 You and I have a disagreement in the 12 context assumption. Like I assumed indicator would be 13 unreliable. You assumed all the indicator perfect.
14 So that was an advantage with this documentation 15 process.
16 Okay, so I'll quickly turn you to James.
17 Yes, so we said earlier we want to make sure that 18 users in ECA will document all the information 19 systematically. So we have IDHEAS-ECA report which 20 included guidelines, worksheet, and all the HEP and 21 the PIF tables, and three or four examples. Two of 22 them, we applied the method to U.S. benchmark study.
23 And these five worksheets are documented 24 information in the boxes in the previous page. So 25 only after you finish these five boxes, document has NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
286 1 information, then you can turn into the two boxes of 2 software which will help you do the math.
3 MEMBER KIRCHNER: May I just point out 4 just something that you might check for consistency?
5 On the previous view graph, you used different 6 nomenclature. You were using HFE.
7 DR. XING: Oh.
8 MEMBER KIRCHNER: On the next one, you're 9 calling it --- I'm not a practitioner in this, so the 10 more acronyms makes for more confusion for me. If the 11 practitioners know that an IHA equals and HFE, fine.
12 But I notice you call it one thing there, and now 13 you're calling it something else. So you ---
14 DR. XING: Thank you.
15 MEMBER KIRCHNER: -- you might look at ---
16 DR. XING: Yes. It was in ---
17 MEMBER KIRCHNER: -- whether your 18 nomenclature ---
19 (Simultaneous speaking.)
20 MEMBER KIRCHNER: -- is consistent.
21 DR. XING: -- because the analysts started 22 the analysis with HFE data.
23 So that was a recommended use if you need 24 to analyze the event or document them in the 25 worksheet. And then use the HRA tool box for NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
287 1 calculating HEP. Having said that, I'll turn to James 2 to show the software.
3 CHAIR BLEY: I'm sorry. Back up one. The 4 NRC HRA tool box?
5 DR. XING: That's a new term.
6 CHAIR BLEY: You haven't explained that.
7 MR. CHANG: Yes, I'm going to explain.
8 DR. XING: He's going to explain. We took 9 the software for ECA, but James actually developed the 10 software, not just ECA, it also has a dependency there 11 and has other ---
12 CHAIR BLEY: And it's up on the NRC web 13 for people to use here at NRC?
14 MR. CHANG: Not yet.
15 CHAIR BLEY: Not yet?
16 MR. CHANG: The software installation 17 package is now with NRC IT, that's what, no ---
18 (Simultaneous speaking.)
19 MR. CHANG: And then after that we will 20 exercise it with EPRI in the project. And then, if 21 everything goes well, that will release. That's the 22 time to release to public.
23 CHAIR BLEY: Okay. We'd like to be kept 24 informed of that process, if you don't mind.
25 MR. PETERS: Absolutely.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
288 1 CHAIR BLEY: It's a good thing. So this 2 is kind of like the PRA tool box that's up already?
3 It's a place to go to get tools, or just one tool?
4 MR. CHANG: It's most equivalent to the 5 EPRI's calculator.
6 CHAIR BLEY: The what?
7 MR. CHANG: For use in the EPRI HRA 8 calculator, but equivalent to that kind of level of 9 software. But it's for implement ECA method at the 10 dependency and SPAR-H.
11 DR. XING: SPAR-H.
12 CHAIR BLEY: Okay. You're going to show 13 us what's in there now?
14 MR. CHANG: Yes.
15 DR. XING: Okay, James. I think everybody 16 wishing for your turn, ha, ha.
17 MR. CHANG: Chairman and the other 18 committee members, correctly state this morning that 19 the event on the HRA, NRC to implement ECA, that 20 there's 28 performance influencing factor. Each 21 factor has a number of the PIF attributes. That's the 22 memory, the mental note, to remember all these 23 methods, these items, and then also the math involved 24 in calculation, especially convolution, math in the 25 calculating the PT, that will be quite correction to NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
289 1 the HRA analyst to implement the method.
2 Without the software to implement the 3 method that we envision that software, this method 4 will be only implemented in probably research 5 projects, not go through, be practical to apply for 6 NRC day-to-day operation.
7 So staff has this variation that we want 8 to develop a software to take a capacity out of the 9 analysts' hands. And then that analyst folks put 10 their efforts, the folks doing the good qualitative 11 analysis, the good documentation, and then that has a 12 good tool for them to implement. And then the tool 13 itself serve a good guidance.
14 And then the data that Jing talked today 15 that we implement in the tool, that's a five data 16 point come to the one recommendation vary where we 17 want to use, we implement in the tool.
18 That's here that the user can, once 19 analyzed that highest documentation, that the tool can 20 operate all the factors that the user check and then 21 watch the factor's effect, individual factor effect, 22 so that the user coming to that has all the clear 23 documentation. And then it was a detail coming in to 24 the HEP calculation.
25 So the screen shown here is that the NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
290 1 software that's up on the screen that on the top it's 2 HRA calculator. And the bottom one is you can use, 3 the one below it is the user that wants the analysis 4 will we able to save in the file. And then we treat 5 them and then inspect later all the codes.
6 Underneath is actually HFE identification, 7 that this is a typical identification we use in the 8 PRA model, why this H-A-S-D-T-R-whatever. And beneath 9 it is the description that describes that the user 10 will be able to provide more description about what's 11 this HFE about.
12 On the top, on the right, that you see 13 that, there's a blue background, that's a white text, 14 this LCA, ECA, oh, that's HEP, the number here that is 15 the -- Yes?
16 CHAIR BLEY: Would you use the pointer, 17 and --
18 MEMBER REMPE: The mouse.
19 CHAIR BLEY: The mouse. Once you move it 20 it'll light up.
21 MR. CHANG: Oh, okay.
22 CHAIR BLEY: And you can show us where 23 you're talking about.
24 MR. CHANG: Okay, thank you.
25 CHAIR BLEY: Thank you. It goes away NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
291 1 after you rest, but --
2 MR. CHANG: That is a much helpful tool.
3 Okay.
4 CHAIR BLEY: Thank you.
5 MR. CHANG: So this a final HEP, we took 6 it from in the ECA method. And then it has two 7 elements. One is a PC and a PT. The PC here that we, 8 okay, let me come back to this data.
9 This is only the independent, but we also 10 implement the dependency model that's documented in 11 the NUREG 1921 fire HRA method that NRC jointly 12 developed with EPRI. That's the dependency model. We 13 implement in this software that user could use that 14 dependency model and then the software will take the 15 independent HEP to calculate dependence, HEP --
16 CHAIR BLEY: I'm sorry, NUREG 1921?
17 MR. CHANG: 1921.
18 CHAIR BLEY: And that's a dependency?
19 MR. CHANG: No, fire HRA method. Within 20 it has a --
21 CHAIR BLEY: Oh, it's the fire HRA.
22 MR. CHANG: Yes, fire HRA. And within it 23 has a dependency that meant a flow chart.
24 CHAIR BLEY: Now, you have an appendix in 25 IDHEAS-G on dependency. Are those two related, or are NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
292 1 they just randomly different?
2 DR. XING: Not related, I would say the 3 appendix dependency, because we put all the source 4 there, how dependency should be addressed.
5 CHAIR BLEY: Are you lit up?
6 DR. XING: Yes. So the appendix in 7 IDHEAS-G report on dependency, I would consider that's 8 not finished. The development hasn't finished. We 9 only laid out our source on how dependency should be 10 created. And the dependency HEP should be calculated 11 if we use IDHEAS-G.
12 However, we haven't completely finished 13 that part. And meanwhile, our branch already has a 14 project further give a better guidance on dependency.
15 So we gave that IDHEAS-G dependency software to our 16 other project. Hopefully, they will take some source 17 there.
18 19 CHAIR BLEY: Okay. Are you bringing the 20 fire HRA dependency ---
21 DR. XING: Yes, so ---
22 CHAIR BLEY: -- into IDHEAS-G, or just 23 into this example?
24 DR. XING: Only in the ECA. We didn't 25 bring that to IDHEAS-G.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
293 1 CHAIR BLEY: Okay, you didn't refer to it 2 either, I don't think.
3 MR. CHANG: But there's a tool box.
4 MR. PETERS: Same thing in the IDHEAS-G 5 document.
6 DR. XING: Yes. In the IDHEAS-G report, 7 we said we didn't do our own development of 8 dependency. We only have some source documented.
9 CHAIR BLEY: Yes, and you documented some, 10 I thought, kind of useful thoughts. But it sounds as 11 if you didn't link to the other work that had been 12 done so that these things are kind of dangling again 13 and not hooked together where they could be.
14 MEMBER KIRCHNER: Just to point out in 15 your previous flow chart where dependency enters into 16 your -- because now you've introduced something else.
17 DR. XING: Yes.
18 MEMBER KIRCHNER: So I thought I 19 understood this calculation, but where is the 20 dependency coming into this flow chart?
21 DR. XING: Yes. This flow chart, because 22 we didn't specifically address dependency, the method 23 will say if you need to address dependency use 24 whatever you choose as the current method. So it's 25 not included in the flow chart.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
294 1 MEMBER DIMITRIJEVIC: Plus, if you 2 understood method, you well, because I am not even 3 close yet. And we are moving in that direction.
4 MR. CHANG: Thank you. So these are the 5 previous information that will constantly show on the 6 top screen of this software. And underneath it, we 7 have the seven tabs. Each tab that's allowed a user 8 to do different functions.
9 The three root tab we implement here, the 10 critical task, that's to calculate cognitive error 11 probability. So this software just now allowed the 12 user to calculate up to three critical tasks.
13 One thing I didn't mention, this software 14 that, it's intentioned use that Jing already 15 mentioned, the HRA is the first dual qualitative 16 analysis that understands that the human failure event 17 want to analyze and then do a qualitative analysis.
18 So this is two estimates of this error probability of 19 these human failure events.
20 So given that we have the three, so we 21 allowed the user to have maximum of three critical 22 tasks. In our conversation with the understanding of 23 what the HFE we're using, the PI model it seems that 24 we expect most of that, we only use the one critical 25 task.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
295 1 And the reason that's coming to the three 2 critical tasks is that when you have detection, 3 understanding, and implementation, that could deploy 4 FLEX and component, that type of scenario, that the 5 context was quite different that we may come to the 6 multiple critical task.
7 Okay, the brown tab is still calculated.
8 I have a slide to talk about this. And then there's 9 a gray slide, the documentation, and then dependency 10 and SPAR-H.
11 MEMBER DIMITRIJEVIC: So this 3.2 minus 12 three is just for Critical Task 1.
13 MR. CHANG: Yes.
14 MEMBER DIMITRIJEVIC: It's not for the 15 human error which you have there, there were ID myHFE.
16 It's not for myHFE, just for critical task identified 17 for myHFE.
18 MR. CHANG: Yes. But I want to clarify.
19 So 3.2 is for this critical task. In this critical 20 task that you see that I have a check for each macro, 21 failure mechanical --
22 DR. XING: Macro.
23 MR. CHANG: Right. So this critical task 24 involved all five macrocognitive functions. And then 25 underneath this blank box means that if that has any NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
296 1 performance influencing or PIF attribute, that means 2 negative ins, that will show on here. And without 3 showing anything here, that means that I did not 4 identify any negative factor or factor performance.
5 So --
6 CHAIR BLEY: I want to stop you right 7 there.
8 MR. CHANG: Yes.
9 CHAIR BLEY: You've done your qualitative 10 analysis, you've identified that all of the 11 macrocognitive functions are involved in Critical Task 12 1.
13 MR. CHANG: Yes.
14 CHAIR BLEY: Now, does the software then 15 let me go in and search for what the important PIFs 16 would be or --
17 MR. CHANG: Yes.
18 CHAIR BLEY: Can you show us that?
19 MR. CHANG: Yes. Later screen that will 20 show that. But now I just provide a high level 21 overview of it.
22 MR. PETERS: It's much easier to see when 23 you actually have a working tool to click buttons ---
24 MR. CHANG: Yes, but I was --
25 (Simultaneous speaking.)
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
297 1 MR. CHANG: -- the device, but not to do 2 a demo in this place so --
3 (Laughter.)
4 MR. PETERS: So what we're going to try to 5 show is pretend that James is clicking through the 6 screens as we're going.
7 CHAIR BLEY: So if you had gone through 8 that process and identified some PIFs that were 9 important ---
10 MR. CHANG: Yes.
11 CHAIR BLEY: -- you would have found them, 12 and they would be showing up on this screen?
13 MR. CHANG: Showing at the top, yes, in 14 the corresponding ---
15 CHAIR BLEY: So since they aren't there, 16 that means this was quantified assuming all the PIFs 17 were good.
18 MR. CHANG: Yes.
19 MEMBER DIMITRIJEVIC: Why does the MFC 20 selection on your left, it only has a detection line?
21 MR. CHANG: Yes. Because this is a, we 22 have limited space ---
23 CHAIR BLEY: I can't see that.
24 MR. CHANG: -- that you switch the form of 25 detection to understanding, to deciding, and then when NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
298 1 you switch, the user switch on the site, the tree will 2 change correspondingly.
3 The tree, PIF tree for each macrocognitive 4 mode are different. So that will change 5 correspondingly.
6 CHAIR BLEY: Have we seen the PIF tree 7 today?
8 MR. CHANG: I'll show of them. That's a 9 later slide, we'll show some of them ---
10 CHAIR BLEY: Okay, because I'm not sure 11 exactly what you're talking about.
12 MR. CHANG: Yes. It is mentioned here 13 that, given this thing, so this critical task involved 14 detection, understanding, deciding, action, teamwork.
15 And each of them checked means that this critical task 16 involving this macrocognitive function. But 17 underneath it there's no item means that that has a 18 HEP. This is the minimum HEP we apply for this 19 cognitive function.
20 MEMBER KIRCHNER: What's your basis for 21 that then, for each of the numbers that are entered?
22 MR. CHANG: Yes.
23 MEMBER KIRCHNER: And you're saying for, 24 let me just pick one. For detection it's the nominal 25 error or the mean error, whatever it is, one times ten NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
299 1 to the minus four.
2 MR. CHANG: Yes. This is another user in 3 there. This is a program already put in there. This 4 is the minimum HEP for detection we put in there.
5 MEMBER DIMITRIJEVIC: So does this come 6 from those 6.2, 3, 4 tables that ---
7 MEMBER KIRCHNER: Yes, that's where I was 8 going. This comes from the tables.
9 DR. XING: From the original table that 10 make ---
11 MEMBER KIRCHNER: But how does it know the 12 complexity of, I'm a little concerned. I know you 13 will have understanding next and such, but the set of 14 inputs you need to detect some of the more extreme 15 events, maybe very low probability, but extreme and 16 complicated events, would have a much higher value.
17 MR. CHANG: Yes. But that means that you 18 likely will have a negative PIF attribute involved.
19 Now we are just talking about, at a minimum, the best 20 quality that, without recovery, what's the HEP at 21 lowest without ---
22 MR. PETERS: James, could you skip ahead 23 to Slide 97? That might clarify things a lot.
24 (Simultaneous speaking.)
25 MEMBER SUNSERI: One more down.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
300 1 MR. CHANG: One more down.
2 MR. PETERS: No, no, go up two.
3 PARTICIPANT: There you go.
4 MR. PETERS: Yes. This one will show 5 where particular performance --
6 DR. XING: That would be in your 7 handout.
8 MR. PETERS: It doesn't show as well on 9 the screen as it does in the handout.
10 MEMBER DIMITRIJEVIC: On this we have, it 11 identifies some, we have identified some of those 12 issues. Understanding information is temporary and 13 incorrect.
14 MEMBER KIRCHNER: So I noticed standing up 15 and looking at --
16 PARTICIPANT: Walt doesn't have it.
17 MEMBER KIRCHNER: That's all right. I 18 looked at the screen up there. There seems to be now 19 a precision, now the numbers changed, right. So now 20 --
21 MR. CHANG: Yes. Once they're selected 22 in, so ultimately using the equation that Jing has 23 shown to calculate all of that into the --
24 MEMBER KIRCHNER: So now it's increased, 25 the error probability is increased to 7.2.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
301 1 MR. CHANG: Yes.
2 DR. XING: Because you have many in the 3 middle, those big white box. You have many PIF --
4 MEMBER KIRCHNER: Yes, so I looked at 5 that.
6 DR. XING: -- actually selected.
7 MEMBER KIRCHNER: Geez, the precision of 8 those numbers, I guess maybe it's the computer model.
9 Thanks.
10 MR. PETERS: We were trying to get more 11 decimal points but, you know ---
12 MEMBER KIRCHNER: Yes. I guess the 13 computer doesn't care, but as an engineer, as I do.
14 CHAIR BLEY: They should round off for us.
15 MR. PETERS: Fully noted.
16 DR. XING: Yes. As an analyst, they can 17 actually hold the ECA report which has all the numbers 18 there. They can see if I changed the complexity for 19 detection from Attribute A to B. They can see a 20 factor there. But for computer part, it only shows 21 the final number.
22 CHAIR BLEY: Yes.
23 MEMBER DIMITRIJEVIC: And where did you 24 enter those?
25 MR. CHANG: When?
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
302 1 MEMBER DIMITRIJEVIC: Where did you enter, 2 on the screen, did you enter those.
3 MR. CHANG: So this screen that I use, see 4 at the bottom left there has, we call it radio button.
5 Now it's lacking in action. Once the user selects 6 action that's on the right, this will show the PIF and 7 their attribute corresponding to action. That user, 8 using this tree to select which PIF attribute applied 9 to performing the action.
10 MR. PETERS: So again, you check box the 11 applicable performance influencing factors, and that 12 will affect your overall calculation?
13 MEMBER REMPE: Well, I hate to ask this, 14 but I'm a little confused. Because under detection, 15 you have SF1.
16 MR. CHANG: Yes.
17 MEMBER REMPE: But under scenario 18 familiarity, you have SF2. Well, how did it get, 19 under detection, I mean, if you look at the bottom 20 screen, what's checked is not consistent with what's 21 in the box in the far left under detection. Why is 22 that?
23 MR. CHANG: Because, okay, we omitted the 24 two slide. Let me come here. Since you're already 25 here.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
303 1 MEMBER REMPE: Which slide is it, 99?
2 MR. CHANG: Yes.
3 MEMBER REMPE: Okay.
4 DR. XING: Yes, 99.
5 MEMBER REMPE: Okay, I'm there.
6 MR. CHANG: So in IDHEAS has identified 20 7 performance influencing factors. But not all these 8 performance influencing factors affect all 9 macrocognitive ---
10 (Simultaneous speaking.)
11 CHAIR BLEY: Macrocognitive, ha, ha, ha.
12 DR. XING: Just say cognitive function.
13 MR. CHANG: Okay function, okay. So this 14 software only implement the PIF relevant to less 15 specific cognitive function. So here you see that, in 16 the detection PIF, that these are PIF affecting 17 detection. Understanding, that's a subset of this PIF 18 affecting the understanding. Each cognitive function 19 has a different set of the PIF of this, under these 20 20 master lists.
21 MEMBER REMPE: So maybe to be a little 22 more specific, am I looking in the bottom screen under 23 action, and that's why I see SF2, and SIC1 checked.
24 MR. CHANG: That's right.
25 MEMBER REMPE: It's not the detection one, NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
304 1 what you have in the lower screen right now.
2 MR. CHANG: Yes.
3 MEMBER REMPE: Okay. I was curious on 4 that, but I just was wondering why is it not 5 highlighted so you know which one you're on or 6 something. How do I know I'm am on the action one 7 instead of the detection?
8 (Simultaneous speaking.)
9 MEMBER REMPE: Oh, the far left, okay. I 10 see it now. Thank you.
11 MR. CHANG: And since we are here, the 12 screen here, even this imagined complexity, this PIF 13 in all macrocognitive functions, but underneath the 14 attributes are different. So here, we see this text 15 complexity sphere for detection that's only C12, C6.
16 These are the attributes. For understanding, there's 17 different list, decision making, different list.
18 CHAIR BLEY: How did we get from this you 19 just showed us to this smaller list?
20 MR. CHANG: You said that I go to the task 21 complexity that says now I'm in detection. I'll go to 22 the check, click the text complexity. It was 23 automated, showed me what's relevant.
24 CHAIR BLEY: So you pick the ones that 25 would apply?
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
305 1 MR. CHANG: Yes.
2 CHAIR BLEY: Okay.
3 MR. CHANG: Yes, sir.
4 CHAIR BLEY: And that's what the next 5 screen was then?
6 MR. CHANG: Yes.
7 CHAIR BLEY: What's left.
8 MR. CHANG: Yes.
9 DR. XING: One feedback we have from users 10 and from our users, they would like this software, 11 they kind of want to make the analyst go through this 12 PIF attribute list. So if they probably think, oh, 13 this is simple, no complexity at all. But when they 14 are forced to go through each of these attributes, 15 maybe one attribute is applicable that wasn't in their 16 initial software.
17 CHAIR BLEY: So when you're using this, 18 and I guess this is kind of what the screen looks 19 like, or part of it is, do I put, like, a number in 20 that box, or is it just a check mark?
21 MR. CHANG: Check mark. But the number is 22 in the software. We can measure once the effect of 23 these factors.
24 CHAIR BLEY: So you don't give a measure 25 of how bad this thing is, it's just bad.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
306 1 DR. XING: We can check the report. The 2 report has that.
3 MR. PETERS: So go back and look at Slide 4 97 real quick.
5 MR. CHANG: Which one?
6 MR. PETERS: I was saying 97.
7 MEMBER DIMITRIJEVIC: Before you do that, 8 I have this question before. I was wondering how come 9 detection doesn't have this information completeness 10 and reliability? Because I would assume that's the 11 thing most important for detection.
12 DR. XING: Okay. How about we let James 13 finish with the software, then we come back to that 14 question. Would that be okay?
15 MEMBER DIMITRIJEVIC: All right.
16 MR. CHANG: Yes. So unless I hear this, 17 that the user selected which cognitive function I am 18 analyzing. And then that once clicked, on this slide 19 that's the worst corresponding with this. This is a 20 cognitive function ---
21 CHAIR BLEY: Do those one at a time.
22 MR. CHANG: Yes, one at a time. And the 23 item, PIF in red is the basic PIF that's mentioned in 24 Jing. And all the blue is the one that has a weight 25 factor attributed. That's just important too for the NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
307 1 user to manipulate the tree.
2 Okay. That I already talked. So that if 3 you look at a high level structure here that's over, 4 it's to calculate human error probability of human 5 failure event. This human failure event contained two 6 elements, time and completed portion, completed 7 portion here that is a critical task.
8 Each critical task is based on 9 contribution from the five cognitive functions. And 10 then each cognitive function, that's how they affect 11 the top is based on that performance influencing 12 factor attribute.
13 The directions that are in the software 14 that the check box user can check off it, is this a 15 critical task involved in this actuality or this 16 macrocognitive function involved in this critical 17 task? So that's a lot. So we'll only take the 18 practical cognitive function and the critical task 19 into the calculation.
20 Okay, the first probable calculation of 21 PT, as mentioned this morning that the PT, it's not an 22 influencing factor. This simply is just under, crew 23 was doing the task based on their normal pace. And 24 then too is titled, given the time available for 25 completing the task, whether there's a possibility of NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
308 1 crew not able to complete a task.
2 So the software implements, in this thing 3 it's based on two distribution. One is time required.
4 That means how much time the crew takes to perform 5 this task. So if say that, one, the operator in the 6 event say that, one, the operator go onsite to open 7 the valve.
8 So taking from the if a symptom occur, 9 that's a control and detected information, and then go 10 through the procedure step to the point the call say, 11 okay, we're needed onsite operator coming in, and then 12 give him the procedure attachment, go to that place to 13 manipulate valve. And then until the valve's 14 manipulated, the time it takes, that's time required.
15 The second parameter is the time 16 available, how much time from the symptom, that before 17 depend on my success criteria, how much time that I 18 have for these tasks need to be completed in order 19 call it a success.
20 So we represent this thing in the two 21 parameters. Both of them is a distribution. We 22 possess several distribution folders and we, I'll come 23 to this, but likely it's that in this case there are, 24 for the user into the time required, it's a blue line, 25 that the user choose the method to enter the 25 points NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
309 1 or 50 percentile. That's what the time. If you have 2 quite a number of operators doing the action, what's 3 the time that it's in this distribution?
4 And we implement the number that the user 5 entered into the step function you see on the graph.
6 That's a step function. So the zero percent to five 7 percentile, because user did not enter, I simply take 8 the probability density function between five percent 9 and 25 percent, divide by two. That's the PDF you 10 see. And this arrow, that represents zero percent to 11 five percent. And then that's based on the numbers 12 here.
13 So the user enter that PDF we simply take, 14 even for distribution, that's a five percent based on 15 the number here, five percent to 25 percent, what's 16 the probability density function that's here, so the 17 software property, the distribution step function.
18 MEMBER DIMITRIJEVIC: So that's usually, 19 that is, like, how many points they want to do this 20 type of functioning?
21 MR. CHANG: Currently, we only ask them to 22 enter five data points.
23 MEMBER DIMITRIJEVIC: All right. And 24 where does the red thing come from?
25 MR. CHANG: Where does this 5/25?
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
310 1 MEMBER DIMITRIJEVIC: No, the time 2 available come from?
3 MR. CHANG: Well, this typical face on the 4 thermal hydraulic analysis, that's what's the time 5 available before something happen if operator didn't 6 do any action. That's typically what we do in PRA.
7 Yes?
8 MEMBER DIMITRIJEVIC: That doesn't come as 9 a distribution, so the ---
10 MR. CHANG: Yes. So in this case, we also 11 implement as one constant user can enter a single 12 value. Yes, I just said that, okay. Well, showing 13 that the distribution that the user entered into this 14 Weibull distribution and then provide an safe and 15 escape.
16 And the software takes these two 17 distributions, usually a Monte Carlo simulation to 18 calculate the probability less the time required --
19 MEMBER DIMITRIJEVIC: Yes. But this is 20 pretty straightforward, the inputs --
21 MR. CHANG: Yes. So when you take one 22 million data sample, and the Monte Carlo took care of 23 the PT, it's showing that we implement the different 24 distribution for time required and time available.
25 Both of them have them had the normal distribution, NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
311 1 gamma distribution, Weibull distribution, and then 2 provided the five data points, the percentile. For 3 the time available, we added one option for user, 4 simply enter a constant number.
5 MEMBER DIMITRIJEVIC: Well, if you enter 6 distribution, then this software will calculate 7 convolution, right?
8 DR. XING: Yes.
9 MR. CHANG: Yes. But it's implement 10 convolution the way inference is among the Monte Carlo 11 simulation.
12 MEMBER DIMITRIJEVIC: Oh, okay, well, if 13 you know other way, that will be interesting.
14 (Laughter.)
15 MR. CHANG: Yes. Well, now we have 16 computer, we don't need, that will be harassing for 17 the HRA user to calculate that convolution.
18 Okay, so we already talked that on the 19 left-hand side that analyst goes through, apply the 20 practicable macrocognitive function are practicable to 21 the situation and then select, no, come into the tree 22 here to select what's the PIF attribute apply.
23 Committee Member Joy, that you earlier 24 this, you mentioned about this, the purpose of this 25 SRM is to reduce variability. My viewpoint is that, NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
312 1 in the earlier letter, they tried benchmarking 2 project, that we upload one thing, is the gap between 3 the qualitative analysis and quantitative analysis.
4 But something did a really good 5 qualitative analysis. But the way that the inside, 6 they were not able to apply to the method they 7 selected for analysis.
8 And my fear was that we've got to have a 9 straight line from the qualitative analysis to the 10 quantintative analysis that guides you not only to a 11 good qualitative analysis but also the analysis result 12 that you can carry to the quantification.
13 I see that things here, the ECA, the way 14 it implement, the number of the PIF and the 15 attributes, that's very much a different level than 16 the current HRA method is.
17 And there are, one thing I want to mention 18 is specific of this PIF. If you see that the GS 19 report is not, each PIF attribute is not just this 20 short description, there is some additional 21 information. That additional information was 22 implemented in the software when the user hovered the 23 mouse on the top of this item. Now the screen has 24 shifted description.
25 I have example of that basic question for NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
313 1 the user that will kind of remind the user to say, 2 okay, now the first thing you check at the high level 3 which PRA that is more, no, that applies. And then 4 they go into the detail. That's a PRA, give you each 5 attribute that you check where they're practicable.
6 I think that the information at this level 7 were able to carry the qualitative information coming 8 to the quantitative result. That reduce the 9 probability. That's my personal two cents.
10 Okay. So this simply just say that we 11 implement whatever the equation that you see in the 12 report, in the software. We talk these --
13 DR. XING: Maybe you can go to the 14 example.
15 MEMBER DIMITRIJEVIC: Yes. I was on 97 16 going to the what's happening here. Because I note 17 you're already in the hundred one.
18 MR. CHANG: Before going to the example, 19 let me talk one thing. Among these, I think, one 20 morning, 160 PIF attributes despite macrocognitive 21 function, there's a PIF that most of PIF we simply 22 infer into a point effect. Say that you check this 23 one, that one will increase the ten percent of the HEP 24 or that would, you check these attributes in the task 25 capacity that the HEP will increase this amount.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
314 1 There is some PIF has a range of, in fact, 2 and then most of them that the uncertainty, the range 3 is more than one order of difference. So in this case 4 that's here, that the software provided that reference 5 point.
6 As you see here, this was implemented in 7 the C1. I'm sorry, I cannot read that either. That's 8 a two-starter there that's provided. That two-start 9 in one item means that the user needs to specify 10 which, given the scale that for user to spend a point, 11 when to apply for this PIF attribute effect.
12 I was, user created this attribute that 13 this screen will pop op. Let me show you that, okay, 14 these are the things you were analyzing. And then 15 this allows the user to, we use the scale from one to 16 ten, and then underneath that provider, we provide 17 information. Say okay, one, we present that you have 18 have information at seven. You know, three is this, 19 this, this. This gives you the reference value for 20 user to choose the number.
21 And what user need to do is simply adjust 22 this number, so that's okay. The software will take 23 this number, that two, use a linear, that slope and 24 then to calculate whatever that HEP or the effect on 25 HEP into the equation. And then that eventually shows NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
315 1 on the top, that HEP number.
2 MEMBER KIRCHNER: For clarification, you 3 can't do multiple choice?
4 MR. CHANG: For this one?
5 MEMBER KIRCHNER: Can you, well, I'm just 6 looking at the box. Can you check C1, C4, and C6?
7 MR. CHANG: Yes.
8 MEMBER KIRCHNER: Okay.
9 MR. CHANG: Now this one just for C1, once 10 I click okay, then I can go to check C4. And another 11 pop-up screen for C4 will show to me that I checked 12 it, wherever that I want to use for C4.
13 MEMBER KIRCHNER: But doesn't that 14 contradict your claim about the variability? I 15 thought it would force you to say this is a complex 16 problem or it's not, so to speak. You know, but if 17 you pick many, and then a different user can pick a 18 different set, then you're going to get a lot of 19 variability.
20 MR. CHANG: Well, I think this is a detail 21 that, when you say the complex or not, that's everyone 22 can ---
23 MEMBER KIRCHNER: Yes.
24 MR. CHANG: But now we come to here. The 25 complex, what's the level of complexity? This, I feel NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
316 1 this should give you an attribute, different aspect of 2 complexity, at least the part that will apply, that 3 user check more. That kind of implies that all this 4 is more complicated. If you check less, that simply 5 is not so complicated.
6 MEMBER KIRCHNER: Well, I just worry or 7 wonder whether the system double-counts things in 8 multiple places.
9 DR. XING: Yes. That's one reason I would 10 use, actually we had, probably every other week, a 11 user come to evaluate the software. He want us to add 12 those boxes, what you clicked.
13 MEMBER KIRCHNER: Yes.
14 DR. XING: So when everything you clicked 15 showed on the box, they will say, oh, I already 16 counted this context at this. I shouldn't double 17 count it.
18 MEMBER KIRCHNER: Yes.
19 DR. XING: So they can look at everything 20 you clicked, you selected, and detect the double 21 counting.
22 MEMBER KIRCHNER: It might be interesting 23 in some kind of data-like test to have multiple users 24 attack the same problem and see how convergent the 25 answers are.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
317 1 DR. XING: I will ---
2 (Simultaneous speaking.)
3 MEMBER KIRCHNER: Are you doing that?
4 DR. XING: -- talk that. That's our 5 ongoing project.
6 MR. PETERS: Yes, we're doing that now.
7 MEMBER KIRCHNER: Okay.
8 MR. CHANG: Okay, fine. In documentation, 9 once the user does all the checking and then that's 10 got documentation, that will load this information 11 shown at the time. And then the HFE was, the 12 calculation was that the PIF attribute for each 13 different cognitive function there.
14 And then the time analysis, what's the 15 parameter entered, what's the HEP for PT information 16 so that the user can save this information to 17 Microsoft Word to add additional information to 18 justify why the attribute is selected, you know, that 19 type of information that provides documentation.
20 CHAIR BLEY: Two related questions. One, 21 have you written up a short report on using the 22 software, or a conference, or journal paper on it?
23 MR. CHANG: Not yet.
24 MR. PETERS: We just finished the software 25 a couple of weeks ago.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
318 1 CHAIR BLEY: Okay.
2 MR. PETERS: Yes. We're just not to that 3 stage yet.
4 CHAIR BLEY: And the other related one 5 was, although you didn't bring it for the meeting, 6 would you be willing to come some lunchtime and demo 7 the software?
8 MR. CHANG: Yes.
9 CHAIR BLEY: Okay. Maybe we can find a 10 time for us to do that for anybody who's interested.
11 Okay, thanks. I think it would be helpful.
12 MR. CHANG: Okay. So now --
13 CHAIR BLEY: And when you write it up, 14 we'd like to see your paper.
15 MR. CHANG: Oh, paper, okay.
16 The example, the example is simple, that 17 we took this example from the U.S. HRA benchmark study 18 that is a very straight forward steam generator tube 19 rupture event of the 500 GPM, because its purpose is 20 demonstrated how we implement software in this 21 analysis.
22 The HFE we analyzed was, given the steam 23 generator rupture, then from a PI perspective, the 24 interest thing of the human actions, operator is able 25 to identify which steam generator has tube rupture and NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
319 1 correctly isolate broken steam generator, and then 2 depressurize and cool down to prevent that steam 3 generator pilot operating valve release. Because that 4 release, that will, radioactivity would release to the 5 outside environment. So even here later, we have a 6 clear, what's our reaction, and then what's the 7 success criteria?
8 Given this here that you see in the ECA 9 that was, at the beginning we talked about what's the 10 initial condition, it's a full power, under power 11 operation. And then we have the fire operating crew 12 operator that's inside the control room, one shift 13 supervisor, two reactor operators, they are men, crew 14 to respond to the event.
15 And then we have a shift technical 16 advisor, STA, during the event here will perform 17 independent check to see that these three crew that 18 they are running to the right direction. And then we 19 have shift managers that stay on there as overseer of 20 the scenario. That's the crew, that aspect, condition 21 we have.
22 The initial event is very straightforward, 23 just 500 GP and steam tube rupture. That is not 24 beginning has a leak, okay, it's just rupture. And 25 then find the condition that, we all assume that NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
320 1 there's no component, no instrumentation failure. So 2 everything responds as designed.
3 And then, as we all know, that simulator 4 training, because this is a steam generator rupture 5 event, design basis event crew is trained, at least 6 once a year, or even twice a year, with a variation of 7 the steam generator rupture if or in combination with 8 different components of instrumentation failure that's 9 for them to respond to the situation.
10 So from the symptom-wise, that the crew, 11 we expect the crew is quite familiar with the symptom 12 occurred in the steam generator rupture event. Given 13 that, for this particular plant, if the chief rupture 14 that occurred did not manually trip the reactor, then 15 four minutes after the event that reactor will be 16 automatically tripped.
17 So that we talk about HFE, and there's a 18 scenario context briefly talk through.
19 MEMBER KIRCHNER: Isn't that really two 20 events, two HFEs?
21 MR. CHANG: Depends on how you model it.
22 That's ---
23 MEMBER KIRCHNER: We do get a different 24 answer with your software if you modeled it 25 differently. I would separate these two personally, NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
321 1 but ---
2 MR. CHANG: Two means --
3 MEMBER KIRCHNER: So if you lump them 4 together it doesn't matter.
5 MR. CHANG: Two means that when you say 6 the two --
7 MEMBER DIMITRIJEVIC: They isolate and 8 depressurize.
9 MEMBER KIRCHNER: Isolate the steam 10 generator is different than controlling the RCS 11 pressure.
12 MR. CHANG: Yes.
13 DR. XING: We can try that in the U.S.
14 empirical study. This was given to analyst as one 15 HFE.
16 MEMBER KIRCHNER: All right. Well, you're 17 leading the witnesses when you do that. Because 18 you're telling them to do two important steps, and 19 you're putting them together.
20 MR. CHANG: Let me come to that --
21 MEMBER KIRCHNER: Okay.
22 MR. CHANG: -- when we come to address the 23 issue.
24 MEMBER REMPE: Would the results differ if 25 you did the other way?
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
322 1 MEMBER KIRCHNER: Yes. Well, that's what 2 I would want to know, yes.
3 CHAIR BLEY: We'd had earlier meetings on 4 the empirical studies. But when they were done, I was 5 part of them, there were separate groups. There were 6 a group of analysts who didn't get to see what went 7 on. And then there were a number of crews who went 8 through. And the crews did them as you'd expect in 9 the simulator. They just, they didn't know what was 10 coming.
11 MEMBER KIRCHNER: Yes.
12 CHAIR BLEY: And they'd get events as they 13 occurred. The other folks were asked to quantify, I 14 think, both of these events. So they did them 15 separately. But, you know, that was compared with 16 what really happened in the simulator.
17 Of course, there are only four of five 18 data points in the simulator, so it's a limited kind 19 of comparison. But you learned a lot of things from 20 it. It was useful. Anyway, that's the way they did 21 it.
22 MEMBER KIRCHNER: Okay.
23 CHAIR BLEY: So the crews weren't told 24 they're being judged on two different things. The 25 analysts were told to evaluate two different things.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
323 1 I think that's right. Go ahead.
2 MR. CHANG: So now this is where I start 3 to talk about how we calculated the PT that's given 4 this action. In the U.S. HRA benchmark that has three 5 crew run through the scenario, and then we got the 6 three data points, there's 30 minutes, 36 minutes, 40 7 minutes, ten seconds, 43 minutes, ten seconds. This 8 is a time that these three crew completed the cool 9 down and isolation at the time.
10 So I simply took these things and turned 11 these three data points within 33 percentile, 67 12 percentile, and 95 percentile. And then that using a 13 linear tube, come out to the five percent, 25, 50, 75, 14 95 percentile. That was the time used for the time 15 required.
16 And in the HRA benchmark report also say 17 that the success criteria typically was between two 18 and three hours. It's only one sentence, so I took 19 the normal distribution that pick a mean number, 2 and 20 3, and then gone through the software, that's the 21 distribution. And then come to the FPT is equal to 22 zero. So that's for PT.
23 Now we come to a calculated --
24 MEMBER DIMITRIJEVIC: Back up.
25 MR. CHANG: Yes?
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
324 1 MEMBER DIMITRIJEVIC: Well, you're talking 2 here about pressurizer pore which I don't know really 3 why, I mean what is your measure for that, to prevent 4 what? I mean, you know ---
5 MR. CHANG: If they ask, reactor coolant 6 system, they are up, depressurize RCS system. The RCS 7 system, now eventually that will pressurize steam 8 generator and then push the pump open.
9 MEMBER DIMITRIJEVIC: Yes, but that's not 10 bypassed for the inside containment. You worry about 11 releases outside.
12 MR. CHANG: Some of that pore that go to 13 the steam line, steam line may directly go to ---
14 MEMBER DIMITRIJEVIC: But that's, well, 15 here you said pressurizer pore which is in, that's why 16 I say here. There you say steam generator pore, 17 whatever that is. But I think it's like you are 18 mixing little apples and oranges in this.
19 MR. CHANG: Okay. So given this, the 20 human action, that's HFE. We talked on that operator 21 successfully isolated a broken steam generator and 22 then depressurize RCS and cooled down RCS to prevent 23 the pump open. This slide ---
24 CHAIR BLEY: It should be the steam 25 generator.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
325 1 MR. CHANG: Oh, steam ---
2 (Simultaneous speaking.)
3 MR. CHANG: Yes.
4 MR. PETERS: We didn't catch that when we 5 were putting the slides together. Thank you.
6 CHAIR BLEY: And I think when you weren't 7 here it came up that, you know, we're talking about 8 a steam generator tube rupture. A steam generator 9 rupture is a different thing. That's a primary 10 rupture. And I don't know what a broken steam 11 generator is. So you're talking about old tube 12 ruptures. And you ought be consistent.
13 MR. CHANG: The steam generator has the 14 tube rupture. Yes, that's what we talk that.
15 CHAIR BLEY: But you've also got a broken 16 steam generator up there too, on the fourth bullet.
17 MR. CHANG: Yes. There are two 18 successfully acute to decide the action. Isolate 19 broken steam generator and depressurize RCS, yes. So 20 your question --
21 CHAIR BLEY: Broken steam generator's odd 22 language for this sort of thing. It would be the 23 steam generator with a tube rupture.
24 MR. CHANG: That's a common term I heard 25 from the NRC staff. I just used it.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
326 1 CHAIR BLEY: There you go.
2 (Laughter.)
3 MR. CHANG: Ruptured steam generator, that 4 talks about the ---
5 MEMBER DIMITRIJEVIC: Failed, usually it 6 fails.
7 MR. PETERS: So we should use the word 8 ruptured, obviously it's a ruptured steam generator 9 event.
10 MEMBER DIMITRIJEVIC: Or you could say 11 failed steam generator.
12 (Simultaneous speaking.)
13 MR. PETERS: The slides aren't quite --
14 MR. CHANG: Sorry, my NRC ---
15 MEMBER REMPE: It's in the report.
16 MR. CHANG: Ruptured steam generator means 17 that the main steam line of this corresponding steam 18 generator has a rupture. And the broken steam 19 generator is the tube of that steam generator.
20 CHAIR BLEY: That's new language to me.
21 MR. CHANG: Oh, okay.
22 CHAIR BLEY: I'd call them both broken, 23 ha, ha, ha.
24 MR. CHANG: Okay. So given this task, the 25 detection, looking at the, given this condition that NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
327 1 what's the key information that operator needs to 2 observe in order for the operating crew able to 3 succeed to the action.
4 Okay. So these, including that 5 information, then you could be aware, okay, there's 6 something wrong. And then the main immediate path for 7 them is to, they will need to be able to enter E zero, 8 and enter from E zero transfer to the E-3, and then 9 cut power to E-3 to perform that required action.
10 The key information helping operator to go 11 to this path, there is key information. What's the 12 likelihood that the operator is not able to detect 13 this information that would lead into how the action 14 is not performed or not performed on time.
15 The probability of doing this was going to 16 the detection CFM, that tree there, to check what are 17 the PIF attributes practicable to this context. So 18 that's a detection.
19 Understanding here is that, given the crew 20 identify all able to detect all the critical key 21 information, and operator is able to follow the right 22 procedure path, and then some of them, they might need 23 to make sudden judgements, or information integration, 24 and then follow that next is logical, whatever, that 25 the two come out, two correct analysis. Okay, this is NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
328 1 a steam generator rupture event, and then identify 2 what's the broken steam generator. That's the 3 diagnosis we talk here.
4 MEMBER DIMITRIJEVIC: But that's for 5 isolation. That's the indication for isolation.
6 MR. CHANG: Yes.
7 MEMBER DIMITRIJEVIC: Under 8 pressurization, any steam generator broken will lead 9 to that pressurized primary by opening usually for 10 something. So that's two different indications 11 depending what action you're doing which was discussed 12 before.
13 MR. CHANG: Right. So our push here, I 14 note the key information that's able, for our ability 15 to do this, that not only the secondary radiation 16 alarms are also disabled, to check what's the steam 17 generator water level and incident pressure that's 18 all, the different steam generator come out to the 19 understanding that that's okay, that the other 20 probable steam generator has an issue. Now, this is 21 key information, and then it's able to make the 22 diagnosis.
23 Issues you're making here, let's assume 24 operator has correct analysis of a steam generator 25 bubble has problem. That's only one procedure, doing NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
329 1 isolating, this thing to isolate that broken steam 2 generator, and then follow the procedure to cool it 3 down and depressurize it. So that decision making 4 within this context procedure only provide one path.
5 So there's no alternative.
6 So in this case, that the decision is not 7 applicable, because the procedure just only one path 8 to make the decision. And the action execution here 9 is now we are talking about action, yes, isolate 10 broken steam generator. And now I understand I need 11 to isolate the steam generator bubble, I need to turn 12 this flow closed, and then the other things. How 13 reliable I can perform that task?
14 And then I need to depressurize and cool down 15 RCS in a proper manner that this thing, how reliable.
16 We are looking at this activity as a whole, going to 17 the action CFM there, the three, to check what are the 18 PIF attributes we applied for doing this action.
19 And teamwork, because in the ECA is 20 talking about different team. That mostly this 21 activity is performed inside the, action that a 22 complication has happened within the main control 23 room. That might be something to talk about, that the 24 outside operator to close a LOCA steam generator rod 25 that's a mechanical test and to check.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
330 1 But within this context, we think that 2 inter-team activity is not contributing to that 3 defense, so that we did not include the teamwork in 4 this HEP estimate.
5 MEMBER DIMITRIJEVIC: You know, just to be 6 picky, you know, E-0 is not a number.
8 MEMBER DIMITRIJEVIC: Oh. EOP-0?
9 MR. CHANG: Yes. Emergency Operating 10 Procedure 0. And Emergency Operating Procedure is 11 reactor safety procedure. That's once the reactor 12 operator forced go into that procedure. And then from 13 E-0, in the steam generator event, will lead them to 14 jump into E-3, EOP-3, that's a steam generator block 15 for ---
16 MEMBER DIMITRIJEVIC: To be honest with 17 you, I thought you were talking one minus, you know, 18 one minus three. Because in the previous tables you 19 have a similar ---
20 MR. CHANG: Okay, thank you for pointing 21 that out. Yes?
22 DR. XING: That's a lower probability.
23 MEMBER SUNSERI: Is that a cognitive 24 error, I don't know.
25 MEMBER DIMITRIJEVIC: Yes, well.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
331 1 MR. CHANG: Okay. So the way with the 2 understanding we come to the software to calculate 3 HEP. So the user here, let's simply go to each one of 4 them going to a decision detection. And then in the 5 detection we didn't identify any negative PIF 6 attributes.
7 And each tree, the first one of attributes 8 has no impact. And then the user has to check every 9 PIF and then to check no impact. That's just 10 demonstrated that the user, you know, goes through 11 each PIF to check were there any PIF attributes.
12 And then the same thing here about 13 understanding, given the context that a user will take 14 the understanding this radial pattern and then, within 15 the tree here, to check what's the PIF attribute 16 applied. In this case it's also now and then.
17 So we said that the decision and teamwork 18 is not applicable. So that's unchecked. The user 19 unchecked that, will make grade. It's just human 20 factors, consideration make it more apparent that ---
21 MEMBER KIRCHNER: I might observe 22 something, and I don't know how you would how you 23 would factor this in.
24 MR. CHANG: Yes.
25 MEMBER KIRCHNER: This is all true in NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
332 1 simulator space. This is not true if you're in the 2 control room of a real reactor and the blank hits the 3 fan. And this is a really serious event.
4 And so I wouldn't have some blanks in some 5 of these categories, because if this actually happens, 6 this is, you know, the tension level, there are a lot 7 of dynamics that change from a well rehearsed 8 simulator scenario. Just an observation.
9 MR. PETERS: Absolutely.
10 MEMBER KIRCHNER: And I don't know how you 11 factor that in.
12 MR. PETERS: Well, it's easy to factor in 13 if the analyst knows if they're mulling a real 14 scenario. They would go and select the challenging 15 context that they see in the real scenario. So I 16 think it's a good comment.
17 DR. XING: And we picked this example just 18 like in the U.S. study. We won't have a baseline 19 example. This is suppose everything is perfect --
20 MEMBER KIRCHNER: No, I understand.
21 DR. XING: -- how we go. But we know in 22 reality --
23 MR. CHANG: Yes, with the information ---
24 DR. XING: -- we probably never have that 25 perfect.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
333 1 (Simultaneous speaking.)
2 MEMBER KIRCHNER: There is another aspect 3 that occurs to me.
4 MEMBER BALLINGER: It has happened, and 5 you can get data on that.
6 MEMBER KIRCHNER: Yes.
7 MR. PETERS: Well, the interesting part is 8 ---
9 MEMBER DIMITRIJEVIC: It's happened in 10 Japan.
11 MR. PETERS: This is a retrospective 12 analysis ---
13 MEMBER BALLINGER: Well, there's one in 14 the U.S. for sure that I know of.
15 MEMBER DIMITRIJEVIC: Yes. Right, the 16 Japan decision making was the most important factor 17 which is here blackened at the top.
18 MEMBER BALLINGER: Yes. They have, what, 19 two in the U.S. and one in Japan, right.
20 MR. PETERS: So the nice thing about 21 this --
22 MEMBER BALLINGER: North Anna, Indian 23 Point, and I don't know about Japan.
24 MEMBER KIRCHNER: No, I'm just trying to 25 add something.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
334 1 MEMBER BALLINGER: No problem.
2 MR. PETERS: You know, I was just going to 3 say, the nice thing about this tool is the intent of 4 this tool is for a retrospective analysis use at the 5 NRC. So what we will have going into the scenarios, 6 we will know the contextual information, what actually 7 happened at control.
8 And we'd be using this tool to help us do 9 our, submit a determination process, enhanced 10 analyses. So that would give us basically this 11 contribution of human error probability on the known 12 conditions.
13 So if we wanted to apply it to a 14 prospective analysis, it's a little bit different.
15 And that's where you would kind of get these vanilla 16 sunny day scenarios. But our original intent is for 17 staff use. And like I was saying earlier, highly 18 scientific number, 90 percent of what we do is 19 retrospective, that we would have that context to fill 20 in.
21 MEMBER KIRCHNER: Yes. I guess the other 22 thing that I'm maybe jumping ahead of you a little 23 bit, in this particular example, the PT calculation 24 makes a contribution of zero.
25 But again, in such a real life scenario, NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
335 1 although the time required to do it, this distribution 2 that you show, it probably broadens and moves out 3 because of delays and such. So I was just curious.
4 Do you have a sense for when the PT might, 5 for this scenario, it's a zero contribution, but when 6 you get into situations where the time for detection, 7 the time for understanding, the decision making, the 8 execution and such, is different than the clean, we've 9 done this drill before. We know it takes 20 minutes.
10 But now you've got all those variables. Am I ---
11 MR. CHANG: Let's come back to the 12 definition of PT.
13 MEMBER KIRCHNER: Yes.
14 MR. CHANG: PT is that known capacity they 15 followed in the, you know, just the way the team ---
16 MEMBER KIRCHNER: Yes.
17 MR. CHANG: It's what's the reason that --
18 -
19 MEMBER KIRCHNER: Exactly, you gave us.
20 You know, you gave a nice example.
21 MR. CHANG: Fine.
22 MEMBER KIRCHNER: You had a bunch of 23 teams, you had data, it took him this long.
24 MR. CHANG: Yes.
25 MEMBER KIRCHNER: I'm just thinking of NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
336 1 when the situation gets real and complicated, the time 2 required probably dilates.
3 MR. CHANG: Yes, but that's a ---
4 MEMBER KIRCHNER: And you could then, I 5 guess, explore it by changing this percentile on times 6 to something that would reflect the delay in response 7 because of either detection, whatever. You see where 8 I'm going? I am trying to --
9 MEMBER DIMITRIJEVIC: This is a very 10 complex action response. Because you're going to, in 11 150 minutes you're going to lose 75,000 gallons, I'd 12 say, we found that. And you're going to get the LOCA 13 signal. Your interaction systems are going to come 14 in.
15 MEMBER KIRCHNER: Exactly.
16 MEMBER DIMITRIJEVIC: It's going to be 17 difficult to depressurize. You have to look, you have 18 a small LOCA read, the high pressure injection bumping 19 in, you have to shut the charging before you start.
20 This is very complex thing, just as the action itself.
21 And the thing is, to say the time doesn't 22 really, you have competing factors, you know, LOCA 23 signal, systems coming in.
24 DR. XING: Well, we have a table we had 25 given --
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
337 1 CHAIR BLEY: They started this, and they 2 had thermal hydraulics to back it up for the plant 3 that was involved. But if you isolated the steam 4 generator within 40 minutes, I forgot what the time 5 was, you'd get it done before you had the small LOCA.
6 And it's an unusual plant, I think, in that regard.
7 MEMBER DIMITRIJEVIC: You can at least, 8 on, the primary side, there were rooms --
9 CHAIR BLEY: I'm sorry?
10 MEMBER DIMITRIJEVIC: Nobody has, you 11 know, LOOP isolation valves. I mean, you cannot 12 isolate primary side of the steam generator. You 13 cannot easily treat water and steam like that.
14 CHAIR BLEY: Yes. And you have to, hum, 15 I forget what time they gave you to isolate it.
16 MR. CHANG: Around 40 minutes.
17 CHAIR BLEY: Yes, which, for most of the 18 plants ---
19 MR. CHANG: Yes, from the beginning of the 20 --
21 CHAIR BLEY: -- here in the U.S., you get 22 much less time than that before you get a LOCA.
23 MEMBER DIMITRIJEVIC: But that will not 24 stop the loss of coolant until you depressurize.
25 Insolation itself will not stop loss of coolant.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
338 1 CHAIR BLEY: Yes. That goes ---
2 MR. CHANG: Sorry, the time was a quarter 3 and depressurize. That's isolated, that's all 4 happened earlier. So that did not ---
5 (Simultaneous speaking.)
6 CHAIR BLEY: There we go, that's better.
7 DR. XING: This is everything now. They 8 actually, in the study, they listed the detail, the 9 time for every major activity. So that was already --
10 -
11 MEMBER KIRCHNER: No, I was just curious 12 that there is some interplay. Even though this one 13 was a nice clean example, based on the data and the 14 analyses, once they start overlapping and making a 15 contribution, implicit in some of the CFMs, is a time 16 element as well.
17 DR. XING: Yes. In fact, in the ---
18 MEMBER DIMITRIJEVIC: And in a sense, 19 those two actions are completely different now, 20 isolating and depressurize.
21 DR. XING: Yes.
22 MEMBER DIMITRIJEVIC: So, I mean, they 23 should not be analyzed as one. I mean, it's easy for 24 you to separate. It seems like you are analyzing this 25 relation not the categorization.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
339 1 DR. XING: Yes. I can talk an example.
2 In the another scenario in the U.S. benchmark study, 3 the loss of CCW, that one, operator needed to do two 4 actions. They need the instructions. They need to 5 stop the RCS within one minute of you detect the loss 6 of the --
7 CHAIR BLEY: The pumps.
8 MEMBER DIMITRIJEVIC: The pumps, yes.
9 DR. XING: Yes. And then start the PDP 10 within, you start the PDP before temperature exceeded 11 230 degree F. And the estimate, it will take, after 12 you lost the ECW, it would take seven to nine minutes 13 for the temperature to reach that level. However, for 14 all the four crews, it took them average, I think the 15 shortest one was four minutes, the longest was eight 16 or nine minutes. It took them that long to detect the 17 loss of the symptom.
18 So none of them started. I mean, all of 19 them eventually started the RCP, I'm sorry, stopped 20 the RCP. But they all exceeded the one minute 21 criteria. So you can see the failure. Worse than 22 that, because they spent so much time doing the 23 detection diagnosis up front, they have the procedure 24 to start a PDP. But by the time they ---
25 MEMBER DIMITRIJEVIC: What's a PDP?
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
340 1 CHAIR BLEY: Positive displacement 2 charging from ---
3 (Simultaneous speaking.)
4 CHAIR BLEY: -- that that plant used and 5 ---
6 MEMBER DIMITRIJEVIC: What is that going 7 to do ---
8 CHAIR BLEY: This is an abnormal 9 procedure. It's not an emergency procedure.
10 DR. XING: Yes. It's abnormal procedure.
11 But they followed the procedure, but they have seven 12 to nine minutes before they do that. But they spent 13 all this time in up front information detection. So 14 they all failed that second action too, because of 15 that complication in time.
16 MEMBER DIMITRIJEVIC: Well, for us this a 17 complicated action, and we are not calculating HEP, 18 and we can question your HEP until, like, you know, 19 the, we are just trying to see how your method works.
20 DR. XING: Yes. So this example, we just 21 want to, let's just make hypothetics in our --
22 MEMBER DIMITRIJEVIC: Right.
23 DR. XING: -- you know, perfect world.
24 MR. CHANG: Yes. That is actually the 25 things about it that reduce variability. Now they use NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
341 1 the check located action that I only have one negative 2 PIF attribute here. And this is very specific, that 3 everyone can argue that while a specific attribute may 4 play in the action this year, that, I think, is one 5 mechanism to improve the consistency.
6 Okay. So what done is that they simply 7 selected a document that's all, and in this report was 8 load into this place that user can either add it here, 9 that the provided location why I select it. There's 10 no impact, otherwise why I check this performance 11 attribute, et cetera, this information.
12 I want to let the committee, that each 13 bullet, it's a performance PIF attribute checked. And 14 then underneath, below that, that bullet is the 15 effect. But in this case, there's no effect. So you 16 see that the basic HEP is zero, and then that the 17 weight factor is zero.
18 But in the earlier slide we see that here.
19 You see that while the detection, that C2 was checked, 20 that it provided a basic HPOs, Y minus three. And 21 then that C3 was checked. That has a contribution of 22 HEP, basic HEP of 1E minus three.
23 For detection, that SIC information was 24 checked. That it has the weight factor of 0.1. So 25 this they come to ---
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
342 1 MEMBER DIMITRIJEVIC: What page is this?
2 MR. CHANG: Huh?
3 CHAIR BLEY: I'm trying to find it.
4 MR. CHANG: It's 103, Page 103. Okay, I 5 see that on the upper right.
6 CHAIR BLEY: Oh, yes.
7 MR. CHANG: Yes. This is just the 8 information we provided once the user's done the 9 analysis. The complex math and the data that's 10 software to manage it. But to come to the end, the 11 user has all this information that he can -- This PIF 12 actually did this type of contribution. Is that the 13 right contribution that, no.
14 CHAIR BLEY: This is a completely 15 different example than the one we ---
16 (Simultaneous speaking.)
17 MR. CHANG: Yes, yes.
18 (Simultaneous speaking.)
19 MEMBER DIMITRIJEVIC: It's myHFE.
20 MR. CHANG: Yes, myHFE.
21 MEMBER DIMITRIJEVIC: Is it yours? It's 22 myHFE, but it's the steam generator. Well, do you 23 have page for steam generator and have the time? But 24 there is not too much on that page.
25 MR. CHANG: Yes.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
343 1 MEMBER DIMITRIJEVIC: Because everything 2 is zero, and you don't have that one.
3 MR. CHANG: Right. The purpose that I'll 4 try to show was that user check every PIF and then 5 make sure that none of the attribute applies on the 6 check, that they're not applicable for each PIF.
7 MEMBER DIMITRIJEVIC: But the active here 8 you have the one applicable, didn't you, like for the 9 action?
10 MR. CHANG: Yes. But it was too, under 11 part and it didn't show up on the screen.
12 MEMBER DIMITRIJEVIC: But you have the 13 page that you passed out.
14 MR. CHANG: Yes. This is partial screen 15 shot. Okay, that's all I have.
16 DR. XING: We turn to our boss.
17 MR. PETERS: Thank you, this is Sean 18 Peters again, Chief of Human Factors and Reliability 19 Branch. And I'm going to go very quick to try to make 20 up time. My DeLorean is broken right now, so I won't 21 be able to get us back on schedule. But we'll move as 22 quick as we can.
23 So I'm going to tell you a little bit 24 about the testing that's going on right now. We have 25 an NRC and industry joint team. This is a team of us NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
344 1 and EPRI. This is wholly supported by the BWR and PWR 2 owners groups who have volunteered members to members 3 to be part of this team and volunteered their plants 4 for us to go and do modeling.
5 The purpose of our testing is to model 6 FLEX scenarios using our IDHEAS-ECA tool to help our 7 future regulatory decision making. This includes 8 license amendments, NOEDS, credit, and like 9 termination processes and analyses, things like that.
10 And it's basically because industry would 11 like to get credit for the installed equipment. They 12 spent all these millions of dollars installing FLEX, 13 and training on it. They want to get some PRA credit 14 for that.
15 What we're going to be getting out of this 16 testing is a better understanding of the human 17 challenges and flexible mentation. So we're actually 18 going to go through the plants and see how they 19 implement FLEX tools and see where there are 20 challenges of implementing those.
21 And we hope to get feedback for our method 22 improvement. So as users click through our buttons, 23 as they use this qualitative analysis process, we want 24 to get feedback from them on how it went.
25 So there are two sets of teams. We have NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
345 1 scenario developers out there helping us define, 2 clearly define the scenario. And we have teams of HRA 3 practitioners. Both teams are roughly 50/50 split.
4 The scenario developers have half NRC folks, half 5 industry folks, same with the HRA teams, half 6 industry, half NRC.
7 We're doing two sites so ---
8 CHAIR BLEY: Any operators from the 9 industry on your teams?
10 MR. PETERS: Any operators, I think there 11 are. I'm trying to remember the specific background.
12 Most of the people have, from the implementors, they 13 are the people that actually implemented FLEX at their 14 various plants. But what they're not modeling, is 15 they're not modeling their plant. They're modeling 16 Peach Bottom and Surry. So these are people that have 17 installed, and operated, and trained on FLEX 18 equipment.
19 CHAIR BLEY: I hope you've got some real 20 operators involved on your teams.
21 MR. PETERS: That's what I'm understanding 22 on the scenario development team especially. HRA 23 team's, obviously not necessarily operators on the HRA 24 team but for the scenario development. And then we 25 also have a bank of experts that we draw on if we have NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
346 1 questions associated with when we're doing the 2 modeling of HRA, which are the people that are plant 3 familiarity.
4 CHAIR BLEY: We've a long history of, one, 5 saying we absolutely have to have operators on our HRA 6 teams and, two, knowing that if we don't, we get funny 7 results.
8 MR. PETERS: I think, so somebody has 9 worked with you a long time. Dr. Susan Cooper is 10 leading the team, and she's probably the person I've 11 heard say we need operators more than anybody ever in 12 the history of HRA. So she is leading this program 13 and our staff. So we, but I have to go back and 14 specifically get you the expertise.
15 The site visits are right now. I'm 16 missing Dr. Cooper here to answer that question.
17 Because she is currently at Peach Bottom running down 18 FLEX equipment today. They'll be there until 19 tomorrow. And they're starting Surry the first week 20 of October. So I think it's October 2nd and 3rd we'll 21 be at Surry looking at their facility.
22 Scenario modeling, so they're going to be 23 taking information we've got through the run downs, 24 and developing our scenario models in this October, 25 November timeframe and then testing those models in NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
347 1 the November, December timeframe with the HRA teams.
2 And then we'll be documenting the results 3 of that testing and finalizing our ECA tool based upon 4 that feedback. I don't like to put exact dates on 5 that. It just depends on how extensive the feedback 6 is and what changes we want to make. But we're 7 shooting for the summer of 2020.
8 But in the interim, our tool will be out 9 there for public use and consumption in the next few 10 months. And so we'll give the, we can give our ACRS 11 counterparts a demonstration of the tool, even access 12 to it, once we get it acceptable to be put on the NRC 13 network.
14 CHAIR BLEY: Great, okay.
15 MEMBER REMPE: So out of curiosity, is 16 this effort part of a user need for IDHEAS, or is it 17 for a different user need that you're addressing?
18 MR. PETERS: We're faster than the user 19 need process. So inevitably, we have, as part of the 20 SRM, it was creating a set of methods for the Agency 21 to use. We actually don't have a user need for this, 22 but we have users who have expressed need, shall I 23 say.
24 (Laughter.)
25 MEMBER REMPE: So it's a part of the user NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
348 1 need for the SRM is why you're ---
2 MR. PETERS: It's basically part of the 3 SRM ---
4 MEMBER REMPE: A mission directive ---
5 MR. PETERS: Yes. Our users are full 6 onboard. They want this thing done. They're actually 7 a part of our team. Well, we already take their, yes, 8 I know, it's a good joke. We already do take their 9 money, and we're already developing this out of their 10 pile of money. So they're directing us.
11 This is the highest importance to our NRR 12 counterparts at the office level and the staff level.
13 And it's very high importance to the industry. So 14 everybody's onboard. We just don't have the formal 15 documentation calling this a user need, long story 16 short.
17 So that takes care of the testing, and 18 we'd love to get back with you and tell you what we've 19 gotten out of this in the near future.
20 CHAIR BLEY: That'll probably be next fall 21 though, given what you just had on the last slide.
22 MR. PETERS: Yes. If you want us towards 23 showing you what the finalized tool is, that would be 24 a good timeframe.
25 CHAIR BLEY: And the analyses, right?
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
349 1 MR. PETERS: Yes.
2 CHAIR BLEY: The test cases, yes.
3 MR. PETERS: Absolutely.
4 CHAIR BLEY: Okay.
5 MR. PETERS: So just a real fast recap to 6 try to minimize the time. So we've developed a 7 general methodology. All the white parts and the 8 orange parts of the previous slide that you thought 9 was informative, it models our underlying, the 10 cognitive processes underlying our human actions. And 11 it's application independent, so it's user-centered 12 under the general methodology.
13 We've enhanced the scientific basis. We 14 based it on the state of the art cognitive sciences.
15 We model activities in a teamwork and organizational 16 environment which is lacking in some of the HRA models 17 out there.
18 And our cognitive mechanisms are built-in, 19 and they explain why humans fail in these actions.
20 But we've developed enhanced guidance to help reduce 21 HRA variability. As we discussed, you're not going to 22 be able to completely eliminate variability based upon 23 inherent biases in the modelers themselves.
24 But you can help guide them down a path 25 where at least they can discuss why they have NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
350 1 different results from different analysts and 2 specifically point to where that variation exists.
3 And that's what we find helpful in the NRC 4 process, that if we talk with licenses about where our 5 differences are, we can then come to a common 6 agreement at that point and make those changes. And 7 here's our agreed upon consensus result.
8 And then we also have a built-in interface 9 with HRA data. That's where we saw a lot of lacking 10 in some of the other methodologies where they have 11 very highly structured qualitative analysis or maybe 12 not structured at all, just a kind of implicit 13 qualitative analysis, and have data capabilities. So 14 we've developed this interface between the qualitative 15 analysis and the data.
16 And I just want to express, we're not, I 17 can't say we're building anything new. Like I said, 18 the whole intent was to try to take pieces that 19 already exist out there. And we are enhancing those 20 in some way, but these concepts that we have are not 21 just magically thought up by our HRA team. This is 22 off of years and years of development and walking on 23 the backs of people who've made such great strides in 24 development earlier.
25 Other specific improvements, we talked NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
351 1 about qualitative analysis, operational narrative, 2 description of scenarios, progression and context. We 3 have guidance for searching, and documenting context, 4 and how to model that context with PIFs. We model 5 human failures.
6 You know, I guess I don't want to go 7 through these slides too much more. It's just taking 8 a lot of time. We talked a lot about time and 9 certainly analysis in our HEP quantification tool.
10 And then we have guidance for how to perform HRA 11 practice.
12 But I guess what I really want to get to 13 is the closure of SRM-M061020. And from a development 14 point in the Office of Research, what we think our end 15 point for development, from our perspective, will be 16 a successful completion of IDHEAS-G and IDHEAS-ECA.
17 These are the primary tools that we see 18 the NRC will be utilizing. So IDHEAS-G primarily 19 utilized by the Office of Research for building new 20 tools, IDHEAS-ECA is what's going to be primarily used 21 by our NRR and region-based customers, and even some 22 of our research customers who do ASP analyses.
23 We think this tool will be robust enough, 24 and we think the software tool will be robust enough 25 and liked enough that we'll want to look towards the NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
352 1 replacement of SPAR-H.
2 I may be putting my personal hope out 3 there. It's sometimes hard to get ingrained users to 4 use a new method and model. But the FLEX scenarios 5 will require them to use a new model, because SPAR is 6 not specifically built for that. It doesn't have that 7 capability. And this tool will.
8 What that puts us towards is what's the 9 future. And this is where I get throw the ACRS under 10 the bus. I think we've done over a decade of research 11 and work trying to move this path forward, answering 12 kind of what's on the SRM.
13 But at some point, the SRM is really 14 directed to the ACRS. And the ACRS will need to go 15 back and decide what does it want to do. Does it want 16 to endorse a method or set of methods for the Agency 17 to use? My hope is that the ACRS will choose to 18 endorse at least this general framework of IDHEAS and 19 say yes, we think this is a way to go forth with doing 20 that, doing HRA modeling for the future.
21 From our perspective, from an NRC staff 22 perspective, where we would go forth is a possibility 23 of issuing a regulatory guide. We've done all this 24 work on the HRA best practices, comparison, and 25 methods to best practices, benchmarking with respect NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
353 1 to international and U.S.. And some of the thoughts 2 around that are we have a lot of information about the 3 capabilities of the various HRA methods out there.
4 Where I personally think a regulatory 5 guide could be useful is where we could then spell 6 out, yes, this method is good here, this method is 7 good here, specifying clearly where the methods are 8 applicable and where they're not applicable, and then 9 stating how IDHEAS-G, we think, is applicable in these 10 scenarios. And ECA and our at-power applications are 11 good in most places.
12 So our concept is this is a possibility.
13 I understand that the Agency is not really big on 14 creating regulatory guides, but regulatory guides is 15 a way to tell our staff we think these are acceptable 16 methods to use in these scenarios.
17 CHAIR BLEY: Two things. You've said what 18 you'd like our letter to address. Do you see it 19 coming after you get IDHEAS-ECA in a state that you've 20 got something written down on paper about it?
21 And the second question is do you think 22 IDHEAS-G, as in the version we got, is all done? Or 23 are there some things you'd like to fix up before we 24 get to a full committee in a letter that would address 25 it?
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
354 1 MR. PETERS: Yes. I think the staff has 2 heard, I don't think ECA needs to be fully complete 3 before we could get a letter of endorsement from the 4 ACRS. Because inevitably, it's a down selection of 5 IDHEAS-G. If the ACRS says that IDHEAS-G is a way we 6 think is a good path forward for future method and 7 model development, we'll have that common framework 8 that the Commission has kind of directed us to have.
9 Now, the second question, do we want to 10 take IDHEAS-G and work on it a little bit more, I 11 think the staff does, based upon some of the feedback 12 that I've heard in the hallway conversations, that 13 there may be a couple little changes that we want to 14 make. And even the dependency comments might be 15 something we want to put into the report.
16 I've also heard in the back rooms that our 17 meeting has already been postponed from November to a 18 later date, so feasibly a February full committee 19 time. So that could give us some time to make the 20 final changes.
21 CHAIR BLEY: That would be enough time to 22 ---
23 MR. PETERS: I think so.
24 CHAIR BLEY: Because it's kind of hard --
25 (Simultaneous speaking.)
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
355 1 MR. PETERS: -- we also have a concurrent 2 deadline to get this ECA tool ready for use here in 3 the next, you know, next month or so. And so I don't 4 like to overload Jing too much.
5 CHAIR BLEY: The holiday's coming up.
6 MR. PETERS: Yes, Jing hasn't had a 7 Christmas in about ten years so it's, but I can work 8 with the ACRS staff and determine how far along we are 9 here in the near future.
10 CHAIR BLEY: Well, if you do that, I think 11 that'll be good, and keep us informed. The reg guide, 12 I'm not exactly sure what you're going to focus on 13 there, but ---
14 MR. PETERS: It's just a thought I'm 15 throwing out there. Because it's not, I'm not saying 16 a well thought out thought, but if we want guidance on 17 which methodologies to use, and which scenarios for 18 situations, it might be good, something coming from 19 the NRC staff versus, I guess, versus not having 20 something.
21 CHAIR BLEY: Okay. Well, we'll talk among 22 the members later. I think one thing that makes me a 23 little, two things that make me a little uncomfortable 24 are, one, we've got the historical data and the 25 analysis that Jing's put together that's now part of NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
356 1 IDHEAS-G, and as part of it, as a quantification tool.
2 And we just haven't seen much about that.
3 MR. PETERS: Yes.
4 CHAIR BLEY: And I'm not sure what you can 5 give us when that would boost our confidence in what's 6 there, because it's kind of amorphous. You know, 7 I've heard about it for years. Most of the members 8 probably haven't heard about it before. So that one 9 kind of lets me hang a little bit.
10 Seeing IDHEAS-ECA carried through to a 11 point that gives confidence that, yes, the spinoff 12 from IDHEAS-G into ECA is actually working, is another 13 thing that I think would give members more confidence.
14 And, you know, it sounds like you're not going to 15 have scenarios ready until the end of the year.
16 MR. PETERS: Yes, until November. I mean, 17 they're on a very tight time crunch for this thing.
18 CHAIR BLEY: And do we have anything done 19 in, you know, roughly that same timeframe that would 20 be a little more complete on IDHEAS-ECA?
21 MR. PETERS: The ECA tool has to be ready 22 in November also, kind of --
23 CHAIR BLEY: In November, okay.
24 MR. PETERS: -- when we start doing the 25 testing. Yes. So we would have a complete draft of NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
357 1 the user's manual, how to --
2 CHAIR BLEY: You would, okay.
3 MR. PETERS: Oh, yes. Yes, this is to say 4 I have a ---
5 CHAIR BLEY: And it would be available to 6 play with and see how ---
7 MR. PETERS: Absolutely.
8 CHAIR BLEY: Yes.
9 MR. PETERS: Yes, for the testing teams.
10 CHAIR BLEY: That helps me. We'll let the 11 other members chime in later.
12 MEMBER REMPE: So I have a question in 13 saying, well, the full committee meeting's been 14 delayed until February. Are we going to have another 15 subcommittee meeting, is the vision, before we go to 16 full committee? Because I think that might be 17 helpful.
18 CHAIR BLEY: Let's talk about that when we 19 go around the table. It's not in the plan right now.
20 And I'm not sure what it would do.
21 MEMBER REMPE: Having a manual for ECA 22 just discussed at the full committee meeting, and 23 there were a lot of questions raised today just all 24 brought up at a full committee meeting, knowing how 25 many years we've interacted, I would, again, we can go NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
358 1 around and talk about it at the closing comments, but 2 I just kind of thought maybe that was something that 3 you'd already thought, oh, we probably should have 4 one. But you're saying no, we haven't thought about 5 that at all?
6 CHAIR BLEY: I've looked at a lot of 7 computer manuals, and I'm not sure --
8 MEMBER REMPE: If you're going to bring up 9 that data --
10 CHAIR BLEY: -- any of them are worth a 11 committee meeting.
12 MEMBER REMPE: You've talked about the 13 data, and again, you're ---
14 CHAIR BLEY: But they're not going have a 15 data report until another couple of years.
16 MR. PETERS: I mean, when do you think, 17 Jing? I mean, Jing's always optimistic, so add about 18 six months to whatever she says.
19 CHAIR BLEY: I know. This was done about 20 ten years ago, ha, ha, ha.
21 DR. XING: Well, if we didn't completely 22 revise the IDHEAS-G report for five times, we would 23 have it done in 2014.
24 CHAIR BLEY: You had to revise IDHEAS-G.
25 DR. XING: Yes. For the data report, it's NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
359 1 going to take a lot of time to, like, I showed you two 2 examples today. For one big point I explained each 3 item. So you can imagine if I had a thousand of that 4 kind of paper, documented them to that level of 5 detail, it's going to take a lot of time.
6 CHAIR BLEY: Yes. That's not coming up 7 soon.
8 DR. XING: Yes, it's not.
9 (Simultaneous speaking.)
10 MEMBER DIMITRIJEVIC: You did tables when 11 I was --
12 DR. XING: Yes, I have tables --
13 MEMBER DIMITRIJEVIC: Yes.
14 DR. XING: -- I can share. But without 15 the detailed documentation, people say the paper only 16 say a number. They always want to have more.
17 MEMBER DIMITRIJEVIC: Background, yes.
18 DR. XING: Yes, backup.
19 MEMBER DIMITRIJEVIC: I have one question.
20 But you are applying this now to FLEX in two plants?
21 The plant, you integrate that FLEX in PRA, and would 22 that be acceptable? They're using different human 23 action methods in the, you know, internally. Whereas 24 the other PRAs were with FLEX.
25 DR. XING: Yes.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
360 1 MEMBER DIMITRIJEVIC: That would be okay, 2 I mean, to have ---
3 DR. XING: Yes. That's actually, in the 4 testing team they can talk about that. Some people 5 say, okay, I will use the ECA method. But I also want 6 simultaneously try a different method, see how it 7 works.
8 MEMBER DIMITRIJEVIC: Okay. But one is 9 they integrate the FLEX into that PRA, they will have 10 two methods in the PRA.
11 MR. PETERS: Yes. I mean, I don't want to 12 speculate too much on what industry is doing. I know 13 that they've had their own FLEX explicitation that 14 they used. And I think that they just came up with 15 human error probabilities that they want to plug into 16 their PRA models. So I don't even think they're even 17 going to say use a method. They're just going to put 18 in placeholder values that for ---
19 CHAIR BLEY: I saw something sometime back 20 from NEI, wasn't it?
21 MR. PETERS: Yes.
22 MEMBER DIMITRIJEVIC: Well, because you 23 wanted ---
24 CHAIR BLEY: Pretty fuzzy.
25 MEMBER DIMITRIJEVIC: You think that, I NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
361 1 would like you to say something about where that can 2 be used, right?
3 MR. PETERS: Yes.
4 MEMBER DIMITRIJEVIC: So that's a very 5 important thing. Most of the current facilities have 6 developed PRA, and not any reason to go to more 7 expensive method, with adding FLEX. And the question 8 is, is it okay to have two different methods?
9 So it's not applicable for the, it's not 10 likely to be applicable for current rate. For the new 11 coming plant, when you don't have a closing 12 advertised, this is like shooting the, you know, as 13 I say, chicken with a gun. I mean, you know, we don't 14 have enough information. And maybe we should use 15 simpler methods, simplest method to do HEPs for the 16 new plants.
17 So now the question is where would we 18 visualize this application? The one other thing it 19 could be, there will be plants coming to be the 20 completely new, you know, the not light water, new 21 designs where we are not familiar. And maybe this is 22 a good general method to be applicable for different 23 stuff.
24 DR. XING: Yes.
25 MEMBER REMPE: Perhaps a plant might be so NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
362 1 safe they say I don't need FLEX for many days ---
2 MR. PETERS: The level that I'm hearing 3 right now is they're saying they don't even need 4 operators. And so that's a whole nother ball game.
5 There are no human actions to model. So this is, yes, 6 I don't know the future. And that's the other part of 7 my branch is trying to work through some of those 8 issues.
9 But, yes, but where the general 10 methodology would be utilizable, I think it could be 11 utilized in new applications. We could use the at-12 power methodology for certain applications. So yes, 13 or even ECA, we could probably utilize in those new 14 applications also.
15 DR. XING: What our vision like is they 16 want to use IDHEAS-ECA. Even back to 2017, there was 17 a public meeting. Four plants indicated that they 18 wanted to send an application for modifying their PRA 19 model in which they gave an extra credit for using 20 FLEX equipment. For some point in outage, you use the 21 FLEX generator, diesel generator.
22 MEMBER DIMITRIJEVIC: I know, we 23 understand. They want to credit FLEX.
24 DR. XING: Yes, they --
25 MEMBER DIMITRIJEVIC: Based on some of the NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
363 1 recovery actions in the end of event tree. Now, the 2 question is, should they have a mix and match of 3 different methods, you know?
4 DR. XING: We cannot ---
5 MR. CHECK: Jing, I think I want to --
6 DR. XING: -- dictate what --
7 MR. CHECK: -- caution what we say here, 8 right. So we cannot dictate what method the industry 9 would use to calculate the HEPs or even to get the 10 data, what kind of data they would use for the 11 equivalent reliability.
12 We can have tools and methods available 13 for our staff to use. And we can make those available 14 for industry to use. But we're not supposed to be 15 dictating the tools and models that industry would 16 use.
17 DR. XING: Thank you.
18 MEMBER DIMITRIJEVIC: Yes. But you should 19 visualize the application. I mean, why would NRC use 20 this to review what?
21 MR. CHECK: Understood. So I think one 22 very potential application, obviously, is in FLEX.
23 Another one could be something like, you know, we had 24 a NUREG on the alternate shutdown panels for Appendix 25 R. And we can actually test that out with the NUREG NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
364 1 we have that used expert elicitation.
2 And so there were other control room 3 actions, we can also use it. We have a Level 3 model 4 that we can always, in the office, pilot IDHEAS-ECA 5 for those actions. And so we can have a lot of 6 different applications where we can now document the 7 potential users for them. Again, that would be what 8 the staff could see them being used, but there's no 9 way we could dictate if they're used that way.
10 MR. PETERS: And the other users that we 11 would have is all of our significant determination 12 process and ASP analyses that we do, even for in 13 control room type action. So there's a very good use 14 for IDHEAS-ECA.
15 CHAIR BLEY: Let me try something. We 16 could do this under the auspices of a subcommittee for 17 those who are interested from the member side. And 18 one would be to have an hour, hour and a half of demo 19 of the software tool.
20 Another would be to provide us with the 21 three tables, the HEP table, the PIF weight table, and 22 the PIF interaction table, with at least a key to what 23 the columns mean. And you must have acronyms or 24 something in there, you know, a key to what the 25 acronyms mean without having the report to go behind NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
365 1 it. And we can ask you some questions about it. So 2 that'll give us a sense of the completeness and the 3 kind of things that are --
4 DR. XING: Yes.
5 CHAIR BLEY: -- that are here.
6 DR. XING: I can provide that if you 7 accept that it's in progress.
8 CHAIR BLEY: I would. I'm thinking of 9 just, I assume those are computer files, three 10 separate computer files.
11 DR. XING: Files, but there are, like, a 12 lot of short names, for example, like a broken steam 13 generator operator. Other people may take it for a 14 different meaning.
15 (Laughter.)
16 CHAIR BLEY: Is there so much of that that 17 it would be hard to give a cross list of the names 18 that are in the table and what they really mean? Is 19 that a big deal?
20 DR. XING: I don't know. I have to check.
21 CHAIR BLEY: Okay.
22 MR. CHANG: Because right now, while I 23 work on both tables, sometimes I got lazy. Like, I 24 will just write a D. I know that means detection.
25 CHAIR BLEY: Yes.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
366 1 DR. XING: At another time, I wrote a word 2 D, it may mean something different. I have to do some 3 check. But I think you wouldn't mind to see part of 4 the table. I'll make sure, if I only give you a small 5 portion of each table, you can at least get a sense of 6 how the table looks.
7 MR. PETERS: I even want to clarify. It 8 probably just depends on when you're looking for the 9 meeting. Jing could feasibly get it all together 10 within a year, within a half year, something like 11 that. But if you're looking for it in the next month 12 or two, then that would be the more broken up tables.
13 CHAIR BLEY: Well, I guess the question 14 is, are there actually three computer files that are 15 the three tables?
16 DR. XING: I actually have a lot more 17 computer files, like ---
18 CHAIR BLEY: So they're in pieces.
19 DR. XING: -- when the file gets to long, 20 I got too lazy. I just break it down into small 21 files. Otherwise, when I'm looking for something I 22 have to ---
23 CHAIR BLEY: So it would be pretty hard to 24 show them to us? And I guess that's ---
25 DR. XING: Yes. But look like this, you NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
367 1 have at least 15 sub-tables when I classify the 2 information by each PIF. The 15 means I collapse all 3 the PIF related to the environment in one computer 4 file. So I can provide those tables.
5 CHAIR BLEY: Okay. Well, let's give it 6 some thought and talk with Chris. And then I'll work 7 with Chris on that.
8 DR. XING: And also I'd like to clarify 9 what your comment early. In IDHEAS-G, we on purpose 10 do not claim we have all this data here. We only, the 11 Chapter 6 is all about we use IDHEAS-G as a method.
12 So we establish this table. But putting the data in 13 the table is an ongoing and lengthy process 14 CHAIR BLEY: Yes, I got that.
15 MEMBER REMPE: When you talk about a 16 potential or potential IDHEAS for a future 17 subcommittee, I think and update on IDHEAS-ECA 18 progress would be also useful to discuss in such a 19 subcommittee. What do you think, since you're over 20 this thing, and not me?
21 CHAIR BLEY: I don't see them having much 22 to discuss in that area until --
23 MR. PETERS: Summer time.
24 CHAIR BLEY: Yes.
25 MEMBER REMPE: They're doing the plant NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
368 1 visits, I think you said, this week and the next week.
2 CHAIR BLEY: But it's going to be several 3 months before they work out all the scenarios they're 4 doing. And then it's going to be several more months 5 before they use the tool.
6 MR. PETERS: You know, they're looking at, 7 I think, the preliminary results in the December to 8 January timeframe. So, I mean, it's on a very tight 9 schedule. Because --
10 CHAIR BLEY: Wow.
11 MR. PETERS: Yes. So using --
12 CHAIR BLEY: But to be able to come here 13 and talk about them --
14 MR. PETERS: Yes. Getting into the proper 15 state of documentation, that's a different piece.
16 MEMBER REMPE: I really see it's a problem 17 to have nice view graphs and all that, but a 20-minute 18 status report, like, in this project, we've had 19 informal meetings where I thought we said you don't 20 have to have slides even. Or it maybe came to Level 21 3 --
22 (Simultaneous speaking.)
23 MR. PETERS: We've given draft reports and 24 really ---
25 CHAIR BLEY: You've had experience ---
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
369 1 (Simultaneous speaking.)
2 MR. PETERS: -- and we've had some pretty 3 bad experiences.
4 (Laughter.)
5 MEMBER REMPE: But just some sort of 6 status. I just am curious on how it's going. I mean, 7 you brought up what kind of experts do you have at 8 this, and what are you thinking. Off the wall, this 9 is where we're at would be kind of nice. But again, 10 it wouldn't be more than 15 or 20 minutes.
11 CHAIR BLEY: A slide show with ---
12 MR. PETERS: That wouldn't be hard at all.
13 And we could give you preliminary results in the slide 14 show, is what we're finding with the tool use and ---
15 CHAIR BLEY: I don't know when we have an 16 opening to have a subcommittee meeting if we have it.
17 And we'll talk some more.
18 MR. PETERS: Yes. I don't think that 19 would be a challenge.
20 MR. MOORE: Mr. Chairman?
21 CHAIR BLEY: Sir?
22 MR. MOORE: With regard to the opening, I 23 just point out that the NuScale schedule's going to 24 get really busy during the spring, especially during 25 the March, April, May timeframe. And that'll take up NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
370 1 many full committee meetings and some subcommittee 2 meetings. So it's just going to get busy during that 3 time period. We'll have to look for an opening.
4 MR. PETERS: We might be able to do a 5 February timeframe if you're just looking for a 6 presentation on the status.
7 CHAIR BLEY: The status and seeing maybe 8 a demo.
9 MR. PETERS: Oh yes, easy. A demo will be 10 easy by then.
11 CHAIR BLEY: And maybe seeing some of the 12 tables in a little more length to understand what's 13 there. I think that would be good if we can do that.
14 But then, you were also thinking about 15 when you'd have a cleaned up IDHEAS-G a little bit in 16 the areas you talked about. And is that kind of in 17 that same timeframe?
18 MR. PETERS: It seems like it.
19 CHAIR BLEY: Would that be before that?
20 MR. PETERS: Yes, it seems like that same 21 timeframe.
22 DR. XING: Yes.
23 MR. PETERS: But I need to go back and 24 really, we need to go back and really analyze it, but 25 it does seem like the February timeframe would be good NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
371 1 for a cleaned up IDHEAS-G.
2 DR. XING: It's doable.
3 MS. LUI: Okay, sorry.
4 CHAIR BLEY: No, just talk into it but 5 from very close.
6 MS. LUI: Sorry. I just wanted to point 7 out that this time the reason why we didn't do a live 8 computer mode because the software apparently only 9 exists on the research network server. So in order 10 for the staff to do a light demo, we have to be able 11 to connect with a research server and somehow get that 12 projected into, I mean, out to a screen in this 13 particular room. So in terms of ---
14 CHAIR BLEY: That's hard to do.
15 MS. LUI: So in terms, I mean, we'd 16 probably be here until midnight for that to get worked 17 out. So in order for that to actually work we need to 18 make sure that the software would be portable so that 19 we'll be able to set it up in each room. Otherwise, 20 we have to look for another venue for that to become 21 operational. I just wanted to point out that there 22 was some logistic issues that we all need to work out 23 in addition to a technical progress.
24 MR. PETERS: Okay, thanks, Chris.
25 MEMBER REMPE: One option might be if we NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
372 1 could get back into White Flint Three. Don't they 2 have an internet connection? And we've had 3 subcommittee meetings there, perhaps.
4 CHAIR BLEY: I'm wondering, I didn't want 5 to come back here at all in January, but somebody else 6 is making me come back in January if we could have a 7 subcommittee meeting then.
8 I'm not sure we even need the 9 subcommittee. If we could put this in a steering 10 committee meeting --
11 MR. PETERS: We could probably just give 12 you guys access to the tool on the network, and you 13 could play around with it if you like. And that's 14 another ---
15 CHAIR BLEY: That could work for me.
16 MEMBER REMPE: I would like to find out 17 more about what's happening with the ECA stuff.
18 MR. PETERS: Yes. I mean, as soon as we 19 get it cleared through our IT department to load it 20 into the NRC network.
21 CHAIR BLEY: I think a briefing on the ECA 22 stuff could happen at our full committee meeting.
23 MEMBER REMPE: There's going to be a lot 24 of members who aren't here. They're going to have to 25 come up to speed with what happens in the full NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
373 1 committee meeting. But it's up to you.
2 CHAIR BLEY: Yes.
3 MEMBER REMPE: Actually, January, adding 4 a half day, is not a big deal. I don't think we need 5 a full day but, I mean ---
6 CHAIR BLEY: Well, we'll talk some more 7 about that. I don't know if we need a subcommittee 8 meeting or not.
9 MR. PETERS: Okay.
10 CHAIR BLEY: But if we're aiming toward 11 February, that would mean we'd need revisions to 12 IDHEAS-G in January to support that.
13 MR. PETERS: And ruining Jing's Christmas.
14 Yes.
15 CHAIR BLEY: Or we could slip it for a few 16 months. Because it's true, Scott's right. We have 17 hard schedules on ---
18 MR. PETERS: Totally understand.
19 CHAIR BLEY: -- on NuScale which just 20 aren't going to yield any space. I don't know what it 21 does to you guys if we delay until summer.
22 MR. PETERS: Well, I mean, I don't think 23 it has much effect on us for delays. Because we are 24 going to progress along the path we laid forward, 25 completing IDHEAS-G, completing IDHEAS-ECA.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
374 1 CHAIR BLEY: If that's the idea, we have 2 full committee sometime after the bulk of the NuScale 3 stuff slides by. You have more stuff done.
4 MR. PETERS: Yes.
5 CHAIR BLEY: And we'll be in a much better 6 place to write a letter for you.
7 MR. PETERS: Jing wants to be done now.
8 DR. XING: I personally don't favor that 9 plan, because I have several other projects coming up.
10 If I wait that long, we talk that PIF early, if you 11 wait too long to do something, there's a high chance 12 of facing a new set of error in the report. So I 13 would rather forget about the Christmas thing, as 14 early as we can do it. I don't mind we do it on 15 December 23rd.
16 MR. PETERS: So we tell Jing that we're 17 doing it December 23rd, but we actually do it next 18 summer. That's a good plan.
19 (Laughter.)
20 MR. PETERS: But we'll figure it out.
21 We'll figure it out offline.
22 CHAIR BLEY: Talk some with Chris, and I'm 23 okay whichever way we go.
24 MR. PETERS: Okay.
25 CHAIR BLEY: I think February is probably NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
375 1 it. We can either do it in February or we so it in 2 summer.
3 MR. PETERS: Okay.
4 DR. XING: Okay.
5 CHAIR BLEY: And if there's a slot in 6 January to have either a subcommittee or some people 7 participate in a small working session with you guys 8 that might do it too. So see what works out.
9 You guys are all done?
10 MR. PETERS: I have one last slide real 11 quick. We don't have to talk about it. I'm just 12 telling you what's the future of our work once IDHEAS-13 G is done. And so obviously, we have a lot of 14 documentation, technical basis work which we've 15 already talked about, ECS implementation support.
16 So once you have a tool out there, a 17 method out there, once people start utilizing it, 18 there's a lot of support with plans to come forth with 19 it.
20 And so dependency, recovery, minimum joint 21 HEP, these are areas where we didn't do a lot of 22 innovation in the IDHEAS-G methodology. And we get a 23 lot of feedback from users that these are areas of 24 concern to them. And this is something that we will 25 be looking into in the near future.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
376 1 And Jing mentioned that we have a side 2 project looking into the challenges, just 3 characterizing the challenges of dependency right now.
4 And once we have that characterized, we'll go into 5 trying to figure out better ways to model it. But we 6 did get good feedback from the ACRS that maybe we 7 solved a lot of the problem here with the IDHEAS-G 8 methodology. But we were ---
9 CHAIR BLEY: We didn't talk much about 10 dependency. But to me, if you do, I think you've 11 called them base case scenarios, then you do variance 12 with them, they'd create difficult conditions. I 13 think that's really where dependency crops up in a big 14 way. And that links together things because of the 15 extreme context that can come up.
16 And I think that's a major piece of the 17 dependency, you know, the little tricks for saying 18 one person, two person kind of stuff is small 19 potatoes. Something that changes a scenario so that 20 we're looking at very difficult spots for the operator 21 gets us into the kind of things that have gone wrong 22 when it's real tough to recover.
23 And for me, that is where most of the 24 dependency is hiding, in those severe context events 25 which are variants on the others. Go ahead.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
377 1 MR. PETERS: And the last thing, data, 2 data, data.
3 CHAIR BLEY: Okay. We know we have the 4 June data over some years. We know SACADA is going to 5 chug along and keep creating data. What else are you 6 thinking?
7 MR. PETERS: So we're starting a data 8 program right now which, now that we have the base 9 structure of IDHEAS-G laid out, it will show us where 10 we're lacking. And we're going to be doing a cross 11 comparison of what data we need in IDHEAS-G, versus 12 what we have available, versus our current methods, 13 and what we have, I'm sorry, versus our current 14 databases. And then we'll be doing searches out there 15 for data in external counterparts, you know, like 16 military data, NASA data, things like that.
17 CHAIR BLEY: And this would, in principle 18 come into the Jing ---
19 MR. PETERS: This will come into Jing's 20 stack of thousands of reference documents that we 21 would then be back incorporating into the IDHEAS-G.
22 CHAIR BLEY: You still have stuff going on 23 with Halden?
24 MR. PETERS: We do. So Halden is one 25 thing. I'm also looking to propose a new task on our NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
378 1 international workgroups or the working group on the 2 interorganizational factors and WGRISK. We're looking 3 at putting a new CAPS out there that would get 4 international partners in to contribute data. So we 5 have interest from several other parties in that.
6 So the long term picture, I think, is that 7 data piece is really filling in the missing gaps in 8 HRA by finding what's out there and having a 9 coordinated program for getting it.
10 And then I always like to leave the TBD, 11 because I just don't know what's coming down the pike 12 next. Fukushima changed a lot for us from an HRA 13 perspective. So I don't know what the next big thing 14 will be from the NRC or industry's perspective.
15 A lot of it may be modeling, people that 16 just go out and push the easy button on these new and 17 advanced reactors with no human actions otherwise. I 18 mean, we just don't know.
19 CHAIR BLEY: Okay. Well, thank you.
20 MR. PETERS: Thanks.
21 CHAIR BLEY: I think we're going to go to 22 public comments, and then we'll go around the table.
23 But first we have comments from Mr. John Stetkar who's 24 here as a member of the public today. John, I think 25 you should stand for this.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
379 1 MR. STETKAR: I'll come to the microphone 2 and be clearly or whatever it says. First of all, my 3 name is John Stetkar. And full disclosure, I'm a 4 former member of the ACRS but speaking today as a 5 member of the public.
6 I'll try to keep this as brief as I can.
7 And I only have one page. So it'll be pretty quick.
8 I read the NUREG. I like a lot of parts of it. I 9 think it emphasizes the importance of the qualitative 10 analysis. I think it emphasizes very well the 11 importance of the scenario context. And it provides 12 a method to link the contextual elements of the 13 scenario to factors that, in principle, can be 14 evaluated objectively and related to human error 15 probabilities.
16 Four items that I'd like to highlight, 17 first of all, the NUREG does not contain any 18 methodology or guidance for performing a feasibility 19 assessment. And I think that's an important omission.
20 Performing a feasibility assessment is a fundamental 21 part of any human reliability analysis methodology.
22 And furthermore, earlier drafts of the 23 methodology I had, in my opinion, was pretty good 24 guidance. And it's just gone. So I'd hope that you'd 25 consider resurrecting that and adding it.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
380 1 The second comment is the methodology does 2 a fairly thorough job in identifying those detailed 3 cognitive failure modes. I found that really, really 4 useful. I'm thinking in terms of a potential 5 practitioner. You say people can redefine them, you 6 can go down to the details, you can stay at the middle 7 level, you can stay way up at the macrocognitive 8 functions. But I found the detail set was very good.
9 In a similar manner, you defined the 20 10 performance influencing factors, you develop a set of, 11 I think, pretty well thought out attributes for each 12 of those.
13 What I found lacking, and what the report 14 actually discusses, is that in many cases analysts, if 15 they use, for example, a three variant rating scale of 16 a performance influencing factor, it's good, it's 17 average, it's bad. Well, my average might be 18 different than your average. Or your good might be my 19 average.
20 It would seem helpful to me anyway if the 21 report made an attempt to provide a scale for each of 22 those performance influencing factors, in a sense 23 arrange from zero, you know, as good as it can get to 24 as bad as it can get, with some examples so that it 25 would help practitioners to anchor.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
381 1 You know, if you used a scale from zero to 2 ten, let's say, it could look at descriptions of 3 combinations of the attributes and say, oh okay, 4 that's kind of what they were thinking about for a 5 seven. So when I went in and said, well, this thing 6 is kind of a seven, we would be able to speak on at 7 least a similarity of scale. You might think it's a 8 five, and I might think it's a seven. But we're not 9 talking about the difference between, you know, a two 10 and an eight.
11 And looking, I know nothing about the ECA.
12 It's the first I've heard about it today. I think you 13 may have already done some of that in the background 14 of whatever is wired into the computer model for ECA.
15 You almost have to have done that. If it's done at 16 that fine structure level in the computer model, it 17 would strike me that you ought to be able to pull it 18 back out at the higher level in the general 19 methodology so that there is that set of information 20 in the general methodology.
21 Somebody wants to use ECA as an 22 application, then fine. If somebody wants to use a 23 different X, Y, Z application, they still have that 24 more comprehensive set of scaling thoughts. I don't 25 want to call them guidance, because that's a phrase NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
382 1 that I don't want to use in the general methodology.
2 So I just make that thought.
3 The third point is that I think that there 4 should be what I'd call a healthy skepticism about the 5 quantification model as it's presented in the general 6 methodology.
7 As a professional cynic, I look at that 8 quantification model and I say, yes, this is yet 9 another algorithm to relate ratings and weights of 10 performance influencing factors to human error 11 probabilities. And at that level, it's no different 12 than THERP, or HCR/ORE, or SLIM for anybody else's 13 methodology that has been roundly criticized as not 14 having a fundamental basis or an anchorage for data.
15 That anchorage might be there in your 16 analyses of the data. There might be good 17 justification for why those three performance 18 influencing factors are the predominant means of 19 determining the relationship with a human error 20 probability and why some linear combination of the 21 other 17 principle performance influencing factors are 22 a secondary effect, if you will.
23 But I just can't get it. I certainly 24 can't get it from what's in the general methodology, 25 and I don't get a good sense of confidence in it. I'm NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
383 1 concerned that, because it's presented in the NUREG 2 and now apparently has been implemented in ECA, that 3 it is now suddenly the recommended, NRC recommended 4 model for quantifying human error probabilities.
5 Because again, I personally think that a 6 healthy amount of skepticism is prudent until somehow 7 you can provide a lot more confidence in that model, 8 the linkages and the logical structure of that model.
9 The fourth comment that I wanted to make, 10 and this is a focused comment, it's on Example 1 in 11 Appendix M. It's the example on the, if I don't put 12 it down in notes, I don't remember, the H.B. Robinson 13 fire example. And in particular, I have some real 14 problems with the way the important human actions and 15 success criteria are defined in that example.
16 And in particular, the example says that 17 an important human action is to trip the reactor 18 coolant pumps. That must be done to prevent seal 19 damage. And furthermore, that it alone is a necessary 20 insufficient to prevent seal damage. I do not believe 21 that's true.
22 So I know that the abnormal operating 23 procedure at that particular plant instructs the 24 operators to trip the pumps. I believe that's to 25 prevent failure of a really expensive piece of NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
384 1 equipment in the plant and not to prevent damage to 2 the pump seals.
3 So I would urge you very, very strongly to 4 look at that particular requirement and the way it's 5 reflected in that example. Why is that important?
6 Because there's only two examples in the whole NUREG.
7 And people are going to look at those examples and 8 examine them in exceeding detail as NRC 9 recommendations for how to do HRA. So those examples 10 had better be crisp, and clear, and well thought out.
11 And if people have fundamental questions 12 about how you've defined the important human action, 13 how you've defined the success criteria in the context 14 of the only two examples you provide, you're going to 15 be in trouble.
16 So again, my recommendation is look at 17 that example pretty carefully, and in particular for 18 that set of success criteria. And with that, I 19 promised I'd be short.
20 CHAIR BLEY: But don't go away.
21 MR. STETKAR: Okay, I won't go away.
22 CHAIR BLEY: Out of all the things you had 23 to say, the first one doesn't sit great with me, that 24 they really need a feasibility study. Now, most of 25 the feasibility studies I've seen were done to NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
385 1 essentially see if there's enough time to get this 2 done. And they're doing that. So what is it you want 3 in a feasibility study that isn't getting picked up 4 in the process as it's laid out?
5 MR. STETKAR: Okay. Time is only one 6 element. If the former guidance was there, and in 7 fact, in NUREG 2199, he had power.
8 CHAIR BLEY: That's the fire?
9 MR. STETKAR: No, 2199 is the at-power 10 application --
11 CHAIR BLEY: Oh, okay.
12 MR. STETKAR: -- of the general 13 methodology. Lays out in that NUREG, the things to 14 think about for a feasibility study. Do you have 15 enough people, not are the people stretched, do you 16 just have enough people? Do you have the tools? Do 17 you have, time is one element that's looked at.
18 So there are, I don't remember how many 19 different ---
20 DR. XING: There are eight criteria.
21 MR. STETKAR: -- criteria, eight criteria.
22 I was going to say about a half a dozen.
23 CHAIR BLEY: There's nothing among them 24 though that doesn't need to be looked at when you 25 carry out the processes that have defined it.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
386 1 MR. STETKAR: That's true, but there are 2 many people who have defined, this is part of defining 3 human failure events in your model. There are many 4 people who have defined human failure events without 5 even thinking about the feasibility and then gone into 6 a lot of effort to analyze them and later found out 7 that they weren't feasible, that they wasted a hell of 8 lot of time building a model for something that 9 couldn't be done.
10 CHAIR BLEY: Good Lord.
11 MR. STETKAR: Because they hadn't --- it 12 doesn't require a lot of work.
13 CHAIR BLEY: So then you leave this up to 14 prevent overwhelming stupidity from causing problems.
15 MR. STETKAR: It's only my opinion.
16 CHAIR BLEY: It's true. Okay, thank you.
17 MR. STETKAR: And it was, as I said, it 18 was in ---
19 CHAIR BLEY: Yes.
20 MR. STETKAR: -- an earlier version of --
21 CHAIR BLEY: Yes. I probably recommended 22 to get rid of it, because they ought to look at that 23 automatically.
24 (Laughter.)
25 CHAIR BLEY: But I don't know if I did or NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
387 1 not.
2 MR. STETKAR: Okay. But that's ---
3 CHAIR BLEY: I headed elsewhere.
4 MR. STETKAR: Those are the four kind of 5 bigger ones.
6 CHAIR BLEY: Thank you.
7 MR. STETKAR: You're welcome.
8 DR. XING: Thank you.
9 CHAIR BLEY: Anybody else have anything 10 for John?
11 MR. PETERS: I just want to thank John for 12 all his hard work and guidance as an ACRS member.
13 MR. STETKAR: I was paid, you know, very 14 well as an ACRS member for all of that. I ain't paid 15 now. I'm going back to sit down.
16 CHAIR BLEY: You have a different sense of 17 being paid well than some of us.
18 (Laughter.)
19 MR. STETKAR: One final comment. I'm 20 retired. My income is precisely zero. So relative to 21 zero, you know.
22 (Simultaneous speaking.)
23 MR. PETERS: Yes. That's when those 24 tables will be useful, to figure out what the averages 25 are for people.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
388 1 CHAIR BLEY: We're getting the phone lines 2 open. Is there anybody else in the room who would 3 like to make a comment? We'd love to hear from you.
4 Okay, is there anybody on the phone line 5 who like to make a comment? If so, state your name 6 and give us your comment, please.
7 Thank you very much. As we go around the 8 table with the members, I'm interested in any general 9 comments you have but also do we need a subcommittee 10 meeting, and what's your feeling about that? And what 11 should be in it if we have one? Are you leaning 12 towards saying, yes, this is pretty good, and with a 13 little cleanup it could satisfy the SRM and our own 14 view of that. And I won't ask you if we need to write 15 a letter, because we're going to have to write a 16 letter on this one. Ron?
17 MEMBER BALLINGER: Well, I should say that 18 I now know about 75 percent of the acronyms.
19 CHAIR BLEY: He thought.
20 MEMBER BALLINGER: All kidding aside, I do 21 understand it now. But I'm kind of in favor of an 22 update kind of thing, of something related to this ECA 23 issue. Because I think that might just close ---
24 CHAIR BLEY: In a full committee or in a 25 subcommittee?
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
389 1 MEMBER BALLINGER: Well, I don't know.
2 You have to have discussions about how to do that.
3 CHAIR BLEY: they're going to work on 4 that, but you aren't ---
5 MEMBER BALLINGER: To me, it would sort of 6 close the loop on this a little bit.
7 CHAIR BLEY: Yes.
8 MEMBER BALLINGER: Anyway, and maybe I'll 9 know 80 percent or 90 percent of the acronyms by then.
10 CHAIR BLEY: Just a minute before we go 11 on. Harold?
12 MEMBER RAY: By the way, I calculated that 13 this presentation would have been an hour longer if 14 they had not used acronyms.
15 (Laughter.)
16 CHAIR BLEY: I thought you had something 17 to do this afternoon, Mr. Ray.
18 MEMBER RAY: Well, I think this is 19 essential to risk informed regulation generally. I 20 expressed myself earlier thinking that where I see its 21 great value is in what lies in the future as opposed 22 to application to current plants. But I may be wrong 23 about that. So I'll just leave it at that. But in 24 any event, there's a lot of progress been made, and I 25 think it's going to be valuable.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
390 1 As far as how to get from here to the next 2 step, I won't opine on that, Dennis. I don't feel 3 confident to that.
4 CHAIR BLEY: Thank you, Harold. Matt, and 5 that thank you for the extraordinary effort to get 6 here today. I appreciate it.
7 MEMBER SUNSERI: So it's seems like it's 8 good work to me. And, you know, I think it's well 9 thought out, and designed, and put together.
10 I guess as far as what more involvement we 11 have is, I'm unclear what our objective is. I mean, 12 what is the purpose of our going forward with reviews 13 if the reviews cite the aspects? Is it to produce the 14 operating manual? Or, you know, I'm just somehow not 15 clear on it. So I can't answer the question about 16 should we have a subcommittee or not. If maybe you 17 could clarify what our objective is.
18 CHAIR BLEY: Thank you. Joy?
19 MEMBER REMPE: So I think, in order to 20 have confidence in endorsing this, I would like to 21 have another subcommittee meeting to improve my 22 confidence in it.
23 And that involves seeing how the NUREG has 24 been cleaned up, it means understanding what's 25 happening from the people involved in this ECA NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
391 1 evaluations to say how is this really working, so I 2 have a little more confidence.
3 The data, it's also, if you're using it as 4 a quantification method, and you really are in it, I 5 think the data are important.
6 And what was the, oh, I'm less into the 7 software tool but, yes, I'd be curious to see it. But 8 it's not as essential to my endorsing the method.
9 Because I don't think we're endorsing the software 10 tool. But it would be interesting to see.
11 CHAIR BLEY: Thank you.
12 MEMBER REMPE: Oh, I do want to thank you 13 for your presentations today. Over the time I've 14 participated in these meetings, I've seen growth and 15 confidence. And I think that it helps with it. So 16 you should be commended on that too.
17 CHAIR BLEY: Dr. Kirshner?
18 MEMBER KIRCHNER: Yes, thank you. I think 19 the first thing I'd like to do is quantify the human 20 failure of these damn microphones switches. But it's 21 very high. I think it's about 3.967 with a 95/95 22 confidence. But it's very high for a simple switch to 23 watch how often everyone struggles with, one, 24 remembering to turn it on and, two, to actually 25 function. It would be a nice, rich, dataset, NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
392 1 actually.
2 So on a more serious note, thank you. I 3 think I was at one or two of these IDHEAS 4 presentations when I had first joined the committee.
5 And I see a lot of progress. I think I made most of 6 the points I would like to have made during the course 7 of the presentation.
8 So on the question of subcommittee versus 9 full committee, we almost have the full committee 10 here. I would be, whether it's subcommittee or full 11 committee, much interested in the ECA implementation 12 out at the two sites and how that exercises both the 13 methodology and your software. Because then it would 14 be, for me, a little more tangible in terms of results 15 and application.
16 So again, thank you for the presentations.
17 Thank you, Dennis.
18 CHAIR BLEY: Thank you. Vesna?
19 MEMBER DIMITRIJEVIC: Okay. First, we 20 always say thank you.
21 CHAIR BLEY: Microphone, please.
22 MEMBER DIMITRIJEVIC: Oh, my goodness.
23 (Laughter.)
24 CHAIR BLEY: I don't know, we got a lot of 25 data on Vesna today.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
393 1 MEMBER DIMITRIJEVIC: We always say thank 2 you, but I want to say special thank you. You have 3 been so responsive, and open, and brought everything 4 with you, and it was a pleasure. So thank you very 5 much. That was really a lovely presentation.
6 DR. XING: Thank you.
7 MEMBER DIMITRIJEVIC: I cannot really form 8 my opinion. This letter goes against everything I 9 believe in.
10 (Laughter.)
11 PARTICIPANT: Tell us how you really 12 feel.
13 MEMBER DIMITRIJEVIC: No, I don't know.
14 I have not have a chance to develop, I only have the 15 chances to participate, develop on one risk informed 16 methodology. And I thought during the develop to keep 17 things simple, because less you know, more complicated 18 we want to make things. But that's why we actually 19 have to keep them simple, in my opinion.
20 So I strongly believe I succeed to keep 21 fracture mechanics, macro models, all kind of things 22 out of this methodology so it can be easily 23 applicable. Because what was the main goal that was 24 important?
25 So from my perspective, we're making NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
394 1 guesses. We don't really know what human probability 2 error is. That's not scientific fact, but I cannot be 3 calculated based on question, so we make guesses.
4 Now, are we better when we make guesses on 5 ten staff instead of one? So like this is beginning 6 of the PRA. You know, we broke four, three, so we can 7 make, but we didn't make guesses. We had data. We 8 brought them to the thing where we have data.
9 Here, we are breaking it. So what we do, 10 maybe we make a better guess, but what we are 11 introducing its high uncertainty, and we are missing 12 interaction between those guesses.
13 So for me to think is that good or not, I 14 have really to think this over, look at examples and 15 things that, you know, to say okay. Because I'm also 16 not sure what is the goal of our Letter. It's not 17 going to say that he said that this matter is valid 18 and right, documenting well, blah, blah, blah, but 19 also where we suggest to be applied.
20 And I will maybe, when I'm convinced it's really 21 doing good things, I will suggest it to apply to some 22 important human actions related somewhere. That's why 23 I would use it to track something. But I'm not sure.
24 So what I was thinking, I don't even think 25 we need subcommittee meeting in my opinion. Because NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
395 1 if we have informal meeting, we check the data and 2 software so we get a little more familiar with that.
3 I think for me that will be enough. However I think 4 we need subcommittee meeting between ourselves, some 5 short lunch meeting, so that we understand what will 6 be goal of our letter.
7 CHAIR BLEY: Yes.
8 MEMBER DIMITRIJEVIC: What do we want to 9 say in the letter, and what do we envision saying?
10 Actually, I'm all new for this. I may even excuse 11 myself depending how the letter goes with that.
12 Because I am just coming on the end of the process.
13 But anyway, thank you very much.
14 DR. XING: Thank you.
15 MR. PETERS: And if we wait long enough, 16 there will not be any of the original commissioners 17 nor any of the ACRS members, or no staff members left 18 to answer from the original intent of the ---
19 (Laughter.)
20 CHAIR BLEY: We're getting close.
21 MEMBER BALLINGER: Don't mess with my 22 fracture mechanics.
23 (Laughter.)
24 CHAIR BLEY: Well, I too want to thank 25 you. Everybody who was here. And everyone did a NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
396 1 great job. The report has really progressed from what 2 it was to something that one can read from cover to 3 cover and understand what's going on there.
4 My opening comments I stand by. I think 5 it would be a big help to people to have some of that 6 simple clarification up front about qualitative 7 analysis and how that links through all these pieces 8 that are coming. And I think it's a big thing.
9 I'm on a different side than Vesna on 10 this, because I spent a lot of time looking at bad 11 events, things that have gone really bad, nuclear 12 plants and other places. And they don't go really bad 13 because an operator, you know, forgot to turn a 14 switch. They don't go really bad for all the simple 15 things we see and people usually recover from pretty.
16 They go when the context lines up to make it really 17 hard to get out of the situation. It fools you.
18 You know, some of the events that were 19 looked at in the empirical studies were designed to be 20 events that masked what was going on. It looked like 21 one thing, ended up being something else. The one 22 Jing focused on near the end was not that way. It was 23 pretty clear what was going on. But the speed at 24 which it got them in trouble was a bit surprising.
25 And, you know, there was another crew who did the NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
397 1 test case, and they did it fine, no trouble. So then 2 we had four cases that were in there.
3 If we're going to have a method that helps 4 us do risk assessment, and helps us find the things 5 that are risky, the kind of things that have turned up 6 when we've had really bad events, then I don't know a 7 way other than digging into the context to really help 8 you find them. And figuring out how to identify the 9 context is part of it. And you have a story in there 10 on that.
11 I think pointing so heaving to the 12 quantification method before the documentation exists 13 to show its quality and pedigree is awkward and could 14 leave us in a, you know, if we never get that report 15 done for one reason or another, we're sitting there 16 with numbers that we just don't have anchors to.
17 That goes back, as we heard, to many of 18 the early methods that claimed they had a basis, but 19 whenever you tried to dig into it, it kind of 20 vaporizes. And they're just numbers.
21 These could be very good, and the story, 22 the verbal story is good. But we don't have a way to 23 ---
24 MR. PETERS: So what Dennis is saying is 25 at least without the documentation we're at least as NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
398 1 good as the existing methods for publication, ha, ha, 2 ha. And we're working towards the documentation.
3 CHAIR BLEY: I think you've come a long 4 way. You know, you guys could talk with Chris and 5 work it out. I think, I'm thinking right now, without 6 more thought, that a subcommittee-like working 7 meeting, I don't think we need an official transcript 8 of it, but to let us go through the three areas that 9 people have talked about, I think it will be 10 worthwhile if we can do it.
11 And if we can see that and see a February 12 date for the letter, that's okay. If we can't, then 13 it's going to move off to summer. And you'll get more 14 details on that from Chris.
15 In any case, we will be waiting to hear 16 when it would be convenient or possible for you and us 17 to get back together again and hear those three 18 things. But you've got to have real confidence that 19 you'll have the tool in a fashion we could play with 20 it, that you'll have something you can show us on the 21 data, and that you'll have an update on the ECA 22 project and what it looks like.
23 And, you know, informal, without formal 24 presentations. Well, I think you'd need something 25 formal on the ECA status, but without just a lecture NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
399 1 but really a talk through, I think, would be adequate 2 on that. So we'll see how that comes together. And 3 once again, thanks a lot.
4 MR. PETERS: Okay, thank you.
5 CHAIR BLEY: We are adjourned.
6 (Whereupon, the above-entitled matter 7 went off the record at 6:20 p.m.)
8 9
10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
The General Methodology of an Integrated Human Event Analysis System (IDHEAS-G)
NRC Staff Presentation to the Advisory Committee on Reactor Safeguards Reliability and PRA Subcommittee September 18, 2019
Acronyms ACRS Advisory Committee on Reactor Safeguards HRA human reliability analysis ANS American Nuclear Society HSI human-system interface ASME American Society of Mechanical Engineers NPP nuclear power plant CFM cognitive failure mode NRC U.S. Nuclear Regulatory Commission CT critical task NRR Office of Nuclear Reactor Regulation D detection (macrocognitive function) PIF performance-influencing factor DM decisionmaking (macrocognitive function) PORV power-operated relief valve E action execution (macrocognitive function) PRA probabilistic risk assessment ECA event and condition assessment PZR pressurizer EOP emergency operating procedure RCS reactor coolant system FLEX flexible and coping (strategies) RO reactor operator gpm gallons per minute SACADA Scenario Authoring, Characterization, and IDHEAS Integrated Human Event Analysis System Debriefing Application IDHEAS-ECA Integrated Human Event Analysis System SG steam generator for Event and Condition Assessment SGTR steam generator tube rupture IDHEAS-G General Methodology of an Integrated SM shift manager Human Event Analysis System SRM staff requirements memorandum IHA important human action SS shift supervisor I&C instrumentation and control STA shift technical advisor HAMMLAB HAlden HuMan-Machine LABoratory std standard deviation HEP human error probability T teamwork (macrocognitive function)
HFE human failure event U understanding (macrocognitive function) 2
HRA RESEARCH PROGRAM OVERVIEW Sean E. Peters, Chief Human Factors and Reliability Branch Division of Risk Analysis Office of Nuclear Regulatory Research 3
Why are we here?
SRM-M061020 The Committee should work with the staff and external stakeholders to evaluate the different Human Reliability models in an effort to propose either a single model for the agency to use or guidance on which model(s) should to be used in specific circumstances.
4
Timeline of HRA Development PRA Policy IDHEAS At-Power Evaluation of Statement HRA Methods HRA Cognitive Basis HRA Good vs Good International HRA U.S. Empirical Study Practices Practices Empirical Study 2011 2020 1995 2005 2006 2009-2014 2016 2017 SRM M061020 Fukushima IDHEAS-G Daiichi IDHEAS-ECA HRA Toolbox 5
Timeline References
- PRA Policy Statement " (60 FR 42622) - The use of PRA technology should be increased in all regulatory matters to the extent supported by the state of the art in PRA methods and data, and in a manner that complements the NRC's deterministic approach and supports the NRC's traditional defense-in-depth philosophy.
- NUREG-1792 - Good Practices for Implementing [HRA] (ML051160213)
- NUREG-1842 - Evaluation of [HRA] Methods Against Good Practices (ML063200058)
- NUREG/IA-0216 - International HRA Empirical Study (ML093380283, ML11250A010, ML14358A254)
- NUREG-2127 - The International HRA Empirical Study: Lessons Learned from Comparing HRA Methods Predictions to HAMMLAB Simulator Data (ML14227A197)
- NUREG-2114 - Cognitive Basis for [HRA] (ML16014A045)
- NUREG-2156 - The U.S. HRA Empirical Study (ML16179A124)
- NUREG-2199, Vol. 1 - [IDHEAS] for [NPP] Internal Events At-Power Application (ML17073A041) 6
IDHEAS-G Development
- All evaluated HRA methods had strengths and weaknesses
- 1. Application scope
- 2. Scientific basis
- 3. Reduction of HRA variability
- 4. Use data for HRA 7
Intended Uses of IDHEAS-G
- High-level guidance for developing application-specific HRA methods or tools
- Platform to generalize and integrate human error data from various sources for HEP estimation
- Perform HRA for all nuclear applications
- Systematically analyze human events, including identifying human failures and root causes 8
IDHEAS-G Framework Cognitive basis framework (NUREG-2114)
Literature and human factors practices IDHEAS General Human error Existing HRA methods Methodology data (NUREG-2198) generalization Operational experience IDHEAS Internal At- IDHEAS for actions Other specific HRA Power Application outside the control room applications (NUREG-2199, Vol. 1) (e.g., IDHEAS for Event and Condition Assessment)
Plant-specific PRA models 9
Outline Part I. The General Methodology of an Integrated Human Event Analysis System (IDHEAS-G)
Part II. IDHEAS-G Applications Part III. Demonstration of an IDHEAS-G Application IDHEAS for Event and Condition Assessment (IDHEAS-ECA)
Jing Xing, Senior Human Performance Engineer Human Factors and Reliability Branch Division of Risk Analysis Office of Nuclear Regulatory Research 10
Overview of IDHEAS-G Cognitive Basis Structure Cognition Model PIF Structure Human error data Stage 1 Stage 2 Stage 3 Stage 4 Scenario Modeling of HEP Integrative analysis important human actions quantification analysis 11
Cognition ModelOverview Cognition Model for Human Performance and Reliability Important Cognitive Basis Structure Human Critical Action Tasks and Context of PIF Structure the Action 12
Cognitive Basis Structure Human action Task 1 Task 2 Task 3 and cognitive activities and cognitive activities and cognitive activities Under- Decision- Action Detection Teamwork standing making execution 13
Macrocognitive Functions
- Detection is noticing cues or gathering information in the work environment.
- Understanding is the integration of pieces of information in the work environment with a persons mental model to make sense of the scenario or situation.
- Decisionmaking is selecting strategies, planning, adapting plans, evaluating options, and making judgments on qualitative information or quantitative parameters.
- Action execution is implementation of the decision or plan to make a change in some physical component or system.
- Teamwork is various teams interacting and collaborating on tasks.
14
Cognitive Basis Structure (continued)
Macrocognitive Cognitive PIFs Processors functions mechanisms Processor - D1 Cognitive Detection mechanism PIF 1 Processor - D5 Cognitive Processor - U1 mechanism PIF 2 Understanding Processor - U5 Cognitive PIF 3 mechanism Task and Decision- Processor - DM1 cognitive making Cognitive PIF 17 activities Processor - DM6 mechanism Action Processor - E1 Cognitive PIF 18 execution mechanism Processor - E5 PIF 19 Processor - T1 Cognitive Teamwork mechanism Processor - T7 15
Detection Detection cognitive Detection processors Detection cognitive mechanisms activities D1. Initiate detection - D.a. Mental model of the Establish the mental cues model for information D.b. Perception of sensory to be detected information D2. Select, identify, and D.c. Attention attend to sources of information D.d. Working memory
- Detect cues D3. Perceive, recognize, D.e. Vigilance
- Acquire (gather) and classify information D.f. Information foliage information D.g. Pattern recognition D4. Verify and modify the outcomes of detection D.h. Shared cognition within a team D5. Retain, document/record, or D.i. Infrastructure for communicate the exporting the outcomes information detected Return to Cognitive Basis Structure Go to detection CFMs 16
Understanding Understanding cognitive Understanding processors Understanding cognitive mechanisms activities U1. Assess/select data
- Maintain situational U2. Select/adapt/develop awareness the mental model U.a. Data
- Assess status U3. Integrate data with the based on indirect mental model to U.b. Selection of data information generate the outcome U.c. Mental model
- Diagnose problems of understanding (situational U.d. Integration of data with and resolve conflicts awareness, diagnosis, mental model in information resolving conflicts) U.e. Working memory
- Make predictions or form expectations U4. Verify and revise the U.f. Shared cognition within for the upcoming outcome through a team situation iteration of U1, U2, development and U3 U5. Export the outcome Return to Cognitive Basis Structure Go to understanding CFMs 17
Decisionmaking Decisionmaking cognitive Decisionmaking processors Decisionmaking cognitive mechanisms activities DM1. Adapt the infrastructure of decisionmaking
- Make a go/no-go DM.a. Decisionmaking decision for a pre- DM2. Manage the goals and model specified action decision criteria DM.b. Data for
- Select among DM3. Acquire and select decisionmaking multiple options or data for strategies decisionmaking DM.c. Selection or judgment
- Change or add to a DM4. Make decision pre-existing plan or (judgment, strategies, DM.d. Cognitive biases strategy plans) DM.e. Deliberation or
- Develop a new DM5. Simulate or evaluate evaluation of strategy or plan the decision or plan decision DM6. Communicate and authorize the decision Return to Cognitive Basis Structure Go to decisionmaking CFMs 18
Action Execution Action Execution cognitive mechanisms E.a. Physical movement and motor skills Action Execution cognitive activities Action Execution processors E.b. Mental model of the actions and the systems to be acted on
- Execution of a E1. Assess action plan and E.c. Working memory cognitively simple criteria E.d. Attention action E.e. Vigilance E2. Develop or modify
- Execution of a action scripts E.f. Sensory feedback of motor cognitively complex movement action E3. Prepare or adapt E.g. Automaticity infrastructure for action
- Long-lasting action E.h. Action programming implementation E.i. Executive control
- Control action E4. Implement action E.j. Error monitoring and correction
- Fine motor action scripts E.k. Initiation of action execution
- Physically E5. Verify and adjust E.l. Spatial precision or accuracy of strenuous action execution outcomes action execution E.m. Timing precision of action execution E.n. Coordinate 19motor movement of action execution personnel Return to Cognitive Basis Structure Go to action execution CFMs 19
Teamwork Teamwork activities Teamwork processors Teamwork cognitive mechanisms T1. Establish or adapt teamwork infrastructure T2. Manage information T.a. Teamwork infrastructure T3. Maintain shared situational awareness T.b. Command
- Communication T.c. Control T4. Manage resources
- Cooperation T.d. Line of communication T5. Plan interteam
- Coordination collaborative activities T.e. Data processing and T6. Implement decisions and information commands management T7. Verify, modify, and T.f. Shared mental model control the implementation Return to Cognitive Basis Structure Go to teamwork CFMs 20
Cognitive Basis StructureFailure of Human Actions Macrocognitive Cognitive Processors PIFs functions mechanisms
- Failure of any macrocognitive function leads to the failure of the task and the human action.
- Failure of a macrocognitive function results from errors of one or more processors.
- Errors of a processor may occur if one or more associated cognitive mechanisms do not work properly or reliably.
- PIFs affect the capacity limits of the cognitive mechanisms.
21
PIF StructureIntroduction Cognition Model for Human Performance and Reliability Important Human Cognitive Basis Structure Action Critical and Tasks Context of the action PIF Structure
- Context are the conditions that affect human performance of an action.
- PIFs are used to model the context.
22
PIF StructureIntroduction (continued)
- The NRC staff established the following criteria to develop the PIF structure:
- Pertinence and comprehensiveness
- Independence
- Specificity
- Explainable
- Assessable
- Quantifiable 23
PIF StructureIntroduction (continued)
- Starting point for PIF structure development:
- All PIFs in existing HRA methods
- Also, considered the ASME/ANS PRA standard
- Developed new PIFs to address criteria discussed in previous slide 24
PIF Structure Environment Context and Situation System Personnel Task
- Accessibility/habitability *Staffing *Information availability
- System and I&C and reliability of workplace including *Procedures, guidance, transparency to *Scenario familiarity travel paths and instructions personnel *Multitasking,
- Workplace visibility *Training
- HSI PIF *Noise in workplace and
- Equipment and
- Teamwork and interruptions, and communication organizational factors distractions tools *Task complexity pathways *Work processes
- Cold/heat/humidity *Mental fatigue
- Resistance to physical *Time pressure and movement stress
- Physical demands PIF *Poor lighting in *Tools are difficult to use *Procedure is inadequate workplace *Tools are unfamiliar to personnel *Procedure is difficult to *Sustained high-demand attributes *Glare or reflection on *Tools do not work use cognitive activities Note: The PIF attributes physical structure *Tools or parts are unavailable *Procedure is available, *Long working hours shown are examples *Smoke or fog- *Document nomenclature does not but does not fit the *Sleep deprivation and correspond to the induced low visibility agree with equipment labels situation PIFs highlighted in red.
Links to cognitive See Section 3.4 and Appendix B mechanisms 25
PIF Structure Environment and Situation Context
- Consists of the conditions in personnels work environment and the situation in which actions are performed.
- Accessibility/habitability of workplace including travel paths
- Workplace visibility
- Noise in workplace and communication pathways
- Cold/heat/humidity
- Resistance to physical movement 26
PIF Structure Systems Context
- Systems are the objects of the actions, through which the work missions are achieved.
- System and I&C transparency to personnel
- Human-system interface
- Equipment and tools 27
PIF Structure Personnel Context
- Personnel are the people who perform the action and include individuals, teams, and organizations.
- Staffing
- Procedures, guidance, and instructions
- Training
- Teamwork and organizational factors
- Work processes 28
PIF Structure Task Context
- An action may consist of one or more discrete tasks.
- Information availability and reliability
- Scenario familiarity
- Multitasking, interruptions, and distractions
- Task complexity
- Mental fatigue
- Time pressure and stress
- Physical demands 29
Example PIF - Human-System Interface
- Definition: HSI refers to indications (e.g., displays, indicators, labels) and controls used by personnel to execute actions on systems.
- Attributes: HSI attributes depend on the specific interfaces used in an application. New HSI technologies may introduce additional attributes.
- The source of indication (e.g., indicators, labels) is similar to other sources nearby.
- The indications have low salience.
- Indications are confusing or nonintuitive.
- Controls are difficult to maneuver.
- Labels on the controls do not agree with document nomenclature.
- Controls are not reliable, and personnel are unaware of the problem.
30 Return to System PIFs
PIF Attributes
- They describe ways that the PIF challenges the cognitive mechanisms and increase the likelihood of human error.
- They were identified from cognitive and behavioral studies, as well as human error data from various sources.
- Attributes have the capability to link to existing human error data for HEP quantification.
- Using attributes to specify a PIF allows HRA analysts to examine, if not completely eliminate, the interdependency between PIFs.
- The list of PIF attributes is considered a living document
- New technologies and changes in conduct of operations can introduce new PIF attributes (e.g., use of computerized procedures) 31
- IDHEAS-G needs data to explain the following:
- Assessment of PIF attributes
- Quantification of the change in HEP when the PIF attribute changes
- Combination of multiple PIFs on the HEP
- Metadata analysis* was performed to gain insights into these three aspects.
- A metadata analysis combines data from multiple studies to arrive at a 32 conclusion or obtain insights into the answer of an inquiry.
PIF Effects on HEPs (continued)
- Assessment of PIF attributes
- Some HRA methods use discrete levels (e.g., low, medium, high) to model the state of a PIF
- Assessing, for example, task complexity as low, medium, or high does not link its impact on HEPs
- Cognitive studies assess specific PIF attributes
- Quantification of HEP should be based on PIF attributes and how they change from a baseline where there is no impact on the HEP 33
PIF Effects on HEPs (continued)
- Cognitive studies measure human error rates while systematically varying the attributes of one or more PIFs
- Measurements: no impact or low impact versus high impact on the error rate
- PIF weight factor:
=
- error rate at a given PIF attribute
- error rate when the PIF attribute has no or low impact 34
PIF Effects on HEPs (continued)
- Holistic
- Combination of individual effects
- Using the PIF weight factor, the NRC staff observed that the effect of the combination of multiple PIFs can be roughly estimated by adding the effect of individual PIF weights.
- Future research, including extensive metadata analysis, should be performed to establish the cognitive basis for combining the effects of multiple PIFs.
35
PIF StructureConclusion
- Covers all PIFs in the reviewed HRA methods and factors reported in the literature and nuclear-specific human event databases
- Pertinence and comprehensiveness
- PIF context
- Independence
- PIF attributes
- Specificity, explainable, assessable, and quantifiable
- Sharing the PIF structure should increase the consistency of different HRA methods and allow comparison of the HRA quantification results from different methods.
36
Cognitive Basis Structure and PIF Structure PIF Structure Cognitive Basis Structure Context Processor PIFs PIF Cognitive Processor Detection attributes mechanism Environment Cognitive and situation mechanism Processor Understanding System Processor Scenario Cognitive Decision- Task mechanism Processor making Personnel Cognitive Processor mechanism Action execution Task Cognitive Processor mechanism Processor Teamwork Cognitive mechanism 37
From the Cognition Model to HRA Cognition Model for Implementing the Cognition Model Human Performance and Reliability In an HRA process Cognitive Basis Structure IDHEAS-G process PIF Structure 38
AN INTEGRATED PROCESS FOR HRA WITH IDHEAS-G Jonathan DeJesus, Reliability and Risk Analyst Human Factors and Reliability Branch Division of Risk Analysis Office of Nuclear Regulatory Research NoteThe time uncertainty analysis is covered in 39 this section as part of the IDHEAS-G HRA process.
IDHEAS-G HRA Process Stage 1: Scenario analysis Scenarios Scenarios
- Develop operational narrative
- Identify scenario context
- Identify important human IHAs IHA 1 IHA 2 IHA 3 actions Critical Critical task 1 Critical task 2 Critical task 3 Stage 2: Modeling of tasks important human actions
- Identify and analyze critical Macro-tasks
- Identify applicable CFMs cognitive Action Detection Understanding Decisionmaking Teamwork Uncertainty
- Assess PIFs functions execution analysis and CFMs CFMs CFMs CFMs CFMs CFMs CFM1 PIF1 Stage 3 - HEP quantification HEPs due to CFMs
- Estimation of CFM2 PIF2
- Estimation of HEP due to HEP HEP CFM3 PIF3 time uncertainty Stage 4 - Integrative analysis Dependency CFM4 PIF4
- Document uncertainties
- Assess dependencies adjustment 40
Stage 1 Develop Operational Narrative Scenario narrative
- Overview of the event
- Beginning status
- Initiating event
- Initial conditions
- Boundary conditions
- Progression and end Baseline scenario state Timeline
- Date/time Operational
- System response, human narrative of a response, data for human event situation awareness, and notes Deviations from the Additional scenarios baseline scenario 41
Stage 1 Identify Scenario Context
- Search for the conditions that challenge or facilitate human performance in the scenario
- Search process should focus on conditions that affect the macrocognitive functions
- Environment and Situation
- Systems
- Personnel
- Task 42
Stage 1 Identify Important Human Actions
- Identification
- Actions required in the scenario progression to achieve the goal of the event (e.g., achieve safe and stable state)
- Search in baseline and deviation scenarios
- Definition
- Success criteria of the action
- Consequence of the HFE
- Cues and indications (including their timing)
- Relevant procedure guidance
- Time available 43
Stage 2 Identify and Analyze Critical Tasks
- Objective
- Identify critical tasks
- Characterize critical tasks
- Critical tasks
- Essential to the success criteria of the IHA
- Basic units of HEP quantification
- Failure of any critical task = Failure of the IHA 44
Stage 2 Identify and Analyze Critical Tasks (continued)
Scenario Human action IHA/HFE Task Critical Task and cognitive activities Under- Decision- Action Detection Teamwork standing making execution 45
Stage 2 Identify and Analyze Critical Tasks (continued)
- Identification Use task analysis
- Task Diagrams
- Help identify critical tasks and their relations, cues, and timing information
- Show success path and the order of tasks Detect a RCP S S Open FCV-626 S Start abnormal Enter AOP-018 1S or Trip RCPs alarm F F F 2F 3F 4F AOP = abnormal operating procedure RCP = reactor coolant pump F = failure S = success 46 FCV = flow control valve
Stage 2 Identify and Analyze Critical Tasks (continued)
- Breaking down IHA (or HFE) into critical tasks
- Use as few critical tasks as possible to represent the IHA
- Further break down the IHA only when the PIF attributes vary for different portions of the IHA
- An IHA should be broken down into critical tasks at a level that retains the context of the IHA and can be represented with macrocognitive functions
- Stop breaking down at a level where there are performance indications or empirical data available to inform the HEP 47
Stage 2 Identify and Analyze Critical Tasks (continued)
- Characterization
- Define scenario context and PIFs
- Identify the cognitive activities involved in the critical task; use taxonomy of cognitive activities
- Cognitive activities determine the macrocognitive functions and processors required for the critical task
- Basis for identifying the CFMs applicable to the task 48
Stage 2 Identify and Analyze Critical Tasks (continued)
Task characteristics Description Task goal The expected outcome of the task with respect to the success criteria of the action.
Specific requirements Specifications on the task goal such as timing requirements, criteria of task outcomes, and how the task goal should be achieved (e.g., monitoring parameters at a certain time interval, using secondary cues when the primary cues are not available, cooling down the RCS within a certain rate).
Cues and supporting The cues to initiate the task and key information needed to perform the task. A cue could information be an alarm, an indication, a procedure instruction, or others (e.g., an onsite report). The supporting information is in addition to the cue required to perform the task.
Procedures Available procedures, guidance, or instructions designed for the task.
Personnel Types of workers needed for the task, minimum staffing required, special skillset required.
Task support Job aids, tools, and equipment needed.
Location Places where the task is performed, special environmental factors about the locations.
Cognitive activities Cognitive activities that are involved in the task and that place demands on their corresponding macrocognitive functions.
Concurrent tasks Concurrent tasks (critical or noncritical) that compete for personnels cognition and resources.
Teamwork considerations Interteam collaborative activities required for the task and requirements for communication facilities (e.g., equipment, tools, devices).
49
Stage 2 Identify Applicable CFMs
- Completeness
- Non-overlapping
- Specificity and sensitivity
- Observability
- Since IDHEAS-G is a general methodology, the CFMs in IDHEAS-G should be independent of HRA application.
50
Stage 2 Identify Applicable CFMs (continued)
- Basic set of CFMs in IDHEAS-G
- High-level CFMs Failure of the macrocognitive functions
- Middle-level CFMs Failure of the processors of the macrocognitive functions
- Detailed CFMs Behaviorally observable failures of the processors
- NRC staff developed a reference set of detailed CFMs
- Specific HRA applications may develop its own set of detailed CFMs from the middle-level CFMs or adapt the reference set 51
Stage 2 Assess PIFs
- PIFs have a baseline where there is no impact on error rates
- Determine the applicable PIF attribute based on the context identified in scenario analysis and task characterization.
- When the context
- challenges task performance, it maps to applicable PIF attributes
- facilitates task performance, it moves PIF to baseline 52
Stage 3 HEP QuantificationOverview Time available
=
Time required CFMs PIF attributes HEP of = = (, , )
CFM 1 an IHA Critical
= (, , )
task 1 CFM 2 Critical = (, , )
task 2 CFM 3 Critical = (, , )
CFM 4 task 3 53
Stage 3 HEP QuantificationPc
- Probability of CFM, , can be estimated in one, or a combination, of the following three ways:
- Data-based estimation
- Expert judgment
- HEP quantification model 54
Stage 3 HEP QuantificationPc (continued)
- HEP quantification model PIF2 PIF1
- Information availability E-2 and reliability, task complexity, and scenarioE-3 familiarity
- Modification PIFs E-4
- Remaining 17 PIFs (orange and red curves) Attributes of Base PIFs 55
Stage 3 HEP QuantificationPc (continued)
- HEP quantification model (continued)
- 2. Linear combination of PIF effects 1
= 1 +
=1 HEP from Base PIFs Recovery factor; set PIF weight factors from to 1 unless data Modification PIFs suggest otherwise Recall = PIF interaction error rate at a given PIF attribute factor; set to 1 with error rate when the PIF linear combination attribute has no or low impact 56
Stage 3 HEP QuantificationPt
- In response to ACRS 0.25 Time Required comments, time Time Available uncertainty model was 0.2 developed probability density 0.15
- Convolution of time available and time 0.1 required distributions 0.05
- = >
- Pt is proportional to this 0 10 15 20 25 30 35 40 area time (minutes) 57
Stage 3 HEP QuantificationPt (continued)
- Time available ( )
- No credit for situations with excessive
- Experimental studies (e.g., [1], [2]) show that it does not have an impact on human error rates in task performance
- However, extra makes recovery possible, but does not guarantee it.
- Factors affecting distribution of
- Thermal-hydraulic calculations
- System time window and time delay for the cue
- Crew-to-crew variability in performing actions
- May affect of subsequent actions in the scenario
[1] H. Topi, et al. The effects of task complexity and time availability limitations on human performance in database query tasks, Int. J. Hum. Comput. Stud., Vol. 62, No. 3, pp. 349-379, March 2005. 58
[2] M. A. DeDonno, Time Pressure and Decision Making, Ph.D. Dissertation, Case Western Reserve University, 2009.
Stage 3 HEP QuantificationPt (continued)
- Time required (or needed) ( )
- Assumes action is performed at a normal work pace
- Time pressure is accounted for in Pc
- Factors affecting distribution of
- Crew-to-crew variability 59
Outline Part I. The General Methodology of an Integrated Human Event Analysis System (IDHEAS-G)
Part II. IDHEAS-G Applications Part III. Demonstration of an IDHEAS-G Application IDHEAS for Event and Condition Assessment (IDHEAS-ECA)
Jing Xing, Senior Human Performance Engineer Human Factors and Reliability Branch Division of Risk Analysis Office of Nuclear Regulatory Research 60
IDHEAS-G Applications
- 2. A platform for generalizing and integrating human error data to support HEP estimation
- 3. A methodology for developing application-specific HRA methods 61
- Event context - conditions that challenge or facilitate human performance Human IDHEAS-G
- Important human actions and critical tasks event process - potential failures and consequences
- Relevant CFMs and PIFs - How an human action may fail and what contributes to failure Piloting studies using IDHEAS-G for human event analysis:
- Fukushima Daiichi accident
- A set of seismic events
- Digital I&C events
- Halden simulator experiments 62
- 2. IDHEAS-G as a platform to generalize and integrate human error data
=
- Ideal world:
- The same task for a failure mode is repeated thousands of times with the same people under the same context;
- Do this for all possible contexts Failure modes # Occurrence Context Variety Well-defined Known, sufficient Context clearly Sufficient data for all failure modes number of task defined and failure modes and occurrences repeated contexts 63
Human error data: The real world
=
- Reality:
Good news - There is human error data in human error analysis, operational databases, cognitive and human factors experiments.
Not so good news X Failure modes not analyzed nor in great variety X Context undocumented and/or unrepeated X Limited coverage - limited failure modes / context tested X Not talking to each other 64
Generalizing human error data to inform HEP estimation Data source 1 Data source 2 Tasks Context Tasks Context Failure PIFs Failure PIFs modes modes A generic, adaptable set of failure modes and PIFs 65
IDHEAS-G Modeling human errors and estimating HEPs Human failure event Modeling HEP estimation Cognitive Human action / failure modes tasks (CFMs)
f(PIF1, PIF2, PIF3)
Performance Context influencing factors (PIFs) 66
Generalizing Human Error Data with IDHEAS-G CFMs and PIFs
- 1. Evaluate data 2. Interpret and 3. Consolidate data into IDHEAS-G source represent data Human Error Data Tables Cognitive failure HEP Table - Error rates Human action modes (CFMs)
/ tasks PIF Weight Table -
Performance Context influencing factors (PIFs) PIF Interaction Table - C 67
IDHEAS-G HEP Tables
- Cognitive activities involved in the tasks of a data source are mapped to corresponding IDHEAS-G CFMs of different levels.
The HEP Table documents the following dimensions of information for every data point:
CFMs Human error rate or HEP PIF attributes applicable to the data point Time information, (with or without time-constraint, adequate/inadequate time).
Brief narrative of the task or types of human failure in the data source Uncertainties in the data source and interpretation 68
Example 1: Human error rates in a flight simulator study The effect of incomplete information on decision-making in simulated pilot de-icing Task: Make decision on de-icing in flight simulation under icing weather Context: Pilots were provided with incomplete or inaccurate information for handling icing encounters; They performed time-critical decisionmaking.
Failure mode: Failure of decisionmaking.
PIFs: Incomplete or unreliable information (30%), time pressure Accurate and Accurate and Inaccurate
% error complete incomplete and complete information information information
% Stall 18.1 30 89
% recovery 26.7 63.8 75
- ERDM (adequate information + inadequate time) = 0.18
- ERDM (inadequate information + inadequate time) = 0.3
- ERDM (inadequate info + unreliable info + inadeq. Time) = 0.89 69
IDHEAS-G PIF Impact Table
- The PIF Impact Table documents the data points of which the human error rates or HEPs of a task are measured for two or more PIF states, typically a base or low-impact state and a poor PIF state with one or more attributes.
- The weight of a PIF between a poor and base state can be calculated.
A PIF sub-table contains the following dimensions of information:
The context or PIF description in the original data Relevant PIF attributes Error rates or HEPs PIF weight, calculated as the error rate for an attribute divided by the error rate at the base or low-impact PIF state.
CFMs associated with the data point Uncertainties in the data and mapping to PIF attributes 70
Example 2: The effect of long working hours (mental fatigue)
Reference:
Effects of sleep loss on team situation assessment (J. V. Baranski, et al. 2007)
Task: Team makes judgment of threat on a military surveillance task (situation assessment)
CFMs: Failure of Understanding - Incorrect situation assessment PIF attributes: Long working-hour; no feedback information, no supervision / peer-checking Results: Sleep loss affects assessment accuracy and time needed.
Data: Assessment error rate (%)
No sleep loss Sleep loss Full feedback 4.2 5.5 No feedback 4.5 6 Solo 6 8 Team 4.5 5.5 Generalized PIF weights:
- W (mental fatigue) = (5.5-4.2)/4.2=0.31
- W (no supervision / peer-checking) = (6-4.5)/4.5= 0.33 71
- W (no feedback) = (4.5-4.2)/4.2= 0.07
IDHEAS-G PIF Interaction Table
- The PIF Interaction Table documents data points in which there are human error rates measured as two or more PIFs vary independently and jointly.
- A data point in the table consists of human error rates in a 2x2 or larger matrix for individual or combined PIFs.
- The combined effect of multiple PIFs can be inferred from the data point.
PIF 1 Low impact High impact PIF weight PIF2 Low impact R11 R12 W1 =R12/R11 High impact R21 R22 PIF weight W2 =R21/R11 W3 =R22/R11 No interaction (linear sum): W3=W1 + W2 Multiplicative interaction: W3=W1 x W2 72
Example: PIF Impact Table on Multitasking/Interruption/Distraction IDHEAS-G Tasks and macro- Context in the Error Rate PIF PIF attribute functions original data (% of weight incorrect)
Multitasking - D - missing cue Single vs. dual task 2.8% vs 21% 7.5 intermingled D - missing changes Single vs. dual task 5% vs 20% 4 U - Wrong diagnosis Single vs. dual task 1% vs 37% 37 Excessively frequent or E- sequence task 0, 3s, 30s 2%, 4%, 16% 2, 8 long interruption E - nonsequence 0, 3s, 30s 2%, 2%, 2% 1 task Interruption by same No interruption vs 4% vs. 8% 2 task modality interruption Interruption by D No interruption vs 4% vs. 4% 1 different task modality weak interruption Distraction - irrelevant D - Monitoring Without vs with 2.5% vs. 7% 2.8 to the task target distraction 73
SACADA (Scenario Authoring, Characterization, and Debriefing Application)
The NRCs tool to collect operator performance data in simulator training to estimate the HEPs
- A software
- Standalone (not connected to the simulator or other systems)
- With an user interface and database
- Simulator data are entered before and after simulator training
- All data/information are entered by plant staff
- A tool for operator training, e.g.,
- Expedite communication of crew performance in training
- Reports: identify training focuses, trending crew or individual performance
- A database for HRA
- Plants: plant-specific data
- NRC: generic data (integration of plant-specific data) 74
SACADA Software Main Functions
- SACADA-1
- Design simulation scenarios (Authoring)
- Characterize performance challenges (Characterization)
- Debrief performance results (Debriefing)
- Output data (Reporting)
- SACADA-2
- SACADA-1
- Job Performance Measures (JPM)
- Written exams
- Crew notebook features
- Simulator Guide Preparation 75
76 An Example of SACADA Data on PIFs (total 7042 Data Points)
PIF Status 1 Status 2 Status 3 Status 4 Data Points Cognitive Type Response 1990 Planning (R.P.)
R.P. Basis Skill Procedure Knowledge 1282 R.P. Familiarity Standard Adaptation Anomaly 959 Required R.P. Uncertainty Clear Uncertain Competing Conflicting 861 Priority Guidance Workload Normal Concurrent Multiple 523 demand concurrent demand Time Criticality Expensive Normal Barely 408 adequate Communication Normal Extensive Extensive 226 Required Within MCR Onsite Miscellaneous Non-Standard Noisy Coordination Communicator 201 Background Unavailable Memory Demand
77 An Example of SACADA Data on PIFs (total 7042 Data Points)
PIF Status 1 Status 2 Status 3 Status 4 Data Points Cognitive Type Response 1990 Planning (R.P.)
R.P. Basis Skill Procedure Knowledge 1282 R.P. Familiarity Standard Adaptation Anomaly 959 Required R.P. Uncertainty Clear Uncertain Competing Conflicting 861 Priority Guidance Workload Normal Concurrent Multiple 523 demand concurrent demand Time Criticality Expensive Normal Barely 408 adequate Communication Normal Extensive Extensive 226 Required Within MCR Onsite Miscellaneous Non-Standard Noisy Coordination Communicator 201 Background Unavailable Memory Demand
Mapping SACADA Taxonomy to IDHEAS-G
- IDHEAS-G and SACADA taxonomies share the same cognitive framework so their elements can be mapped to each other (not necessarily a one-to-one mapping).
- The error mode statistics of SACADA is generalized to IDHEAS-G HEP Tables; data gathering SACADA context factors is generalized into the PIF Impact Table and PIF Interaction Table.
- The scopes of the functions in SACADA-1 are specifically for NPP control room actions performed by licensed crews, so SACADA error modes and context factors are mapped to only a subset of IDHEAS-G CFMs and PIF attributes.
SACADA-2 will provide data in the broader scope.
78
Summary of IDHEAS-G for Generalizing Human Error Data
- Human error data of various sources are generalized into three IDHEAS-G Human Error Data Tables: HEP Table, PIF Impact Table, and PIF Interaction Table
- SACADA data can be generalized into IDHEAS-G Human Error Data Tables.
- The generalized human error data can be integrated to inform HEP estimation.
- Data generalization is an on-going, continuous effort; Data integration should be periodically updated.
79
IDHEAS-G Applications
- 2. A platform for generalizing and integrating human error data to support HEP estimation
- 3. A methodology for developing application-specific HRA methods 80
IDHEAS-G as a Methodology for Developing Application-Specific HRA Methods
- Different HEP models may be adapted for specific HRA applications IDHEAS-G vs. application-specific method IDHEAS-G Application-specific method Applicable to all nuclear applications Specific for the application Comprehensive but low usability Concise and easy to use Referencing the Human Error Data Calculating HEPs of human actions Tables 81
Approach to Develop Application-Specific IDHEAS Methods
- Define the scope of application, requirements, and available sources for the intended use
- Keep the qualitative analysis the same as that in IDHEAS-G
- Develop application-specific sets of CFMs and PIFs and HEP models IDHEAS-G Application-specific method Scenario analysis
- Same guidance as in IDHEAS-G
- Specifications on guiding questions Human action and Task analysis
- Same guidance as in IDHEAS-G
- Specifications on developing task diagrams Failure analysis Failure analysis
- A basic set of CFMs
- A subset of CFMs
- The comprehensive PIF /
- A subset or an adapted set of PIFs /
attribute list attributes
- Guidance on estimating HEPs
- Time uncertainty analysis HEPs
- Time uncertainty analysis - same 82
IDHEAS-ECA Define IDHEAS-ECA by NRR users:
- Requirements: Easy to use, not over-burden HRA analysts
- Resources: IDHEAS-G HEP Tables, NRC 2018 FLEX-HRA Expert Elicitation DELTA between IDHEAS-G and IDHEAS-ECA - modeling failures and calculating HEPs IDHEAS-G IDHEAS-ECA A basic set of CFMs in three Five high-level CFMs (failure of D, U, DM, E, and T) levels of details 20 PIFs and their attributes
- All 20 PIFs preserved
- A compressed set of PIF attributes based on human error data available (combining attributes with similar effects)
IDHEAS-G Human Error Data Base HEPs and PIF weights integrated from IDHEAS-G Tables Human Error Data Tables, allowing HEP calculation for given failure modes and PIF attributes 83
HEP Quantification in IDHEAS-ECAPc
- HEP quantification model 1
= 1 +
=1 Base HEPs Recovery factor; set PIF weight factors from to 1 unless data Modification PIFs suggest otherwise PIF interaction Recall = factor; set to 1 with error rate at a given PIF attribute linear combination error rate when the PIF attribute has no or low impact 84
Integrated Base HEP Values and PIF Weights for IDHEAS-ECA Example: Base HEPs for Information availability and reliability PIF Attribute D U DM E T Inf1 Information is temporarily incomplete or not readily NA 5E-3 5E-3 NA NA available Information is moderately incomplete - a small NA 5E-2 5E-2 NA NA portion of key information is missing Information is largely incomplete NA 2E-1 2E-1 NA NA
- Key information is masked or indications are missing Inf2 Unreliable or uncertain NA E-2 E-2 NA NA
- Personnel is aware that source of information could be temporally unreliable
- pieces of Information change over time thus they become uncertain by the time personnel use them Moderately unreliable or uncertain NA 5E-2 5E-2 NA NA
- Personnel recognize information unreliable
- Conflicts in key information Key information is highly uncertain NA E-1 E-1 NA NA Extremely unreliable - Key information is NA E-3 E-3 NA NA misleading 85
Integrated Base HEP Values and PIF Weights for IDHEAS-ECA Example: Generalized PIF Weights for Multitasking/Interruption/Distraction PIF Attribute D U DM E T MT0 No impact 1 1 1 1 1 MT1 Distraction by other on-going activities that 1.2 - 2.8 1.1 1.1 1.2 - 2.8 1.2 - 2.8 demand attention MT2 interruption taking away from the main task 1.1 - 4 1.1 - 1.7 1.1 - 1.7 1.1 - 4 1.1 - 4 MT3 Concurrent visual detection and other tasks 2 - 10 NA NA NA NA MT4 Concurrent auditory detection and other tasks 10 -20 NA NA NA NA MT5 Concurrent diagnosis and other tasks NA 3-30 NA NA NA MT6 Concurrent Go/No-go decision-making NA NA 2 NA NA MT7 Concurrently making Intermingled, complex NA NA 5 NA NA decisions / plans MT8 Concurrently executing action sequence and NA NA NA 2.3 NA performing another attention/working memory task MT9 Concurrently executing intermingled or inter- NA NA NA 5 NA dependent action plans MT10 Concurrently communicating or coordinating NA NA NA NA 5 multiple distributed individuals or teams 86
IDHEAS-ECA Processsame as IDHEAS-G PIF attributes of every CFM Scenario for every CT PRA context and Calculate model list of (3.3.2) applicable Assess attributes of Analyze scenario context PIFs every applicable PIF (3.1.2)
(3.2.3)
List of Develop scenario narrative HFE and its applicable Calculate Develop scenario timeline definition CFM(s) for overall HEP (3.1.1) the CT(s) (3.3)
HFE and its List of definition Break down CT(s) Characterize the CT(s) and Identify and define HFE (3.1.3) HFE into CT(s) select applicable CFMs (3.2.1) (3.2.1 and 3.2.2)
HFE and its definition Estimate parameters and of distribution Analyze HFE timeline (subset of scenario timeline, if there (3.3.1)
Calculate are multiple HFEs in the scenario) Estimate parameters (3.3.1) of distribution (3.3.1) and and = mean and standard deviation of and = mean and standard deviation of CFM = cognitive failure mode PIF = performance-influencing factor = time required CT = critical task PRA = probabilistic risk assessment = time available HEP = human error probability = error probability due to CFMs HFE = human failure event = error probability due variability in and 87
IDHEAS-ECA Products
- IDHEAS-ECA report - including guidance, worksheets, base HEPs and PIF weights, and three full examples
- The worksheets document the analysis results and basis
- Worksheet A: Scenario analysis
- Worksheet B: Modeling of important human actions - Definitions of the actions and critical tasks
- Worksheet D: Summary of HEP calculation
- Worksheet E: Time uncertainty analysis
- Recommended use:
- Analyze the event and document the results in IDHEAS-ECA worksheets
88
Summary of IDHEAS-G Applications
- Analyzed and documented 10+ real and simulated human events
- 2. A platform for generalizing and integrating human error data to support HEP estimation
- Developed IDHEAS-G Human Error Data Tables and continued generalizing available data to the Tables
- 3. A methodology for developing application-specific HRA methods
- Developed IDHEAS At-Power Application (2017) and IDHEAS-ECA 89
Outline Part I. The General Methodology of an Integrated Human Event Analysis System (IDHEAS-G)
Part II. IDHEAS-G Applications Part III. Demonstration of an IDHEAS-G Application IDHEAS for Event and Condition Assessment (IDHEAS-ECA)
James Chang, Human Reliability Engineer Human Factors and Reliability Branch Division of Risk Analysis Office of Nuclear Regulatory Research 90
91 IDHEAS-ECA Implemented in the NRC HRA Tool Box
IDHEAS-ECASoftware
- Objective: To reduce the analysts effort of using the IDHEAS-ECA method
- Scope:
- To calculate the HEP of an HFE in an analysis
- Include dependency analysis (based on NUREG-1921) 92
IDHEAS-ECASoftware HEP Calculation Structure HFE Time (Pt) Cognition(Pc)
Critical Critical Critical Task Task Task Action Detection Understanding Decisionmaking Teamwork Execution Attributes of Performance Influencing Factors A switch that the users can open and close 93
94 IDHEAS-ECASoftware Calculate Pt
IDHEAS-ECASoftware Calculate Pt (continued)
- Pt is the probability of being unable to complete the task within the specified time
- Assume the operator responses are as expected
- Calculated by two parameters
- The time required for the operator to complete the tasks (Time Required)
- The system time window within which the tasks have to be completed (Time Available)
- Time Required and Time Available are distributions
- Monte Carlo sampling is used to calculate Pt (sample size: 1,000,000) 95
IDHEAS-ECASoftware Calculate PtApplied Distributions
- Time Required
- Normal distribution
- Gamma distribution
- Weibull distribution
- Percentile (5, 25, 50, 75 and 95 percentile)
- Time Available
- Normal distribution
- Gamma distribution
- Weibull distribution
- Percentile (5, 25, 50, 75 and 95 percentile)
- Constant 96
IDHEAS-ECASoftware Calculate Pc 97
IDHEAS-ECASoftware Calculate Pc (continued)
- IDHEAS-ECA implements 20 PIFs (the master set)
- Each PIF consists of a set of PIF attributes
- PIF attributes are the basic element to calculate Pc
- An HFE may contain one or more critical tasks
- HEP(HFE) = 1 - (1 - HEP(CT1)) * (1 - HEP(CT2))
- A critical task may contain one or more macrocognitive functions
- HEP(CT) = 1 - (1-HEP(CFM1)) * (1-HEP(CFM2))
- A CFM is only affected by a subset of PIFs and a subset of the PIFs attributes 98
PIFs Affecting Each CFM As Implemented in the Software Detection PIFs Understanding PIFs Decisionmaking PIFs Action Execution PIFs Teamwork PIFs 99
Task Complexitys Attributes for Different Macrocognitive Functions Detection Understanding Decisionmaking Action Teamwork 100
IDHEAS-ECASoftware Assessment of PIF Attributes applicable to a CFM
- Effect specification: A single value or a range
- Apply the effect of the PIF attribute with a range effect
- The software provides anchor points (from 1 to 10 with descriptions)
- The analyst picks a value between 1 and 10.
- The software calculates the corresponding effects on Pc.
101
Specify the Effect of Task Complexity Attribute C1 on Failure of Detection 102
103 IDHEAS-ECASoftware, Documentation
IDHEAS-ECAExample 1 SGTR Event (Based on Empirical HRA Study, NUREG-2156)
- Initial condition
- At 100% power operation
- A five-operator crew (SS, RO(2), STA, and SM) in the main control room
- Initiating event:
- A SGTR event of 500 gpm occurred in one SG
- Boundary condition
- All components and equipment function as designed
- Simulator training on SGTR event variation is routinely conducted for all crews
- Reactor trips automatically four minutes after the SGTR if not manually tripped 104
IDHEAS-ECA Example SGTR Event
- Scenario context
- Environmental and situational context
- System context
- Personnel context
- Task context
- Alarm pattern recognition in addition to EOP instruction 105
Time
- Three data points on the time the operator completed RCS depressurization based on simulator exercises
- Data: 36:00, 40:10, 43:10 (in mm:ss)
- Represented by 32.6(5%), 35.1(25%), 39.8(50%), 41.0 (75%), and 43.17(95%)
- It is estimated the RCS depressurization needs to be complete within two to three hours after the tube rupture to prevent PZR PORV from opening
- Represented by Normal distribution: Mean: 150; std: 10 106
Calculate Pt 107
CFMs Description
- Detection CFM: Fail to detect the key system information for the diagnosis to enter E-0 and E-3
- Understanding CFM: Correctly detected all key information but failed to understand this is a SGTR event or identify the broken SG
- Decisionmaking CFM: Not applicable because the procedure directs the action after having a correct understanding of the problem.
- Action execution CFM: failed to successfully execute the decided actions(isolate the broken SG and depressurize RCS)
- Teamwork CFM: Not applicable because not in a multiple team environment.
108
Calculate Pc 109
Documentation 110
IDHEAS-ECA TESTING AND HRA FUTURE WORK Sean E. Peters, Chief Human Factors and Reliability Branch Division of Risk Analysis Office of Nuclear Regulatory Research 111
IDHEAS-ECATesting
- NRC - Industry joint team
- Better understanding of human challenges in FLEX implementation
- Feedback for ECA method improvement
- 2 teams - Scenario developers, HRA practitioners
- 2 sites - Peach Bottom, Surry
- Site visits Sep/Oct 2019
- Scenario modeling - Nov/Dec 2019
- Testing documentation/final ECA tool - Summer 2020 112
HRA Improvement with IDHEAS-G
- A general methodology
- Models cognitive process underlying human actions
- Application-independent
- Enhanced scientific basis
- Based on state-of-the-art cognitive science
- Models cognitive activities in a teamwork and organizational environment
- Cognitive mechanisms are built-in and explain why humans may fail an action
- Enhanced guidance to reduce HRA variability
- Structured HRA process with detailed guidance for consistently analyzing an event and transparently documenting the results
- A built-in interface with HRA data
- Models cognitive elements of human behavior
- Data from different events or domains can be mapped to the cognitive elements 113
IDHEAS-G Specific Improvements Over Existing HRA Methods
- 1. Qualitative analysis (2015 ACRS recommendation)
- Guidance for operational narrative: description of scenario progression and context
- Guidance for searching and documenting context and how to model context with PIFs
- 2. Modeling of human failures
- Cognitive basis structure provides a structured and systematic understanding of success and failure of human actions.
- PIF structure is the most comprehensive model of conditions leading to human failure.
- Basic set of CFMs improves the completeness, independence, specificity, and sensitivity of modeling human failures.
114
IDHEAS-G Specific Improvements Over Existing HRA Methods (continued)
- 3. Time uncertainty analysis (2015 ACRS recommendation)
- Includes the contribution of the probability that an action cannot be completed in the available time window to the overall HRA results
- Models the effect of time on the HEP
- 4. HEP quantification model
- Based on data and findings in the cognitive literature
- Allows use of human error data, which is generalized with IDHEAS-G, to compute HEPs
- Moves HRA from expert judgment to data-based HEP quantification
- Allows continuous improvement as more data sources (e.g.,
SACADA) become available 115
IDHEAS-G Specific Improvements Over Existing HRA Methods (continued)
- 5. Guidance for HRA Practices
- Guidance on how to perform different HRA elements (e.g., HFE identification and task analysis) is provided
- The guidance improves HRA practices, as demonstrated in the piloting of human event analysis using IDHEAS-G.
116
Closure of SRM-M061020
- Complete IDHEAS-G
- Complete IDHEAS-ECA
- ACRS letter
- Regulatory Guide?
117
HRA Future Work
- Documentation and technical basis work
- IDHEAS-ECA implementation support and updates
- Dependency
- Recovery
- Minimum joint human error probabilities
- Errors of commission
- Data, data, data, data..
- TBD 118
BACKUP SLIDES 119
Detection Detection cognitive Detection processors Detection cognitive mechanisms activities D1. Initiate detection - D.a. Mental model of the Establish the mental cues model for information D.b. Perception of sensory to be detected information D2. Select, identify, and D.c. Attention attend to sources of information D.d. Working memory
- Detect cues D3. Perceive, recognize, D.e. Vigilance
- Acquire (gather) and classify information D.f. Information foliage information D.g. Pattern recognition D4. Verify and modify the outcomes of detection D.h. Shared cognition within a team D5. Retain, document/record, or D.i. Infrastructure for communicate the exporting the outcomes information detected Return to Cognitive Basis Structure Go to detection CFMs 120
Understanding Understanding cognitive Understanding processors Understanding cognitive mechanisms activities U1. Assess/select data
- Maintain situational U2. Select/adapt/develop awareness the mental model U.a. Data
- Assess status U3. Integrate data with the based on indirect mental model to U.b. Selection of data information generate the outcome U.c. Mental model
- Diagnose problems of understanding (situational U.d. Integration of data with and resolve conflicts awareness, diagnosis, mental model in information resolving conflicts) U.e. Working memory
- Make predictions or form expectations U4. Verify and revise the U.f. Shared cognition within for the upcoming outcome through a team situation iteration of U1, U2, development and U3 U5. Export the outcome Return to Cognitive Basis Structure Go to understanding CFMs 121
Decisionmaking Decisionmaking cognitive Decisionmaking processors Decisionmaking cognitive mechanisms activities DM1. Adapt the infrastructure of decisionmaking
- Make a go/no-go DM.a. Decisionmaking decision for a pre- DM2. Manage the goals and model specified action decision criteria DM.b. Data for
- Select among DM3. Acquire and select decisionmaking multiple options or data for strategies decisionmaking DM.c. Selection or judgment
- Change or add to a DM4. Make decision pre-existing plan or (judgment, strategies, DM.d. Cognitive biases strategy plans) DM.e. Deliberation or
- Develop a new DM5. Simulate or evaluate evaluation of strategy or plan the decision or plan decision DM6. Communicate and authorize the decision Return to Cognitive Basis Structure Go to decisionmaking CFMs 122
Action Execution Action Execution cognitive mechanisms E.a. Physical movement and motor skills Action Execution cognitive activities Action Execution processors E.b. Mental model of the actions and the systems to be acted on
- Execution of a E1. Assess action plan and E.c. Working memory cognitively simple criteria E.d. Attention action E.e. Vigilance E2. Develop or modify
- Execution of a action scripts E.f. Sensory feedback of motor cognitively complex movement action E3. Prepare or adapt E.g. Automaticity infrastructure for action
- Long-lasting action E.h. Action programming implementation E.i. Executive control
- Control action E4. Implement action E.j. Error monitoring and correction
- Fine motor action scripts E.k. Initiation of action execution
- Physically E5. Verify and adjust E.l. Spatial precision or accuracy of strenuous action execution outcomes action execution E.m. Timing precision of action execution E.n. Coordinate123motor movement of action execution personnel Return to Cognitive Basis Structure Go to action execution CFMs 123
Teamwork Teamwork activities Teamwork processors Teamwork cognitive mechanisms T1. Establish or adapt teamwork infrastructure T2. Manage information T.a. Teamwork infrastructure T3. Maintain shared situational awareness T.b. Command
- Communication T.c. Control T4. Manage resources
- Cooperation T.d. Line of communication T5. Plan interteam
- Coordination collaborative activities T.e. Data processing and T6. Implement decisions and information commands management T7. Verify, modify, and T.f. Shared mental model control the implementation Return to Cognitive Basis Structure Go to teamwork CFMs 124
Workplace Accessibility and Habitability
- Definition: Workplace has hardware facilities, physical structures, and travel paths to support personnel in task performance. Those should not impede personnel in performing required tasks.
- Attributes:
- Accessibility (travel paths, security barriers, and sustained habituation of worksite) is limited because of adverse environmental conditions, such as steam, high water, fire, smoke, toxic gas, radiation, electricity shock risk, and blocked roads.
- There is limited accessibility because of security system operation.
Doors or components that are normally locked require keys to unlock.
A fire or flood may cause electric security systems to fail locked.
- Habitability is reduced. Personnel cannot stay long at the worksite because of factors like radiation or earthquake aftershocks.
- The surface of systems, structures, or objects cannot be reached or touched (e.g., because they are hot).
- The worksite is flooded or underwater.
125 Return to Environment and Situation PIFs
Environment- and Situation-Related PIFs Other than Workplace Accessibility and Habitability PIF Attributes Workplace - poor lighting in workplace visibility - glare or reflection on physical structure
- smoke- or fog-induced low visibility Noise - loud noise in workplace impeding face-to-face communication
- noise in communication devices Cold, heat, - extreme cold humidity - extreme heat
- high humidity Resistance to - wearing heavy protective clothes and/or gloves personal or vehicle - slippery surface (e.g., icing) physical - traffic impeding vehicle movement movement 126 Return to Environment and Situation PIFs
System and I&C Transparency to Personnel
- Definition: Systems and I&C should be designed for personnel to understand their behaviors and responses in various operating conditions.
- Attributes:
- System or I&C does not behave as intended under special conditions.
- System or I&C does not reset as intended.
- System or I&C is complex, making it hard for personnel to predict its behavior in unusual scenarios.
- System or I&C failure modes are not transparent to personnel.
127 Return to System PIFs
Human-System Interface
- Definition: HSI refers to indications (e.g., displays, indicators, labels) and controls used by personnel to execute actions on systems.
- Attributes: HSI attributes depend on the specific interfaces used in an application.
New HSI technologies may introduce additional attributes.
- The source of indication (e.g., indicators, labels) is similar to other sources nearby.
- The source of indication is obscured or masked in many potentially relevant indications.
- The indications have low salience.
- Related information is spatially distributed or unsynchronized.
- Indications are confusing or nonintuitive.
- Secondary indications are not promptly available, or personnel are not aware of them.
- Controls are difficult to maneuver.
- Personnel do not anticipate the failure modes of controls and their impacts.
- Indications of states of controls are inadequate.
- There is confusion in action maneuver states.
- Controls provide inadequate feedback (i.e., lack of adequate confirmation of the action executed (incorrect, no information provided, measurement inaccuracies, delays)).
- Labels on the controls do not agree with document nomenclature.
- Controls are not reliable, and personnel are unaware of the problem.
128 Return to System PIFs
Tools and Parts Availability and Usability
- Definition: The tools, equipment, and parts assessed in an event include all the things needed to support personnel actions. They should be available and readily usable.
- Attributes:
- Tools are difficult to access or to use (e.g., lack of administrative control of tools).
- Tools are unfamiliar to personnel.
- Failure modes or operational conditions of the tools are not clearly presented (e.g., ranges, limitations, and requirements).
- Critical tool does not work properly because of aging, lack of power, incompatibility, improper calibration, lack of proper administrative control, or other reason.
- Tools or parts needed are missing or not available.
- Document nomenclature does not agree with equipment labels.
129 Return to System PIFs
Staffing
- Definition: Staffing refers to having adequate, qualified personnel to perform the required tasks. Staffing includes the number of personnel, their skill sets, job qualifications, staffing structure (individual and team roles and responsibilities). Adequate and qualified staff is normally expected.
- Attributes:
- shortage of staff (e.g., key personnel are missing, unavailable, or delayed in arrival; staff pulled away to perform other duties)
- lack of backup or lack of peer-check or cross-checking (e.g., an overseer or independent reviewer is not available)
- ambiguous or incorrect specification of staff roles and responsibilities
- inappropriate staff assignment (e.g., lack of skills needed)
- key decisionmakers knowledge and ability are inadequate to make the decision (e.g., lack of required qualifications or experience)
- lack of administrative control of fitness for duty 130 Return to Personnel PIFs
Procedures, Guidance, and Instructions
- Definition: This PIF refers to availability and usefulness of operating procedures, guidance, instructions, and protocols. Procedures should be validated for their applicability and usefulness. Following procedures should lead to the success of important human actions.
- Attributes:
- The procedure is inadequate.
- The procedure design is difficult to use.
- The procedure lacks details.
- The procedure is confusing.
- The procedure is available but does not fit the situation (e.g., it requires deviation or adaptation).
- The procedure is not available for skill-based tasks.
- The procedure is not available; thus, personnel have to find ways to perform the task based on their knowledge.
- The procedure is misleading.
131 Return to Personnel PIFs
Training
- Definition: This PIF refers to training that personnel receive to perform their tasks. Included in this consideration are personnels work-related experience and whether they have been trained on the type of the event, the amount of time passed since training, and training on the specific systems involved in the event. It is expected that adequate training is required for professional staff.
- Attributes:
- Training frequency is low (greater than 6 months between sessions).
- Training duration or the amount of training is not adequate.
- Training on procedure adaptation is inadequate. The training focuses on following procedures without adequately training personnel to evaluate all available information, seek alternative interpretations, or evaluate the pros and cons of procedural action plans.
- Training is inadequate on collaborative work process as a crew (e.g., inadequate supervision in monitoring actions and questioning current mission; inadequate leadership in initiating assessment of action scripts, facilitating discussion, and avoiding tunnel vision).
- Training or experience with sources of information (such as scope and limitations of data and information on the failure modes of the information sources) is inadequate.
- Experience in diagnosis (e.g., not being aware of and coping with biases, not seeking additional information, and not avoiding tunnel-vision) is inadequate.
- There are gaps in team knowledge and expertise needed to understand the scenario.
- There is inadequate specificity on the urgency and criticality of key information such as key alarms, system failure modes, and system design to the level of detail needed for responding to the situation.
- The training is inadequate or practice is lacking in the step-by-step completion of action execution.
- The training lacks practicality.
- Hands-on training on action execution is lacking (e.g., training consists of virtual training, classroom training, or demos only without hands-on practices).
- Experience or training is lacking on procedures, guidelines, or instructions for the type of event (e.g., use nonoperators to perform some actions outside the control room).
- The action context is infrequently part of training or personnel rarely perform the actions under specific context (greater than 6 months between performance).
- Personnel are not trained on the procedures or for the type of actions.
132 Return to Personnel PIFs
Teamwork and Organizational Factors
- Definition: Teamwork refers to everything affecting team communication, coordination, and cooperation.
- Attributes:
- Inadequate team information management
- Inadequate teamwork resources
- Distributed or dynamic operational teams
- Inadequate team decisionmaking infrastructure
- Team coordination difficulty
- Authorization difficulty
- Inadequate communication capabilities between teams
- Inadequate teamwork practices or drills together 133 Return to Personnel PIFs
Work Processes
- Definition: Work processes refer to aspects of doing work, supervision, management support, policies, and safetyconscious work environment at the organizational level.
- Attributes:
- poor work prioritization or scheduling
- lack of or ineffective instrumentation (e.g., prejob briefing) to inform personnel of potential pitfalls in performing the tasks
- lack of or ineffective instrumentation (e.g., supervision) for safety issue monitoring and identification
- lack of or ineffective instrumentation for safety reporting
- lack of or ineffective instrumentation for corrective actions 134 Return to Personnel PIFs
Information Availability and Reliability
- Definition: Personnel need information to perform tasks. Information is expected to be complete, reliable, and presented to personnel in a timely and user-friendly way.
- Attributes:
- Updates of information are inadequate (e.g., information perceived by one party who fails to inform another party).
- Information from different sources is not well organized.
- Conflicts in information
- Information updates are inadequate
- Different sources of information are not properly organized
- Personnel are unfamiliar with the sources or meaning of the information.
- Pieces of information change over time at different paces; thus, they may not all be current by the time personnel use them together.
- Feedback information is not available in time to correct a wrong decision or adjust the strategy implementation.
- Information is unreliable or uncertain.
- Primary sources of information are not available, while secondary sources of information are not reliable or readily perceived.
- Information is misleading or wrong.
135 Return to Task PIFs
Scenario Familiarity
- Definition: The scenario is familiar to personnel, with predictable event progression and system dynamics, and does not bias personnel in their understanding of what is happening.
- Attributes:
- Scenario is unfamiliar.
- A bias or preference for wrong strategies exists.
- Personnel are unfamiliar with system failure modes.
- Personnel are unfamiliar with worksites for manual actions.
- Plans, policies, and procedures to address the situation are lacking.
- Unpredictable dynamics
- Dynamic decisionmaking is required
- Shifting objectives 136 Return to Task PIFs
Multitasking, Interruptions, and Distractions
- Definition: Multitasking refers to performing concurrent and intermingled tasks. Distraction and disruption refer to things that interfere with personnels performance of their critical tasks.
- Attributes:
- excessively frequent or long interruption during the continuous performance of critical tasks
- distraction by other ongoing activities that are relevant to the critical task being performed
- concurrently detecting (monitoring or searching) multiple sets of parameters when the parameters in different sets may be related
- concurrently diagnosing more than one complex event that requires continuous seeking of additional data to understand the events
- concurrently making decisions or plans that may be intermingled
- concurrently executing intermingled or interdependent action plans
- command and control multitasking 137 Return to Task PIFs
Task Complexity
- Definition: Task complexity, also referred as cognitive complexity, measures task demand for cognitive resources (e.g., working memory, attention, executive control). Nominal complexity refers to the level of complexity that does not overwhelm personnel.
- Attributes:
- Detection criteria are complex.
- Detection overloading.
- Detection requires sustained attention.
- Cues for detection are not obvious.
- Multiple causes for situation assessment
- Relations of systems involved in an action are too complicated to understand
- Key information is cognitively masked
- The potential outcome of the situation assessment consists of multiple states and context (not a simple yes or no).
- Decisionmaking involves developing strategies or action plans.
- Decision criteria are ambiguous and subject to different interpretations.
- Multiple, intermingled goals or criteria need to be prioritized.
- Goals conflict.
- Decisionmaking requires integration of a variety of types of information with complex logic.
- Decisionmaking requires diverse expertise distributed among multiple individuals or parties who may not share the same information or have the same understanding of the situation.
- Competing strategies
- Personnel may need to unlearn or break away from automaticity of trained action scripts.
- Controlled actions may require monitoring of action outcomes and adjusting action accordingly.
- Action criteria are difficult to use
- Action requires out-of-sequence steps.
- Long-lasting, noncontinuous action sequences, or long-time gap between the cues for execution to initiation of the execution are necessary
- Action sequences are parallel and intermingled.
- Action execution requires close coordination of multiple personnel at different locations.
- Action execution requires long sustained attention 138 Return to Task PIFs
Mental Fatigue
- Definition: In the normal status of mental fatigue, personnel do not experience decrement of vigilance and abilities to perform complex cognitive tasks.
- Attributes:
- sustained, high-demanding cognitive activities
- long working hours with cognitively demanding tasks
- sleep deprivation, exposure to noise, disturbed dark and light rhythms, and air pollution 139 Return to Task PIFs
Time Pressure and Stress
- Definition: Time pressure refers to the sense of time urgency to complete a task, as perceived by personnel.
This sense of time urgency creates psychological pressure affecting personnel performance.
- Attributes:
- reluctance to execute an action plan because of potential negative impacts (e.g., adverse economic impact or personal injury)
- high time pressure because of perceived lack of adequate time to complete the task
- emotional stress (e.g., anxiety, frustration)
- physical stress (e.g., noise, disturbed dark and light rhythms, air pollution) 140 Return to Task PIFs
Physical Demands
- Definition: Physical demands indicate that a task requires extraordinary physical effort, such as twisting, reaching, dexterity, or strong force.
- Attributes:
- Action execution requires highly accurate fine motor skills, fine motor coordination, or skills of craft.
- Fine or difficult motor actions, such as installing or connecting delicate parts, must be performed.
- The task is physically strenuous (e.g., lifting heavy objects, opening or closing rusted or stuck valves, moving heavy things in water or high wind).
- There is resistance to motor movement (e.g., wearing heavy clothing; lifting heavy materials; opening or closing rusted or stuck valves; executing actions in water or high wind, in extreme cold or heat, or on unstable ground).
141 Return to Task PIFs
Taxonomy of Cognitive Activities Macrocognitive function Types of cognitive activities Detection
- Detect cues (through carefully monitoring, searching, inspecting, or comparing, etc.).
- Acquire information (checking, reading, communicating or chatting, computing, etc.).
Understanding
- Maintain situational awareness.
- Assess status based on indirect information.
- Diagnose problems and resolve conflicts in information
- Make predictions or form expectations for the upcoming situation development.
Decisionmaking
- Make a GO/NO-GO decision for a prespecified action.
- Select among multiple options or strategies.
- Make changes or additions to a preexisting plan or strategy (e.g., changes of personnel, criteria, subgoals).
- Develop a new strategy or plan.
Action Execution
- Execute complex actions.
- Execute simple actions.
- Execute fine motor actions.
- Execute strenuous dexterous actions.
Teamwork (within-team and
- Communicate.
between-team interaction)
- Coordinate (including command and control).
- Cooperate.
142 Return to Task Characterization
High-Level CFMs Macrocognitive Cognitive Failure Mode function Detection Failure of detecting cues/information Understanding Failure of understanding/assessing situation Decisionmaking Failure of making decisions/planning actions Action execution Failure of executing planned actions Teamwork Failure of interteam teamwork Return to Basic set of CFMs 143
Detection CFMs High-Level CFM: Failure of Detection Middle-Level CFMs Detailed CFMs for Detection Fail to initiate D1-1 Detection is not initiated (e.g., skip steps of procedures for detection, forget to check detection information, fail to realize the need to check information, no mental model for detection)
D1-2 Wrong mental model for detection (e.g., incorrect planning on when, how, or what to detect)
D1-3 Failure to prioritize information to be detected Fail to select, D2-1 Fail to access the source of information identify, or attend to D2-2 Attend to wrong source of information sources of information Fail to perceive, D3-1 Unable to perceive information recognize, or classify D3-2 Key alarm not perceived D3-3 Key alarm incorrectly perceived information D3-4 Fail to recognize that primary cue is not available or misleading D3-5 Cues not perceived D3-6 Cues misperceived (e.g., information incorrectly perceived; failure to perceive weak signals; reading errors; incorrectly interpret, organize, or classify information)
D3-7 Fail to monitor status (e.g., information or parameters not monitored at proper frequency or for adequate period of time, failure to monitor all of the key parameters, and incorrectly perceiving the trend of a parameter)
Fail to verify the D4-1 Fail to self-verify the perceived information against the detection criteria perceived D4-2 Fail to peer-check the perceived information information Fail to communicate D5-1 The detected information not retained or incorrectly retained (e.g., wrong items marked, wrong the acquired recording, and wrong data entry) information D5-2 The detected information not communicated or miscommunicated Go to components of detection Return to Basic set of CFMs 144
Understanding CFMs High-Level CFM: Failure of Understanding Middle-Level CFMs Detailed CFMs for Understanding Fail to assess or U1-1 Incomplete data selected (e.g., critical data dismissed, critical data omitted) select data U1-2 Incorrect or inappropriate data selected (e.g., failure to recognize the applicable data range or recognize that information is outdated)
Incorrect mental U2-1 No mental model exists for understanding the situation model U2-2 Incorrect mental model selected U2-3 Failure to adapt the mental model (e.g., failure to recognize and adapt mismatched procedures)
Incorrect integration U3-1 Incorrectly assess situation (e.g., situational awareness not maintained, and incorrect of data and mental prediction of the system evolution or upcoming events) model U3-2 Incorrectly diagnose problems (e.g., conflicts in data not resolved, underdiagnosis, failure to use guidance outside main procedure steps for diagnosis)
Fail to iterate the U4-1 Premature termination of data collection (e.g., not seeking additional data to reconcile gaps, understanding discrepancies, or conflicts, or failing to revise the outcomes based on new data, mental models, or viewpoints U4-2 Failure to generate coherent team understanding (e.g., assessment or diagnosis not verified or confirmed by the team, and lack of confirmation and verification of the results)
Fail to communicate U5-1 Outcomes of understanding miscommunicated or inadequately communicated the outcome Go to components of understanding Return to Basic set of CFMs 145
Decisionmaking CFMs High-Level CFM: Failure of Decisionmaking Middle-Level CFMs Detailed CFMs for Decisionmaking Inappropriate DM1-1 Incorrect decision model or decisionmaking process (e.g., incorrect about who, how, or when to make decision model decision, the decision model or process does not support the decision goal DM1-2 Incorrect decision criteria Incorrect goals or DM2-1 Incorrect goal selected priorities DM2-2 Unable to prioritize multiple conflicting goals Data are under- DM3-1 Critical information not selected or only partially selected (e.g., bias, undersampling of information) represented DM3-2 Selected information not appropriate or not applicable to the situation DM3-3 Misinterpretation or misuse of selected information Incorrect judgment or DM4-1 Misinterpret procedure planning DM4-2 Choose inappropriate strategy or options DM4-3 Incorrect or inadequate planning or developing solutions (e.g., plan wrong or infeasible responses, plan the right response actions at wrong times, fail to plan configuration changes when needed, plan wrong or infeasible configuration changes)
DM4-4 Decide to interfere or override automatic or passive safety-critical systems that would lead to undesirable consequences Failure to simulate or DM5-1 Unable to simulate or evaluate the decisions effects (e.g., fail to assess negative impacts or unable evaluate the decision/ to evaluate the pros and cons) strategy/plan DM5-2 Incorrectly simulate or evaluate the decision (e.g., fail to evaluate the side effects or components, or fail to consider all key factors)
DM5-3 Incorrect dynamic decisionmaking Failure to DM6-1 Decision incorrectly communicated communicate or DM6-2 Decision not authorized authorize the decision DM6-3 Decision delayed in authorization Go to components of decisionmaking Return to Basic set of CFMs 146
Action Execution CFMs High-Level CFM: Failure of Action Execution Middle-Level CFMs Detailed CFMs for Action Execution Fail to assess action E1-1 Action is not initiated plan and criteria E1-2 Incorrect interpretation of the action plan (e.g., wrong equipment/tool preparation or coordination)
E1-3 Wrong action criteria E1-4 Delayed implementation of planned action E1-5 Incorrect addition of actions or action steps to manipulate safety systems outside action plans (e.g., error of commission)
Fail to develop or E2-1 Fail to modify, adapt, or develop action scripts for a high-level action plan modify action scripts E2-2 Incorrectly modify or develop action scripts for the action plan Fail to coordinate E3-1 Fail to coordinate the action implementation (e.g., fail to coordinate team members, errors in action implementation personnel allocation)
E3-2 Fail to initiate action E3-3 Fail to perform status checking required for initiating actions Fail to perform the E4-1 Fail to follow procedures (e.g., skip steps in procedures) planned action E4-2 Fail to execute simple action E4-3 Fail to execute complex action (e.g., execute a complex action with incorrect timing or sequence, execute actions that do not meet the entry conditions)
E4-3A Fail to execute control actions E4-3B Fail to execute long-lasting actions E4-4 Fail to execute physically demanding actions E4-5 Fail to execute fine-motor actions Fail to verify or adjust E5-1 Fail to adjust action by monitoring, measuring, and assessing outcomes action E5-2 Fail to complete entire action scripts or procedures (e.g., omit steps after the action criteria are met)
E5-3 Fail to record, report or communicate action status or outcomes Go to components of action execution Return to Basic set of CFMs 147
Teamwork CFMs High-Level CFM: Failure of Teamwork T1 Fail to establish or adapt the teamwork infrastructure T2 Fail to manage information T3 Fail to maintain shared situational awareness Middle-Level T4 Inappropriately manage resources CFMs T5 Fail to plan or make interteam decisions or generate commands T6 Fail to implement decisions or commands T7 Fail to control the implementation Go to components of teamwork Return to Basic set of CFMs 148
IDHEAS-ECASoftware Dependency (based on NUREG-1921) 149 Return to IDHEAS-ECA software objective and scope
NRC HRA Tool Box Implementation of SPAR-H 150 Return to IDHEAS-ECA software objective and scope