ML18135A047

Transcript of Advisory Committee on Reactor Safeguards Reliability and PRA Subcommittee Meeting - April 4, 2018, Pages 1-255

ML18135A047
Person / Time
Issue date:	04/04/2018
From:	Christiana Lui Advisory Committee on Reactor Safeguards
To:
Lui C
References
NRC-3626
Download: ML18135A047 (255)

Text

Official Transcript of Proceedings NUCLEAR REGULATORY COMMISSION

Title:

ACRS Reliability and PRA Subcommittee Docket Number: N/A Location: Rockville, Maryland Date: April 4, 2018 Work Order No.: NRC-3626 Pages 1-255 NEAL R. GROSS AND CO., INC.

Court Reporters and Transcribers 1323 Rhode Island Avenue, N.W.

Washington, D.C. 20005 (202) 234-4433

1 1

2 3

4 DISCLAIMER 5

6 7 UNITED STATES NUCLEAR REGULATORY COMMISSIONS 8 ADVISORY COMMITTEE ON REACTOR SAFEGUARDS 9

10 11 The contents of this transcript of the 12 proceeding of the United States Nuclear Regulatory 13 Commission Advisory Committee on Reactor Safeguards, 14 as reported herein, is a record of the discussions 15 recorded at the meeting.

16 17 This transcript has not been reviewed, 18 corrected, and edited, and it may contain 19 inaccuracies.

20 21 22 23 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 www.nealrgross.com

1 UNITED STATES OF AMERICA NUCLEAR REGULATORY COMMISSION

+ + + + +

ADVISORY COMMITTEE ON REACTOR SAFEGUARDS (ACRS)

+ + + + +

RELIABILITY AND PRA SUBCOMMITTEE

+ + + + +

WEDNESDAY APRIL 4, 2018

+ + + + +

ROCKVILLE, MARYLAND

+ + + + +

The Subcommittee met at the Nuclear Regulatory Commission, Two White Flint North, Room T2B1, 11545 Rockville Pike, at 1:00 p.m., John Stetkar, Chairman, presiding.

COMMITTEE MEMBERS:

JOHN W. STETKAR, Chairman RONALD G. BALLINGER, Member DENNIS C. BLEY, Member VESNA B. DIMITRIJEVIC, Member WALTER L. KIRCHNER, Member JOSE MARCH-LEUBA, Member DANA A. POWERS, Member NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

2 HAROLD B. RAY, Member JOY L. REMPE, Member MATTHEW SUNSERI, Member DESIGNATED FEDERAL OFFICIAL:

CHRISTIANA LUI ALSO PRESENT:

PAUL AMICO, Jensen Hughes SUSAN COOPER, RES KAYDEE GUNTER, Jensen Hughes*

J. S. HYSLOP, NRR JEFF JULIUS, Jensen Hughes ASHLEY LINDEMAN, EPRI SEAN PETERS, RES MARY PRESLEY, EPRI MARK SALLEY, RES

• Present via telephone NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

3 T-A-B-L-E O-F C-O-N-T-E-N-T-S Opening Remarks....................................4 John Stetkar, ACRS Status of Joint NRC/EPRI HRA Project for MCRA Scenarios in Fire Events..........9 S. Cooper, RES Updates to NUREG-1921, Supplement 1...............15 A. Lindeman, EPRI M. Presley, EPRI Overview of NUREG-1921, Supplement 2..............54 S. Cooper, RES Supplement 2 - Decision to Abandon LOC Scenarios.....................................92 M. Presley, EPRI Supplement 2 - C&C Impact on Safe Shutdown........58 S. Cooper, RES MCRA HRA Project Status and Path Forward.........197 S. Cooper, RES A. Lindeman, EPRI M. Presley, EPRI Public Comment...................................200 Discussion Among Members.........................201 Adjourn..........................................204 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

4 1 P R O C E E D I N G S 2 1:00 p.m.

3 CHAIR STETKAR: The meeting will now come 4 to order. This is a meeting of the Reliability and PRA 5 Subcommittee, the Advisory Committee on Reactor 6 Safeguards.

7 I'm John Stetkar, Chairman of the 8 Subcommittee Meeting. ACRS members in attendance are 9 Ron Ballinger, Dennis Bley, Harold Ray. I believe 10 we'll be joined by Dana Powers and Matt Sunseri, Jose 11 March-Leuba, Walt Kirchner will join us, Joy Rempe, and 12 Vesna Dimitrijevic. Christiana Lui of the ACRS staff 13 is a designated federal official for this meeting.

14 The subcommittee will hear presentations 15 on the Joint NRC EPRI project on main control room 16 abandonment, human reliability analysis in fire 17 probabilistic risk assessments, in particular, the 18 draft quantification guidance.

19 The subcommittee will gather information, 20 analyze relevant issues and facts, and formulate 21 proposed positions and actions, as appropriate for 22 deliberation by the full committee.

23 The ACRS was established by statute, and 24 is governed by the Federal Advisory Committee Act.

25 This means that the committee can only speak through NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

5 1 its published letter reports. We hold meetings to 2 gather information to support our deliberations.

3 Interested parties who wish to provide 4 comments can contact our office requesting time, after 5 the meeting announcement is published in the Federal 6 Register.

7 That said, we also set aside some time for 8 spur-of-the-moment comments from members of the public 9 attending or listening to our meetings. Written 10 comments are also welcome.

11 The ACRS section of the USNRC public 12 website provides our charter, bylaws, letter reports, 13 and full transcripts of all open full committee and 14 subcommittee meetings, including slides presented at 15 these meetings.

16 The rules for participation in today's 17 meeting were previously announced in the Federal 18 Register. We've received no written comments or 19 request for time to make oral statements from members 20 of the public regarding today's meeting.

21 We have a bridge line established for 22 interested members of the public to listen in. To 23 preclude interruption of the meeting, the phone bridge 24 line will be placed in the listening mode during the 25 presentations and the committee discussions. We'll NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

6 1 un-mute the bridge line at a designated time to afford 2 the public an opportunity to make a statement or provide 3 comments.

4 A transcript of the meeting is being kept 5 and will be made available as stated in the Federal 6 Register notice. Therefore, we request that our 7 participants in this meeting use the microphones 8 located throughout the meeting room when addressing the 9 subcommittee.

10 Speakers should first identify 11 themselves, and speak with sufficient clarity and 12 volume, so that they may be readily heard. Please make 13 sure that the green light on the microphone in front 14 of you is on when you're speaking, and turn it off when 15 it's -- when you're not. Helps our transcript.

16 At this time please check all of your 17 cellphones, any other beeping devices, annoying 18 electronic equipment, and so forth, and turn them off.

19 Otherwise, we will smash them.

20 We will now proceed with the meeting, and 21 I call upon Mark Henry Salley of the NRC staff to begin.

22 Mark.

23 MR. SALLEY: Thank you, and good 24 afternoon. I'm Mark Henry Salley. I'm the Branch 25 Chief of Fire and External Hazards and the Office of NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

7 1 Research. I have Sean Peters with me. He's the Branch 2 Chief for Human Factors and Reliability Analysis.

3 I'm going to give you a little opening for 4 this today, and this is another NRC EPRI project that's 5 worked under the joint MOU. I think it's important 6 when we start out, that a number of successes we've had 7 working together with EPRI, in fire PRA, fire modeling, 8 post-fire safe shutdown circuit analysis, and in fire 9 HRAs, you're going to hear today.

10 So, there's a string of things. We work 11 together for a common goal, and we can pull our 12 resources, as you'll see today, and we can come out with 13 a high-quality product.

14 You're going to be hearing from Dr. Susan 15 Cooper from the Office of Research, as well as Ashley 16 Lindeman and Mary Presley from EPRI.

17 But before I turn it over to them, I just 18 wanted to give a little background here to kind of set 19 the stage for this. In 2005, again, the first big 20 project we worked with EPRI in the fire area, was 21 NUREG/CR-6850. 6850 had a little bit of HRA -- just 22 a bit, I believe, in task 12 -- and there was clearly 23 identified a need to do more detailed HRA in the 24 risk-inform performance-based environment.

25 So, we got together, we put a joint team NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

8 1 together. In June 2009, if you remember, they came to 2 the subcommittee and presented the work they had done 3 to-date. In December of that year we put a Federal 4 Register notice out with the draft report. We received 5 public comment on it.

6 We came back in April and September of 7 2011, and presented to you. And then we finally 8 published the report -- NUREG-1921, which is also 9 EPRI-1023001, in July of 2012.

10 So, that was a huge step forward for fire 11 HRA to use in the PRA applications. The one thing with 12 that report where it kind of stops short with the area, 13 and that was main control room abandonment. That 14 wasn't included in that first version of 1921.

15 And it's interesting, if you look at 16 deterministic regulations if you recall, which for the 17 plans that are the, as we call, Appendix R plans, 18 Section 3G and 3L talk about dedicated and alternative 19 shutdown, which is your main control and abandonment, 20 we needed something for the risk-informed 21 performance-based, and also we haven't looked at that 22 topic, if you think about it, since 1981.

23 So this was an area that clearly a lot of 24 work could be done. So, we got the group back together 25 to look at this, and -- check my dates here -- on NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

9 1 May 4th of 2016 we presented to the group, and August 2 2017 we published Supplement 1 to NUREG-1921. That's 3 also EPRI 3002009215.

4 And that, of course, is the qual- -- excuse 5 me, the qualitative HRA approach for main control room 6 abandonment.

7 So, the next piece to come up is what you're 8 going to hear about today. And this is the 9 quantitative piece. Before I turn it over, there's one 10 other area I'd like to point out in fire HRA, where we've 11 done quite a bit of work, that some of you may be 12 familiar with, and that's new reg 2180, which is the 13 incipient fire detection.

14 And as we got into that project, we looked 15 at it, we saw that HRA was a key element into doing the 16 incipient fire detection.

17 So, without further ado, I'd like to turn 18 it over to the group here, and get started.

19 CHAIR STETKAR: Green light on, talk, 20 green light off, don't.

21 DR. COOPER: All right, thank you, Mark.

22 Susan Cooper, Office of Research. I'm going to go 23 ahead and kick things off here with the presentation, 24 which, as Mark said, focuses on main control room 25 abandonment, and more particularly, we're going to NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

10 1 spend most of our time on the most recent 2 report -- draft report on quantitative guidance for HRA 3 main control room abandonment scenarios.

4 So, for today, in general, we're going to 5 provide an update to the overall project and, as I said, 6 we're going to want to focus on the second product of 7 the research for main control room abandonment.

8 This particular work is based on -- built 9 on the original fire HRA guidelines, which Mark 10 mentioned, which is NUREG-1921, that established the 11 basic process of how to do fire HRA.

12 And then, again, as Mark mentioned, 13 NUREG-1921 supplement 1, which is the qualitative HRA 14 guidance for main control room abandonment scenarios, 15 was published in August of 2017.

16 So, those two were the basis for the next 17 product on main control room abandonment. As with 18 those other products, the quantification guidance is 19 built on the experience of the authors, as well as other 20 experts or expertise that we accessed with respect to 21 developing NUREG-1921, performing NFPA 805 22 submittals, reviewing NFPA-805 submittals, so on and 23 so forth.

24 This is the last piece of the research that 25 we planned for main control room abandonment, and that NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

11 1 is the quantification guidance.

2 So, the agenda for today is to do a little 3 bit of a review of the project's history and background, 4 including introducing the project team. Because 5 supplement 1, which is on the qualitative analysis, did 6 undergo some changes after we presented to the 7 subcommittee back in May of 2016, we wanted to talk a 8 little bit about what those updates were, responses to 9 comments here from the ACRS.

10 In addition, we had a peer review and peer 11 review comments to respond to, as well. So we will 12 spend a little bit of time talking about those updates.

13 And then, as I mentioned, we like to spend 14 most of our time talking about supplement 2, the new 15 product, which you have in draft form, which is the 16 quantitative guidance for HRA for main control room 17 abandonment scenarios.

18 We're going to give a bit of an overview 19 of how we went about this project, and what guidances 20 we provide. In particular, we divide up, and we'll 21 talk a little bit more about this, divide up the 22 scenarios into what happens before abandonment occurs, 23 what happens during the decision to abandon the control 24 room, specifically on loss of con- -- for 25 loss-of-control scenarios. And then also, what NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

12 1 happens once you left the control room and you're now 2 implementing your safe shutdown strategy.

3 Then, we're going to wrap up with a quick 4 mention of our status future work and any closing 5 remarks. So, that's the agenda we propose for today.

6 Now --

7 MEMBER BLEY: Susan?

8 DR. COOPER: Yes.

9 MEMBER BLEY: Let me just make a little 10 minor, minor point here. I hope you folks have noticed 11 that you haven't been consistent through this report 12 in defining your phases 1, 2 and 3. There are quite 13 a few places where you say, phase 1 ends at the decision 14 to abandon the control room, which is old speak, I 15 think.

16 DR. COOPER: Okay. We will look for that, 17 but that is true. You're right.

18 MEMBER BLEY: There's at least two or 19 three --

20 DR. COOPER: That does need to be --

21 MEMBER BLEY: -- places that you'll 22 catch.

23 DR. COOPER: Okay. Okay, good. Thank 24 you. That is definitely an error that will need to be 25 corrected. Thank you.

NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

13 1 Okay, project history and background.

2 So, for -- specifically for the main control room 3 abandonment project, the fire events, the objective is 4 to provide HRA guidance specifically for those types 5 of scenarios where the fire conditions have been -- you 6 know, have created the instance where you have to leave 7 the control room -- the operation has to leave the 8 control room.

9 And there are two different cases that are 10 considered. There's what's called the loss of 11 habitability -- or LOH -- and that's where it's -- the 12 conditions in the control room are not such that people 13 can stay there safely.

14 So, that's one case. And the other one, 15 which is going to be of more interest for today, is going 16 to be loss of control, with the acronym of LOC. And 17 we'll talk a little bit more about those later.

18 We identified in NUREG-1921 when it was 19 published, that we needed to do more work in main 20 control room abandonment, that we had only gone so far 21 in that -- with that guidance.

22 And part of the reason why we have left it 23 until now is that it is a pretty complex topic with a 24 lot of plant-specific differences, a pretty wide range 25 of capabilities for the remote shutdown panel, which NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

14 1 also has an associated impact on the procedures and the 2 strategy that you have for shutting down, so that does 3 make it more difficult to develop generic guidance for 4 those -- some of those reasons.

5 So, this is the project team specifically 6 for supplement 1. You see on the NRC side Tammie 7 Rivera and I are on the team. Stacey Hendrickson could 8 not make it today, but she's is on one of the bridge 9 lines that Chris has set up, so I think it's separate 10 from the public line, so if we want to talk to her, we 11 can talk to her. John Wreathall is not available 12 today.

13 And then, on the upper side, both Mary 14 Presley and Ashley Lindeman are here today with me, and 15 Paul Amigo and Jeff Julius are there in the audience, 16 and Kaydee Gunter is also, I think, on the line with 17 Stacey, and Erin's not made it with us.

18 So, this is the same team we had for 19 supplement 1, with the exception of Nick Melly of NRC 20 is not on the team for supplement 2, which is more 21 HRA-related. He provided -- there is a stronger PRA 22 element to supplement 1, and they needed his support 23 there.

24 He's going to be a peer reviewer for 25 supplement 2. All right.

NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

15 1 MEMBER BLEY: Mr. Stetkar. We did not 2 write a letter on supplement 1, did we?

3 CHAIR STETKAR: We did not.

4 MEMBER BLEY: Are we planning to write a 5 letter on the whole thing, now?

6 CHAIR STETKAR: I think that's something 7 that we should discuss after this.

8 MEMBER BLEY: Okay. They haven't asked 9 for it.

10 CHAIR STETKAR: They haven't asked for it.

11 MEMBER BLEY: Okay.

12 DR. COOPER: Okay. We're now going to 13 discuss some of the updates to supplement 1, the 14 qualitative analysis, and Ashley's going to take over 15 on that.

16 MS. LINDEMAN: Okay. So, we were here 17 about two years ago in May of 2016, when we presented 18 our first draft of supplement 1. And since then, we've 19 done a lot of work. We took the comments from the 20 meeting here and addressed them, that we could. There 21 was other comments that we took that had a longer time 22 frame to resolve.

23 But, we went with a slightly revised 24 version in peer review in the summer of 2016. So, I 25 asked the peer reviewers to review for technical items, NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

16 1 and if it meets the needs of the intended users.

2 The peer review comprised a variety of 3 stakeholders, PRA practitioners from both the industry 4 and the NRC, HRA analysts, likewise from the industry 5 and the NRC. And then we also reached out to some 6 cognitive and behavioral science analysts.

7 So, we also got a lot of feedback from the 8 peer review, which, you know, we incorporated into our 9 final draft. So, EPRI published the report in August 10 of 2017, and as I understand that the NRC version of 11 the report is currently in publications.

12 There's been a -- several changes since 13 the version that you guys last formally received in May.

14 Various edits and clarifications from both the ACRS 15 comments and the peer review comments, so didn't want 16 to go through those kind of one-by-one, but wanted to 17 acknowledge that there were several places where we 18 found clarification wasn't good.

19 We also felt that we needed some way to walk 20 through the process. In NUREG-1921 there's a good 21 framework of the HRA process, and we felt since 22 supplement 1 used basis from 1921, we also felt it was 23 necessary to walk through and guide the analysts on what 24 pieces are relevant from 1921, and what pieces of the 25 HRA process are enhanced in supplement 1 in that NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

17 1 guidance.

2 We did add a dual unit timeline example in 3 section 7, which is the timing and timelines. I have 4 a picture of that and we'll go over some of the important 5 tidbits and links between actions and communications.

6 There was a --

7 MEMBER RAY: By dual unit you mean a dual 8 unit control room?

9 MS. LINDEMAN: Yes.

10 MEMBER RAY: Okay.

11 DR. COOPER: Dual unit shutdown.

12 MS. LINDEMAN: Yes.

13 DR. COOPER: But --

14 MEMBER RAY: Okay, there's so few dual 15 unit control rooms, that's why I asked. There's only 16 one that I know of, so I assume you're not talking about 17 that. Or both units are controlled from the same 18 control room.

19 DR. COOPER: Susan Cooper. Yes, it is for 20 a shared space control room.

21 MEMBER RAY: Okay.

22 DR. COOPER: So, it could be two separate 23 control boards within a shared space.

24 MEMBER RAY: Yeah.

25 DR. COOPER: There are other ones.

NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

18 1 MEMBER RAY: All right.

2 DR. COOPER: There are other ones.

3 MEMBER RAY: Right. Well --

4 MR. HYSLOP: On the west coast.

5 MEMBER RAY: I wasn't sure. As I said, I 6 just wanted to ask what dual unit, whether it meant two 7 units side by side, or --

8 MR. HYSLOP: This is J.S. Hyslop. Yeah, 9 from data 5, you know, just because they're not shared 10 from the same space doesn't mean that you don't have 11 a dual unit shutdown. There's some places that don't 12 have alternate shutdown boards and they just have 13 different panels and things like that, and so you have 14 multiple shutdowns going on at the same time. That's 15 possible.

16 MEMBER RAY: Yeah, absolutely. And I was 17 trying to distinguish between that and the shared 18 control room space.

19 MS. LINDEMAN: All right. One of the 20 other comments that we received from the ACRS was 21 addressing the variability and the capability of the 22 remote shutdown panel design. So yeah, there's a new 23 section that looked at a BWR and a PWR and the type of 24 function and capability and instrumentation in the RSDP 25 design, recognizing that each panel, each plant, is NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

19 1 weak.

2 CHAIR STETKAR: Be careful, by the way, 3 Ashley, and folks up front. You've received no 4 comments from the ACRS on supplement 1.

5 MS. LINDEMAN: Oh, I'm sorry.

6 CHAIR STETKAR: You've received comments 7 from individual members in a subcommittee meeting.

8 So, be careful, because we're on public record. The 9 ACRS has not made any recommendations formally on 10 supplement 1.

11 MS. LINDEMAN: Thank you for clarifying 12 that. Additionally, since the draft in 2016, we 13 continue development on appendix B, which is command 14 and control, so that was more sought out and more 15 developed at the time of publication.

16 We also added appendix Delta, which was 17 insights from operator interviews conducted both by the 18 NRC and the industry.

19 Supplement 1 also introduced three phases 20 of main control room abandonment, and I'll briefly 21 discuss each phase. We have phase 1, which is 22 associated with actions taken prior to the decision to 23 abandon.

24 So, in this scenario we're very early on, 25 and actions are directed, as usual, from the main NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

20 1 control using the EOPs. That's probably the last time 2 we're really going to spend on phase 1, as the 3 supplement 2 really focuses on the quantification 4 differences for phase 2 and phase 3.

5 Phase 2 is the decision to abandon, and 6 there's a little bit different terminology, depending 7 on if the habitability criteria are met, so you're on 8 a loss of habitability situation, or a loss-of-control 9 situation.

10 And phase 2 is really important for the 11 loss-of-control scenarios where there's time to make 12 the decision, the cues come in and they're developing 13 into a situation where the plant may not be controlled.

14 How we've defined the loss-of-control 15 situations is, it's not something like the multiple 16 scurrilous operations less than in fire PRA, where 17 there's a set of equipment involved in a flow diversion 18 or other -- a loss-of-control situation is 19 plant-specific, and it requires an iterative process 20 between the plant operations and the fire PRA and the 21 fire HRA team.

22 So -- so usually the PRA team would come 23 and define conditions that may require abandonment, and 24 they would discuss with the operators to see times or 25 conditions where fires can develop and needs would NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

21 1 exist.

2 So, this is a good example of the iterative 3 feedback between training and procedures.

4 CHAIR STETKAR: And actually, it's 5 presented in the guidance that way, that the PRA people 6 know everything about the scenario and they present it 7 to the operators and ask them, will you leave.

8 That's not the way the real world works.

9 The operators are faced with deteriorating conditions 10 in the control room. They will decide to leave or not.

11 The PRA has to deal with that decision. So, the PRA 12 people should never say, here is the scenario, will you 13 abandon the control room or not, from the scenario 14 perspective of the PRA. Because the PRA is a piece of 15 paper.

16 The operators should be faced with a 17 scenario of, here are the indications that you have, 18 here are the effects of the fire, including 19 non-safety-related stuff, including secondary stuff, 20 including coordinations with whatever's going on with 21 the fire brigade. And then ask the operators, under 22 these conditions when do you think you would leave?

23 When do you think you would leave. Not, will you leave?

24 If they say, never, that's good. If they 25 say, well, we don't think, you know, these conditions NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

22 1 are tenable, we would leave, then the PRA has to deal 2 with that, and that's a fundamental problem with the 3 way it's presented in supplement 1, because it's always 4 the PRA analyst presents a PRA scenario to the 5 operators, and asks them, what would you do during this 6 PRA scenario? Not, what would you do during a fire.

7 I just make that comment. It's too late 8 for us to have any impact on supplement 1. We might 9 be able to pull that into supplement 2 somehow.

10 DR. COOPER: Susan Cooper, and I'll let 11 Mary talk, too. We'll take a look at that again, but 12 I -- in the -- I think it's section 4 of supplement 1 13 that specifically addresses the decision to abandon, 14 although there's some discussion in section 3, which 15 is the PRA modeling part. But there are two principle 16 criteria for being able to credit loss-of-control 17 scenarios that we talk about.

18 And one is, that there is explicit 19 procedural guidance that obviously has to have been 20 developed not just from the PRA, but operations has to 21 have agreed to, and they made some decisions.

22 The other one is based on interviews, you 23 get a consensus opinion from the set of operators. And 24 we have even questions -- guidance questions -- on how 25 to interview.

NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

23 1 And at least it was not my intention -- nor 2 did I think it said that, but we'll look -- we'll 3 certainly look and make a correction if needed -- but 4 the idea was not to feed or lead the operators, per se.

5 It was -- because otherwise you're not 6 going to get the answer you want. You're not going to 7 say, well this is happening, aren't you going to leave, 8 then?

9 No, we don't do that. We try to understand 10 what kinds of situations they think. What's -- and it 11 might be system-based, it might be equipment-based.

12 It depends on plants. So, you're right, if it comes 13 across that way, we should fix it, but that was not what 14 we intended.

15 CHAIR STETKAR: Take a look at the 16 guidance for the operator interviews. My personal 17 opinion is they're very leading. They stress the 18 importance of the PRA expert stressing the importance 19 of the PRA scenario to the operators, so that the 20 operators know it's really important that they know 21 when to leave. Because this is the PRA scenario.

22 DR. COOPER: Okay.

23 CHAIR STETKAR: So, be careful.

24 MS. PRESLEY: Was that last -- sorry, this 25 is Mary Presley. Was that last comment with respect NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

24 1 to supplement 1 or supplement 2?

2 CHAIR STETKAR: Supplement 1, because the 3 guidance for the interviews is in supplement 1.

4 MS. PRESLEY: Okay, because there are some 5 specific things, and I think that's an area we've timed 6 for some improvement in supplement 2. So, it 7 wouldn't -- when you look at --

8 CHAIR STETKAR: I'm trying to keep 9 separate --

10 MS. PRESLEY: When we look at phase 2, if 11 you think we still are using that language, please let 12 us know, because I think -- hopefully, we'll have fixed 13 that.

14 CHAIR STETKAR: I quite honestly -- we'll 15 get to supplement 2. I didn't see it as much in 16 supplement 2, because supplement 2, you focus more on 17 quantifying human performance within the context of PRA 18 scenarios.

19 I saw it more in supplement 1, and in 20 particular, in the guidance for the interviews, where 21 I felt that they were too focused on a set of predefined 22 PRA scenarios in asking the operators their response 23 within the context of that PRA scenario.

24 You know, you've had a fire that's disabled 25 all feedwater, and you may or may not have control from NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

25 1 the main control room over the pressurizer PORVs.

2 Would you leave?

3 You know, it's not as blatant as that, but 4 that type of notion, rather than saying -- there's a 5 whole bunch of alarms. Half of the control room is 6 black. At what point do you lose confidence in your 7 indications, and decide to leave.

8 Regardless of whether or not you have main 9 feedwater, or can control the PORVs, because that might 10 not be the thing that prompts them to leave, you know.

11 We have to get back away from this specific PRA event 12 cut-set mentality and --

13 DR. COOPER: Okay.

14 CHAIR STETKAR: So, look at that guidance 15 for the interviews.

16 DR. COOPER: Okay. Yeah, and -- Susan 17 Cooper here. Yeah, the team members that are here 18 today, we had some discussion this morning about moving 19 forward with supplement 2 and what improvements we 20 would do next.

21 And we are examining possibilities of 22 providing a roadmap that -- even better than it does 23 now, shows where 1921 guidances is overarching.

24 Supplement 1 is good enough, supplement 2 has done a 25 better job, and whatever.

NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

26 1 So, this feedback is very helpful. So, 2 thank you. We'll do what we can with it.

3 MEMBER SUNSERI: So, I'm just getting 4 involved and I haven't looked at all the previous stuff.

5 So, let me just ask for my confirmation here. We're 6 talking about situations not solely associated with 7 habitability of the control room, but situations where 8 it may not be the best decision to operate the plant 9 from the main control room. Is that right?

10 MS. LINDEMAN: That is correct, yeah.

11 DR. COOPER: That is correct.

12 CHAIR STETKAR: These, in principle, 13 could be fires that do not occur inside the main control 14 room itself. Could be out in some instrument and 15 control paneled room that disables, you know -- I'll 16 use the technical term, snarky. Makes all of your 17 displays look kind of snarky.

18 MEMBER SUNSERI: Yeah. No, I get this 19 spurious in -- stuff like that. Yeah, I get it.

20 CHAIR STETKAR: Open circuit's hot, you 21 know.

22 MEMBER SUNSERI: I just was wanting to 23 make sure I understood.

24 CHAIR STETKAR: Or it could be a fire in 25 the control room itself.

NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

27 1 MEMBER SUNSERI: You could be having those 2 symptoms before you even know there's a fire.

3 MS. LINDEMAN: Yeah, okay. All right, 4 and then the final phase that we'll talk about in the 5 main control room abandonment scenario is phase 3, and 6 those are the actions once command and control and left 7 the main control room.

8 So, this is a graphic of -- not to scale 9 of the three time phases of main control room 10 abandonment. Just as I presented earlier, we have 11 phase 1, which is the period before abandonment, 12 phase 2, which is the time period for making that 13 decision to abandon the control room, and then phase 3, 14 which is the time period factor.

15 CHAIR STETKAR: Ashley, leave that up for 16 a moment. I was going to wait until we got to here 17 because I think this is my only opportunity to whine.

18 We had some discussion two years ago about why does the 19 quantification and qualitative analysis in 20 supplement 1, for the decision to abandon the control 21 room, the things that occur during that phase 2 block 22 there, why does that apply to loss-of-control 23 scenarios? Why does it not apply to loss of 24 habitability?

25 In supplement 1 you've added additional NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

28 1 rationale for loss of habitability and why it is 2 absolutely certain that they will absolutely leave, and 3 that you only need to know, do they leave too late.

4 I'm concerned about their leaving too 5 early. I'm concerned -- and why is that important?

6 It's important because it shifts that vertical line 7 between phase 1 and phase 2 to the left.

8 And maybe they don't have enough time to 9 accomplish the stuff in the main control room, that 10 you're taking credit for them doing in the main control 11 room, and that when they abandon, they find out they 12 can't do it as well remotely.

13 They either don't have the amount of 14 indications, they have to rely more on distributed 15 local people doing things, and that can be important.

16 So, for example, if I have a fire in the 17 main control room, you say, well, according to our 18 criteria, the great gods of PRA and people who wrote 19 NUREG-CR6850, say, when I get to a certain visibility 20 or I get to a certain heat-release rate, or something 21 in your plant-specific procedures, I am told to leave.

22 And I do that, because otherwise I'm going to die.

23 Well, I don't necessarily know that, and 24 I don't necessarily trust procedures when I think I 25 might die. And furthermore, if I have a fire in the NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

29 1 main control room and my fire brigade is running around 2 with fire extinguishers, or whatever they're doing, 3 bumping into and distracting me, maybe I decide that 4 I really would like to leave. Long before it gets to 5 hot or too smoky.

6 You talk about people putting on Scott 7 Air-Paks, you know, and staying in there until things 8 get so that they can't see. I was toying with the 9 notion, if you've never worn a Scott Air-Pak and tried 10 to communicate, understandably at all you'd understand 11 that people cannot do that. They're meant to make it 12 so that you don't die before you get out of where you 13 are.

14 So, why this artificial distinction about 15 loss of habitability requires no cognitive decisions.

16 There's only an uncertainty about, do they stay too 17 long, versus loss of control, where you do say, there 18 can be a much broader variability in the decision time, 19 because there might not be those specific criteria on 20 when do you discuss your indications enough to decide 21 to leave. To me, there's no fundamental difference.

22 DR. COOPER: Susan Cooper, and I'm going 23 to start answering the question, and I may defer to my 24 colleagues up here, or maybe throw a lifeline there to 25 the audience.

NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

30 1 So, the first part that I'm going to 2 identify, and that is, why not do something more 3 detailed for loss of habitability than, you know, 4 something similar to what we do for loss of control.

5 And I'm going to -- but the first answer, 6 which probably is not going to fly, and that is that 7 HRA is doing what the PRA needs, and they haven't asked 8 us to do that. They don't think it's important enough 9 that we dig into those details.

10 If they do different calculations to look 11 at whether or not there makes a difference, it's not 12 going to make a difference.

13 The second part is, that at least for the 14 plants we have now, the control rooms we have now, the 15 remote shutdown panels we have right now, the 16 abandonment strategies we have, the reluctance to leave 17 the control room is maybe even more of a concern than 18 we might have thought.

19 I mean, I think everyone who's done 20 anything with fire and has talked to operators, you've 21 heard something like, well I'm not going to leave until 22 my feet are on fire, or something like that.

23 And I've heard stories like that, and I've 24 heard people tell me that they're -- you know, about 25 their reluctance. But we did look into that as part NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

31 1 of the development of the quantification guidance for 2 supplement 2, and the result from the experts that we 3 got was that that is the driving factor.

4 And it has to do with the familiarity and 5 the capability -- well mostly the familiarity of the 6 control room, and being there, and not wanting to go 7 someplace else. That is the driver. Now -- so I mean, 8 that's kind of the basis.

9 Now, if you had a chance to take a look at 10 supplement 1, we did go back and add some material into 11 the section on the decision to abandon related to 12 habitability. And in particular, we went back and 13 tried to explain some things that apparently haven't 14 been documented elsewhere, and Nick Melly was -- and 15 I think Tammie Rivera also was more instrumental in 16 writing up the section, trying to better fill in the 17 blanks on what was in 6850 about the criteria for 18 habitability, and how that links back to actual human 19 subjects, and what you would be experiencing physically 20 if those habits, you know, those kinds of conditions 21 exist in the control room.

22 And, you know, some of the author team has 23 also compared that with other environmental factors, 24 like, you know, trying to control equipment and local 25 plant stations where the panels were hot to the touch NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

32 1 and, you know, what are those temperature limits, and 2 stuff like that.

3 And so, you know, we did go back and look 4 carefully at that, and we recognize that it's not a 5 sharp cut in time.

6 CHAIR STETKAR: I hope you read the 7 transcript, and read your words as an outsider.

8 Because you're talking about this as a dispassionate 9 fire modeler might talk about it. You're not talking 10 about it in the context of a fire in the control room.

11 And I'm not talking about something that 12 you spit on and it goes out. It's obviously a 13 meaningful fire. It's generating smoke. It might be 14 generating a lot of heat. It may have a combination 15 of environmental conditions with smoke, and effects on 16 my instrumentation.

17 And there are people who are going to be 18 trying to put the darn thing out. And that is not me, 19 as an operator, because if it is me, then I'm violating 20 some other PRA rule that says I can't do this. Or some 21 procedure rule that says I can't do this, despite the 22 fact that I might want to do it.

23 I'm certainly being distracted, and at 24 some point my cumulative distraction leads me to decide 25 maybe it's better that I do leave. Let them put the NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

33 1 darn fire out.

2 Maybe I can come back if there's enough.

3 But maybe I don't trust that other stuff well 4 enough -- perfectly, let's say -- and I decide 5 to -- see, I don't see philosophically why this big 6 focus on loss of control, and total ignorance of loss 7 of habitability. It's conditions that the operators 8 are faced with.

9 DR. COOPER: So, the other piece is 10 operational experience.

11 CHAIR STETKAR: We've had a lot of fires 12 where people have decided not to leave the control room 13 until --

14 DR. COOPER: We haven't had any fires 15 where we'd have left the control room.

16 CHAIR STETKAR: Yeah. Okay, and that's 17 my whole point. Or we're pushed to leave the control 18 room.

19 DR. COOPER: Well, there 20 were -- internationally there may have been one where 21 they left the control room. But there certainly have 22 been fires in the US where they have not left the control 23 room, and we've spent some time looking them.

24 Browns Ferry -- as a matter of fact, Mark 25 Sallie dug up or found an info video interviewing some NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

34 1 of the Browns Ferry operators in some of the conditions 2 that they were experiencing -- hot panels, plastic 3 falling -- you know, popping out of the instruments, 4 and stuff like that.

5 So, we don't have a lot of operational 6 experience, but the operational experience we have also 7 shows that they're not going to leave, even if it 8 is -- even if the conditions are not optimal. Now, I 9 know we do have examples of people putting on -- you 10 know, using the Scott Air-Paks and staying in the 11 control room.

12 We've also had a few interviews that said 13 that they were going to do that.

14 CHAIR STETKAR: No no no. I don't care 15 what they say they're going to do. I'm 16 interested -- you say you have experience that people 17 have done that? I'd be really interested to hear -- to 18 read about that, because I'm not aware of people that 19 have done that.

20 DR. COOPER: They did in Browns Ferry.

21 CHAIR STETKAR: Browns Ferry was 22 30 -- god, I can't -- 33 years ago. I'm sorry, 40 23 years, 75, 43 years ago.

24 MEMBER BLEY: Now, I've talked to lots of 25 people -- a number of people in control rooms who say, NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

35 1 no problem with the Scott Air-Paks. I haven't talked 2 to anybody who's actually had to try to do that. We 3 don't do -- I'll be Charlie today -- what's done in the 4 Navy We don't put people out in fires anymore. At 5 least not many plants.

6 Some years ago some good send their people 7 to firefighting school and have to do this stuff.

8 These days, not so much.

9 CHAIR STETKAR: Is that right?

10 MEMBER BLEY: Yeah, that's right. At 11 least the ones I've talked to. And so, most of them 12 have never been in that hot, smoky environment with 13 breathing apparatus on and trying to see. They just 14 kind of been told it's okay. But if there is real 15 experience there that's documented, it'd be real 16 interesting to see it.

17 MS. PRESLEY: So, I just -- I think one 18 other maybe already obvious point is that HRA, we only 19 have so much resolution we can go into. We can 20 postulate how the operators might behave, and we can 21 interview them and what they think they're going to 22 behave, and we can look at the sparse operating 23 experience and try to extrapolate, but we recognize 24 this is something that has uncertainty.

25 And we talk about the uncertainties NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

36 1 associated it and how you have to look at that. But 2 for the other -- there are other consequential 3 decisions that we've seen in operating experience, 4 where operators have kind of delayed it as long as 5 possible.

6 So, this is a model, and it's the best we 7 can go forward at this time with the experience and 8 information we have. I think it's the best --

9 CHAIR STETKAR: We'll talk more about 10 uncertainty later. I mean, you can just get one shot 11 --

12 MEMBER BLEY: I'd like to turn John's 13 question around a little bit then, and ask you about 14 your interviews. Did you find plants -- and I assume 15 you talked with trainers as well as operators, and where 16 folks are essentially trained to believe they're never 17 going to have to abandon the control room. I was 18 looking at your discussion of reluctance, so that's 19 fine.

20 Is there a belief out there that this 21 never -- it's not a real scenario, we're never going 22 to have to do this? Do plans actually go through the 23 drill and move the control out to the distributed or 24 local control areas?

25 DR. COOPER: Susan Cooper. All NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

37 1 plant -- all interviews that we've done -- and I think 2 that includes the rest of the team, including those out 3 in the audience -- have indicated that they practice 4 at -- train on certain aspects of abandonment.

5 Usually, doesn't in- -- almost always never includes 6 the actual decision.

7 But it does include all the moving parts 8 once they've left. So, that's -- and that's 9 Appendix R. So, they've been doing that for some time.

10 Since I'm at the NRC, I'm not doing the actual work.

11 I may need to throw this to either what EPRI or some 12 of our consultants in the audience.

13 But I -- for another project I do know that 14 that particular plant, they had their own fire training 15 facility onsite, which included the capability of 16 producing fairly large fires.

17 All of the field operators there were 18 trained on fire brigade, and they were moving toward 19 getting more of their licensed operators also trained 20 in fire brigade, even some people in management.

21 So, that familiarity was there. That 22 having been said, the capability of the remote shutdown 23 panel was such that I think they didn't really imagine 24 needing to leave, because they could transfer control 25 of individual pieces to their remote shutdown panel, NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

38 1 and just basically have another local panel that's 2 taking some of the load of control without 3 actually -- because for us abandonment means command 4 and control, all operations leave the control room, not 5 that some pieces of it have been farmed out elsewhere.

6 So, in that particular case the way -- and 7 there are other plants that probably have that sort of 8 capability, where they can, you know, select specific 9 systems and trainings and equipment, and, you know, 10 selectively move the control from the control room to 11 their remote shutdown panel. And then they don't have 12 to leave.

13 MEMBER BLEY: At the bottom line, I think, 14 from John's point of flipping this around, the 15 habitability, it just really doesn't sound like 16 anybody's thought real hard about the habitability 17 issue.

18 MR. SALLEY: Hey Susan, if I could just 19 jump in --

20 MEMBER BLEY: Yeah.

21 MR. SALLEY: -- for a second on the 22 habitability, and let's take our trip back in time to 23 Browns Ferry. If you go back to the Browns Ferry fire, 24 '75, and you look at the habitability, of course nobody 25 wanted to leave the control room then, and they did a NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

39 1 lot of heroic actions to not do that.

2 They breached a lot of barriers and opened 3 fire doors so that they can move the smoke. They went 4 as far as taking the plant air system and bringing in 5 air lines and chucking them open to literally blow smoke 6 out of the main control room.

7 So, they clearly weren't going to leave 8 the, you know, leave the main control room. That's 9 pretty much their safe zone. That's what they're 10 trained to do.

11 After Browns Ferry, we also made some 12 design changes. A lot of plants have smoke purge. So, 13 we can now align the HVAC and we can do a purge so that 14 they can, you know, stay there longer. Breathing 15 apparatuses -- the Scott Air-Paks -- was brought up.

16 Some plants even brought in line masks 17 where they have, you know, air cylinders that are 18 compressed and you open your line mask, and you have 19 an unlimited amount of time.

20 So, I think there is a lot to do that. The 21 key, I think, in the decision point -- and when I was 22 reviewing this with Susan, was when that SOS makes that 23 decision to throw the transfer switch. Because one he 24 throws that transfer switch, enters those procedures, 25 then he's locking himself into, this is how I'm going NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

40 1 to do shutdown now.

2 So, even though they may leave some people 3 in the main control room to monitor things and assist 4 them to do work, once they make that procedural jump, 5 they start that process, then they start into their hot 6 shutdown. So, that's kind of the key there.

7 DR. COOPER: Yeah, I guess -- this is 8 Susan here -- just to add to that, not all, but many 9 of the plants' safe shutdown strategies also are a 10 single path to success.

11 So, not only are you giving up all of the 12 indications and other capability you have in the 13 control room, but now you're on, you know -- there's 14 only one train of equipment that you can rely on for 15 shutdown, and if anything goes wrong, you know, you 16 don't have -- at least not procedurally 17 identified -- options to do other things.

18 So, that can also be considered a 19 deterrent, you know, so far as, you know, what I have 20 in the control room. What are my options in the control 21 room, versus when I leave? And I go to a different set 22 of procedures and different capabilities of equipment.

23 CHAIR STETKAR: I think we need to be a bit 24 cognizant of the time. But I did want to dwell on this 25 a little bit, because we won't have a chance in the NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

41 1 supplement 2 discussion.

2 Mary mentioned this is our model and this 3 is what we've had today. There's another way of 4 formulating the model. You mentioned uncertainties.

5 We'll talk more about uncertainties in supplement 2, 6 or I will.

7 One could develop an uncertainty 8 distribution, and I don't care whether it's loss of 9 control or loss of habitability, or some amalgam of the 10 two, that says there's some probability that they will 11 leave at time key 1. There's some probability that 12 they'll leave at time key 2, and there's some 13 probability that they'll leave at time key 3.

14 That's pretty simple. I didn't try to get 15 any more sophisticated than that. They don't have to 16 be precise probabilities. Each of those 17 keys -- key 1, key 2, key 3 -- has implications about 18 what you can accomplish in what you're calling phase 1, 19 and what you can accomplish in what you're calling 20 phase 3.

21 So, they have implications on your 22 quantification of human error probabilities, that will 23 eventually feed into the PRA.

24 Now, key 3 is too late for phase 3. That 25 makes the quantification in phase 3 pretty darn easy.

NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

42 1 If key 1 is too early for phase 1, maybe you don't get 2 a chance to do some of the stuff that you're 3 already -- you're taking credit for them doing very, 4 very reliably in phase 1.

5 I'm not talking -- but that's a different 6 type of model than you've established. It's a 7 different modeling thing. I'm curious why your 8 experts didn't propose that model. That -- why you've 9 latched onto this particular type of model.

10 DR. COOPER: There isn't any interview 11 that I can recall -- I'll ask Mary and Ashley to 12 remember too -- I don't remember anyone bring up the 13 notion of leaving too early.

14 MS. LINDEMAN: I have one comment loss of 15 habitability.

16 CHAIR STETKAR: Is that because you only 17 asked them what's the latest you might leave under these 18 conditions?

19 DR. COOPER: No.

20 MS. PRESLEY: The bias towards reluctance 21 was so strong that there was no -- I mean, we tried to 22 ask the question in many different ways, and -- Paul 23 Amico from Jensen Hughes 24 CHAIR STETKAR: Paul, we don't know who 25 you are, so make sure you identify yourself.

NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

43 1 MR. AMICO: I'm Paul Amico from Jensen 2 Hughes. So, there's no question that this is a 3 simplification. Okay, I mean, you know -- and as are 4 many things are in PRAs are simplifications. We don't 5 do T-1, T-2, T-3 for different options for feed and 6 bleed, for example.

7 We always say, well what is the least 8 amount of time you have available for it to get to feed 9 and bleed, and that's the way we do it.

10 In this particular case, the least amount 11 of time is, the longer they wait in the control room 12 for a habitability situation, that's the least amount 13 of time to accomplish everything else in phase 3. So, 14 that tends to maximize that part of the probability.

15 As to leaving too early, I guess we're a 16 little bit conservative in that too, in that we actually 17 give very, very little credit -- we don't say, we'll 18 they can stay for 16 minutes, so they do 16 minutes worth 19 of stuff.

20 Generally, we only give them credit for 21 those first few actions, like trip the reactor, trip 22 the turbine, those things. And then we say, well if 23 this turns into an abandonment scenario, basically we 24 assume they've got to do everything in the abandonment 25 shutdown process.

NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

44 1 We really don't give them credit for the 2 fact that before they left the control room, maybe they 3 started this pump, or maybe they did this thing. We 4 still say they have to do that, as part of the execution.

5 So, it's a simplification which I would 6 say, you know, tends to be on the conservative side.

7 We minimize the amount of time they have to execute 8 phase 3, and we also give very little credit for doing 9 things in the control room before they have to abandon.

10 And that's really the simplification.

11 CHAIR STETKAR: Thank you. We'll talk 12 more about phase 1 when we get into supplement 2.

13 MEMBER BLEY: Let me just ask something to 14 make sure I understand the model. Just back up one 15 slide, please. Phase 1 here is described as the 16 associate -- time associated with actions taken before 17 the decision to abandon. Now go to the next one.

18 Actually, the decision to -- it depends on 19 what you mean by the words. The decision to abandon 20 is right at the end of phase 2. But I think what you're 21 saying is, at the end of phase 1, you've decided 22 to -- at least your words say, see if you are to abandon.

23 DR. COOPER: So -- Susan Cooper here.

24 Not exactly that. It's actually -- I think the way we 25 described it is, if you were to know when all the cues NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

45 1 were available and occurring -- you know, 2 existing -- indications of any kind, environmental 3 conditions, or whatever -- the point at which that 4 information is available in some way, that means that's 5 the cue for abandonment.

6 So, what that end of phase 1 is really the 7 cue for abandonment, which is, in fact, probably a 8 collection of cues, and they accumulate over time. So, 9 figuring out exactly where that is, is a little bit 10 tricky.

11 So -- but that's really the end of phase 1, 12 is when the so-called cue for abandonment occurs. But 13 it's a lot fuzzier than a cue.

14 CHAIR STETKAR: The difference, when I 15 think of loss of habitability -- and at least the way 16 I read the guidance -- is that the cue to abandon the 17 loss of habitability is when you meet those magic 18 criteria for visibility or heat, temperature, things 19 like that.

20 And those things are calculated by fire 21 models. And it might take quite a while for those 22 things to develop. But it said in the guidance that 23 once they make that decision, the time -- from that 24 condition until they make the decision -- is minuscule, 25 so that phase 2 -- the time in phase 2 for loss of NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

46 1 habitability -- is essentially zero.

2 You get the cue, you decide to leave, 3 you're definitely going to leave when you get the cue, 4 and you're not going to leave before you get the cue, 5 and until those conditions are met, people are sitting 6 in the control room doing, happily, everything that 7 they would normally do, and it might be 30 minutes.

8 And I will tell you that if I was using the 9 guidance as it's written, I would do that in my PRA, 10 because that's what the guidance tells me to do.

11 MEMBER BLEY: And I read it differently in 12 different places.

13 CHAIR STETKAR: Okay.

14 MEMBER BLEY: The words didn't seem 15 consistent as I went through, on what these things 16 actually mean, and how you --

17 CHAIR STETKAR: But it seemed pretty clear 18 that the loss of habitability cue is determined by fire 19 modeling, either smoke density or temperature, and that 20 once you meet those crit- -- or whatever -- there might 21 be different plant-specific criteria -- but once you 22 meet those criteria for fire modeling, there's not a 23 degrading condition where there's vagueness. It's, 24 people decide to leave and they leave. And before 25 that, they decided not to leave --

NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

47 1 MEMBER BLEY: But only if those things are 2 codified in the procedure for telling them --

3 CHAIR STETKAR: That's true. They have 4 to be - yeah.

5 DR. COOPER: Susan Cooper. With respect 6 to habitability, the criteria that are calculated doing 7 the fire modeling end up being pretty severe. So, 8 you're right, it's not -- it's actually an accumulation 9 of effect. They may actually make the decision before 10 they reach the actual criteria as we calculate.

11 CHAIR STETKAR: Yes.

12 DR. COOPER: Because, for example, for the 13 visibility criteria, as I recall, that criteria said 14 that you cannot see this far. In other words, if the 15 indications are rubbing the panel here, you cannot see 16 those indications, because the visibility is just not 17 there.

18 So, if they needed to read that and they 19 thought this was getting to be a problem, they 20 probably -- depending on how the scenario evolved and 21 where the smoke was, and stuff like that -- they may 22 leave earlier than what the calculation would say.

23 So, the calculation is a simplification 24 for the longest time they would stay in the control 25 room, which, at least as far as phase 2, gives them the NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

48 1 least amount of time to implement their shutdown 2 strategy.

3 So, as Paul said, it's a simplification.

4 You know, we could -- there are other things we could 5 do, but that's -- there are tradeoffs.

6 CHAIR STETKAR: But I -- you know, my --

7 MEMBER KIRCHNER: Can you go to the next 8 slide, Susan? So, you don't put some kind of 9 distribution on each of these vertical lines. My 10 thought would be -- maybe it's a variant of what you 11 were saying earlier --

12 CHAIR STETKAR: Yeah.

13 MEMBER KIRCHNER: -- that they aren't 14 going to be that precise. Even if they are calculated 15 precisely -- at 38 minutes, for example, or whatever 16 those lines are -- the reality is, even in your case 17 of visibility, it might be billowing smoke in one corner 18 of the room, and the other half of the room is fine.

19 So, they're not going to go until the whole control room 20 is filled with smoke, or whatever the scenario is.

21 So, isn't there some distribution that 22 should be put on these to look at the sensitivity or 23 uncertainty, and propagate that in execution of what 24 is then subsequently analyzed? I don't understand.

25 MS. LINDEMAN: So, for loss of habitabil-NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

49 1 ity -- right? T equals zero is the start of the fire.

2 And fire -- and analyzing it, we have a distribution 3 of heat release rates based on, you know, certain 4 cabinet type.

5 And when you run your calcs for loss of 6 habitability, you'll discretize that distribution, so 7 you may fires of 50 kilowatts, 100 kilowatts, 200, etc.

8 And as a result, there are different times when the 9 habitability criteria are met, you know, on maybe the 10 order of eight to nine minutes anywhere, to roughly 30, 11 depending on the fire growth.

12 So, in that aspect there is a range of times 13 for loss of habitability, and I'm not sure if we did 14 a good job of explaining that before.

15 MEMBER KIRCHNER: I presume the same would 16 obtain for a loss of control.

17 MS. LINDEMAN: So, we'll talk about loss 18 of control timing -- so, if we go to the slide -- I don't 19 want to go to the slide before because I don't want to 20 go backwards. But that --

21 CHAIR STETKAR: You don't want to.

22 MS. LINDEMAN: That figure is a construct 23 to help analysts think about the response in phases.

24 And the way we use the timing information is maybe the 25 more important point. And when you see how we use the NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

50 1 timing information in phase 2 and 3, and 1, maybe this 2 will be little bit more clear.

3 Because I think we're getting caught up on 4 lines on a diagram which are constructs, not 5 hard -- they weren't meant to be hard lines, and we'll 6 see there's some squish, and this is actually an item 7 that we have to describe more of what that squish is 8 used for.

9 DR. COOPER: Mary's exactly right. This 10 is Susan. The important of the phases, though -- the 11 time phases -- is that we treat them differently in HRA, 12 and the way we develop the quantification guidance, the 13 kinds of qualitative analysis you need to do, what 14 actions operators are taking and where, what procedures 15 we're using, those are all different.

16 And that's why the different phases are 17 important to us. So -- but they are constructs not 18 necessarily measurable in all places.

19 MEMBER SUNSERI: So, I was going to ask 20 another question. I mean, you know, I hear talk about 21 loss of control and habitability and all that stuff.

22 But I think from an operator -- from being a former, 23 you know, operator in one of these plants, it always 24 seemed to me like it wasn't a matter of that, but it's 25 a matter of, is this a fast-moving issue, or a NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

51 1 slow-moving issue. Right?

2 And so, it all comes down to the cues, not 3 times. Right? So, an operator on a fast-moving 4 thing, maybe all they can do is tell the guy that's at 5 the board, trip the reactor, we're leaving. And that's 6 the initial step of the shutdown process, or the 7 transfer process.

8 The other situation may be if the 9 slow-moving thing, the shift supervisor says, hey look, 10 we're losing this thing. We're going to have to go in 11 a few minutes. I want you to do these three things, 12 which is in accordance with my shutdown guidance.

13 Right?

14 When we leave I want the reactor tripped, 15 I want the auxiliary feedwater pump started, whatever 16 it is, you know? And then, you know -- but once again, 17 it's not so much time-driven. It's cue-driven.

18 MS. LINDEMAN: Mm hmm. Okay.

19 MEMBER SUNSERI: And, you know, it 20 shouldn't be, you know, when the visibility is down to 21 whatever the lumens, it's going to be when you can't 22 see the controls anymore, or whatever the thing is.

23 Right?

24 DR. COOPER: You're right. It's very 25 context-specific. And so, there are variations of NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

52 1 main control room abandonment scenarios, including, 2 you know, what's happening, and why is it happening that 3 way. So, yes.

4 MEMBER SUNSERI: And, by the way, I 5 wouldn't ever recommend this as like an ongoing thing, 6 but I've been involved in the startup of a couple of 7 plants, and one of the things that we had to demonstrate 8 was that you could trip the reactor and establish a 9 cooldown rate from outside the control room, and it 10 wasn't as scary as we thought it would -- I mean, the 11 operators -- it wasn't as anxious of a -- now granted, 12 there was a control situation, a lot of oversight, and 13 all that kind of stuff. But, you know, having walked 14 through that --- but, I mean, you know, with the modern 15 instrumentation of the plants and the shutdown transfer 16 panels and the remote shutdown, it's a quite doable 17 thing.

18 MS. LINDEMAN: Okay. All right, well, I 19 guess due to time we'll just kind of skip to the next 20 slide, which is the timeline for doing it is at the end 21 of May. And I realize that this graphic is really 22 difficult to see, and, you know, even the point of 23 showing this isn't to demonstrate that you can 24 construct a timeline that includes all the actions for 25 two units.

NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

53 1 What I'd like to highlight is, you know, 2 one of the outcomes of supplement 1, which was having 3 a narrative and a picture for the analysts to identify 4 all of the actions, but also all of the critical 5 communications and staffing locations, and, you know, 6 we believe that's, you know, one of the most powerful 7 outcomes of supplement 1 is giving this tool to the 8 analysts, not only for feasibility, but, you know, the 9 entire scenario.

10 DR. COOPER: Susan Cooper here. If you 11 can't read the graphics, the blue lines are where 12 communications are occurring between operators, and 13 would indicate, you know, probably some kind of 14 coordination that's required.

15 So, that's one of the keys of this kind of 16 timeline. It doesn't have to be dual unit, but to show 17 that. But this is just trying to show all the activity, 18 and try to get a handle around what all's going on, which 19 could be important for staffing considerations also.

20 MS. LINDEMAN: Time, sequence, actions.

21 I think with that we'll all kick it back over to -- well, 22 we're currently under development, which is 23 supplement 2, which is the quantification guidance.

24 DR. COOPER: Susan Cooper. I'm going to 25 give you an overview of supplement 2 first. And then NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

54 1 we'll get into some of the details.

2 So, supplement 2 has been developed to be 3 a companion document to both NUREG-1921, which is the 4 overall fire HRA guidance, and then, supplement 1 to 5 1921, which provides a qualitative analysis guidance 6 for main control room abandonment scenarios.

7 The high-level approach that we used, 8 although we -- you know, we developed a construct of 9 the different phases, at least for the purpose -- the 10 focus of quantification, we looked mostly at phase 2, 11 which is the decision to abandon, and that's 12 specifically for loss of control.

13 And then, phase 3, which is implementation 14 of the main control room abandonment safe shutdown 15 strategy.

16 We used supplement 1 to try to identify key 17 issues that should be represented in quantification for 18 both of these phases. We used these consensus list of 19 issues, and compared them to existing methods, to see 20 how well existing methods could be used to represent 21 what we thought was important for main control room 22 abandonment.

23 As a result of those comparisons, the 24 author has developed some strawman approaches for 25 quantification. And then, what we did with -- and it NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

55 1 was a little bit different between the two phases. In 2 phase 2, which is the decision to abandon, the team 3 developed some candidate decision trees to be used for 4 quantification for command and control issues.

5 Specifically, for shutdown after leaving the control 6 room, we just had that list of key issues.

7 And then, we brought in some experts and 8 had a panel to discuss those issues, and confirm or 9 modify the strawman approaches. And then we used that 10 input to complete the quantification approaches for 11 both phase 2 and phase 3.

12 I want to talk a little bit about the expert 13 panel. There were a number of things that we wanted 14 to do. We wanted to keep a balance between the NRC and 15 the industry, so far as number of people on the panel.

16 So, we had two people from the NRC and two people 17 actually from the author team.

18 In both cases, we wanted to make certain 19 that the experts had knowledge of operations and 20 training, fire-related experience, but specifically, 21 related to main control room abandonment.

22 So, we wanted people that had experience 23 with either developing the fire models, doing the HRA, 24 or, from the operations side, had an understanding of 25 what those kinds of operations involved.

NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

56 1 So, that -- those were the guiding 2 principles for the experts. The other thing is, we 3 wanted to -- down the line we talked about the fact we 4 wanted to make certain that the experts had a wide range 5 of understanding, not just for one or two specific 6 plants, because we were developing a method that was 7 going to be generic. So, we wanted to make sure it 8 wasn't just, you know, I just know this one plant. So, 9 that was important also.

10 So, the experts we had -- from the NRC side 11 we have Harry Barrett, formerly of NRR, and he was an 12 NFPA-805 reviewer. He's since retired. We got him 13 just before he left.

14 And then, also, we had Jim Kellum from NRO, 15 who has decades of experience as operations trainer, 16 so he represents lots of plants. And then, from the 17 EPRI side, industry side, we had two of our 18 authors -- Jeff Julius and Erin Collins, both with 19 Jensen Hughes, which, between the two of them, 20 represented a lot of plant and plant analyses.

21 We used the experts differently between 22 phase 2 and phase 3. For phase 2 we started off by 23 confirming, or focusing, whether or not we got the key 24 issues right. That should be represented in 25 quantification.

NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

57 1 After that we had the experts take a look 2 at the strawman decision trees. As a result of that 3 they made some modifications and prioritizations. And 4 then, we went ahead and did an actual HEP development, 5 starting with worst-case scenarios, best-case 6 scenarios, intermediate, and so on and so forth.

7 For phase 3, which was the impact of 8 command and control on the implementation of safe 9 shutdown strategies. We didn't use the experts to 10 develop HEPs for a quantification tool. We simply used 11 them to confirm or prune our key issues list, and then 12 to identify priorities and associated context for what 13 we wanted to include in our guidance. So, that's how 14 we used the experts in the two different phases.

15 MEMBER KIRCHNER: May I ask you a 16 question? Did you then kind of take a representative 17 plant of different types and just run through it to see 18 if this is what the experts decide this is what the 19 operators might do?

20 DR. COOPER: We did not -- this is Susan 21 Cooper -- we did not have specific example scenarios 22 to go through in doing these developments. But the 23 discussions naturally led to plant-specific examples 24 that were brought into the discussion.

25 But the end results were not based on a NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

58 1 plant-specific input, you know. It was -- and the 2 answers were consensus answers. So even if one expert 3 brought in, you know, I have this information that I 4 know about this plant, and then another expert will 5 offer something else, and then all of those opinions 6 were brought together into a consensus, so far as, you 7 know, in the case of phase 2 -- the decision to 8 abandon -- there were consensus HEPs that were 9 developed.

10 And then, for phase 3 there was just a 11 consensus that across the board these are the things 12 that we think would be most important, you know, 13 generically.

14 So, the end result is generic. The actual 15 discussions in getting to that answer might have 16 brought in some plant-specific information.

17 So, how to use supplement 2, the purpose, 18 of course, for supplement 2 is to provide that 19 quantification guidance that's beyond the scope of 20 NUREG-1921, which does provide quantification guidance 21 for other fire scenarios.

22 For the most part, although there's some 23 types of operator actions and human failure events that 24 we identify in supplement 2 where 1921 guidance is 25 sufficient, and in the case of supplement 1, there are NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

59 1 places where we repeated that guidance because we 2 thought it was important, there's some cases where 3 supplement 2 expanded on concepts that were started in 4 supplement 1, and so there may be some redefinition.

5 Command and control is definitely one of those 6 instances.

7 And, at least in the draft that you have, 8 we've tried to indicate where 1921 certainly in those 9 cases, if that's the guidance you need to use, we've 10 indicated that. And we've tried to also indicate where 11 you need to go back and look at supplement 1.

12 We've already mentioned that we're 13 considering trying to provide some additional help to 14 the analysts on how to sort those things out, and of 15 course welcome your comments on how we can continue to 16 improve that. Next slide.

17 So, that's the end of the overview. And 18 then, we're going to go into, specifically, discussion 19 and decision.

20 CHAIR STETKAR: Let me intercept you here.

21 DR. COOPER: Okay.

22 CHAIR STETKAR: Because you're going to 23 skip phase 1 and I'm not. The guidance in supplement 2 24 for phase 1 is, I'll call it thin. Let me quote a 25 couple of passages.

NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

60 1 During phase 1 operator actions are 2 directed from the main control room. Similar to other 3 fire scenarios, these actions are not necessarily 4 unique to main control room abandonments, since the 5 cognition and execution for these actions are very 6 similar, if not identical, to fire scenarios where the 7 fire is not inside the main control room, or even 8 internal events PRA actions following a reactor trip.

9 Section 2.2 says, up until the point of the 10 decision to abandon, the operating crew is responding 11 to the fire from the main control room, so the guidance 12 in NUREG-1921 is applicable and sufficient to evaluate 13 and quantify phase 1 actions.

14 I look at 1921 -- 1921 says, ah, main 15 control room abandonment is a different issue, and 16 that's why we're having this meeting today. 1921 17 focuses an awful lot on distributed actions being 18 guided from the main control room with people running 19 around in the plant doing things.

20 It doesn't talk much about fires inside the 21 main control room. Inside the main control room.

22 Which brings me back now to my loss of habitability.

23 How is it that I can use the guidance in 1921, or 24 guidance for HRA, for internal events, to quantify my 25 actions inside the main control room, when, as Walt NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

61 1 said, over in the corner there's smoke billowing out.

2 Now, it hasn't got -- I can still see 3 things. And, oh, by the way, the fire brigade is 4 pushing me out of the way because they want to get to 5 the corner, and the only way is through me.

6 So, how is the guidance for quantifying 7 fires outside of the main control room, I can almost 8 say, that the guidance in 1921 applies because as long 9 as I'm not being distracted by people trying to put out 10 the fire, if one of those people who might be me.

11 Or, potentially in fear for my own 12 survivability, I can say, well, it doesn't make too much 13 difference if the fire is out in a cable-spreading room, 14 or out in some other location, because I'm just reacting 15 to whatever indications are coming in, which 1921 does 16 a fairly decent job.

17 I was hoping, during phase 1, that you 18 would say, well, for fires inside the main control room, 19 you have to think about much different things. But you 20 don't say that. You say that I just use standard, 21 everyday human reliability analysis until I have to 22 abandon, and everybody knows that the abandonment thing 23 is the most important stuff.

24 So, can you address that? Why does this 25 phase 1 not need separate guidance to alert people to NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

62 1 the fact -- especially for fires inside the main 2 control room -- that that is a different beast than 3 anything else that has been addressed in the guidance 4 to-date?

5 DR. COOPER: So, especially with you 6 reading things directly, we need to make a correction 7 that it's not up until decision to abandon --

8 CHAIR STETKAR: That's what it says, 9 though. If I'm going to --

10 DR. COOPER: We need to correct that.

11 CHAIR STETKAR: -- read the guidance --

12 DR. COOPER: We need to correct that. So, 13 I would say that it's shorter than it should be. I will 14 agree that we need to add something there. I would say 15 that for fires in the control room, that 1921 probably 16 can still address that, because we do address 17 environmental factors and their influence on operator 18 performance. However --

19 MEMBER BLEY: That's typically --

20 DR. COOPER: -- it doesn't -- it's 21 typically outside the control room.

22 MEMBER BLEY: It's typically outside the 23 control. I wanted to be --

24 DR. COOPER: It's typically outside the 25 control room. So --

NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

63 1 MEMBER BLEY: But in their behalf, it does 2 identify the things that it does to you. It's just not 3 in the control room. Which is important. I'm not 4 disagreeing.

5 DR. COOPER: Yes.

6 CHAIR STETKAR: Typically, the 7 environmental factors, though, are presented in terms 8 of accessibility, you know, stay times, and things like 9 that, which can, in principle, be extended to the main 10 control room, but --

11 DR. COOPER: There -- yeah, there's some 12 other influences. But yes. Yeah. So, I think you're 13 correct. We should add some more guidance there. We 14 need to clean up the way we discuss the construct of 15 the time phases is tripping us up there.

16 But definitely, we need to add some words 17 so that 1921, which, as you say, usually the 18 environmental factors is coming in on plant locations 19 at -- local plant locations.

20 MR. AMICO: Yeah. And --

21 CHAIR STETKAR: Paul, you have to -- every 22 time you have to identify yourself.

23 MR. AMICO: I know. I'll get to that.

24 This is Paul Amico from Jensen Hughes again, because 25 I can't sit down and keep quiet. So, we have been using NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

64 1 1921 for -- keep in mind the wording may be a little 2 wrong, but the point being that the actions in phase 1 3 are up until the point where you have the cues, where 4 now you might consider abandoning.

5 Up until that point, the situation is the 6 same. Whether it eventually becomes an abandonment 7 scenario or not, you're operating only from the EOPs 8 and the fire response procedures.

9 Now, when I say we use 1921, 1921 instructs 10 you to address the context under which the operators 11 are performing. And we do look at what kind of fire's 12 occurring in the control room, how big it is -- because 13 we do do the analysis, even for non-abandonment 14 scenarios about what's happening -- and we make 15 adjustments to the PSFs in the existing models.

16 So, for the HCRORE model, we would say, 17 okay, their fighting the fire in the control room.

18 That's a distraction. We're going to add four minutes 19 to time delay. Or, we're going to add three minutes 20 to the cognition time, because these other things are 21 occurring.

22 So, we actually do quantify those aspects 23 of the actions in the control room. We may add to 24 execution time because somebody -- because there's a 25 big fire and there's a big commotion.

NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

65 1 So, we do actually consider that the stress 2 of the operators in performing the actions on the 3 control panels will be higher. And so, we'll address 4 the THERP stress factors.

5 So, all that really is addressed by 1921, 6 and we do, in fact, use different adjustments to the 7 things that happen in the control room -- if there's 8 a fire in the control room, versus if the fire is 9 someplace else and they're performing the same actions.

10 I think that's the context we're talking 11 about. And we do try to base that on actual operator 12 interviews, to the extent that that's something that 13 is reasonable to do.

14 So, that's, I think, the context that 15 they're trying to lay here is, 1921 covers the things 16 you do in the control room when there's a fire in the 17 control room, up until the point you reach the cues, 18 where you may want to abandon. That's phase 1.

19 MS. PRESLEY: We could probably stand to 20 make that a little more explicit in our guidance.

21 CHAIR STETKAR: That's -- you know, I did 22 a quick search of 1921, and it's -- to my mind it's not 23 as clean as Paul makes it sound, because it -- 1921 24 focuses mostly on main control room abandonment, and 25 it -- I can read it to make it sound that way, but since NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

66 1 we're now development guidance explicitly for these 2 scenarios, some of which are fires in the control room, 3 some of which are not fires in the control room, but 4 all affect the integrated performance of people in the 5 control room, or decisions to leave.

6 See, quite honestly, I'd feel more 7 comfortable about this notion of loss of 8 habitability -- have to be careful here because I don't 9 feel comfortable about it at all -- but if there was 10 enough guidance to say that the evaluation of team 11 performance in the main control room for fires that 12 occur in the main control room, has to be really, 13 really, really careful to account for the environment, 14 the stress, everything that Paul just mentioned --

15 MEMBER BLEY: I think you're at a spot --

16 CHAIR STETKAR: I'd feel more 17 comfortable.

18 MEMBER BLEY: -- where the developers in 19 the method know exactly how to use it and the place they 20 want to use it, but a third party coming to the guidance 21 and trying to do an analysis based on the guidance, may 22 be confused.

23 CHAIR STETKAR: If I -- I tend to read 24 it --

25 MEMBER BLEY: Or take the wrong turn.

NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

67 1 CHAIR STETKAR: I tend to read it as that 2 third party. I would pick up this guidance, quite 3 honestly, as a human reliability analysis 4 practitioner, if you will, and say, well, okay, if one 5 of the actions -- if I have enough time in phase 1, is 6 to try to start an auxiliary feedwater, and if I can't 7 do that, try to start main feedwater. If I can't do 8 that, try to depressurize and start condensating.

9 If I can't do that, try to initiate feed 10 and bleed, and maybe I got 27 minutes to be able to do 11 that, and I've got 27 minutes because the smoke density 12 doesn't get up to whatever it's supposed to be until 13 36 minutes.

14 Well, I have procedures, I have training, 15 I've done this oodles of time for my internal events 16 at power risk assessment. I'll just use that human 17 error probability, because you say I can.

18 MEMBER BLEY: And that's -- honestly, 19 that's what I would do.

20 MS. PRESLEY: We can make that link more 21 explicit. But I think also the integrated timeline is 22 an important piece, because it shows who's where, doing 23 what. And if there's a part that's being taken off for 24 the fire brigade and fighting the fire in the control 25 room, that would be reflected, and we would take that NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

68 1 into account in the analysis.

2 CHAIR STETKAR: That's not -- I'm worried 3 about -- I'm not part of the fire brigade. I'm worried 4 about the fire brigade people pushing me out of the way 5 because the only way to the corner is through me, and 6 me having to battle with them, because I have to look 7 at these indicators here.

8 So, it's not depleting my resources for the 9 fire brigade. That's -- take a separate fire 10 department, you know. They're still going to want to 11 disrupt me to put the fire out, not to mention 12 whatever's going on with the fire, itself.

13 MR. JULIUS: Jeff Julius from Jensen 14 Hughes. The pieces missing in that discussion is, 15 looking at the overall -- all the phases, because a lot 16 of times the procedures in phase 2, or once you've made 17 that decision, you go and you turn off the feedwater, 18 you turn off everything you do.

19 And so you pick it up, and so you give 20 people -- fire protection people and people at the 21 plant that don't like the PRA people, because I was 22 telling them, I'm not giving you any credit for this 23 in phase 1 because you were taking quick -- you were 24 following procedure guidance and you're turning these 25 up, and we're only going to model the failure to restart NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

69 1 it in phase 3.

2 So, some of those -- you're right. It's 3 probably a case, as Dennis said, that as a developer, 4 we know when we've seen that, but when you look at these 5 in isolation -- you can't look at them in isolation.

6 You have to look at the overall scenario.

7 MS. PRESLEY: So, you didn't even get to 8 phase 2. So, this -- I guess that goes with --

9 MEMBER BLEY: I'm sorry to interrupt you 10 already. Have you let anybody else try to use this?

11 MS. PRESLEY: No.

12 MEMBER BLEY: Not yet.

13 MS. PRESLEY: Not yet. No. Our peer 14 review -- we'll talk about peer review in the next --

15 MEMBER BLEY: Different.

16 MS. PRESLEY: Yeah. Well -- okay, so to 17 echo what Susan had discussed about high-level 18 approach, the high-level approach is the same. We kind 19 of took -- looked top-down and bottom-up. We 20 developed a list of factors that we thought were 21 important to the quantification for the decision to 22 abandon and loss of control. We did that based on the 23 information from the interviews that the analysts have 24 done and that we had -- the team had done with the NRC 25 folks, as well as our understanding of the kinds of NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

70 1 literature.

2 So, we've pulled all of that together, and 3 for each factor, we tried to kind of understand the 4 range of what we've seen and what might be seen. So, 5 we kind of about what's the best-case, what's the 6 worst-case, and what are some of the intermediate cases 7 we've seen.

8 And then we looked at a set of HRA methods, 9 and we looked at failure modes and mechanisms, and those 10 methods, to see if there was anything related to either 11 issues or failure modes that we need to add our list, 12 or anything that we could use kind of -- I would say 13 off-the-shelf -- without a ton of modification.

14 So, we found that there's one that we 15 thought maybe we could use, and we started playing with 16 it, trying to adapt that, and found really it was 17 stretching the method too much, and ended up developing 18 our own decision trees based on the factors that the 19 team thought were important through the reviews and the 20 issues list development.

21 And then we took that strawman to our 22 experts and got some more feedback, and that ended up 23 consolidating our three trees into one tree, and 24 getting some ATP values to quantify the tree with.

25 So, that was the overall process, and I'll NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

71 1 go through a little bit more detail on each step.

2 So, the first -- when I said we had our 3 issues tabled, this is one of three slides, and I'll 4 go through this fairly quickly. The first issue that 5 we came up with was procedure content. Whether or not 6 there was an explicit definition of what the criteria 7 were for abandonment.

8 We saw quite a variation from, if there's 9 a fire in this area, the supervisor should consider 10 abandoning. So, that was kind of our worst-case. It 11 was very open-ended. And the best-case would be, these 12 are the systems you should look at. If they're not 13 functional, then abandon.

14 So, we saw quite a range. We didn't see 15 anything super explicit, which would have been our 16 best-case. So, most of them fell under the 17 intermediate or worst-case, where they provided some 18 indication of what location is important, or when the 19 fire is identified and confirmed, or whether or not your 20 emergency procedures are working the way they should 21 be.

22 So, that was the first -- our first thought 23 was, the procedure was very important. We'll see what 24 the experts had to say, but we had parsed it into three 25 layers.

NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

72 1 Then, we looked at time available, and 2 across the set of analyses, we saw that the time 3 available was typically 30 minutes or less from what 4 we've seen. So, we parsed that up into best/worst/

5 intermediate case.

6 And then, when we looked across the 7 training spectrum, the worst-case was only classroom 8 training at the minimum level -- I think once every two 9 years or something -- you might have some classroom 10 training on the criteria to abandon.

11 The best-case is realistic training in a 12 simulator. Maybe it's integrated with the actual use 13 of the shutdown panels, that -- I'm not sure we've seen 14 any instances of that. So the best-case in this case 15 is quite a high bar that's not really seen in the 16 industry.

17 The intermediate cases where you have 18 detailed classroom training which talks through the 19 criteria. Maybe we'll talk about, you know, what 20 exactly are the criteria? How do you know you've met 21 them? You might have some simulator training where 22 they actually simulate the conditions, and you have to 23 figure out you've abandon- -- you have to abandon.

24 So, there's a range on training, and the 25 best-case we don't really see. So, cues and NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

73 1 indications was another issue that we thought was 2 really important.

3 So, the procedure tells us when we need to 4 abandon. The cues and indications tell us how do we 5 know that that criteria's been met? There's a ton 6 happening. We don't know exactly, necessarily, what 7 exactly the operators are going to see, because we only 8 trace some cables -- the ones that are important in the 9 PRA.

10 And some fires may be fast-growing, some 11 may be slow-growing, and this was -- the cue and 12 indication is really not one like triple little HRA, 13 where you have a parameter or a couple of parameters 14 that you're looking at that's trending, and you see the 15 parameter, it matches the level, you go do an action.

16 This is not that. This is a unfolding 17 scenario that's getting progressively worse. And so, 18 the cues and indications, being able to interpret what 19 you're seeing against your criteria, was one important 20 aspect. And we're going to talk about that a little 21 more when we talk about the timeline for phase 2, and 22 how you build that.

23 We talked about reluctance, and initially 24 our thought was that reluctance was highly tied to the 25 capability of the remote shutdown panel, and the NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

74 1 comfort and familiarity with the main control room, and 2 just this inability -- like, it's a bad day, we don't 3 think we'll ever actually abandon.

4 And we saw, through some of the operator 5 interviews, that there was -- there's kind of a -- it's 6 hard to get operators to see some of these PRA 7 scenarios, that they were so Appendix R-minded that it 8 was really beyond some of their imaginations to get to 9 this scenario.

10 Now, AB&C is what we thought -- or our 11 initial thoughts -- the capability of the remote 12 shutdown panel for phase 2 turned out, through our 13 expert opinions, to be one of those areas where they 14 didn't think it was necessarily that important, so 15 we'll talk about why that is and what happens to that 16 information, in a couple of slides. But that was our 17 first cut, was based on those three factors. Next.

18 Staffing and Communication. So, this has 19 to do with how much backup you have in helping make that 20 decision. Best-case is you have the STA or other crew 21 that are monitoring actively the criteria to abandon, 22 and helping you watch that, so you have -- ready to make 23 that decision once you get to that point. The 24 worst-case is, if Justin's your supervisor, everyone's 25 tacked out doing other things, and he needs to make that NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

75 1 decision with little input.

2 And then, finally, transfer the procedure 3 with that main control room abandonment criteria. So, 4 you need to get to the main control room abandonment 5 procedure to abandon. It wasn't always clear in the 6 scenarios that we had, that there was an explicit 7 procedural path that gets you there.

8 And some of our operator interviews found 9 that there were situations where that was true, and we 10 thought that was pretty important to bring up. Again, 11 with our expert opinion, we found that was actually not 12 necessarily a driving factor for this HEP, and so I'll 13 talk about that a little bit when we talk about how we 14 end up pruning some of the trees.

15 MEMBER REMPE: Again, this isn't my field, 16 but when I was reading your report on page 34, you talk 17 about a situation where the staff needs -- there are 18 certain staff members -- like the shift technical 19 advisor -- needs to be in the control room to make the 20 decision to abandon. And there's a certain time that 21 they have to be there after the start of the event.

22 And if this person isn't there, you can't 23 make that decision to abandon. And is this a situation 24 that ought to be cleaned up if you found this to occur 25 at a plant? Am I misreading your report?

NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

76 1 MS. PRESLEY: So, some clarification is, 2 the shift technical advisor might support that 3 decision. So, in order to credit his support of it, 4 he will never be making the decision.

5 MEMBER REMPE: But he needs to be there for 6 them to go forward?

7 MS. PRESLEY: No, he doesn't need to be 8 there. He's not required.

9 MEMBER REMPE: It says, required to 10 support the decision. He has to be there if --

11 MS. PRESLEY: That's poor wording.

12 MEMBER REMPE: Okay. Well, again, I 13 don't know --

14 MS. PRESLEY: Yes, thank you.

15 MEMBER REMPE: -- a lot about that.

16 That's why I said that to start.

17 PARTICIPANT: What section was that?

18 MS. PRESLEY: Page 34.

19 MEMBER REMPE: You know, again, I'm just 20 like going, well if that's the case, this reminds me 21 of other things we've heard of where critical folks 22 aren't there to make a decision, and -- so never mind 23 I guess is the answer to my question.

24 MS. PRESLEY: Well, thank you for pointing 25 out that --

NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

77 1 MEMBER REMPE: Yes, thank you for clearing 2 that up --

3 MS. PRESLEY: -- poor wording.

4 MEMBER REMPE: -- bad choice.

5 DR. COOPER: Not what we meant.

6 CHAIR STETKAR: Some shift supervisors 7 would say it would be a good thing if the shift technical 8 advisor was out, you know, cleaning the ice off the 9 windshields.

10 MS. PRESLEY: Next slide? Okay, so --

11 CHAIR STETKAR: That would be Zion.

12 MS. PRESLEY: We're not naming it in here.

13 So, our beautiful timing fire growth picture, which you 14 guys probably cannot read the text on, which is okay 15 at the moment, because I'll walk you through it, 16 but -- so the point is, is that the timing is not as 17 clear cut as the typical read-the-parameter, 18 take-an-action, that the definition of T -- we have 19 T-delay and T-cog is our typical timing nomenclature.

20 And T-delay is the time to get the cue, and 21 T-cog is the cognition time. So, between the time you 22 get the cue and the time you make the decision, and so 23 that's like phase 1 and phase 2.

24 So, that is not a clean line, and it's 25 highly linked with your procedure and your training.

NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

78 1 It's -- and your cues and indications. Those three 2 things are intractable from each other. So, you have 3 to understand how the procedures work and how the 4 operators are trained, and what the expectations are, 5 versus the progressions scenario.

6 So, we came -- we actually -- I have a 7 printout -- so we came up with a half-dozen of these 8 fire growth scenarios. But basically, it walks 9 through -- you have -- the fire starts at reactor trip, 10 then you have your immediate memorized actions 11 complete.

12 And then you have that fire that happens.

13 You might have had an alarm. So then, you have 14 an -- you've acknowledged the alarm. You know that 15 there's -- you've gotten a fire alarm, you're going to 16 go send someone to go --

17 MEMBER KIRCHNER: Do you always assume 18 reactor trip with the fire starting?

19 DR. COOPER: It won't be modeling it with 20 PRA if you --

21 MEMBER KIRCHNER: Mm hmm.

22 DR. COOPER: It won't be modeled in the PRA 23 if there's no reactor trip. So, the fires we care about 24 will have a reactor trip.

25 CHAIR STETKAR: In your opinion, it would NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

79 1 not be modeled in the PRA if there's no reactor trip.

2 PRAs do model fire scenarios that do not result in a 3 reactor trip, but disable, for example, necessary 4 cooling water systems.

5 MEMBER KIRCHNER: Right. And then the 6 reactor trip --

7 CHAIR STETKAR: I have done that said 8 PRAs, so I can say PRAs do that. Perhaps none that you 9 ever did.

10 MEMBER KIRCHNER: And then the reactor 11 trip may come well after the fire.

12 CHAIR STETKAR: Right. That's right.

13 And that's why it's important. But it is important 14 that the start of their scenario -- it's one of the 15 things I look for -- is the initiation of the fire. And 16 indeed, the reactor trip can occur sometime after that.

17 Whether it's caused by the fire, whether 18 it's coincident, whether it's somebody manually trips 19 the reactor, that begins a certain type of transient.

20 But the fire can cause a tran- -- can cause a disruption 21 of cooling without tripping the reactor.

22 MEMBER KIRCHNER: Yeah.

23 CHAIR STETKAR: And that -- that is --

24 MEMBER KIRCHNER: So, my question is, are 25 all the scenarios you look at reactor-trip-coincident NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

80 1 with the fire alarm? Or is this just an example?

2 DR. COOPER: I want to defer that 3 question.

4 MS. PRESLEY: I would say --

5 MEMBER KIRCHNER: I mean, the more likely 6 thing is that you would have a fire somewhere in the 7 plant, and it would take sometime either to recognize 8 you had a problem, or to do grade equipment, or to get 9 to reactor trip. But I wouldn't see them coincident.

10 MS. LINDEMAN: Yes, that's correct. A 11 common simplification is the fire starts at T equals 12 zero, and there's also a reactor trip at T equals zero.

13 I believe in supplement 1, we acknowledged that that 14 simplification may not be appropriate in all instances, 15 and, you know, it's dependent on a lot of things.

16 Right?

17 The fire can happen like a transient, and 18 that's not associated with a piece of equipment, so it 19 can take time for that fire to grow and develop. Or 20 you can have a fire on the right location and a reactor 21 trip --

22 MEMBER KIRCHNER: So, let's go back to 23 Browns Ferry. When did they discover the fire, and 24 when was the reactor tripped?

25 MS. LINDEMAN: I would say the fire was NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

81 1 detected rather quickly, based on my recollection.

2 MR. SALLEY: Browns Ferry -- so it was a 3 manual trip on Unit 1. After they had the fire and they 4 fought it for quite a while. So, there was an amount 5 of time, and the key on Browns Ferry was they, at that 6 point, knew they lost all their cooling systems. So, 7 they were pretty much done.

8 Now, the interesting thing is Unit 2, 9 because Unit 2 was a sympathetic trip. There's -- you 10 know, it was sympathy that they were going to lose it 11 too, so they tripped that manually.

12 CHAIR STETKAR: But the point is that the 13 fire damaged -- the effects of the fire damage begin 14 at time T zero when the fire occurs. The reactor trip 15 is simply some of that that occurs. It could happen 16 at zero, it could happen, you know, 17.3 minutes after.

17 That doesn't necessarily start the PRA 18 scenario. The PRA scenario starts when you get some 19 damage to equipment that can affect the nuclear power 20 plant. Cooling the core in particular.

21 MS. PRESLEY: So, that point is 22 not -- this timeline's a simplification. So --

23 CHAIR STETKAR: But see, in supplement 1, 24 indeed, that's one of the things I looked at. And 25 supplement 1, perhaps, is not belabored as much as we NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

82 1 just belabored it. But it is -- that distinction is 2 made in supplement 1, that the reactor trip doesn't 3 necessarily need --

4 MEMBER KIRCHNER: Yeah.

5 CHAIR STETKAR: -- to have gotten 6 started --

7 MEMBER KIRCHNER: Because coincident was 8 your fire start.

9 CHAIR STETKAR: T zero starts the fire.

10 MR. AMICO: Paul Amico from Jensen Hughes 11 again. Yeah, but again, is it -- it's more like a 12 typical PRA assumption that we -- that we don't -- we 13 look at everything. For example, all the failures that 14 occurred due to the fire, we assume they happen 15 immediately, as was brought up.

16 It's similar to when we look at a failure 17 to run, for a pump. It could really occur any time.

18 And if it occurs anytime, it could change the amount 19 of time you have left to do some action. So, we don't 20 get into that. A failure to run, we assume it happens 21 at T zero, whether it does or not.

22 And so, the thing here is, we're talking 23 about is that in the PRA assumption, we bring everything 24 back to happening simultaneously, so that you get the 25 worst effect -- the least amount of time, the least NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

83 1 amount of heat removed from the plant.

2 If you say everything happens at T zero, 3 I've got the most latent heat, I've got the most decay 4 heat, I've lost all my equipment at that time. And 5 that's not unique to the abandonment scenario.

6 That's not unique to buyers. That's 7 pretty much the way we do all of our PRAs.

8 CHAIR STETKAR: Okay, that's -- and 9 again, you're speaking of the way you do all of your 10 PRAs. I've done PRAs that do look at the timing of fire 11 damage.

12 MR. AMICO: And indeed PRAs do look at the 13 timing of fire damage. It's called fire growth 14 detection, fire ignition detection, growth and 15 suppression, so that you get a sequence of extending 16 damage, depending on how long the fire exists, and where 17 it is.

18 And indeed it's not unique to fires. For 19 example, if I look at losses of offsite power and 20 station blackouts, I parse my models -- and I'm saying 21 now, I, rather than we, as the royal 22 everyone-does-it-this-way -- I parse my model in terms 23 of, what's the likelihood that I recover offsite power 24 within X minutes? And if I do, I have certain 25 opportunities.

NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

84 1 Why minutes? I have different 2 opportunities -- zed minutes -- I have three 3 opportunities, and maybe, you know, one more. So, this 4 course distinction of times is not what everyone does.

5 I mean, it's not what everyone does on 6 these time-sensitive things, to always presume 7 everything is worse happening at T zero. That's a 8 holdover from LOCA analyses. It's not necessarily the 9 best thing to do for fires.

10 MS. PRESLEY: And because it shows on this 11 view graph, that doesn't mean that we are requiring --

12 CHAIR STETKAR: No.

13 MS. PRESLEY: So, I want to clarify. So, 14 for this example, we're not requiring that. We 15 acknowledge it as a typical simplification. But we're 16 not requiring it. But this --

17 MEMBER KIRCHNER: What I was objecting to, 18 I guess, is making Rx trip coincident with fire starts, 19 or fire alarm. The fire might go on for quite a while 20 before you realize you have a problem. And then, much 21 later in the scenario, you trip the reactor, manually 22 or automatically.

23 And then, you may have a heck of a lot less 24 time to do other actions to mitigate the problem. So, 25 I'm just -- maybe this is misleading me and this is not NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

85 1 the way you actually do the analyses, but putting Rx 2 trip coincident with the fire alarm seems magical.

3 DR. COOPER: Susan Cooper. I agree. I 4 agree.

5 MEMBER KIRCHNER: Magical, in fact.

6 DR. COOPER: I'm not sure that's the point 7 that she wanted to make with this graph. And I 8 guess -- but I agree with you. Probably at sometime 9 after the fire is acknowledged and the local operator 10 has been found out what -- after he made his 11 investigation and calls back to the control room and 12 they make an assessment as to where that fire is now 13 located, and now they'll have some guidance about 14 whether or not they're going to have a reactor trip.

15 So, I think there's -- and we'll fix this.

16 I don't think this graph is actually in the report.

17 CHAIR STETKAR: That is not.

18 DR. COOPER: Yeah, it's not in the report 19 yet. It's something we're trying to use to help 20 explain some things. So, the feedback you've given 21 here to help will help us decide whether we want to use 22 it or not.

23 But the point is that we're trying to 24 illustrate that we understand that there is a 25 progression of things. And this is -- and she's got, NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

86 1 I think she said, 12. So, we do have multiple cuts.

2 It's not just one picture.

3 CHAIR STETKAR: When you think of these 4 things, I always -- you know, we -- we get trapped in 5 this too, so I can say we. We tend to get pigeonholed 6 into things. This, for some reason, seems to be the 7 guidance for quantifying main control room abandonment 8 for fires that happen only during full power operation.

9 Now, I think -- the fire doesn't know 10 what's going on. The people in the control room live 11 in the control room 24/7. So, think about shutdown 12 conditions. There's no reactor trip. Reactor trip is 13 shut down. On the other hand, the fire has 14 all -- causes some damage that results in a trajectory 15 of the plant, and results in responses of the people.

16 Now, if the plant happens to be operating 17 at power, one of the things that the fire might do is 18 trip the reactor. One of the things that the people 19 might do is trip the reactor.

20 But that's irrelevant to guidance for how 21 to think about this in the sense of a PRA. Fire damage 22 begins at zero.

23 DR. COOPER: We have not develop guidance 24 for shutdown fire.

25 CHAIR STETKAR: But no. My point is that NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

87 1 shutdown fire is no different -- the fire doesn't know 2 and the people don't know.

3 DR. COOPER: So --

4 CHAIR STETKAR: So, why do we have to 5 pigeonhole the guidance for, you know --

6 MS. PRESLEY: The point of this is to say 7 that timing is not as clear cut as in -- and I think 8 we've all -- we've just furthered the demonstration 9 that this is true. So, even with my simplified 10 version, which is not clear cut, it could be even more 11 less clear cut.

12 So, at some point you're going to have a 13 fire. At some point you're going to have an alarm.

14 Somewhere you're going to have a reactor trip. Then 15 you do your immediate memorized actions. When you get 16 your alarm, you send someone to confirm it locally.

17 That person eventually will come back and 18 report with, you know, what's going on and where the 19 fire is, and confirm that you have a fire. Then, you're 20 going to start seeing at some point instrumentation 21 failure.

22 And then, at some point you're going to 23 decide that that instrumentation failure is 24 significant enough that your major systems and 25 instrumentation has been impacted.

NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

88 1 And then, you're going to get to the point 2 where, yes, I think I've lost control. I need to 3 abandon. So, that's -- so, the order of those things 4 happening, and how compressed or elongated they are, 5 depends on the fire modeling piece and what scenario 6 you're modeling.

7 But you can't just look at, you know, this 8 cue says I need to look at these three pieces of 9 equipment. When does my modelings of these three 10 pieces of equipment are gone?

11 You have to look at the totality of the 12 picture. So, thank you for proving my point for me.

13 MS. LINDEMAN: We'll get a better picture 14 --

15 MS. PRESLEY: Yeah. And there's, you 16 know -- like I mentioned, this is one snapshot of one 17 kind of progression we looked at. We looked at a 18 half-dozen of these, and it depends on what your cue 19 says.

20 So, if your cue says consider abandoning 21 after a fire in the cable spreading room after local 22 confirmation, then in this particular graph, local 23 confirmation comes before you have major systems 24 identified.

25 Well, are they really doing to do that? Or NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

89 1 are they going to wait a little bit? Are they 2 going -- you have to start asking questions about 3 training.

4 Are you trained to confirm any particular 5 equipment? Or do you take this verbatim? So, the 6 procedure progression and the fire growth progression 7 need to be matched up so you understand what's 8 happening.

9 That was kind of the point of this graph.

10 So, if you want to go to the next one.

11 MEMBER BLEY: But before you -- before 12 you --

13 CHAIR STETKAR: I'm sorry. I'm sorry I 14 was away for a little while.

15 PARTICIPANT: I have one comment.

16 CHAIR STETKAR: Hold on. Jazz, let 17 him --

18 PARTICIPANT: Oh, I'm sorry.

19 CHAIR STETKAR: -- finish talking, 20 please.

21 MEMBER BLEY: It's hard to hear back 22 there. If you want to keep something like this, you 23 know, the words here make sense. The discussion makes 24 sense. But I'd really urge you, if you're going to keep 25 something like this, that you keep at least a couple, NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

90 1 to say, there's a couple of different ways this could 2 happen.

3 There are lots more. But showing one kind 4 of anchors in somebody's head that, yeah, the first 5 thing that ever happens is that the reactor gets 6 tripped. And then we start watching the fire. And 7 they happen in this order, and that's not -- your point 8 was they don't.

9 MS. LINDEMAN: And I think what's in the 10 report is actually just a bullet list of the things that 11 you consider.

12 CHAIR STETKAR: In supplement 1 -- I'm 13 staring at it right now -- in chapter 7 of 14 supplement 1, there is a discussion about this relative 15 timing, and it's clear that zero is when the fire 16 starts, and things happen after that.

17 In the latter part of section 7, it says, 18 PRAs often simplify this by assuming the reactor trip 19 occurs at zero, but the construct in supplement 1 20 actually does distinguish between the -- JS.

21 MR. HYSLOP: Yeah, this is JS. I guess 22 I'm not sure I need to say what I was going to say since 23 you just did, but often, yeah, the trip is assumed at 24 T equals zero, often. And some of the reasons 25 sometimes is just we haven't traced all the cables, so NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

91 1 we're not sure what will happen.

2 But there is a fire growth modeling that 3 occurs, and if you wanted to take advantage of that and 4 say you didn't damage in the target sets, I guess you 5 could delay it.

6 CHAIR STETKAR: Any more discussion on 7 step one, because if there's not, I'm going to call for 8 a break, because we're going to shift gears here to talk 9 more in terms of models and methods, and that sort of 10 stuff, and we need a break. So, let's take a break and 11 return at 3:15.

12 (Whereupon the above-entitled matter went 13 off the record at 2:56 p.m., and resumed at 3:16 p.m.)

14 CHAIR STETKAR: We're back in session.

15 MS. PRESLEY: Okay. So, Step 2 which we 16 did in parallel with the issues list, actually a little 17 bit of an iterative process, but we listed several HRA 18 quantification approaches to see what was out there, 19 what might be used, and what we might be able to pull 20 from. And the ones on the screen are the ones we looked 21 at.

22 The cause-based decision method and 23 decision tree method, HCR/ORE, which is a time 24 reliability method.

25 We looked at IDHEAS At-Power.

NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

92 1 We also looked at -- we're familiar with 2 IDHEAS G and we looked at the cognitive literature stuff 3 underlying it. So while that wasn't one of the 4 quantification methods we looked at specifically, we 5 did look at the failure mechanism there.

6 We looked at SPAR-H.

7 We looked at NARA and CREAM.

8 So if you want to go the next.

9 MEMBER BLEY: NARA's still not public; 10 right? Except for a couple papers?

11 DR. COOPER: Susan Cooper. Yes, that's 12 correct.

13 And we did, we'll see later that we did use 14 NARA. And that was from one of the papers. Among the 15 authors on the team we do have access to the non-public 16 version and can make internal comparison. So, it's 17 good.

18 MS. PRESLEY: So, our initial, so after 19 we'd looked at this set of methods we thought maybe we 20 could adapt the cause-based deficiencies. At the high 21 level, the failure modes, the operator, information 22 interface and the operator procedure interface, 23 failure modes seem to be potentially applicable.

24 So we started there and we went through the 25 review process. And we went tree by tree and we asked NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

93 1 is the failure mode of the tree still applicable? We 2 looked at each branch point and kind of performance 3 shaping factor associated with the tree and we said, 4 Are those the right ones for the failure mode? Are they 5 still applicable? And are there dominant failure 6 modes or mechanisms missing from the set of trees, from 7 the set of CBDT trees that we should, we should be 8 accounting for?

9 And as we went through we found that at the 10 failure mode level it was pretty applicable. But once 11 we started going through we were stretching the method 12 more than maybe it was intended to. We were 13 reinterpreting things.

14 For example, four of the trees talk about 15 the operator procedure interface. But what procedure 16 step are you really talking about? We found we were 17 convoluting the transfer to get to the abandonment 18 procedure and then the actual decision to abandon cue.

19 We were conflating the two as we went through the 20 case-based decision trees.

21 We saw some, or there were binary decisions 22 where we thought more gradation might be warranted.

23 And some of the trees just required more 24 reinterpretation than we were willing to make.

25 So we said, okay, we'll pull what we can NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

94 1 from it and what we think is important, but we're really 2 not going to go with the existing method and try to just 3 adopt that wholesale.

4 So we ended up with three new decision 5 trees:

6 Failure to transfer to the main control 7 abandonment procedure; 8 Failure to understand the criteria have 9 been met; 10 And then, so, if we get to the criteria then 11 we have to understand the criteria have been met. And 12 then once you understand the criteria have been met, 13 you have to actually make the decision.

14 So the last one is reluctance of delay in 15 making the decision.

16 MEMBER BLEY: In the report itself isn't 17 there just one?

18 MS. PRESLEY: In the report there is one.

19 So this is what we went into from the discussion.

20 So, so that formed the discussion that the 21 skeleton of what we discussed with the -- if you want 22 to go to the next slide.

23 MEMBER SUNSERI: Wouldn't the reluctance 24 to delay just be a subset of one of the other ones?

25 MS. PRESLEY: Well, that's what the NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

95 1 experts told us. So, so yeah.

2 We, we thought it might be a separate 3 factor after we looked at it, but it turns out that 4 reluctance is primary driving factor from the expert 5 elicitation is that reluctance is key. And it 6 underlies everything. So it will always be there. So 7 it's built into the base HEP.

8 There wasn't a separate branch point 9 called out reluctance, or there weren't levels of 10 reluctance based on the quality of the remote shutdown 11 panel because an operator doesn't know that plant X has 12 a great remote shutdown panel, but theirs is really 13 crumby. They just know that they have a remote 14 shutdown panel and it's not the control room.

15 So they, they might not -- they would not 16 necessarily have more or less reluctance than another 17 operator at a more or less capable plant.

18 So, so one of the, one of the key points 19 was that reluctance needs to be built into a base HEP, 20 and we don't need to look at factors around that.

21 We also learned that timing, which we 22 thought would be important, was going to be a key 23 differentiator.

24 And then the other piece was we thought 25 procedures were really important: how you get to the NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

96 1 procedure and the quality of the procedure. And there 2 was a lot of discussion about the process the operators 3 go through of integrating information to always be 4 looking ahead at what's coming, and that they would have 5 an idea of what the criteria are based on their 6 training.

7 And so there are prudent actions that all 8 operators they need to understand as part of their 9 fundamental training. And if they get -- start seeing 10 indications of spurious operations, they're going to 11 start thinking about the abandonment criteria.

12 And so that was more important more of a 13 dominant factor than just the procedures themselves.

14 So, so that was also, from an offsetting perspective, 15 built into the base HEP. And it actually goes with 16 training as well. So we pulled out training as one of 17 the branch points in the final, in the final tree.

18 MEMBER BLEY: Let me ask a kind of broad 19 question about that.

20 Let's say 20, 30, 40 years ago we would 21 argue the same things about the emergency operating 22 procedures, the LOCAs and all that kind of stuff. And 23 we did.

24 And then we saw more and more. And some 25 utilities ran tests of their old experienced operators NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

97 1 who produced relevant, newer procedures. And pretty 2 soon everybody was coming on board. But they have a 3 better chance to do well with more elaborate 4 procedures.

5 But that only came after a lot of 6 exercising the procedures and getting used to them. We 7 don't exercise these procedures very much. I don't 8 know how much people even look at them when they're not 9 in the midst of it. And most of them are never in the 10 midst of it.

11 And do you have a sense for whether that's 12 what's going on here, or is it just there's something 13 about this that doesn't lend itself to writing a linear 14 procedure that will be helpful to you?

15 MS. PRESLEY: So, in the process of doing 16 the HRA some of our team members have found that there 17 was great potential to improve the procedures. There 18 were extra actions in there that people, well, while 19 you're there we're going to do it, but it would 20 challenge the time, the feasibility from the timing 21 perspective.

22 So there was definitely a lot of refinement 23 possible. But from a cognitive perspective I think we 24 don't understand the exact progression that is here and 25 exactly what the operators will be seeing. And that NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

98 1 was talked about a lot. So, having some high level 2 criteria and training the, training that could be a 3 little more agile is important where, where we just 4 don't know how the scenario might unfold.

5 One of the things the experts did talk 6 about was management philosophy and whether or not the 7 operators are trained to think at the higher level and 8 encouraged to do that. So, I think, I think in this 9 particular scenario we can't be so prescriptive because 10 we just don't know what it's going to look like. And 11 I don't know if --

12 DR. COOPER: Yeah. Susan Cooper. I'll 13 just add to that.

14 There seems, despite the fact that we had 15 experience among the author team with making changes 16 to procedures, getting certain utilities to buy into 17 changing the procedures to have more explicit criteria 18 for the decision to abandon the loss of control, when 19 we got the expert panel together there seemed to be some 20 pushback on making, you know, funneling analysts to 21 saying that's the solution to this problem as opposed 22 to looking more at the experience of the operators and 23 not getting sort of fenced in into particular criteria.

24 Exactly where that's coming from is hard 25 to say. But certainly we don't have any experience to NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

99 1 show us which is the right way at this point. I don't 2 think we've run across any plant that is doing training 3 on the decision to abandon, except in the classroom.

4 MEMBER BLEY: Thank you.

5 MS. PRESLEY: There is one that was 6 brought up during the -- there was one plant that was 7 brought up during our elicitation that they were doing 8 it. But even in that case they said, you know, we do 9 the classrooms training and then we go into the 10 simulator to do it. So they're sort of seeded to think 11 abandonment already. So maybe it's not realistic.

12 MEMBER KIRCHNER: How often is the remote 13 shutdown panel tested, "tested" at a typical plant?

14 Once a week, once a month, once a shutdown, once a 15 refueling?

16 MS. PRESLEY: What do you mean by 17 "tested," like trained on? Or even observed and 18 checked.

19 MEMBER BLEY: To make sure the switches 20 work and that kind of stuff?

21 MEMBER KIRCHNER: Everything, yeah.

22 MR. AMICO: This is Paul Amico again from 23 Jensen.

24 So, the question is like to actually 25 operate things? Because they don't --

NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

100 1 MEMBER KIRCHNER: Well, or check that it's 2 operating.

3 MR. AMICO: Yeah. I mean, they don't 4 actually go through and start pumps and things like that 5 using that panel.

6 MEMBER KIRCHNER: No. But I mean at least 7 inspect it.

8 MR. AMICO: Right.

9 MEMBER KIRCHNER: Ensure that it's --

10 MR. AMICO: It will vary from plant to 11 plant. Jeff, I don't know if you have the -- you know, 12 at least during the refueling outage they would check, 13 they would energize it to check the meters come on and 14 everything like that. But every plant would do it 15 differently.

16 DR. COOPER: Susan Cooper. There are 17 some plants that have their remote shutdown panels 18 energized all the time.

19 MR. AMICO: That's true.

20 MEMBER KIRCHNER: That's what I would 21 expect. That's why I was asking.

22 DR. COOPER: But they still have to 23 transfer control from the control room to the remote 24 shutdown panel. I don't know when that testing is 25 done. I seem to remember that Paul or one of our other, NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

101 1 other authors in the audience, has run across that that 2 may not happen so often.

3 MEMBER SUNSERI: So I don't, I mean I may 4 be mixing up, and I'm sure I am mixing up some things 5 here, but I'll ask for some clarification. It seemed 6 to me like I recall during triennial fire inspections 7 every three years a reg guy goes out to the plant, 8 inspects. And they walk the operators through a lot 9 of these fire scenarios and actually time them to make 10 sure that the timed responses are within the regulatory 11 or the analyzed limits.

12 It would seem to me that that exercise or 13 that piece exercises some elements of what you're 14 talking about up here. So I guess I'm surprised that 15 operators then are reluctant, because they do get a 16 chance periodically to walk through these procedures 17 and become more comfortable with them.

18 Can you comment?

19 DR. COOPER: Susan Cooper. So, so the 20 part where they've left the control room and they're 21 taking actions, either at the remote shutdown panel or 22 at local plant stations, yes, it has been and is being 23 exercised quite a lot. You could say it's familiar in 24 a certain sense.

25 The parts that aren't exercised are things NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

102 1 like the decision to abandon the control room. What 2 we've heard I think almost universally is the way that 3 they train abandonment scenarios starting in the 4 simulator is that the trainers for the operators will 5 say you have an abandonment scenario, show me what 6 you're going to do next. Which really is about 7 exercising all of the steps that are needed to be taken 8 in the control room before you can leave.

9 Like what actions do you have to take to 10 transfer control; what other pieces of equipment you 11 might need to operate if you haven't tripped; or 12 whatever, those kinds of things that will be exercised 13 but not the decision.

14 The other piece that's not exercised, 15 which we're going to talk about after Phase II, and that 16 is what's happening at the remote shutdown panel with 17 the supervisor who is trying to keep track of all the 18 things that are going on either at the remote shutdown 19 panel and the variety of local plant stations out at 20 the plant. You know, his, his communication with them, 21 the way he keeps track of going to the procedure and 22 so forth, that also -- you know, there have been some 23 plants that have said they've done that, but that's 24 rare.

25 So that piece also is not trained often.

NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

103 1 MS. PRESLEY: Okay. And so the last piece 2 is that we just wanted to point out, because this came 3 up in the expert elicitation and the bounds of it, our 4 estimation was based on current U.S. fleet. We did 5 include performance working factors that may be like 6 training, above what the current fleet is doing, to show 7 that there are potentials for improvement. But we 8 didn't look at what new designs were. They might have 9 a second main control room which has, you know, 10 virtually the same capabilities.

11 So that wasn't in the scope of the what 12 we're looking at.

13 So, Step 6. Our drumroll please.

14 Our final tree that we came out with with 15 the -- and consolidating the three trees into one. And 16 as I mentioned, the reluctance, you don't see a branch 17 for that. That was baked into the base HEPs, which is 18 why they all start fairly high. And the way to -- I'll 19 talk across the branches.

20 The first was abandonment logic explicit 21 in the procedures. So is there some indication of what 22 the criteria are?

23 MEMBER BLEY: Do you mind if I interrupt 24 you just a second.

25 MS. PRESLEY: Yes?

NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

104 1 MEMBER BLEY: For the other members, if 2 you haven't looked at decision trees and HRA methods, 3 don't get confused. They are not like event trees.

4 They're really just a truth table of conditions. And 5 when you analyze your plant, you know what the 6 conditions are. You come into this tree and say for 7 this scenario, under these conditions it's a certain 8 place here. It's just saying, yeah, I do have the 9 criteria document in the procedure. It's only 10 classroom training and it's not -- if you work your way 11 through this tree you come in with your conditions, 12 match it up, and read the number off the end.

13 I'm sorry. Just this confuses people.

14 MS. PRESLEY: Thank you very much. Yeah, 15 thank you very much for that background.

16 So the first question that we ask then is, 17 is the abandonment logic explicit in the procedures?

18 Now, we mentioned that it's not as much of 19 a driver as we initially thought, but it's still an 20 important driver. And instead of three gradations, 21 which we originally had, we pared it down to one. And 22 is it really purely judgment-based or is there some 23 criteria that are documented in the procedure that the 24 operators can refer to and monitor?

25 So you can go up or down based on your NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

105 1 answer to that question.

2 The second question asks for simulator or 3 talk-through training specifically on the decision to 4 abandon. So everyone kind of gets the training on the 5 abandonment procedure but have they actually simulated 6 what they're going to see and how they're going to think 7 about the decision and how they're going to make that 8 decision?

9 So, while this is not standard practice, 10 this is one area that people could improve. So the 11 question is, is it classroom only or do they go that 12 one step above where they actually do talk-throughs or, 13 again, a simulator run?

14 And so then the third question has to do 15 with training and understanding of the scenario. So, 16 do they have an awareness of the urgency? And this gets 17 to different, different scenarios progress at 18 different rates.

19 So how much time you have is important, but 20 it's equally important to know if the operators 21 understand how quickly they need to make the decision.

22 So this, this gets --

23 MEMBER BLEY: Sorry to interrupt again.

24 MS. PRESLEY: Yes?

25 MEMBER BLEY: When I read something NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

106 1 earlier in these where it said is this time Phase III.

2 Well, this is the time from when they decide to abandon 3 until the PRA shows when they wouldn't be able to get 4 everything done. That's not something in an 5 operator's head.

6 MS. PRESLEY: No. Yeah, so that is the 7 operator will not know if they have 5 minutes, 25 8 minutes. But they would know, they would understand 9 if it's a fast progressing or slow progressing fire.

10 They would be able to tell you what are the types of 11 things they might check in different scenarios before 12 they would make the decision.

13 So the kinds of questions that you should 14 be asking it's do you know you have 5 minutes, or how 15 much time do you think you have? It would be questions 16 like, okay, what would you check before you made the 17 decision? Or how would you make the decision?

18 And then you can start getting an idea of 19 how much time they think they have and what they would 20 do. And that would give you an understanding of 21 whether or not they have an awareness of the urgency.

22 MEMBER BLEY: Question. I haven't looked 23 at these procedures in quite a few years so I don't know 24 what they look like now. Do they give hints of the 25 urgency and time available to carry out all this stuff NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

107 1 to the operators?

2 MS. PRESLEY: I don't believe they do.

3 But Paul can.

4 MR. AMICO: Okay. Paul Amico again from 5 Jensen Hughes.

6 So, it really varies a lot from plant to 7 plant. Unlike EOPs, these fire abandonment procedures 8 are completely different. There's no Westinghouse 9 approach, or CE approach to an abandonment procedure.

10 So it varies greatly.

11 Some of them are changing. As a result of 12 the fire PRAs and 805 -- and then J.S. can talk to this 13 also because he's familiar -- the plants are realizing 14 that they do need to make improvements. So you are 15 seeing some cases now where it's sneaking into the 16 procedure that you need to make this decision within 17 25 minutes, or some range. So it varies greatly, 18 Dennis.

19 It's just but we're starting to see 20 improvements in that, also improvements in the 21 training. So, so it's one of the benefits.

22 MEMBER BLEY: It's like I remember but 23 it's getting better.

24 MR. AMICO: Yeah. Yeah. It used to -- I 25 mean, some of these procedures were so simple, like when NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

108 1 the shift manager feels that they can no longer control 2 the plant from the control room, consider abandonment.

3 MEMBER BLEY: That's it.

4 MR. AMICO: And, you know, and it's 5 getting better because the PRAs are starting to say 6 these are the things that you are likely to see. And 7 they're putting that in as the cues to consider. And 8 they're giving some indication of the urgency.

9 MEMBER BLEY: That's good. Okay, thanks.

10 But it's still varies.

11 MS. PRESLEY: Yes. So sometimes you have 12 time-critical actions linked with fires in different 13 locations, but it's definitely all over the map still.

14 And then so after --

15 MEMBER BLEY: One more thing. Is the 16 staff doing anything about that to encourage that these 17 become more consistent and useful to the operator?

18 MR. HYSLOP: Well, I was just going to --

19 MEMBER BLEY: You've got to say who you are 20 again. I'm sorry.

21 MR. HYSLOP: J.S. Hyslop. We are seeing 22 in our, you know, we had an application meeting with 23 the licensee and we didn't encourage it because, you 24 know, we were at a certain stage of analysis and stuff 25 like that. But they came in and said we're going to NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

109 1 re-do our analysis. So this is one of the better 805 2 main control analyses that was done.

3 I said, Why are you doing that? You did 4 one of the betters.

5 He said, We're updating our procedures to 6 be more specific on the criteria for abandonment.

7 And so I don't know, I don't think, I don't 8 think the staff is encouraging it at this point, but 9 we're hearing it happen.

10 MEMBER BLEY: Do you have any hints as to 11 whether INPO's pressuring this work or not?

12 MR. HYSLOP: I don't.

13 MEMBER BLEY: Okay.

14 MS. PRESLEY: So then the last, the last 15 thing it asks about is how much, how much time to you 16 have available to make that decision. And the way we 17 calculate time available is we look at how long it takes 18 to actually be successful in the Phase III action and 19 see what the leftover time is. And that's your time 20 margin to make a decision. So, basically, that's how 21 much you can delay before the downstream actions won't 22 be effective anymore, so.

23 DR. COOPER: If I can just add. The point 24 is, as Matthew was saying, the actions taken to shut 25 down the plant after you've left the control room are NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

110 1 practiced. You can get times for those. And you can 2 also talk about you'd need a little extra time for 3 recovery, or you certainly factor in travel time. So 4 you can, you can estimate all those times based on 5 actual demonstrations.

6 And then you can total up all those times, 7 using the integrated time lines to account for overlaps 8 and so forth, and then you can get an estimated time 9 for implementation of the safe shutdown strategy. You 10 might want to give yourself a little bit of a margin.

11 But the point is you have an overall system window that 12 you've calculated for your thermal hydraulics on when 13 you reach core damage.

14 So you subtract off what you think you need 15 for implementing all those act -- executing all those 16 actions for shutdown, and also subtracting off when you 17 got the cue, and then you've got something left over 18 for your decision. That's, so it's a calculated thing.

19 Since it's not trained in the control room, it's not 20 like we can go out and collect data on it or anything 21 like that.

22 MS. PRESLEY: And next slide.

23 MEMBER SUNSERI: Well, I had one question 24 on that slide before you leave it.

25 Does it -- my mind can't go through the NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

111 1 permutations as quickly as you start it -- but does it 2 make a difference on what order you ask these questions?

3 It seems like if you brought the awareness question up 4 earlier it might change the outcome.

5 You know, because if you are unaware that 6 you're in an urgent situation and you're using your 7 judgment, not procedures, to drive the outcome, you 8 might get in a different place than you would if -- see 9 what I'm saying?

10 MS. PRESLEY: Yes. So I should probably 11 describe how the numbers came to be.

12 So, the numbers came to be we did, we asked 13 the experts to give us estimates for the worst case, 14 which is judgment-based cue, only classroom training, 15 and no awareness of urgency, and for 5 minutes and 25 16 minutes. And then they gave us the best case 17 probabilities.

18 And then what we asked them to do was a 19 parallel comparison between each branch point pairs.

20 So they actually went scenario by scenario and said we 21 think this scenario is better or worse than that 22 scenario.

23 So in that case it doesn't matter whether 24 you said it first or not. They thought about the 25 scenario as a whole when they compared it against.

NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

112 1 MEMBER SUNSERI: Okay, so I think I see.

2 It doesn't matter if you ask the three questions in any 3 order, you still get through the branches and you still 4 get through the --

5 MS. PRESLEY: Yeah.

6 MEMBER SUNSERI: I get it.

7 MS. PRESLEY: Yeah.

8 MEMBER BLEY: Your concept might be right.

9 MEMBER SUNSERI: Yes.

10 MEMBER BLEY: But this is simply a truth 11 table.

12 MEMBER SUNSERI: Yes.

13 MEMBER BLEY: And so if both of them are 14 in one direction it's under that condition of both being 15 one way or both being the other. It's not a scenario 16 in the sense of when we think of something happening 17 over time.

18 CHAIR STETKAR: I guess -- don't switch it 19 yet. Leave it on the screen.

20 I very much appreciated the discussions in 21 Supplement 1 and Supplement 2 about how complex all of 22 this is and all the variability and how some plants are 23 better and some plants are worse. And we've heard a 24 lot about that today. What I was really surprised 25 about was that your experts, considering all of that, NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

113 1 have assessed that there's between an 80 percent and 2 98 percent success rate of always doing this successful 3 with enough time left.

4 To me that seems really darned good, given 5 all of the stuff we've heard about variability, about 6 reluctance. This says the worst it can be is they'll 7 be 80 percent successful in leaving with enough time 8 left to do everything they need to do in Phase III.

9 That's the worst it could be, an 80 percent success 10 rate.

11 A lot of students in school would be really 12 happy after a lot of studies to have an 80 percent 13 success rate. So I'm really curious how your experts 14 came up with this very, very highly likely success, with 15 no uncertainty by the way, because Appendix A where you 16 discuss the expert elicitation specifically says that 17 you didn't look for uncertainty in the expert 18 elicitation process.

19 MS. PRESLEY: So I'm going to caveat that 20 with it's 80 percent given we've met the feasibility 21 criteria.

22 CHAIR STETKAR: Sure. Obviously, yes.

23 MS. PRESLEY: So you've already met a 24 fairly high bar to get into that. Right? You've 25 weeded out your students that didn't study.

NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

114 1 CHAIR STETKAR: This is a plant where it's 2 left up to the supervisor. If you think it's a good 3 idea, get the heck out of the control room. That you've 4 talked about it in a classroom. And the people aren't 5 aware of the urgency that I need to get out of here, 6 and I, by the way, I have to make that decision within 7 5 minutes, and I'm still 80 percent successful. Eighty 8 percent successful.

9 DR. COOPER: Susan Cooper. Feasibility 10 also requires even if there's judgment in making the 11 decision only, the procedure only has judgment as a 12 criteria, there needs to have been interviews of the 13 operators and a consensus opinion on what constitutes 14 a loss of control where they would leave the control 15 room that has to have been obtained.

16 So there is another piece to that bar. So 17 that's an important piece of the feasibility.

18 So it's not like they have no notion of what 19 it is that would represent or be defined as loss of 20 control.

21 CHAIR STETKAR: Right.

22 DR. COOPER: So there is another little 23 bit more there.

24 CHAIR STETKAR: And in your interviews you 25 make it very clear that loss of control is they have NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

115 1 to know that if the core is going to melt they're going 2 to leave. That's the way you've characterized it, is 3 that loss of control is the plant is on a trajectory 4 that you cannot prevent the core from melting from 5 inside the control room. Look in your guidance.

6 And, of course, if I'm an operator I'm 7 going to say, Well obviously if I know the core is going 8 to melt I'm going to leave.

9 DR. COOPER: Susan Cooper. If it doesn't 10 say it, it should say this because what the operators 11 believe and what we've discussed in detail is that they 12 need to tell us what those conditions are. Not that 13 we tell them that this is it. They have to tell us what 14 those conditions are.

15 And then and it has to be a consensus answer 16 from more than one operator. So it's not like we tell 17 them this is what the situation is, are you going to 18 leave? It's what are the conditions that you think 19 define loss of control? And that has to be a consensus 20 answer.

21 MEMBER KIRCHNER: Susan, one thing on this 22 chart. Is T available for decision? It's a 23 calculated quantity?

24 DR. COOPER: Yes.

25 MEMBER KIRCHNER: From a PRA or whatever?

NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

116 1 DR. COOPER: No. It's calcul -- well, it 2 has a number of different inputs. The first and most 3 important input is going to be some kind of thermal 4 hydraulic calculation associated with the scenario 5 that says when core damage would be reached if you 6 haven't been successful with the shutdown strategy.

7 Other pieces of it are going to be, for 8 example, I mentioned you're going to, for each of the 9 actions that are required for shutting down the reactor 10 after you've left the control room there have been 11 demonstrations on those actions and you can collect 12 times on those. And that was discussed even in 1921.

13 We have some more I think in Supplement 1.

14 So those, those pieces so you can measure 15 out how much time you need for, for Phase III, and you 16 can add in some time for some communications and so on 17 and so forth. And then you need to also subtract off 18 some time to when you think the cue is going to come 19 in.

20 But that's, so that calculated -- that's 21 a calculated time with a number of different timing 22 inputs. But it's a back calculation from all of those 23 different things.

24 MEMBER KIRCHNER: Yeah, I understand 25 that. I'm just going through this logic and thinking NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

117 1 I'm the operator, I don't know whether I'm going to have 2 5 minutes, 5 to 25 minutes, or 25 minutes.

3 DR. COOPER: No, you don't. You don't.

4 It's an HRA calculated to make a distinction between 5 if you have 5 minutes to make the decision, we're making 6 a distinction saying that that's going to be more 7 difficult, or we think it's going to be less likely that 8 they can --

9 MEMBER KIRCHNER: I agree with the notion 10 that if you have more time to make a decision the human 11 error probability of the first order should go down.

12 DR. COOPER: That's what it's, that's what 13 it's supposed to say, yes. That's what I would expect.

14 MEMBER KIRCHNER: But I don't know that as 15 the operator, as this fire commenced --

16 DR. COOPER: No, you don't.

17 MEMBER KIRCHNER: -- and I'm starting to 18 make this decision.

19 DR. COOPER: You don't.

20 MS. PRESLEY: So the part that you know, 21 so everything on this side is things the operator would 22 know. So you would, you would have, either you would 23 or you wouldn't have an awareness of the urgency. You 24 would either understand it as progressing quickly.

25 MEMBER KIRCHNER: Well, I'm going to tell NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

118 1 you as an operator, former operator, if I'm thinking 2 about abandonment, this is urgent.

3 MS. PRESLEY: Yes.

4 MEMBER KIRCHNER: So the answer is yes, 5 I'm already under the gun. I feel the pressure. We 6 need to make a decision.

7 MS. PRESLEY: So all of this is just saying 8 if we have more margin we may be more successful. So 9 this has nothing to do with whether or not the operators 10 understand how much time they have. It's just purely 11 how much margin do you have.

12 CHAIR STETKAR: And if you have -- again, 13 I'm personally very skeptical of these numbers that 14 they've got here.

15 MEMBER KIRCHNER: I am.

16 CHAIR STETKAR: Because if you have more 17 margin, yeah, you get the difference between 80 percent 18 success rate and 98 percent success rate. Both of 19 those to me are very, very successful.

20 Now, if it was a difference between 1 21 percent success rate and 98 percent success rate I'd 22 say, yeah, maybe you thought about the problem a little 23 bit. But I don't think your experts thought about the 24 problem, quite honestly. That is my personal opinion.

25 DR. COOPER: Well, so the -- I mean, I NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

119 1 don't know if it's helpful.

2 CHAIR STETKAR: Because, because I can't 3 read anything in your appendix that tells me what your 4 experts thought about or what their uncertainties were 5 or how they, how they did their evaluations.

6 DR. COOPER: Okay.

7 CHAIR STETKAR: It simply reports their 8 results as point estimates.

9 DR. COOPER: So are you asking for the 10 answer or are you asking us to put the answer in the 11 report?

12 CHAIR STETKAR: I would really like to see 13 -- I mean, if you have the answer that would be great.

14 But I'd certainly like to see it for the benefit of 15 people who use this guidance.

16 DR. COOPER: Sure.

17 CHAIR STETKAR: Because if users look at 18 the rationale that your expert team used and said, Well, 19 gee, maybe -- I'm not sure that anyone would but I'll 20 leave the field open -- I'll look at that rationale.

21 I'm not sure that that really applies for my plant. And 22 maybe, maybe I want to do things a little bit 23 differently. Maybe I want to -- I'll use this 24 conceptual framework but let's skew things differently 25 for my plant.

NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

120 1 Now, I can't do that unless I understand 2 what rationale your experts used and if they did have 3 uncertainties, if there are ranges. I mean, if this 4 is a consensus opinion and two people said it's 1.0 and 5 another person said that it's, you know, .007.

6 MS. PRESLEY: It ranged from .1 to .3, so, 7 for the .2 or the, yeah, the 2 to the minus 5. A very 8 tight range.

9 CHAIR STETKAR: But those are, those again 10 are single point estimates. They aren't --

11 MS. PRESLEY: We only got single point 12 estimated.

13 CHAIR STETKAR: The .3 person I don't know 14 whether there was -- what the range was or what.

15 MS. PRESLEY: Okay.

16 CHAIR STETKAR: It seems awfully for --

17 again, I'll come back without trying to be as 18 antagonistic, if I read all of the words that you have 19 in both Supplement 1 and Supplement 2 about the 20 uncertainty -- and now I'm not talking quantitative 21 uncertainty, I'm talking about the variability from 22 plant to plant, the degrees of sophistication of the 23 training, of the guidance, of the experience of the 24 people, of the fact that, you know, when you say do they 25 experience time pressure? Well, Walt might say, well, NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

121 1 I always will feel that it's urgent for me to get out.

2 Other people might say, well, no. Hey, 3 wait a minute, you know, I think that we should really 4 take our time here and think through this because 5 getting out is a really bad thing.

6 Those are two different opinions. But, 7 but again, within the context of this, that only gives 8 you a differentiation between 80 percent successful and 9 98 percent successful.

10 MS. PRESLEY: So we can look and see what 11 other rationale we can pull for it. Susan?

12 DR. COOPER: Yes, Susan here.

13 There is quite a lot more we could document 14 on the process that we used with the experts. We spent 15 a day with the experts. That might seem like a short 16 period of time actually, but everyone that was -- served 17 the role as an expert had already been either part of 18 this project or aware of the issues. We didn't have 19 that, you know, week of exchanging information, reading 20 material, and so on and so forth. So we, we had an 21 educated group of experts.

22 But, nevertheless, we spent a whole day 23 doing all of this. And we did use a formal process of 24 getting each of the experts view, having them discuss 25 what went behind their numbers, and so on and so forth.

NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

122 1 How much of that to document has been a question. We 2 don't want to document all of it because it would be 3 -- probably we don't need to document all of it.

4 But more needs to be documented. I guess 5 I'm interested that maybe if you could clarify what 6 specifically you think you would like to hear about.

7 I guess you want to hear about what the expert, what 8 kinds of scenarios or contexts they were thinking about 9 when they were coming up with this. Is that what you're 10 asking about?

11 CHAIR STETKAR: I'd be happy to have seen 12 an expert's elaboration of uncertainty. I, I for these 13 conditions today believe that the worst it could be is 14 1.0 for the following reason; and the best it could be 15 is .07 for the following reason; and my best estimate 16 is, I don't know, .23 for the following reasons. None 17 of that rationale is there. So I can't look at the 18 reasons for why you say the range of your experts was 19 .1 to .3.

20 DR. COOPER: So, Susan again. So just I 21 want to clarify this.

22 CHAIR STETKAR: Because it doesn't strike 23 me as a valid -- I mean, we've done expert elicitations.

24 DR. COOPER: Yeah.

25 CHAIR STETKAR: In many cases you ask, NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

123 1 well, for these conditions at least I'd behave 2 differently. How bad do you think it could be, and why?

3 How good do you think it could be, and why? And now, 4 within that range give me your best estimate.

5 But get them thinking about the ranges 6 first rather than just focusing in on a single number.

7 Because people who are very, you said, very 8 knowledgeable, experienced, they know the process, 9 they know everything, tend to focus on very narrow 10 values, and may not have carefully thought about those 11 sources of uncertainty. And that can be dangerous, 12 especially because people are just going to pick these 13 numbers up and use them as the intent of the guidance.

14 DR. COOPER: I'm going to go ahead and let 15 Jeff.

16 MR. JULIUS: Thanks, Susan. This is Jeff 17 Julius of Jensen Hughes. And I was one of the ones that 18 was on that expert panel.

19 So, you're right, when we talked about 20 those we spent the day going over the different aspects.

21 We were pushed and looked at the different ranges of 22 the worst case and the best case, and then coming up 23 with our point estimate. And, yes, the description we 24 could add some text and explanation of the rationale.

25 Part of that is the recognition, or what NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

124 1 I was looking at this, was the action is feasible? It 2 was a fairly high bar with some conservatisms that are 3 in there. But the approach is not only just to use this 4 number, but in Section 5.3 it says use a sensitivity, 5 or set it to 1 and see what the results are. Because 6 there is a lot of complexity and variability in this.

7 And also stretch it the other way.

8 I understand what you're saying. And 9 those reporting, you're right, doesn't give you a good 10 basis for using a multiplier of 2X or 5X on these numbers 11 if you wanted to skew it. And we can improve that.

12 But there was the consideration in the 13 discussion on the pushing the ranges of the conditions.

14 And that most of the time the numbers that are showing 15 up here in this final decision tree were given that it 16 is feasible, and differentiating between a .3 or a .5 17 is, you know, is a difficult decision to make.

18 MEMBER BLEY: Jeff, since we've got one of 19 these people here I'm going to ask you a couple 20 questions.

21 First a comment. I kind of agree with 22 John, when you read the two documents and all the 23 caveats are in there about how bad this could be, I'm 24 surprised that you didn't think these are kind of 25 optimistic given all of those words. That's number NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

125 1 one.

2 Number two is we've separated the 3 habitability and loss of control. But if you're really 4 about to lose control, habitability has probably gone 5 to hell as well. And did you think about the 6 habitability conditions that might be going on in the 7 control room as you approach this point where you think, 8 ah, for control purposes I've got to get the heck out 9 of here? Was that really part of the thinking? Did 10 you guys discuss that as you went through this?

11 MR. JULIUS: This is Jeff Julius, Jensen 12 Hughes.

13 So the second question first. So the 14 comment first. The thinking on the loss of control was 15 more for the cable spreading room or instrument room 16 fires where you're not seeing the environmental 17 impacts. And --

18 MEMBER BLEY: If you're lucky.

19 MR. JULIUS: In general. And that the --

20 MEMBER BLEY: There's a guy over here 21 might say something.

22 Where is that cable spreading room 23 physically?

24 MR. JULIUS: Some are above, some are 25 below the control room.

NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

126 1 CHAIR STETKAR: Assuming it's below --

2 MR. JULIUS: Yeah.

3 CHAIR STETKAR: -- it may or may not have 4 shared ventilation systems. There might be a hole in 5 the floor.

6 MEMBER BLEY: It's something to think 7 about. But anyway, keep going.

8 MR. JULIUS: That's right. And that's 9 right, the fire is in the main control, there are in 10 the panels that are causing loss of control, you do get 11 a combination of the smoke and the system impact.

12 MEMBER BLEY: Were you thinking about that 13 when you made these estimates?

14 MR. JULIUS: Generally goes with 15 afterthought. It wasn't one of the primary.

16 MEMBER BLEY: Yeah. That's kind of what 17 John was talking about.

18 CHAIR STETKAR: What I was talking about 19 in terms of the rationale that it's clear that from what 20 we just heard that at least one of the experts for loss 21 of control had a particular fire location and a 22 particular set of effects on the personnel in the main 23 control room. Those effects not involving 24 habitability concerns at all, simply what's available 25 in front of them and how much confidence they have on NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

127 1 those displays and indications.

2 Could be different for that fire in the 3 main control room that never got to the loss of 4 habitability criteria, whatever those are. But still 5 resulted in enough snarky -- I'll use that term again 6 -- indications and controls that they decide I can't 7 control it or I've lost confidence and have to leave.

8 But that would again be compounded by the 9 habitability, local environment conditions.

10 DR. COOPER: Susan Cooper. Yeah, I've 11 talked with some of the research staff on fire 12 protection about scenarios that could be considered by 13 habitability and loss of control scenarios. The 14 answer I got, which is a simplification, is that 15 probably loss of control criteria would be used first 16 before habitability criteria.

17 However, you raise an interesting point, 18 good point, that some of that habitability stuff could 19 be happening while they're seeing some strange 20 instrumentation. So that's, that an important --

21 that's a good point.

22 I want to get back to the main question 23 about the expert elicitation. I, since I did not have 24 any preconceived ideas about the numbers, nor did I lead 25 them in numbers, I did go through the ATHEANA User's NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

128 1 Guide has some guidance on expert elicitation for 2 quantification I used that guidance, including the 3 controls, if you will, on probabilities.

4 And had some discussion with the experts 5 before we got into the quantification part about what 6 those numbers meant, including the case of the one 7 person who wasn't an HRA person or had any experience, 8 I talked with them about, you know, numbers of crews 9 going through simulators and how that might be related 10 to numbers.

11 So I -- and we did talk about a number of 12 different things. I guess getting back to the point 13 about, you know, how Supplement 1 sounds compared to 14 Supplement 2, I think the same kind of thing sort of 15 ended up happening for Phase III from my perspective.

16 Because I've been hearing about command and control and 17 how all these different things are going on, and they 18 never practiced it, and they've got all these phone 19 calls coming in, and they have no administrative 20 controls about who can open up the door and start 21 bothering you.

22 But then when it came right down to, well, 23 what's important about this, sort of like, Uh-huh, I 24 know, I think there -- the notion that probably with 25 their basic training and so forth that they're going NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

129 1 to get through this despite having any demonstrations.

2 So it did seem to me that when we got down 3 to talking with experts, people who as far as I know 4 would probably be the best or have the best knowledge 5 available in the industry about what might happen 6 because no one's seen this in a simulator, very few 7 people have actually done any kind of demonstrations, 8 integrated demonstrations after they left the control 9 room, when it got right down to it, as far as they could 10 imagine it didn't seem to be as big of a problem as, 11 you know, when you're just sort of asking them for some, 12 you know, like complaints about the process, it didn't 13 end up, you know, when it came down to it that they 14 thought that seemed to be that important.

15 It was a little bit of a surprise to me to 16 be honest, because I was, you know, especially like I 17 said with the command and controls, like, well geez, 18 you know, we don't know what kind of procedures they 19 have. They probably haven't practiced. They've only 20 got one phone or one radio and they've got everyone else 21 calling them. And, you know, it's going to be kind of 22 a nightmare.

23 But when you got right down to it it was 24 sort of like, well, so what's important about that?

25 Not that much.

NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

130 1 So I don't really know how to -- you know, 2 what kinds of messages to take away from that. I think 3 one message for me is that even with the people I would 4 have expected to have the best understanding of this 5 context, that we don't understand very much about this 6 context still. And we probably need to get more 7 information.

8 It would be nice if people, you know, even 9 just motivated by some of the things here that we've 10 written down, that they would start looking at, you 11 know, looking at doing some training, at least having 12 some talk-throughs or whatever, doing some integrated 13 training, you know, once they get to the remote shutdown 14 panel, that sort of thing. And actually, actually sort 15 of see where -- kick the tires and find out where the 16 problems are.

17 Right now we've just had some kind of a lot 18 of people that have been in certain situations with 19 anecdotal comments and concerns. But when it comes to 20 informing error probability there seems to be kind of 21 a more optimistic view than I would have expected.

22 MEMBER BLEY: I guess I'm thinking this 23 thing -- and John's brought it up a few times -- if 24 somehow when people are thinking about this they know 25 they're worried about core melt, and they focus in my NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

131 1 mind would I stay here if I think we could get core 2 damage? I think that makes it real easy.

3 In a real control room with a fire going 4 on, and I'm still driving the plant okay, I'm probably 5 not thinking I'm going to be in a core damage situation 6 pretty soon.

7 DR. COOPER: Yeah. We did --

8 MEMBER BLEY: I don't know if you were able 9 to keep the right mindset on these people. Of course, 10 we weren't there to see that.

11 I'm sorry. I cut off Matt.

12 DR. COOPER: Yes.

13 MEMBER SUNSERI: All I was going to say is, 14 well, may be off key, so but anyway I've heard several 15 times now that the plants don't practice this. And 16 that's not my experience. I mean, my experience is the 17 plants have remote shutdown simulator panels near their 18 simulators that they practice it periodically. I've 19 seen situations where smoke generators are brought into 20 the simulator to --

21 MEMBER KIRCHNER: Don't they do this when 22 they do a FLEX deployment?

23 MEMBER SUNSERI: Well, I don't know about 24 that. But maybe there's aspects of this whole scenario 25 that aren't being exercised. But certainly I think NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

132 1 that operators do the transition, that they practice 2 making the decision to go to the remote shutdown. That 3 they practice stopping the control. And they do it on 4 their simulators.

5 MEMBER BLEY: I think your experience is 6 more to the unique side from what I've heard about in 7 plants.

8 MEMBER SUNSERI: Well, maybe. But I mean 9 I can think of at least three plants --

10 MEMBER BLEY: I mean there's an awful lot 11 of plants not only don't have a simulator for the remote 12 shutdown panel, they don't even have a remote shutdown 13 panel. How many plants have just distributed stuff, 14 they go out and operate? Twenty or 30?

15 MEMBER SUNSERI: Good question.

16 MEMBER BLEY: I'm hoping Mark will have an 17 answer. I know you used to look at this stuff a lot.

18 MR. SALLEY: I'd almost go to J.S. for 19 that. I mean, yeah, a lot of the plants don't --

20 MEMBER BLEY: I'm sorry, I forgot J.S.

21 MR. SALLEY: Yeah. When 81-12 went in a 22 lot of them had load control spaces, is what they called 23 it.

24 MEMBER BLEY: Yeah.

25 MR. SALLEY: So they were doing that at all NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

133 1 multiple places in the plant. For some they put --

2 MEMBER BLEY: Yeah. Local control 3 stations and just using those as a remote shutdown.

4 MR. SALLEY: So 20 or 30 we could 5 guesstimate, J.S., unless you've got a better number.

6 MR. HYSLOP: J.S. Hyslop. There are a lot 7 of -- I don't have a number. I just know we've seen 8 cases where it's distributed. We've seen cases where 9 they've better panels. I'm not sure how many true 10 simulator remote shutdown panels, right, but I haven't 11 seen a lot of those but I know they're out there.

12 I don't have a number so I can't tell you.

13 MEMBER BLEY: It's a mix.

14 MEMBER SUNSERI: Well, I'll take that.

15 So I had another question and maybe this would be a good 16 time to ask it.

17 I've seen some human, or I'm familiar with 18 some human performance studies that show intelligent 19 beings that would have all the right information 20 available to them don't always make better decisions 21 when they have an unlimited amount of time. In other 22 words, having a time limit actually improves their 23 decision making because they don't end up second 24 guessing themselves and making the wrong decisions 25 later.

NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

134 1 But that, your data doesn't seem to 2 indicate that. It seems like given more time they also 3 make a better decision.

4 CHAIR STETKAR: This isn't data. This 5 isn't data.

6 MEMBER SUNSERI: Well, analysis or 7 whatever it is.

8 CHAIR STETKAR: And remember, let me read 9 something. We have to be aware of time here because 10 we need to talk about Phase III and things. But my 11 concern about having the experts and the whole group 12 colored too much about preconceived notions of what 13 this is from not an operator's perspective, from a human 14 reliability analysis widens perspective, is in 15 Supplement 1.

16 It explicitly says "loss of control 17 scenarios are those that will lead directly to core 18 damage if the operators remain in the main control room, 19 i.e., in the absence of operator actions taken 20 following abandonment. For each loss of control 21 scenario, or group of scenarios that share the same 22 characteristics, that would lead to core damage in the 23 absence of abandonment actions the HRA analyst will 24 need to conduct operator interviews to determine if the 25 abandonment procedures and equipment cover these NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

135 1 situations, and also whether the operators would 2 interpret the conditions as a loss of control."

3 Well, if you ask me if the core's going to 4 melt do I have loss of control? Well, hell yes.

5 Will I leave the control room? Well, hell 6 yes.

7 And if your experts know that these are 8 scenarios where the core is going to melt, if they don't 9 decide to leave the control room within 5 minutes, that 10 knowledge will color their evaluations, the same way 11 that the knowledge is not inside the control room, it's 12 in some place outside the control room.

13 DR. COOPER: So what I'm hearing you say 14 -- and because we're not saying that to the operators.

15 This is guidance to the analysts so they understand that 16 they're not to look all over the place for scenarios 17 to model those loss of control scenarios. They can be 18 informed by the PRA and other modeling as to these are 19 the ones that we're concerned about. We'd like to take 20 credit for them in the PRA. And HRA analysts figure 21 out whether or not we can take credit for them in PRA.

22 We're not going to the operators and saying 23 if you don't do this you're going to melt, the core's 24 going to melt, are you going to leave or not? We didn't 25 run the elicitation that way.

NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

136 1 The, you know, I guess, I guess what 2 probably is needed is to be more clear about how we talk 3 with the experts or operators versus what we're giving 4 guidance to HRA analysts. The HRA analyst needs to be 5 informed about this so they can have their, their 6 analysis focus and they can make sure that they're 7 focusing on the right things to support the PRA. But 8 that's not how you interact with the operators. You're 9 absolutely right.

10 So I'm not really sure what, what the 11 answer would be, but probably some beefed-up guidance 12 on how you interact with the operators and so forth.

13 But to be real honest, except for the fact 14 that you need to find out from the operators what they 15 think are the set of conditions that they feel would 16 be loss of control, what they define as loss of control, 17 you know, I mean that has to happen. And I think we're 18 pretty clear about the fact that the operators guide 19 that definition. If that definition doesn't match up 20 with the PR -- what the PRA says you don't get the 21 credit. That's bottom line.

22 So, anyway, and we didn't do the expert 23 elicitation that way either. I'll ask Jeff, one of the 24 experts.

25 MR. JULIUS: Jeff Julius, Jensen Hughes, NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

137 1 member from the expert panel.

2 Just to confirm what Susan said, that it 3 wasn't -- we didn't ask leading questions. We were not 4 asked leading questions in terms of conducting the 5 expert panel.

6 And just a clarification of earlier 7 statements. So the comment, when I was thinking about 8 the fire in the cable spreading room I was 9 conservatively not taking credit for the smoke going 10 up in the control room. Because if there was smoke in 11 the control room, that would give some additional 12 indications to the main control room team that, hey, 13 maybe there should be some more urgency here and maybe 14 we should need to make this decision faster.

15 So I think that's a good example, though, 16 where that type of rationale, you know, the report would 17 benefit from having that kind of rationale or 18 additional detail added in Appendix A.

19 MS. PRESLEY: And I just, just to be very 20 clear, and since I have the notes in front of me, the 21 things that were -- so, so we asked for each criteria 22 kind of what are the best, what are the worst?

23 We discussed, because we had a lot more 24 criteria here to begin with even, we ran through some 25 of them. But some of the things that were coming out NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

138 1 were operators know that there are only a few fire areas 2 that would lead to abandonment. So they are signaled 3 when they have an alarm in one of their areas. They're 4 starting to think right away about what those criteria 5 are.

6 They're thinking about the procedures are 7 important, but as soon as they're starting to see 8 effects they're going to start wondering about, because 9 they know those areas are critical, they're going to 10 start thinking forward. So that was on the why it's 11 reliable side.

12 On the not reliable side they were thinking 13 things like they brought up several historic events of 14 areas where they were reluctance and delays. They 15 talked about the David-Besse AFW failure where the 16 steam generator level was, you know, they kind of 17 wiggled their way out of it was low, it wasn't at the 18 rock bottom but they knew it wouldn't be at the rock 19 bottom because that's not the way that indicator works.

20 So they bought some time and delayed them.

21 Calvert Cliffs they had the -- they were 22 supposed to do the SI injection but the shift supervisor 23 overrode the procedural direction and delayed that.

24 So they, they did bring up they talked 25 about the Robinson fire and the crew that was assembled NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

139 1 and the quality of that crew. And so when they were 2 talking about the HEP they were talking about these 3 sorts of things.

4 So we can put in a little bit of discussion 5 about what we have heard as the background. We didn't 6 specifically say, okay, in the fire's in your control 7 room and you're almost at habitability but not quite, 8 we didn't go quite into that level of detail but we did 9 push the bounds of best and worst. And we only 10 solicited a best estimate number. We didn't ask for 11 5th to 95th percentile because it wasn't, it wasn't that 12 level of expert elicitation.

13 So I just I want to make that clear and we 14 can document that. But the, yeah, the level of 15 resolution you see in there, that's what's in it and 16 that's what's not in it, so.

17 DR. SMITH: Let's, we do have to be a 18 little bit aware of time here. I don't know how 19 flexible people are in running long, but I don't want 20 to get too late.

21 For NRC staff, be aware that these numbers 22 will be interpreted throughout the industry as 23 NRC-approved human error probabilities. These are 24 NRC-approved human error probabilities. Okay, I will 25 just put that -- just be aware of that. It isn't, this NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

140 1 isn't -- once this hits the street these are cast in 2 stone.

3 DR. COOPER: Susan Cooper. One thing I 4 want to add is that because we don't have demonstrations 5 of all of these pieces, especially here we're talking 6 about decision core damage, people are practicing this 7 in the control room. We haven't seen it happen.

8 So I think even with the experts I think 9 there's probably some aspects that they understand 10 about sort of conduct of ops that are not usually 11 touched by HRA but get more into, you know, conduct of 12 operations, you know, sort of your basic bible for how 13 the plant does business, which we don't always get to 14 that level of control, if you will, or operations in 15 HRA because we're -- because some of those typically 16 well-practiced simulator exercises aren't the basis 17 for our understanding of how these operators will 18 perform.

19 There may be some additional things that 20 we need to talk about so far as, you know, what the 21 expectations are of the operators, especially going 22 back to the training and their fundamental 23 understanding of how things are going on, you know, 24 what's going on in their plant. So there may be some 25 more pieces that we need to add in that underline this NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

141 1 in addition to reluctance.

2 MR. SALLEY: Yes, Susan, if I could for a 3 second.

4 John, I understand exactly what you're 5 saying is you put the number out there and then for lack 6 of anything better the rank and file person who's doing 7 the work is going to see this number. Use this as a 8 reference and go with it.

9 What would you recommend that we do, you 10 or Dennis, your thoughts on this to improve it or to 11 make it better or safer, whatever?

12 CHAIR STETKAR: My initial thoughts -- and 13 again, these are personal, you know, it's a 14 subcommittee meeting so there can be ten different 15 interpretations here -- at least bare minimum the 16 experts' understanding of what they were evaluating and 17 their uncertainties.

18 I'll come back to this notion of 19 uncertainty is if they, you know, this point number 20 really doesn't mean anything to me without 21 understanding what their uncertainties are. And 22 saying that three experts, two of which has .1 and one 23 of which had .3 doesn't tell me anything either because 24 they were all people who -- first of all, they were all 25 people who are very familiar with this process and could NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

142 1 be biased that way. They knew, they knew what they were 2 answering. And other folks might not know that the say 3 way.

4 So I think some notion of the fact that 5 there may be more uncertainty in these numbers than 6 people are led to believe, especially for this very, 7 very narrow, the worst it can be is 80 percent success 8 rate, and the best, regardless of what you say about 9 feasibility because the guidance for feasibility 10 actually is pretty good, but just given the fact that 11 it's feasible, if you pass that screen, it still sounds 12 -- it might be in my opinion it might be well justified 13 but I can't understand why they are, given the 14 documentation that you have.

15 Dennis.

16 MEMBER BLEY: I guess I'd start with 17 John's bare minimum. And when you have that I'd draw 18 some conclusions about it.

19 Like, if you hear from all of the people 20 who were on your panel that, no, they didn't consider 21 the environment, the degradation, habitability at all 22 as they looked at this. But put a caveat in that, you 23 know, the scenario we've quantified is with the clean 24 atmosphere.

25 CHAIR STETKAR: Basically for a fire not NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

143 1 in the control room. Now --

2 MEMBER BLEY: Or with smoke leaking in 3 from somewhere else.

4 CHAIR STETKAR: Yes.

5 MEMBER BLEY: No, it's a clean control 6 room.

7 CHAIR STETKAR: It's a clean control room.

8 MEMBER BLEY: I don't know what you'll 9 find when you're talking. That might not be the case.

10 Maybe some of them really thought about that. But if 11 you find other things as you go through that that imply 12 it's a more narrow set than you intended, just be up 13 front about that and say, look, we did this. These are 14 guidance but this guidance really only applies to a 15 selected set of conditions. And for other conditions 16 it will be different.

17 CHAIR STETKAR: The problem is -- and Jeff 18 and Paul perhaps can back this up -- I believe that this 19 issue becomes most important quantitatively for fires 20 inside the main control room. But I could be wrong 21 there. I'm not sure.

22 Cable spreading rooms aren't the same 23 problem that they used to be because cable fire 24 frequencies in part are much lower than what people used 25 to have. And, yeah, you've got to consider transient NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

144 1 combustibles but you have to consider transients in the 2 main control room also.

3 I don't know how much you get into fires 4 in instrumentation and control cabinet rooms driving 5 this. But the problem with those rooms is they often 6 have the same shared ventilation system as the main 7 control room. So, as Dennis said, it's pretty 8 important to understand what the experts knew in terms 9 of the conditions when they established these --

10 MEMBER BLEY: Mark told us something 11 earlier that's a change to me. You know, when I was 12 aboard ship we had ventilation systems. That's the 13 first thing you did if you had a fire was you killed 14 the supply vents and turned the exhaust vents on high.

15 Most plants, when I looked at them, had no 16 separate controls for inlet and outlet and you couldn't 17 adjust them like that. It was ventilation 18 on/ventilation off. You can either suffer in the smoke 19 or feed the fire with more air.

20 So if that's changed, and you were saying 21 you now have purge in a lot of the plants, that's a 22 difference. But I don't think we talked about that 23 sort of thing anyway, did we, in the report? I don't 24 remember seeing that.

25 CHAIR STETKAR: No.

NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

145 1 MEMBER BLEY: It came up in the 2 discussion.

3 But that's a really important point if in 4 fact most of the plants now have the ability to purge 5 smoke, and especially in the control room, that really 6 would help.

7 MS. LINDEMAN: Ashley Lindeman. For lack 8 of habitability usually you would evaluate two cases:

9 one with no ventilation. And if you had purge mode you 10 would take that into account in your fire model.

11 MEMBER BLEY: Ashley, my problem is 12 habitability and loss of control aren't separate 13 things. We parse them separately to talk about them 14 here and then we haven't done much with one. But often 15 they go together. And when they go together, that's 16 important.

17 CHAIR STETKAR: I do want to leave time for 18 Phase III because I have comments on Phase III analysis.

19 And there's a lot of good discussion I know you want 20 to get to in terms of command and control. So if we 21 can, let's go on to the Phase III guidance.

22 DR. COOPER: Okay. This is Susan Cooper.

23 I'm going to talk about our Phase III HRA quantification 24 guidance. And this is for the implementation of main 25 control room abandonment safe shutdown strategies.

NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

146 1 Supplement 2, Section 4 provides the 2 guidance on the quantification of all of the HFEs that 3 would be associated with Phase III .

4 CHAIR STETKAR: Susan.

5 DR. COOPER: Yes?

6 CHAIR STETKAR: Before you get started on 7 this, because you're very quickly going to get into the 8 discussion of command and control, which is a pretty 9 interesting discussion, let me ask you a very leading 10 question, but one that I honestly came away being 11 somewhat confused.

12 Does your guidance recommend that an 13 analyst evaluate only implementation errors in Phase 14 III except for conditions where you identify very 15 specific command and control issues? So, for example, 16 if you say there's no command and control issue 17 according to the guidance here, everything in Phase III 18 is that only implementation or does it also include 19 cognition?

20 In other words, when I think of standard 21 typical HRA I have a cognitive part and I have an 22 implementation part and, you know, I parse it up that 23 way.

24 DR. COOPER: So --

25 CHAIR STETKAR: Because I read it in NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

147 1 different parts of the guidance, whether it's 2 Supplement 1 seems to lead me to believe that there is 3 absolutely no cognition associated with Phase III, that 4 it's strictly rote: follow a procedure, push this 5 button, eat that banana.

6 In Supplement 2 there seems to be a little 7 bit but it's always associated with this make a decision 8 about whether or not there is a command and control 9 issue. And if there's no command and control issue, 10 if it's literally a rote follow the procedure, then it's 11 strictly implementation. But I will submit that there 12 is never a rote follow the procedure. Somebody is 13 always going to be in control.

14 DR. COOPER: So it's a -- there's a little 15 bit, there's a little bit of discussion about this.

16 What we discovered when we were developing this 17 guidance is that in particular there could be 18 differences between PWRs and BWRs so far as how, how 19 their strategies are laid out. For a particular BWR 20 the shut -- main control room abandonment safe shutdown 21 strategy is one path. There is one set of steps.

22 There's no alternative system written in the procedure 23 --

24 CHAIR STETKAR: Right.

25 DR. COOPER: -- to go do if this doesn't NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

148 1 work.

2 So a good portion of our guidance is 3 written for that particular set, that scenario. But 4 it recognizes that there could be other, other 5 strategies that might have decisions between either 6 using different systems, or you might try a system. It 7 doesn't work and you have the option written within the 8 procedure for shutdown, main control room abandonment 9 shutdown, to go to that other system or second system 10 or third system.

11 So there are some plants that have that 12 kind of strategy. But it's very much tied to what the 13 strategy is as documented in their safe shutdown 14 procedure.

15 CHAIR STETKAR: Let me, because I want to 16 get through this, let me just drop this as a suggestion.

17 Take a look at the discussions in both Supplement 1 and 18 Supplement 2 of the guidance of Phase III and be 19 sensitive to whether or not the guidance reinforces the 20 notion that the analyst now should account for 21 cognition.

22 Now, at a particular plant for a particular 23 fire scenario with a particular set of procedures, 24 perhaps consideration of cognition says that it's not 25 a contributor. But only for that particular plant, for NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

149 1 that particular fire scenario, with those particular 2 procedures.

3 If you read the guidance, as I did as kind 4 of a skeptic, I'm led to quickly draw the conclusion 5 of the opposite, that in general you don't consider 6 cognition except for very, very specific cases that you 7 identify as particular command and control issue. And 8 I'm not sure that's what your intent was.

9 DR. COOPER: No, it's not.

10 CHAIR STETKAR: Okay.

11 DR. COOPER: And this is one of those 12 places where there is an evolution from Supplement 1 13 to Supplement 2. And it was actually -- I mean, whether 14 or not it was late for all the authors, it was late to 15 getting to the document so far as there being a 16 distinction between how you treat these different types 17 of strategies so far as whether or not there's decision 18 making going on like there would be, say, like in EOPs 19 where you have, you know, alternative and another path 20 to go in your procedures, and other choices with respect 21 to systems and so on and so forth.

22 So that's, that's we do call it out very 23 briefly in Supplement 2 and we say that if that is the 24 kind of strategy we have, if we have, you know, response 25 not -- maybe equivalent of a response not obtained NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

150 1 column where you would go to another procedure or 2 another system, then this, the rest of this guidance 3 doesn't apply. So we recognize that.

4 The other thing is that we don't say there 5 is no cognition. But the way we treat it for those 6 strategies where you really only have one path, there's 7 only one set of things to do, we do look at the 8 interactions between operators, especially 9 communications between field operators and a 10 supervisor who might be recording information or then 11 later worrying about coordination, and the time that 12 it takes to do that.

13 CHAIR STETKAR: But that's, and the 14 discussion of time I think is done well there. But what 15 I'm talking about is the standard cognition of 16 identification diagnosis decision making. And 17 despite all the things you say that when people abandon 18 the control room I have a set of procedures that tells 19 me to do this, I'll guarantee you that those procedures 20 are not going to match what's happening in the plant.

21 And some supervisor is going to have to make a decision 22 about how do we adapt what we have to match what's going 23 on in the plant.

24 That may involve creative alternatives.

25 It may involve delays for decision making. It may NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

151 1 involve time to go get additional information for 2 people. It's never going to match the presumptions, 3 especially the way the procedures are written now, that 4 you have a certain Appendix R fire that says go do A, 5 B, and C and everything is going to be perfect, because 6 it never will.

7 DR. COOPER: So --

8 CHAIR STETKAR: And that's, I'm not trying 9 to get very specific. I'm trying to step back and say 10 why should the guidance for doing Phase III type human 11 reliability analysis be different from the guidance for 12 doing any other human reliability analysis, except for 13 the fact that there might be more complex things that 14 I'll allow you to get to here in a minute, in terms of 15 coordination of command and control communications 16 that may not apply, certainly do not apply to sitting 17 in the confines of the main control room with a set of 18 emergency operating procedures.

19 DR. COOPER: Well, some of what you're 20 talking about is, again, it's not all doc -- some of 21 the steps, the interim steps we went through are not 22 documented. But we went through all of the pieces of 23 the definition of command and control that we 24 established in Supplement 1, which includes things like 25 situation assessment, and allocating resources, and NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

152 1 making decisions and so forth, and looked at each of 2 those to see what their contribution was for the 3 specific set of strategies that I just mentioned where 4 it's a single path through the procedure.

5 And then identified what kinds of failure 6 modes might be coming from each of those pieces. And 7 that's on top of the execution.

8 And the end point that we reached was what 9 I'm going to be presenting so far as the main 10 contributor of really cognition, thinking, decision 11 making, controlling, that sort of thing. I'm not going 12 to -- I mean, I' not going to say that, you know, whoever 13 is at the remote shutdown panel looking at whatever 14 indications they may have there, they're going to be 15 following their procedure, getting information that 16 they can. They're going to be keeping track of what's 17 going on.

18 But based on the set of strategies that we 19 were trying to address here, and we tried to be -- talk 20 about, you know, this is the scope we're looking at --

21 those steps of strategies do not really specify or 22 codify or formalize any of these kinds of deviations.

23 The reality is, of course, I mean, I realize the reality 24 is going to be different but we've made, we've made that 25 boundary. We've talked about some additional things NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

153 1 that might happen in the future, you know, so far as 2 other equipment that you might be able to credit if you 3 had time to do that, that sort of thing. But we haven't 4 gone beyond what is specified specifically in the main 5 control room abandonment safe shutdown procedures.

6 If it's not in there now, we're not talking 7 about crediting some other innovative idea that might 8 be a recovery to failed system or a failed piece of 9 equipment. We haven't looked at that, no.

10 There's no procedural link to anything 11 else right now. It's not in the procedure right now.

12 There are efforts looking further, but that's, you 13 know, not -- and we mention that there might be other 14 things that are happening in the future, but that's not 15 in this project.

16 CHAIR STETKAR: I just think it's a 17 regression in 2018 to give human reliability analysts 18 the notion that they ought not to think about cognition 19 as a fundamental part of their analysis, regardless of 20 preconceived notions about how specific the procedures 21 may be for a particular scenario. I believe, or some 22 belief is that they should always think it's cognition.

23 If they think about it and dismiss it for a particular 24 scenario, that it's not an important contributor, fine.

25 They've thought about it and dismissed it, and NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

154 1 documented it.

2 If they're surprised, they've thought 3 about it and, gee, they're surprised and they've 4 accounted for it. But the guidance ought to force them 5 to be surprised, not tell them to ignore it unless it 6 slaps them in the face from some other set of criteria.

7 Do you follow me?

8 DR. COOPER: Yeah.

9 CHAIR STETKAR: We've spent too much time 10 over the last decade talking about the importance of 11 cognitive performance and how people really make --

12 people don't make mistakes by pushing wrong buttons, 13 you know. They occasionally push the wrong button but 14 that's not what gets people into trouble.

15 DR. COOPER: I agree. I agree.

16 The difficulty that I have is, again, 17 getting back to the variability on the remote shutdown 18 panels. And now we're not just talking about controls, 19 we're talking about indications. Because that's what 20 is going to drive someone to decide to do something.

21 CHAIR STETKAR: But my point is let the 22 analysts. This is guidance that's supposed to apply 23 to any plant for any fire. So let the analysts struggle 24 with that for their plant and their fire scenario.

25 They have to struggle with their plant, their NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

155 1 capabilities, their operators, their training, 2 whatever their procedures are and how well they fit a 3 particular fire scenario. That's their job.

4 And if the guidance gets too pigeonholed 5 with some preconceived notion about how all of the 6 procedure, you know, the average of all of the 7 procedures that you've looked at and what you think 8 might be important, I think that's a disservice to 9 guidance for performing human reliability analysis.

10 It's kind of back to the old THERP of you follow the 11 procedure and look up numbers in a table, and by 12 definition that's how people perform.

13 DR. COOPER: Yeah, that's not, that's not 14 the picture. But the bottom line is, and as we get into 15 the later slides, for implementation of the safe 16 shutdown strategy you are talking principally about 17 single operators in individual locations by themselves 18 at a panel doing something that's been put out on their, 19 on their procedure. It's not a crew per anymore, at 20 least not the same type of structure anymore. That's, 21 so they're --

22 MEMBER BLEY: They don't, when they're 23 doing that they don't have somebody at some place 24 coordinating all those activities?

25 DR. COOPER: Well, in principle they do.

NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

156 1 But there's a lot of variation, there's a lot of 2 variation between plants so far as whether or not there 3 is actual coordination. In some cases -- and we talk 4 about this some I think in Supplement 1 -- one of them, 5 I can't remember -- but the point is that there could 6 -- or maybe I wrote it and we axed it out -- but I mean 7 in some cases, you know, field operators are given 8 individual attachments. They go out. And the only 9 communication they really have with a supervisor is I'm 10 done, I'm going on to the next thing. You know, that's 11 it. There's no coordination.

12 Or if there is coordination it might be 13 between individual operators, field operators if 14 they're, you know, trying to control, you know, 15 singular level and injection or whatever, maybe they're 16 talking together on headsets. We can do that.

17 And then there are some other instances 18 where there is coordination. But the bottom line is 19 it's a lot of the communication is just reporting.

20 Now, when we get into this, we are trying to focus on 21 those places where coordination is needed and to model 22 something there.

23 CHAIR STETKAR: Why don't we go on because 24 --

25 DR. COOPER: Yes.

NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

157 1 CHAIR STETKAR: -- I'm going to try to 2 finish by about 5:15 to no later than 5:30 certainly.

3 DR. COOPER: Okay.

4 CHAIR STETKAR: But 5:15 or so.

5 DR. COOPER: So, so I guess I'm going to 6 skip to the bottom so we can try to -- of this slide 7 and just say that the principal focus of our research 8 for Phase III was to look at command and control and 9 to try to understand what was important about it for 10 the range of strategies that we were considering and 11 try to identify what needs to be modeled. And 12 sections, there are various sections in Section 4 that 13 talk about. And Appendix B provides some of the 14 background.

15 And there is -- this is one of the most 16 significant areas of additional research that we did 17 in supplement to, you know, expanding on what 18 Supplement 1 had.

19 So I'm just going to very quickly go over 20 this. This is the definition of command and control 21 that we put into Supplement 1. And it's based in a 22 military definition. We haven't found anything 23 better.

24 One of the main take-aways that we have in 25 Supplement 1 in the conclusions is that we still think NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

158 1 there's not much out in the literature anywhere about 2 command and control, certainly nothing specific to 3 nuclear power plants.

4 So command and control is maintaining a 5 coherent understanding of the plant state; timely 6 decision making; allocating resources; coordinating 7 actions; managing communications.

8 And why don't we go to the next slide.

9 So for Supplement 2 we expanded on this 10 understanding of command and control. And as I 11 mentioned, we specifically went through each of those 12 aspects of the definition and tried to examine where 13 there might be failure modes or what aspects of it, of 14 those definitions were relevant or not relevant to the 15 main command and control safe shutdown strategies.

16 And then after that we, like the Phase II, 17 we developed a consensus on the likely relevant issues 18 among the author team before we went to the experts.

19 And the principal thing that we got from the experts 20 was to try to confirm the key issues and then try to 21 hone in on what they thought was the most important 22 thing that ought to be represented as a failure of 23 command and control in the implementation of the safe 24 shutdown strategy.

25 And this process is what I had mentioned NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

159 1 before. Until we got down to saying, so what's going 2 to matter? What's going to result in something, 3 something being done incorrectly? This is where we 4 might have seen a little bit of a disconnect between 5 all these concerns about, you know, how command and 6 control hasn't been actually practiced, you know, in 7 an integrated way with all the field operators and so 8 forth, and then coming to actually something that we 9 need to model.

10 So based on the information we got from the 11 experts we developed a set of screening rules for 12 whether or not you should model something as a, I think 13 we said we can either put it as an additional HEP or 14 failure mode, additional failure mode for an execution 15 action. That comes --

16 MEMBER BLEY: Is this the same experts as 17 before or different?

18 DR. COOPER: It is the same experts.

19 MEMBER BLEY: Okay.

20 DR. COOPER: And then to develop an HEP we 21 looked at existing HRA methods to try to find some 22 overlap of similar issues. There was no actual overlap 23 but there were some words that sort of seemed to sound 24 the same. And we felt like it was addressing some of 25 the same instances, and so we looked at the range of NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

160 1 those values, and then selected, the authors selected 2 a value to use. That's how that process went.

3 So this, I think there's a similar table 4 in --

5 CHAIR STETKAR: I don't think that, just 6 for the record for this meeting, --

7 DR. COOPER: Yes.

8 CHAIR STETKAR: -- you don't publish that 9 value here, do you?

10 DR. COOPER: Sorry, publish what?

11 Publish what?

12 CHAIR STETKAR: In your slides. Oh yeah, 13 you do. I'm sorry, it's on Slide 51. We'll get to it.

14 DR. COOPER: So we have this table was in 15 Supplement 1. Actually, I think this might be a little 16 bit updated by our experts.

17 No, this, I guess this is from Supplement 18 1. We did make some updates from what you had seen.

19 But some of the key things that we wanted to highlight.

20 We found it helpful to think about in control room 21 normal operations first, and how it seemed like command 22 and control was working and what things, what were the 23 things that seemed to be helpful in supporting command 24 and control, and then trying to contrast that with what 25 we understood about main control abandonment NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

161 1 operations once you've left, once you've left the 2 control room.

3 And this just kind of highlights some of 4 the key things.

5 When you're in the control room you have 6 kind of centralized cognitive entity that's 7 represented by the team. You are going to have some 8 coordination with some other plant people, including 9 the plant staff, including the fire brigade.

10 Because you have the heads-up board, and 11 everyone's in the control room for backing each other 12 up, you have shared visual cues. And because of the 13 simulator training and so forth you've got 14 well-rehearsed, tested plans. You know, your 15 resources are anticipating available -- we could argue 16 about the limits of need for flexibility in response, 17 but that's in comparison to main control and 18 abandonment.

19 One of the key things is because we have 20 some of these things, this is recognition-primed 21 decision making model is what my psychologists on the 22 team are telling me.

23 Another really key thing is the 24 communications are mostly face-to-face and, you know, 25 voice communications model real time.

NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

162 1 And because of the protocols for the 2 control room there should be restricted interruptions.

3 I mean, of course there will be phone calls. But you 4 have conduct of operations, or whatever, that limit 5 people coming into the control room. You have to ask 6 permission to get in, and so on and so forth. So 7 there's only so many people that are going to be there.

8 And you probably have some extra help answering the 9 phone.

10 In contrast, when you get to after you've 11 left the control room you are now at the remote shutdown 12 panel and distributed in the field there's the team.

13 The operating crew is distributed through the plant.

14 The supervisor might be alone at the remote shutdown 15 panel. There might be another operator there. And 16 they're going to have to be coordinating with the fire 17 brigade, other plant areas, along with all of those 18 field operators.

19 Each operator, depending on where they're 20 located, is going to have their own view of plant 21 information. Most remote shutdown panels do not have 22 anywhere near the array of information that's in the 23 control room so far as indications. And most of them 24 don't have alarms.

25 And this gets to the point that I think NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

163 1 Matthew made earlier, there are parts of Phase III that 2 are frequently practiced, and that's the field operator 3 actions at the local plant stations. What's not so 4 much practiced is the coordination and interaction with 5 that supervisor who has got the procedure and is at the 6 remote shutdown panel and if he needs to be coordinated.

7 That coordination aspect is not frequently practiced 8 because they don't do that integrated training, they 9 do the pieces, not all together.

10 So let's skip ahead.

11 So, communications are mostly going to be 12 radios or phones. Limit to the confirmatory 13 indications, and they may have to send somebody to go 14 find that information. And we don't really have 15 information on how many interruptions are possible, but 16 there's no control on it that we know of.

17 So, the things that we, we had a set of 18 issues to discuss with the experts ahead of time. And 19 this slide and the next slide are the results of those 20 sorts of things. We actually we talked through with 21 the experts on one day and we continued our 22 conversations actually by phone and email to try to get 23 to this list of that kind of issues and concerns based 24 on what they -- their input.

25 So the role of the shift supervisor or NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

164 1 manager, whoever is in charge of command and control, 2 when they leave the control room they still have the 3 same responsibilities but they don't necessarily have 4 the same indications and controls at the remote 5 shutdown panel that they would have in the main control 6 room.

7 Communications are going to be different 8 and there are going to be time impacts associated with 9 that.

10 Most likely no alarms, and few indications 11 for the remote shutdown panel, so the ability to closely 12 monitor parameters, you know, is going to be different.

13 And the way, you know, the information 14 that's there, you know, you might have to pay more 15 attention to monitoring those parameters and so you 16 might be susceptible to distractions.

17 Coordination might be required, and it's 18 going to be complicated by different operators being 19 in different locations with communication issues.

20 There's a little more expansion on the next 21 slide, but the real, the real focus when we came down 22 to final discussions on what it is that we should be 23 most concerned with, the experts said we should focus 24 on coordination actions, especially things where they 25 weren't done properly that could result in irreversible NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

165 1 damage. This is again getting back to the notion that 2 you have a strategy that has really one train of 3 equipment that you're going to be using for shutdown.

4 That's the design of many of the plant 5 procedures out there. That's what we found.

6 Just a couple things off of this slide so 7 we can try to get through to the end.

8 So while the shift supervisor or shift 9 manager continues to drive the actions that are going 10 to the procedure, for the most part they're not looking 11 at them, they can only monitor some of those actions 12 and record that they're done. But is of importance 13 when they have to coordinate those actions.

14 Allocation of resources, which is one of 15 those definitions of -- one of those aspects of the 16 definition of command and control, that's mostly done 17 by the assignments to the various operators through 18 procedure attachments. That's how the strategy is 19 usually implemented in procedures as there's a master 20 procedure for the supervisor and then individual field 21 operators will have pieces of, you know, or attachments 22 to the overall procedure. But that's their 23 responsibility.

24 Communications are not, typically not 25 face-to-face. And many of those communications may NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

166 1 just be reports that things are done.

2 And then it was noted that communications 3 equipment and associated problems could be a concern 4 with those communications.

5 So, in focusing on sequencing failures in 6 particular, the authors went ahead and developed these 7 screening rules on when to model a command and control 8 failure.

9 We had various iterations on these rules 10 before we settled on this set that's shown in these next 11 two slides. But the first few steps mostly were 12 getting us to t hose sequencing failures. And then the 13 later steps have to do, are kind of related to what you 14 might call performance shaping factors, or whatever.

15 One iteration of this was to actually give 16 different HEPs associated with those different 17 performance shaping factors that because we didn't have 18 enough information to know how important some of these 19 factors would be specifically for this context, we 20 decided to go with a single set of screening values.

21 And from my perspective, because no one's 22 modeled these sorts of things before I would use these 23 screening rules to say, you know, I don't want to model 24 this. How can I avoid this? And I'd look at these 25 screening rules and try to think about, you know, can NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

167 1 I design some compensatory measures by putting 2 something in my procedures so I can avoid having to 3 worry about this particular failure mode.

4 So, anyway, so the first step is just 5 asking whether or not command and control coordination 6 is required. And if it is, you need to continue with 7 the screening rules. They have to be done in order.

8 The next one is whether or not there are 9 sequencing of operator actions that need to be 10 coordinated. And if so, then we need to continue.

11 One thing actually kind of interesting 12 there was some feedback on because we weren't able to 13 come up with anything else, and that is the notion of 14 the irreversible failure within 15 minutes. We can 15 probably write some more up on that discussion, but the 16 idea was that it had to be a relatively quick failure.

17 You had an hour to discover there was something wrong 18 with your pump. We thought that was probably, probably 19 that would be discovered and something would be done 20 before it was irreversibly damaged, so.

21 CHAIR STETKAR: Like they did at H.B.

22 Robinson in the main control room with all kinds of 23 procedures and people there. They didn't discover it 24 in, it was like 39 minutes, that they had no cooling.

25 DR. COOPER: Yeah.

NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

168 1 CHAIR STETKAR: Go on. Continue.

2 DR. COOPER: So I may be very interested 3 inputs on how we can develop a different number or what 4 number or range numbers there would be.

5 CHAIR STETKAR: The important thing is --

6 and for the other members here -- remember, whenever 7 it says "screened from consideration" that means 8 cognitive performance in the context of this 9 evaluation is perfect. The HEP is zero. That's the 10 way I would interpret this guidance as a practitioner.

11 DR. COOPER: No. I wouldn't say that.

12 CHAIR STETKAR: Well, tell me how I should 13 interpret it.

14 DR. COOPER: I would say --

15 MEMBER BLEY: It's the implication of 16 that.

17 CHAIR STETKAR: How would I interpret it 18 otherwise? Because it says if I have no command or 19 control, if it's tossed out by any of these criteria, 20 then all I need to worry about is what I call about 21 implementation slips that can be accreted through a 22 THERP type analysis with recovery factors and, you 23 know, all of that neat stuff. That leads to a really 24 low probabilities typically.

25 DR. COOPER: Well, maybe not so low in this NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

169 1 sense because we don't allow for recovery of many kinds.

2 And there are other things, too. There are some 3 differences in how we handle that, but.

4 MEMBER SUNSERI: Well, there doesn't 5 appear to be anything, at least from my view, unique 6 about these three things from a command and control 7 outside the control room or this, like an operator stuck 8 inside the control room.

9 CHAIR STETKAR: Or inside the control 10 room.

11 MEMBER SUNSERI: That's what I mean.

12 CHAIR STETKAR: When you say we haven't 13 addressed this --

14 MEMBER SUNSERI: Firing up with the 15 suction valve closed, 15 minutes is plenty of time to 16 ruin that --

17 DR. COOPER: Okay. So you would have a --

18 the notion is that, okay, so if it -- that's the short, 19 15 minutes or less. So I mean, I mean like I said, I 20 am actively soliciting input on that what number should 21 be. And if it was too generic. I mean, you obviously 22 have to find something out about the specific 23 equipment. How long does it take this pump to deadhead 24 if you have no suction. That's information you can 25 come up with. And you can compare that to whatever NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

170 1 number this is supposed to be.

2 But the next, the next things, I mean, I 3 mean that's the bottom line is what is the short period 4 of time for them not to notice? And the thing is that 5 we're also then going to ask, you know, what are you 6 doing to monitor this? You have somebody there that's 7 going to notice the pump's vibrating. Do you have 8 indications at the remote shutdown panel or somewhere 9 else that someone's looking at it. So that's the next 10 piece of the detection.

11 So the point is, is there a way you can fail 12 a piece of equipment so that it can't be used anymore?

13 CHAIR STETKAR: I'm sorry. This is 14 guidance, and you just said some words that I don't 15 recall reading anywhere in the guidance. I only read 16 a table that says here are the -- I'm an analyst and 17 I don't want to do anything. I want any excuse for not 18 quantifying anything in the world. That's my world 19 view going in.

20 Anything that I can find here where the 21 Nuclear Regulatory Commission approved guidance tells 22 me I don't have to do anything, by God, I will not do 23 it.

24 So I didn't hear -- read anywhere in the 25 guidance that it says, oh, by the way, you have to as NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

171 1 an analyst make sure that you have enough indications 2 at the remote control area so that you can discover the 3 bad thing, and the people are indeed looking at those 4 indications.

5 DR. COOPER: Well, that's the next step.

6 CHAIR STETKAR: And guiding people to do 7 it.

8 DR. COOPER: That's the next step. Maybe 9 they --

10 MEMBER BLEY: That's one piece of it. The 11 other piece, I want to go back to where you said, no, 12 it isn't like concerning with success. Yes, it is. It 13 is. If you're going to screen it, then maybe the least 14 you ought to do is have some not too low chance of 15 failure that gets stuck in instead of that. Because 16 what you're sticking in is zero if you don't include 17 it in the model.

18 CHAIR STETKAR: Well, if they get done, if 19 they still grudgingly keep it, there's still a 95 20 percent chance that you're successful. So you don't 21 have much margin there. You're either 100 percent 22 successful or 95 percent successful; right?

23 MEMBER BLEY: I think this would whittle 24 away at that 95 percent successful if you put in a 25 reasonable screening value. Screening implies NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

172 1 you're, well, here it implies you're throwing it away, 2 assuming that's --

3 CHAIR STETKAR: No, and that's, and that 4 --

5 MEMBER BLEY: But if you put in .1 or --

6 CHAIR STETKAR: But if I'm 7 misinterpreting something, if I'm now going to use this 8 guidance, my interpretation of the guidance, reading 9 this stuff, is that if I meet in any one of these steps 10 screened from consideration the, I'll call it the 11 cognitive portion of my Phase III human error 12 probability is precisely zero. And I then am led to 13 guidance that says, yeah, I can go look it up and look 14 at a standard implementation type of error.

15 DR. COOPER: Not standard. Not standard.

16 CHAIR STETKAR: Now, I need to worry about 17 the timing.

18 DR. COOPER: Yes. Not standard. And 19 only for this specific set of guidance that we're 20 talking about. And that's because unless coordination 21 is required going through the supervisor, or it could 22 be just going through operators, unless that 23 coordination is required and it results in something 24 that fails a piece of equipment -- that's what we're 25 looking to add is some kind of failure mode, additional NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

173 1 failure mode for a particular piece of equipment that's 2 a key, you know, key piece in the whole sequence of 3 things, that's what we're looking for. So.

4 MEMBER BLEY: I'm going to give you 5 something that will help you here.

6 DR. COOPER: Okay.

7 MEMBER BLEY: It's very different as you 8 go from plant to plant on what they have for a panel 9 or for a distributed system. And in a lot of cases you 10 don't have anywhere near the instrumentation you have 11 in the control room, so it's not clear that you'll 12 always have discharge pressure or that you'll have flow 13 indication. Gee, I got the pump running; maybe that's 14 all you see.

15 So then for your plant you've got to be very 16 careful about this one because unless somebody goes 17 running over near that plant -- pump, you aren't going 18 to know it's clunking away. So it's very plant 19 specific in its implementation.

20 DR. COOPER: Okay.

21 MEMBER BLEY: And I don't remember if you 22 tell us that. I'm sure somewhere you tell us all of 23 this is plant specific, but.

24 DR. COOPER: So, okay, I mean I guess, I 25 mean the key thing on this second one is that if you NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

174 1 get the operator actions that need to be coordinated 2 out of sequence, in other words you don't get suction 3 provided to the pump before you start the pump that you 4 can damage the equipment. But maybe if we just leave 5 out the timing piece that's enough.

6 But the point is that we're looking for 7 ways that the coordination can fail. In particular the 8 experts wanted us to focus on sequencing failure. So 9 if we could get things out of sequence, mostly because 10 you've either lost track of what people have done and 11 you get to the point where you want to start a piece 12 of equipment and you've either not kept track of or got 13 confused in some communications and didn't realize that 14 the first piece of this action has been done.

15 So that's what those, that's what those --

16 MEMBER BLEY: I've never seen anybody do 17 the steps and walk through it, if they walk through it.

18 I always had a vision of if we're going to do this stuff, 19 especially if it was distributed, we're going to have 20 radios or headsets on or something. We're going to be 21 saying, Hey, John, go do this. Or, Charlie, go do that.

22 Susan, go over here and do that. Because then when he's 23 done it and I can keep track of it all and make sure 24 he's sequenced right.

25 What you said is maybe not, maybe they all NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

175 1 just get sheets of paper and go out. And then those 2 numbers we saw earlier get even more --

3 DR. COOPER: Well, yeah, I mean I guess the 4 thing is that even though the actions might appear in 5 order in the procedure, the order in which you get the 6 communications may not be the same. So the point is 7 that you could be sending out operator A, B, and C to 8 do things and you're going to wait on B to get D done, 9 and they're going to report back differently.

10 So that's the error, the kind of thing that 11 we're worried about. And then the sequencing error is 12 the ultimate problem.

13 CHAIR STETKAR: Let me try something for 14 you to think about. And that is -- and Matt alluded 15 to it -- when I read through these things I don't --

16 I know that there are different issues as a human 17 reliability analyst that I need to be aware of in these 18 types of conditions compared to sitting in the main 19 control room with whatever procedures I have, which is 20 difficult to know.

21 And I, I certainly endorse the notion that 22 the guidance should make analysts painfully aware of 23 that.

24 On the other hand, accounting for those 25 conditions now that I'm painfully aware of it, I don't NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

176 1 know why we need such prescriptive screening criteria 2 and prescriptive numerical values to plug in because 3 we don't do that, for example, when I'm in the main 4 control room under strange situations that may not 5 perfectly match a set of procedures. I leave it up to 6 the analyst to evaluate performance-shaping factors, 7 use whatever cognitive reliability model they're going 8 to use and whatever implementation slips and all that 9 kind of stuff.

10 Why do we need to be so prescriptive both 11 in terms of screening criteria and explicit numerical 12 values in this particular guidance? In other words, 13 why doesn't the guidance just simply, given the 14 expertise and the amount of time and effort that people 15 have put into thinking about this, this part of the 16 problem, why doesn't the guidance just simply make it 17 very, very clear, as an analyst you need to be 18 considering these factors in the context of your 19 scenario, in the context of your procedures, in the 20 context of your panels, and in the context of whatever 21 the fire damage is?

22 Because it strikes me that the human 23 reliability analysis, given all of that stuff and the 24 fact that communications might be more difficult than 25 standing in the control room, things might take more NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

177 1 time because I need to do Step A clearly before Step 2 B, but I need to do that in the control room.

3 DR. COOPER: So, I guess I think I 4 understand what you're saying. And I think that the 5 thing is that we, maybe we tried to come up with a 6 distillation of what we understood from the experts to 7 try to come up with a simplified approach. And what 8 I think I hear you saying is that we should open it up 9 a little bit and provide the explanation that went 10 behind it so that an individual analyst can try to 11 better match up to their specific thing.

12 CHAIR STETKAR: I'm just a bit 13 uncomfortable about the prescriptiveness of this and 14 its potential for being misused in ways that you hadn't 15 thought about by people who will say, well, this is the 16 NRC-approved -- I followed the NRC-approved guidance 17 and, therefore, it is perfect.

18 You might not have thought about the ways 19 people, creative people can use guidance in very 20 creative ways if they don't want to do something.

21 MEMBER BALLINGER: I think given the 22 variability in all the plants and things like that that 23 you've described, if you're having a very bad hair day 24 in one of these plants they're going to do things that 25 are going to be different. And so being as NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

178 1 prescriptive as John is saying you're having, it seems 2 to me that it is too prescriptive. It's just too --

3 DR. COOPER: Yeah, I guess that the 4 intention was not so much to be prescriptive as to be 5 simple. And part of the thinking behind the going for 6 simple is because more so than with the decision to 7 abandon, I think our experts and their ability to 8 imagine something that's never been done was, it was 9 beyond their ability to imagine how things could go 10 differently or badly.

11 In other words it was sort of it was more 12 like so we've come to this point, we could -- I mean 13 there are a number of other things that we conjectured 14 so far as errors of commission and doing this and that.

15 And after we came out of the expert panel we still sort 16 of debated some of those things. And we finally came 17 down to this.

18 And part of it was we didn't feel like there 19 was anything in the literature that could help us. And 20 certainly we were stretching to try to find other HRA 21 methods that we could use to develop any kind of HEP.

22 And I didn't think we could push our experts to give 23 us them. I think they were -- they got to the issues 24 and that was as far as they could go.

25 And they realized that this is, this is, NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

179 1 I mean this is no longer in the control room. You know, 2 the point was made, yes, there are some remote shutdown 3 panels in the simulator. But as far as I know, the 4 only, only plant that I've heard that actually has been 5 doing integrated training on safe shutdown after the 6 main control room abandonment is Calvert Cliffs.

7 CHAIR STETKAR: Okay. So if I'm at 8 Calvert Cliffs, maybe, maybe my human error 9 probabilities are much smaller than if I'm at Plant X.

10 But I don't tailor my guidance for how to do human 11 reliability analysis based on those snippets of pieces 12 of information. I tailor my guidance so that if I'm 13 at Calvert Cliffs for a particular fire scenario I come 14 out with a human error probability of, you know, 3.7 15 times 10 to the minus 4.

16 And I'm at Plant X for the same fire 17 scenario I come out with .5.

18 DR. COOPER: Yeah, we, like I said, there 19 was a version of this where we were going to have --

20 I mean more five, five on compensatory measures. That 21 could be procedural, health, or training, or whatever.

22 There was some discussion about having different 23 values.

24 But the problem was that how to make 25 distinctions between, you know, what, you know, how can NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

180 1 we make distinctions in the numbers? And what would 2 be the basis for it other than the fact that we just 3 intuitively know they ought to be different?

4 I mean, I guess if you go back to the 5 decision to abandon tree, I know you were hanging on 6 the best case probability.

7 CHAIR STETKAR: No, no.

8 DR. COOPER: But the main thing for me that 9 came out of that was that there was only a factor of 10 10 difference between the best case and the worst case 11 scenario. And to me that meant that they thought the 12 main thing was the reluctance to leave. And anything 13 that could help, like procedures, they didn't think 14 there was much of an impact.

15 And whether that's because they didn't, 16 couldn't imagine how that would work or what, that was 17 the result.

18 CHAIR STETKAR: And again, you say a 19 factor of 10. I say the difference between the 80 20 percent success and 98 percent success, which is a 21 really small difference. It means it's always really 22 successful.

23 MEMBER SUNSERI: Well, I worry about the 24 1 in 5 chance it was not going to be successful. That 25 seems like pretty high failure rate to me.

NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

181 1 DR. COOPER: Let's just arbitrarily move 2 it. So it was .5 and .4. I mean the point is that there 3 wasn't that much difference between -- or a .5 and --

4 well, anyway.

5 CHAIR STETKAR: Let's see --

6 MEMBER SUNSERI: I want to add one comment 7 though regarding this slide that you have up here which 8 I mean I think the focus of this is on when is your 9 command and control, say you have a failure there or 10 not a failure; right? And my earlier comment, well 11 this looks like, a lot like normal operations is, well, 12 did you think about going back to your criteria?

13 Because when I think about what a command and control 14 failure would look like in a control room abandonment 15 scenario I think about this; right?

16 So if I'm in the control room and I'm 17 dispatching an operator to start a pump he's going to 18 go down, he or she is going to go down there. I'm going 19 to be on the radio with them, or I'm going to be on the 20 telephone with them. And they say, I'm starting the 21 pump. Watch for the flow. I get a report back; right?

22 DR. COOPER: Right.

23 MEMBER SUNSERI: When there's a fire out 24 in the plant and I've left the control room, my radios 25 may not be working, my telephones may not be working, NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

182 1 I may not be able to communicate with that person. So 2 that's when it becomes -- so I would put criteria around 3 that to tell me whether or not I lost command and 4 control.

5 So I would look at these five things you 6 have on page 31 and maybe think about what screening 7 criteria would I wrap around those to tell me I had a 8 loss of command versus these human performance issues.

9 DR. COOPER: At least with respect to the 10 communications that's a feasibility criteria. You 11 have to have the communications and the equipment and 12 all those sorts of things. So that that screen has 13 already been met. So you know that you can 14 communicate.

15 There may be some things about the 16 communications that are not optimal. And we discuss 17 that as being part of the command and control.

18 The bottom line is that we're trying to 19 focus on when there is a need for command and control 20 such that it affects one of these actions in the field.

21 Because they're not doing anything at the remote 22 shutdown panel. At least that's not the situation 23 we're looking at. We're looking at a situation where 24 someone's doing something out in the field, and what 25 can we do to make that fail. And it's either -- and NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

183 1 we're focusing on, I mean we have looked at delays.

2 That was one thing that we looked at. But that's not 3 where the experts led us to look.

4 So we're looking at things where you can 5 get things reversed such that you could fail something.

6 Because a delay is just simply going to change the 7 timing, which we can reflect directly in the rest of 8 our analysis and in communications.

9 MS. PRESLEY: So it's not so much that 10 these things can't happen in normal operations, but in 11 the normal command and control situation you have 12 certain bounds and you have people working together.

13 So this is, it's a bigger vulnerability when you're out 14 in the field because you have a different command and 15 control sector. So we're looking for things that are 16 especially vulnerability now that your command and 17 control has changed.

18 MEMBER SUNSERI: Right. So I think about 19 this, so you know in your control room you've got your 20 shift manager and your supervisor right there. When 21 you go abandon the control room those two people may 22 not be in the same place.

23 MS. PRESLEY: They probably are not.

24 MEMBER SUNSERI: And your ability to make 25 timely decisions is now changed; right?

NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

184 1 MS. PRESLEY: Right.

2 MEMBER SUNSERI: That's command and 3 control.

4 MS. PRESLEY: So the things on Slide 31, 5 we basically had to -- so some of those, the timely 6 decision making, and the, like, the feasibility of 7 communications, the communications, the comms 8 equipment being available, those we've already 9 demonstrated through a feasibility assessment.

10 So we're looking at the additional things 11 that can cause -- that are harder to recover from and 12 become more impactful than they are normally because 13 you have that backup, you have that crew working 14 together the way they're used to. So I guess that's 15 how we got to those five criteria. So these are more 16 vulnerable than they would normally be in normal 17 operations because the command and control is not the 18 same level essentially.

19 DR. COOPER: So, for example, in the 20 control room you had a procedure, and if you're talking 21 about putting a pump into place there's probably one 22 step, one step or substeps that are all related to 23 putting that pump into service. And it will be gone 24 through in order, in sequence, read out loud by the 25 shift supervisor with the board operators responding NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

185 1 as they do things.

2 Now we don't have that. We have people in 3 the field reporting when things have gotten. And you 4 might have in a particular attachment, you might have 5 a note that says this needs, this is a whole point you 6 need to communicate with someone else. Or you need to 7 know that, you know, Step A has been performed before 8 you do what you're going to be doing.

9 But that usually, that's going to run 10 through the shift supervisor. And he actually has to 11 do the job of making sure that they're done in the 12 correct order. So that's what we were focusing on.

13 And how is it possible that it could be, those steps 14 could be done out of order, or Step A never got 15 performed, and you tried to get that pump started or 16 you tried to start that diesel without having, without 17 having cooling water or something like that.

18 So that's the issue that we're, that's what 19 we're trying to get at, could that possibly occur or 20 how could that occur? And so the kinds of things that 21 we're thinking about, well, you know, we just lost track 22 of all the communications that are going, coming in.

23 Maybe he doesn't have place-keeping aids. He doesn't 24 have a place or a good way to write things down. Too 25 many calls come in at once. And when he gets, when he NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

186 1 gets the call from the field operators, wants to start 2 the pump and he wants to know have you started this?

3 Has, you know, operator A started the section head?

4 Can I go ahead and start this thing, start this pump?

5 And the shift supervisor is thinking, well, there is 6 a whole bunch of phone calls that just came in, I've 7 got some scribbles here, I think they got a -- I think 8 you're okay. Go ahead. Using the formal 9 communications.

10 So that's, that's one of the failure modes 11 that we imagined that could happen because of the 12 disconnect between, you know, the beginning of an 13 action and end of an action because they're being 14 performed in the field at different locations, running 15 through the shift supervisor.

16 So that's how it's different is that 17 there's no longer control on the ordering of the steps.

18 Okay, so we -- the first screen is, is there 19 coordination.

20 The second one is are there actions that 21 have to be sequenced, these coordinated actions.

22 The next screen is whether or not you can 23 detect whether or not there's something going on wrong 24 with your equipment.

25 And we give some examples of local NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

187 1 indications or, you know, it's possible the shift 2 supervisor might have a procedure step or note to ask 3 somebody the mod for the equipment from another 4 location if it's not the remote shutdown panel.

5 Then the fourth screen is whether or not 6 the shift supervisor is the one that has the 7 responsibility for all the communications. In other 8 words, he's the final point for all the phone calls.

9 He's not -- and then we have some examples of things 10 that would kick you out. Or, you know, if there's 11 nobody else that's helping them make these, fielding 12 these calls.

13 And then I guess the other thing is if there 14 was a specific set of steps that had to be coordinated, 15 you know, it's not like the shift supervisor is just 16 focusing on these steps. You know, he's got a larger 17 set of duties. So he's, he's the focus of all these 18 communications.

19 And then the fifth and final screening step 20 is any kind of compensatory measures that might help 21 the supervisor with coordination and you don't have 22 those. And we give a couple examples. And we're 23 focusing on procedures in these examples.

24 Things that could help that you don't have 25 that would keep you to model this would be if there's NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

188 1 like a hold point written into the procedure, or a 2 warning, or a caution, that sort of thing.

3 And then another thing would be good 4 place-keeping aids so that the supervisor can record 5 when support systems are being put into service so that 6 when he gets to those, those interactions with 7 operators that are putting front line systems into 8 place he does know that the supporting systems have 9 already been put into service and they can move ahead 10 with these final steps to get this equipment into 11 service.

12 So this is the set of screens that we put 13 together. Those are the concerns that we put together 14 and that have been informed by all the interactions that 15 we've had with experts in other discussions. And as 16 I indicated before, the simplified approach that we've 17 taken is to go ahead and assign a single value as you 18 come through these screens. And the basis for this 19 value is -- comes from three, four different places.

20 We did a review of various HRA methods or 21 guidance to see if it addressed some of the similar, 22 some of the issues that we thought were important.

23 NUREG--2114 which is the cognitive basis for HRA NUREG 24 has some discussion about workload and multi-tasking.

25 The NUREG for IDHEAS At-Power, NUREG-2199, NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

189 1 has a decision tree that addresses workload that 2 includes consideration of distractions and 3 interruptions.

4 NARA, the publicly-available paper has a 5 communications generic task type with some error 6 producing conditions that include that include 7 information overload, which we identify in the 8 instructions as a possible failure mode.

9 And then THERP Table 20-8 also talks about 10 errors in recalling oral communications.

11 So, using these, these inputs, we looked 12 at each of these methods. And the range of HEPs from 13 these methods went 1E to the minus 2 to .1. And we 14 picked the middle of that arbitrarily. But those are 15 the -- you can see the range is not, not wide. But 16 that's what we got in -- that's where we came out in 17 the end.

18 MEMBER BLEY: Susan.

19 DR. COOPER: Yes.

20 MEMBER BLEY: I'm still a little off 21 center on this stuff. But much of this kind of looks 22 reasonable to me. The screening steps 2 and 3 together 23 just feel a little funny to me.

24 DR. COOPER: Okay.

25 MEMBER BLEY: This idea that step 3 is NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

190 1 saying, well, you don't protect things right away.

2 You're off trying to do something else. Which kind of 3 seems in conflict with this business of 15 minutes or 4 less.

5 And then 2 is the one bothers me. If in 6 fact you have a condition that would lead to failure 7 in 15 minutes or less, I guess if you don't have that 8 then we're screening from consideration. And that 9 just feels very optimistic to me.

10 CHAIR STETKAR: Everything is perfectly 11 detectable given more than --

12 MEMBER BLEY: From then on.

13 CHAIR STETKAR: -- given more than 15 14 minutes.

15 MEMBER BLEY: And then I get to the next 16 step and, Oh my gosh, I've run off doing other stuff.

17 DR. COOPER: Well, if you took the 15 18 minutes out does that fix the problem?

19 MEMBER BLEY: You've still got to have 20 something that falls through the screening failure rate 21 at a minimum for me.

22 CHAIR STETKAR: Well, the worst it can be 23 is 95 percent success. Again, it's either 100 percent 24 success because it's screened out, or 95 percent 25 success if it's not screened out. So the worst it can NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

191 1 be according to their guidance is 95 percent success, 2 .05 for failure.

3 DR. COOPER: But the basis for this is the 4 role of command and control in the implementation of 5 the safe shutdown strategy. And in the cases where you 6 don't have to coordinate anything, then there really 7 isn't a role except for the time for the communications.

8 There's no, for the strategies we're looking at there's 9 no deciding to use something else. There's no calls 10 from the supervisor to a field operator to do something 11 else.

12 They all have their pieces of procedure.

13 They're implementing those. That's it. That's the 14 way the procedures are written right now.

15 So there isn't -- you know, the shift 16 supervisor doesn't have to allocate resources. He 17 doesn't have to pick up the phone and say, hey you, field 18 operator, go do this. He's not assigning anybody 19 anything. The procedure doesn't, well, the procedure 20 doesn't have the supervisor making any choices so far 21 as going to another procedure, using a different piece 22 of equipment.

23 MEMBER BLEY: If you go back to number two, 24 if you got rid of the whole phrase after the comma.

25 DR. COOPER: "Such as"?

NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

192 1 MEMBER BLEY: Yeah.

2 DR. COOPER: Yeah, that's -- exactly.

3 MEMBER BLEY: Then it says if there is no 4 thing that can fail without some kind of recovery, yeah, 5 then you could go on, I guess.

6 DR. COOPER: Okay. All right.

7 MEMBER BLEY: I'd have to think about that 8 some more. But it just smells a little funny.

9 DR. COOPER: No, I mean it is -- it was, 10 it was a place where I wanted input because I wasn't 11 sure. I didn't -- we had some discussion about it among 12 the team and didn't have any other answers at the time.

13 So we can certainly take that feedback as --

14 MEMBER BLEY: If you don't -- if you can't 15 have an irreversible failure of equipment --

16 DR. COOPER: That's, that's correct.

17 MEMBER BLEY: Fifteen minutes stuff is in 18 interfering with the next --

19 DR. COOPER: Okay. Fair enough.

20 CHAIR STETKAR: Before we get to the -- Is 21 there anything more on these screening and this topic?

22 DR. COOPER: No.

23 CHAIR STETKAR: I want to bring one more 24 thing up that I intentionally left to the end.

25 I really like the notions of the time lines NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

193 1 that you have. I think they're really good. It keeps 2 people focused on those times.

3 Feasibility assessment. Why -- and we've 4 had this discussion in the past -- why do you not 5 quantify uncertainties in the time available, in the 6 time required and look at the overlaps and the 7 uncertainties as a contributor to the overall human 8 error probability, which we have in fact already in the 9 guidance for at-power ideas?

10 And there's a notion that at least I 11 personally have been trying to instill in this 12 organization going forward is that are there 13 uncertainties? Yes, there are. If there's a 2 14 percent probability, given the uncertainties, that the 15 time required is longer than the time available, 16 there's a 2 percent probability that I'm not going to 17 make it. For the other 90 percent chance I'm now 18 looking at what fraction of that 98 percent chance do 19 I win and do I not win?

20 And that's sort of a conceptual thing that 21 is not -- you don't talk about uncertainties in the 22 times, and you certainly don't talk about it on that 23 time dilution type thing.

24 DR. COOPER: Unfortunately, 25 uncertainties are overlapped, yeah.

NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

194 1 CHAIR STETKAR: Well, it is. And that's 2 why I didn't want to start it but I'll just throw that 3 out there. It's on the record. We don't have time for 4 an in-depth discussion. We've had the discussions.

5 Mary can dredge up her notes. But it something that 6 there is -- it's in the IDHEAS methodology, or it has 7 been.

8 And that's a conceptual notion. It's not, 9 it's not focused on these particular scenarios or 10 anything, it just kind of a way of thinking of things.

11 And now you can go on with your path 12 forward.

13 MS. LINDEMAN: Okay. I'm going to give 14 Susan a break. She's done an excellent job thus far 15 explaining all the technical details.

16 So I think our next step and current 17 status. There's a bullet missing, and I just wrote it 18 down it says digest today's presentation and 19 discussion, so.

20 (Laughter.)

21 MS. LINDEMAN: I mean, personally, I think 22 the feedback is valuable and certainly can be, you know, 23 embraced. Having the discussion today, as you alluded 24 to, has anyone tried this out? So, you know, the next 25 step down the road is seeking peer review.

NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

195 1 And Susan and I have been discussing peer 2 review. So I've solicited a number of volunteers.

3 And I know Susan has done the same. So we'll have a 4 good panel of both NRC expertise and industry and 5 practitioner expertise. So I think next step once we 6 digest this feedback is go to peer review and then go 7 through our publication process.

8 CHAIR STETKAR: You don't have plans I 9 don't see here for a -- I'm assuming -- Well, let me 10 ask you. After you have the peer review of the guidance 11 you will now have Supplement 1 and Supplement 2 and the 12 original 1921. Is there any notion of a pilot 13 application, in particular for the main control room 14 abandonment guidance?

15 DR. COOPER: Susan Cooper. It wasn't our 16 original plans. I think that it's possible, you know.

17 Ashley, and Mary and I need to talk some more about the 18 peer review. Part of our public comment period for 19 1921, which was also part peer review, did -- was 20 actually a pilot of I think it was the Westinghouse 21 Owners' Group. I think they did a pilot. So it might 22 be that some of our industry peer reviewers in 23 particular can take that to be more than just reading 24 the document and providing some feedback, but actually 25 trying to exercise it.

NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

196 1 The other thing that we have discussed that 2 isn't in Supplement 2 are examples. And I think there 3 are places where some of the discussions we had today 4 could be better understood if there were some examples.

5 Some, you know, some of the things that we've talked 6 about today, and we've had some other discussions 7 internally coming up to this, this briefing today where 8 we thought that that probably would be, especially on 9 the timing discussions and the phases and how you 10 calculate time available for Phase II and stuff like 11 that, having some illustrative examples on that would 12 probably assist that.

13 But the current schedule for publication, 14 as shown here in brief, does not have a specific 15 piloting phase. The original schedule did but we kind 16 of stretched out time to get to this point with this 17 document.

18 And I'd like to add that I think that 19 without this being a joint project with EPRI, I think 20 coming up with any kinds of guidance in this area that 21 is not supported by operational experience, by 22 plant-specific training or demonstration, and little 23 literature, I think it would be very, very difficult 24 to come up with anything.

25 CHAIR STETKAR: As Mark Henry said in the NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

197 1 introduction, I think that in these areas that that 2 process, the MOU process benefits both the agency and 3 the industry tremendously because it kept both entities 4 kind of grounded --

5 DR. COOPER: Yes. And I'd like --

6 CHAIR STETKAR: -- in some concept of 7 reality anyway.

8 DR. COOPER: I'd like to, I mean, you know, 9 the, you know, we looked at existing methods in the 10 beginning, but the fact that our industry partners were 11 willing to put CBT aside and start with something new 12 because it technically matched what we thought were 13 concerns, I think is not something that probably would 14 have -- either one of us would independently have felt 15 like we could have done.

16 CHAIR STETKAR: Thank you. Thank you, 17 thank you. Let me take care of a couple of things here.

18 If there's anyone in the room here who'd 19 like to make a comment, come up to the microphone and 20 do so.

21 I'm assuming that there is still some 22 chance that there might be members of the public on the 23 phone line. If there is anyone out there who would like 24 to make a comment, I'm hoping that the system will work, 25 please speak up and identify yourself and make a NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

198 1 comment.

2 (No response.)

3 CHAIR STETKAR: I don't hear anything.

4 I hate to do this because the system is 5 supposed to work. But is there anyone out there 6 listening who can just confirm.

7 MR. BROWN: The line's open.

8 CHAIR STETKAR: It is? Okay, thanks, 9 Theron. Thank you.

10 All right, having heard no comment --

11 MS. GUNTER: This is Kaydee Gunter. I've 12 been listening the whole time.

13 CHAIR STETKAR: Okay.

14 MS. GUNTER: I have no additional 15 comments.

16 CHAIR STETKAR: Great. Thank you.

17 Thank you, thank you. This time of the day and the 18 silence on the line made me a bit uneasy.

19 So, no public comments.

20 I'd like to go around the table and get any 21 final comments or observations from the members. Dr.

22 Ballinger, sir.

23 MEMBER BALLINGER: No comments. Great 24 presentation.

25 CHAIR STETKAR: Dennis?

NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

199 1 MEMBER BLEY: It's been a good discussion.

2 I still have some things to think about. I guess if 3 you're doing revisions of this and you're getting 4 public comments then if we ever want to write a letter 5 it would be after they get these comments I think.

6 But nothing more to question I don't think.

7 CHAIR STETKAR: Matt.

8 MEMBER SUNSERI: It's days like this where 9 I'm glad I sit on this side of the table instead of that 10 side of the table.

11 You guys did a great job of fielding all 12 the questions we asked. And I appreciate all the hard 13 work that goes into this. I know it's a tremendously 14 difficult task trying to figure out how people would 15 act in every situation. So, thank you for your work.

16 CHAIR STETKAR: Jose.

17 MEMBER MARCH-LEUBA: Nothing for me.

18 CHAIR STETKAR: Walt.

19 MEMBER KIRCHNER: No, thank you.

20 CHAIR STETKAR: Joy.

21 MEMBER REMPE: No comments.

22 CHAIR STETKAR: Vesna.

23 MEMBER DIMITRIJEVIC: Well, I was silent.

24 Usually I'm silent when I don't have a question. So 25 I have a comment.

NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

200 1 This was the second reason because I really 2 didn't have a chance to carefully looking at the report, 3 so that's why I was so withholding on the questions.

4 So I noticed that sometimes in this, 5 because it's such a complex problem, and we really 6 admire what you guys are trying to do as such a complex 7 problems by always so many issues. And sometimes I had 8 the feeling that we were discussing not completely the 9 same thing because it wasn't clear exactly what HEP for 10 Phase II, if it's actually abandonment of the control 11 room in the sense that they decide to do that and they 12 do whatever they have to do, turn this switch or not.

13 And in some case this habitability issue 14 comes ten to decide to leave, but it may be worse for 15 them to perform actions they have to perform. So we 16 are often discussing things on different levels.

17 So this is why I was thinking that we should 18 actually, it will really help if you guys have examples.

19 Now these examples can be coming, obviously doesn't 20 have to be specific plan, you just see the remote 21 shutdown panel or procedure, but it could be some 22 different scenarios because a lot of different 23 scenarios will leave completely different responses.

24 And it seems to me that there is a danger 25 we don't identify, but right now we are in the situation NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

201 1 if you have a scenario where we lost any controls, you 2 know, could be component cooling system, could be 3 offset power, could be whatever, any control from main 4 control room, that we have to, you know, make decision 5 to abandon the control, that, it could be for different 6 scenarios, you may see this number. Right now the 7 number is 3 minus 1 in the worst case and 5 minus 2 for 8 CMC, so this is 2.5E minus 1; right? So we have a 25 9 percent status to pay for any scenario which requires 10 to abandon the control room.

11 And if this is how the plants are going to 12 interpret that, then there is certain dangers there, 13 so. You know, so some examples I think may benefit.

14 A couple scenarios go down and see, you know, what's 15 happening. So that's about it.

16 CHAIR STETKAR: Thank you. And I'm not 17 going to say it. Thank you for all the work. And I 18 really appreciate your stamina. I think, I think we 19 had a good discussion.

20 With that, we are adjourned.

21 (Whereupon, at 5:41 p.m., the meeting was 22 adjourned.)

23 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

Supplement 2, NUREG1921 Quantitative HRA for Main Control Room Abandonment Susan E. Cooper (NRC/RES)

Ashley Lindeman & Mary Presley (EPRI)

ACRS PRA Subcommittee Meeting April 4, 2018 Rockville, MD A Collaboration of the Electric Power Research Institute (EPRI) & U.S. NRC Office of Nuclear Regulatory Research (RES)

Status of Fire HRA Modeling for Main Control Room Abandonment (MCRA)

Todays presentation:

- Provides updated information for joint EPRI/NRC-RES HRA research project

- Addresses second product of this research which:

Builds on previous research products

- NUREG-1921 / EPRI 1023001, which established guidance for the conduct of fire HRA

- NUREG-1921, Supplement 1 / EPRI 3002009215, which established guidance for the conduct of qualitative HRA for MCRA Uses the experience developing NUREG-1921 & supporting NFPA 805 submittals Will provide the last piece of the MCRA research, i.e., guidance for the quantification of MCRA HFEs ACRS PRA Subcommittee - April 4, 2018 A Collaboration of U.S. NRC Office of Nuclear Regulatory 2

Supplement 2, NUREG-1921 Research (RES) & Electric Power Research Institute (EPRI)

Todays Agenda Review project history and background

- Project team Review of Supplement 1 (Qualitative Analysis)

- Summary of responses to comments Supplement 2 (HRA Quantification Guidance)

- Overview

- HRA quantification guidance:

Before the fire creates conditions for abandonment (minimal guidance so not discussed today)

For the decision to abandon the MCR on loss of control For implementation of MCRA shutdown strategy Status and future work Closing ACRS PRA Subcommittee - April 4, 2018 A Collaboration of U.S. NRC Office of Nuclear Regulatory 3

Supplement 2, NUREG-1921 Research (RES) & Electric Power Research Institute (EPRI)

Project History and Background ACRS PRA Subcommittee - April 4, 2018 A Collaboration of U.S. NRC Office of Nuclear Regulatory 4

Supplement 2, NUREG-1921 Research (RES) & Electric Power Research Institute (EPRI)

Project History and Background Objective of research is to provide HRA guidance for scenarios involving abandonment of main control room due to fire generated conditions

- Loss of habitability (LOH)

- Loss of control (LOC)

Identified as an area of future research in EPRI 1023001 /

NUREG-1921 Complex topic and plant specific considerations

- Wide range of remote shutdown panel (RSDP) capabilities of and associated procedures

- Therefore, difficult to develop generic guidance ACRS PRA Subcommittee - April 4, 2018 A Collaboration of U.S. NRC Office of Nuclear Regulatory 5

Supplement 2, NUREG-1921 Research (RES) & Electric Power Research Institute (EPRI)

Project Team NRC/RES

- Susan E. Cooper, NRC

- Stacey Hendrickson, Sandia National Laboratory

- John Wreathall, John Wreathall & Co., Inc.

- Tammie Rivera, NRC EPRI

- Ashley Lindeman, EPRI

- Mary Presley, EPRI

- Erin Collins, Jensen Hughes

- Paul Amico, Jensen Hughes

- Jeff Julius, Jensen Hughes

- Kaydee Gunter, Jensen Hughes ACRS PRA Subcommittee - April 4, 2018 A Collaboration of U.S. NRC Office of Nuclear Regulatory 6

Supplement 2, NUREG-1921 Research (RES) & Electric Power Research Institute (EPRI)

Supplement 1 (Qualitative Analysis)

ACRS PRA Subcommittee - April 4, 2018 A Collaboration of U.S. NRC Office of Nuclear Regulatory 7

Supplement 2, NUREG-1921 Research (RES) & Electric Power Research Institute (EPRI)

Qualitative Analysis (NUREG-1921 Supplement 1 / EPRI 3002009215)

Presented to ACRS PRA Subcommittee in May 2016 Held peer review in Summer 2016

- Objectives Review for technical items Does it meet needs of intended users?

- Peer review composition PRA practitioners (industry and NRC),

Human reliability analysts (industry and NRC), and Cognitive and behavioral science analysts (consultants)

EPRI published joint report in August 2017 ACRS PRA Subcommittee - April 4, 2018 A Collaboration of U.S. NRC Office of Nuclear Regulatory 8

Supplement 2, NUREG-1921 Research (RES) & Electric Power Research Institute (EPRI)

Qualitative Analysis (NUREG-1921 Supplement 1 / EPRI 3002009215)

Summary of revision since May 2016

- Edits and clarifications associated with ACRS and peer review comments

- Added analytical process diagram (Section 2.4.1)

- Added dual unit timeline example (Section 7.6.3)

- New section in Appendix A addressing variability in capability of RSDP design (Section A.3)

- Provided additional development on Appendix B - Command and Control

- Added Appendix D - insights from operator interviews ACRS PRA Subcommittee - April 4, 2018 A Collaboration of U.S. NRC Office of Nuclear Regulatory 9

Supplement 2, NUREG-1921 Research (RES) & Electric Power Research Institute (EPRI)

Three Phases of MCRA Phase I - associated with actions taken before the decision to abandon

- Actions directed from MCR using EOPs Phase II - the decision to abandon is plant-specific and requires agreement between plant operations, fire PRA modeling and fire HRA

- Typically the HRA/PRA team will need to define conditions which require abandonment and the time at which these conditions will exist.

Good example of FPRA feedback to training and/or procedures.

Phase III - models actions after the leaving the MCR, similar to NUREG-1921 ex-MCR actions

- Also addresses command & control, coordination and communications ACRS PRA Subcommittee - April 4, 2018 A Collaboration of U.S. NRC Office of Nuclear Regulatory 10 Supplement 2, NUREG-1921 Research (RES) & Electric Power Research Institute (EPRI)

Three Time Phases of MCRA Phase I - Time period before abandonment decision Phase II - Time period for the decision to abandon Phase III - Time period after abandon has been made ACRS PRA Subcommittee - April 4, 2018 A Collaboration of U.S. NRC Office of Nuclear Regulatory 11 Supplement 2, NUREG-1921 Research (RES) & Electric Power Research Institute (EPRI)

Example of integrated timeline for multiple HFEs ACRS PRA Subcommittee - April 4, 2018 A Collaboration of U.S. NRC Office of Nuclear Regulatory 12 Supplement 2, NUREG-1921 Research (RES) & Electric Power Research Institute (EPRI)

Example of Timeline For Phase III (Dual Unit Abandonment)

ACRS PRA Subcommittee - April 4, 2018 A Collaboration of U.S. NRC Office of Nuclear Regulatory 13 Supplement 2, NUREG-1921 Research (RES) & Electric Power Research Institute (EPRI)

Supplement 2 (HRA Quantification Guidance)

ACRS PRA Subcommittee - April 4, 2018 A Collaboration of U.S. NRC Office of Nuclear Regulatory 14 Supplement 2, NUREG-1921 Research (RES) & Electric Power Research Institute (EPRI)

Overview of Supplement 2 Supplement 2 developed to be a companion document to NUREG-1921 and NUREG-1921, Supplement 1 High-level development approach

1. Focused development on Phase II & Phase III Decision to abandon on loss of control (LOC) (Phase II)

Implementation of MCRA safe shutdown strategy (Phase III)

2. Insights from Supplement 1 were used to identify key issues that should be represented in quantification

3. List of issues were compared to existing HRA methods

4. Authors developed strawman quantification approaches Phase II - developed candidate decision trees Phase III - list of issues only

5. Expert inputs were used to confirm and/or modify each strawman and complete quantification methods ACRS PRA Subcommittee - April 4, 2018 A Collaboration of U.S. NRC Office of Nuclear Regulatory 15 Supplement 2, NUREG-1921 Research (RES) & Electric Power Research Institute (EPRI)

Role and Use of Expert Panel Panel composed of:

- Two NRC staff with knowledge of operations and training, including fire-related expertise Harry Barrett (retired; formerly NRR NFPA-805 reviewer)

Jim Kellum (NRO)

- Two HRA/PRA analysts experienced with plant fire-risk studies Jeff Julius, JENSEN HUGHES Erin Collins, JENSEN HUGHES Experts needed to have experience and understanding of a wide range of HRA/PRA and operations issues, especially with respect to MCRA scenarios

- Experience/understanding across a range of NPPs (not just plant-specific for 1-2 NPPs)

Experts used differently for Phase II and Phase III

- Phase II: Confirmed key issues, pruned/modified strawman decision trees using prioritized uses, provided HEPs for best, worst, & a few immediate-case contexts, pair-wise comparison inputs for other decision tree branches

- Phase III: Confirmed/pruned key issues, identified main priorities & associated contexts ACRS PRA Subcommittee - April 4, 2018 A Collaboration of U.S. NRC Office of Nuclear Regulatory 16 Supplement 2, NUREG-1921 Research (RES) & Electric Power Research Institute (EPRI)

Overview of Supplement 2 (continued)

How to use Supplement 2

- Supplement 2 provides quantification guidance on MCRA scenarios that were beyond the scope of NUREG-1921

- Certain types of operator actions & human failure events (HFEs) in MCRA scenarios should be addressed with NUREG-1921

- NUREG-1921, Supplement 1 qualitative analysis guidance Is needed to apply HRA quantification guidance in Supplement 2 (and is repeated, if needed)

Introduced, defined, and illustrated (via event analyses) the key concept of command and control (C&C)

- Supplement 2 expands upon this concept ACRS PRA Subcommittee - April 4, 2018 A Collaboration of U.S. NRC Office of Nuclear Regulatory 17 Supplement 2, NUREG-1921 Research (RES) & Electric Power Research Institute (EPRI)

Phase II - Decision to Abandon ACRS PRA Subcommittee - April 4, 2018 A Collaboration of U.S. NRC Office of Nuclear Regulatory 18 Supplement 2, NUREG-1921 Research (RES) & Electric Power Research Institute (EPRI)

Phase II - Decision to Abandon (LOC only)

High Level Approach Step 1: Developed list of factors important to the quantification of the decision to abandon Step 2: Consideration of existing HRA methods Step 3: Adaptation of existing HRA methods against factors important to quantification Step 4: Development of new decision trees specific for decision to abandon Step 5: Obtained subject matter expert (SME) feedback on decision trees and human error probability (HEP) values Step 6: Document findings and provide guidance to analyst ACRS PRA Subcommittee - April 4, 2018 A Collaboration of U.S. NRC Office of Nuclear Regulatory 19 Supplement 2, NUREG-1921 Research (RES) & Electric Power Research Institute (EPRI)

Step 1: Factors important to decision to abandon (team consensus)

Issue Differentiation Points Procedure - Best case: Explicit statement of severe conditions (such as Content extensive MCR instrumentation failure or equipment failure consistent with fire PRA modeling) due to fire that require abandonment

- Worst case: SS/SM discretion only

- Intermediate case: Procedure provides fire locations that, when identified and confirmed, indicate likelihood of needing to abandon, but still leave it up to SS/SM decision Time available - Best case: Long (~20-25 mins)

(versus time - Worst case: Short (~5 mins) required) - Intermediate case: Moderate (15 mins)

Training - Best case: Realistic training in simulator

- Worst case: Classroom only training at minimum level

- Intermediate case: Detailed classroom training coupled with explicit MCRA criteria provided in procedures ACRS PRA Subcommittee - April 4, 2018 A Collaboration of U.S. NRC Office of Nuclear Regulatory 20 Supplement 2, NUREG-1921 Research (RES) & Electric Power Research Institute (EPRI)

Step 1: Factors important to decision to abandon (team consensus) (continued)

Issue Differentiation Points Cues and - Best case: SS/SM has procedural guidance and training on cues that indications clearly and quickly identify that:

(a) fire has been detected in a location relevant to the abandonment criteria, (b) the MCR is no longer reliable as a source of system/component information and control

- Worst case: No consistent or coherent way of seeing that LOC is occurring using the displays in the MCR.

- Intermediate case: Multiple cues but simulator and classroom training has prepared operators on what to look for and how to make decision Reluctance This is related to the: a) capability of the RSDP, b) operator comfort &

familiarity with the MCR, c) inability of operators to conceive of such a desperate situation.

- Best case: Capable RSDP, explicit MCRA criteria & realistic training

- Worst case: Very limited capability RSDP, no explicit MCRA criteria, &

minimum classroom training

- Intermediate case: Most major systems on RSDP, some MCRA criteria; some training ACRS PRA Subcommittee - April 4, 2018 A Collaboration of U.S. NRC Office of Nuclear Regulatory 21 Supplement 2, NUREG-1921 Research (RES) & Electric Power Research Institute (EPRI)

Step 1: Factors important to decision to abandon (team consensus) (continued)

Issue Differentiation Points Staffing and -Best case: SS/SM aided in decision-making by STA or other crew Communications who are monitoring clear abandonment criteria as would be done with Critical Safety Function Trees

-Worst case: SS/SM discretion only

-Intermediate case: SS/SM receives timely input from ex-MCR operator on severity of fire OR from other in-MCR crew on status of MCR boards and key equipment Transfer to - Best case: Clear transfer to necessary procedure steps (or STA is procedure with assigned to monitor MCRA criteria and MCR indications in order to MCRA criteria achieve quick transfer to procedural steps for MCRA)

- Worst case: No specific transfer steps.

- Intermediate case: Transfer is not distinct.

ACRS PRA Subcommittee - April 4, 2018 A Collaboration of U.S. NRC Office of Nuclear Regulatory 22 Supplement 2, NUREG-1921 Research (RES) & Electric Power Research Institute (EPRI)

Step 1: Timing associated with decision to abandon on LOC Timing not as clear cut as typical internal events actions

- Time available: left over time (after Phase III)

- Time required: gradually unfolding cue, highly dependent on procedures/training ACRS PRA Subcommittee - April 4, 2018 A Collaboration of U.S. NRC Office of Nuclear Regulatory 23 Supplement 2, NUREG-1921 Research (RES) & Electric Power Research Institute (EPRI)

Step 2: Existing HRA Quantification Methods Considered several HRA quantification approaches:

- Cause-Based Decision Tree (CBDTM)

- Human Cognitive Reliability/Operator Reliability Experiment (HCR/ORE)

- Integrated Human Event Analysis System (IDHEAS) At-Power

- Standardized Plant Analysis Risk Human Reliability Analysis (SPAR-H)

- Nuclear Action Reliability Assessment (NARA)

- Cognitive Reliability and Error Analysis (CREAM)

ACRS PRA Subcommittee - April 4, 2018 A Collaboration of U.S. NRC Office of Nuclear Regulatory 24 Supplement 2, NUREG-1921 Research (RES) & Electric Power Research Institute (EPRI)

Step 3: Adaptation of Existing HRA Quantification Methods Initial quantification discussion centered around re-interpreting the CBDTM decision trees Review asked:

- Is the failure mode of the tree still applicable?

- Are there dominant failure modes or mechanisms missing from the set that should be accounted for?

Review yielded:

- CBDTs intended for one main cue. For LOC, the cue is more vague (fire alarm, verification of fire, and verification of LOC)

- Both abandonment procedural step and transfer to abandonment were supposed to be covered and this presented some confusion in re-interpretation

- Decision trees were binary decisions, but the procedure quality range was too large to fit within binary structure ACRS PRA Subcommittee - April 4, 2018 A Collaboration of U.S. NRC Office of Nuclear Regulatory 25 Supplement 2, NUREG-1921 Research (RES) & Electric Power Research Institute (EPRI)

Step 4: - Draft decision trees Developed three new decision trees for decision to abandon

- Failure to transfer to MCRA procedure

- Failure to understand the MCRA criteria have been met

- Reluctance / delay Decision trees formed the skeleton for discussion with SMEs

- Failure to transfer to MCRA procedure - removed from consideration by the SMEs who concluded it was not a significant contributor to failure.

- Failure to understand the MCRA criteria have been met -expanded to address reluctance and timing

- Reluctance / delay - subsumed into criteria tree ACRS PRA Subcommittee - April 4, 2018 A Collaboration of U.S. NRC Office of Nuclear Regulatory 26 Supplement 2, NUREG-1921 Research (RES) & Electric Power Research Institute (EPRI)

Step 5: Summary of Expert Feedback Primary driving factors from expert discussion

- Reluctance to abandon, key driving factor, across all response

- Timing (as discussed earlier)

- General training of crew to integrate information and look forward to what is coming (judgment and understanding)

These considerations are built into the base HEP estimates in the final decision tree HEP estimates and key drivers based on current U.S. fleet

- HRA guidance for MCRA would be substantially different for a new NPP design that uses a substantially different MCRA safe shutdown strategy, including re-constitution of the entire MCR operating crew at essentially a backup MCR ACRS PRA Subcommittee - April 4, 2018 A Collaboration of U.S. NRC Office of Nuclear Regulatory 27 Supplement 2, NUREG-1921 Research (RES) & Electric Power Research Institute (EPRI)

Steps 5 & 6: Final Decision Tree HEP when Tavail HEP when Tavail Simulator or talk through Awareness of decision is less decision is HEP when Tavail Abandonment logic explicit in training on the decision to urgency "time than or equal to 5 between 5 and 25 decision is greater the procedures abandon pressure" minutes minutes than 25 minutes Yes 1E-01 6E-02 2E-02 (a)

Talk- through/ simulator observations No 1E-01 8E-02 3E-02 (b)

Criteria documented in procedure Yes 1E-01 7E-02 3E-02 ( c)

Classroom only No 1E-01 9E-02 5E-02 (d)

Yes 1E-01 9E-02 5E-02 (e)

Talk-through / simulator observations No 2E-01 1E-01 8E-02 (f)

Judgement Yes 2E-01 1E-01 6E-02 (g)

Classroom only No 2E-01 2E-01 1E-01 (h)

ACRS PRA Subcommittee - April 4, 2018 A Collaboration of U.S. NRC Office of Nuclear Regulatory 28 Supplement 2, NUREG-1921 Research (RES) & Electric Power Research Institute (EPRI)

Phase III ACRS PRA Subcommittee - April 4, 2018 A Collaboration of U.S. NRC Office of Nuclear Regulatory 29 Supplement 2, NUREG-1921 Research (RES) & Electric Power Research Institute (EPRI)

Phase III: Implementation of MCRA Safe Shutdown Strategy Supplement 2, Section 4 provides guidance on HRA quantification for all HFEs in Phase III

- Section 4.3 provides process for identifying, modeling, and assigning HEPs to HFEs (including feasibility assessment and analysis of timing)

Most of Phase III HFEs are associated with execution of MCRA procedure steps at local plant stations

- NUREG-1921 guidance for quantification should be used with Supplement 2 Most of the research effort for Supplement 2 focused on when/if additional HFEs (or failure modes for existing HFEs) should be added to represent failures in C&C

- Section 4.1 and 4.2 summarize concerns with respect to C&C

- Section 4.3.3.3 addresses C&C failures

- Appendix B provides additional information regarding C&C research ACRS PRA Subcommittee - April 4, 2018 A Collaboration of U.S. NRC Office of Nuclear Regulatory 30 Supplement 2, NUREG-1921 Research (RES) & Electric Power Research Institute (EPRI)

Phase III: Definition of C&C Supplement 1 to NUREG-1921 (qualitative analysis for main control room abandonment [MCRA]) is based on military definitions but is focused on the following as key functions related to C&C at nuclear power plants (NPPs):

- Maintaining a coherent understanding of the plant state (situational awareness)

- Timely decision-making

- Allocating resources as needed

- Coordination of actions

- Managing communications between team members such that they are timely and effective ACRS PRA Subcommittee - April 4, 2018 A Collaboration of U.S. NRC Office of Nuclear Regulatory 31 Supplement 2, NUREG-1921 Research (RES) & Electric Power Research Institute (EPRI)

Phase III - High-Level Approach to Address Command &

Control Impacts Expanded upon Supplement 1 regarding understanding of C&C for implementation of MCRA safe shutdown scenario Developed a consensus on likely relevant issues Used input from experts to:

- confirm key issues

- focus on a specific C&C failure mode and associated context Developed a set of screening rules for analyst to identify contexts when a C&C failure should be modeled Reviewed existing HRA methods to identify those that addressed issues similar to C&C issues A human error probability (HEP) was selected from range of HRA methods with similar issues ACRS PRA Subcommittee - April 4, 2018 A Collaboration of U.S. NRC Office of Nuclear Regulatory 32 Supplement 2, NUREG-1921 Research (RES) & Electric Power Research Institute (EPRI)

Supp 1: C&C differences between Normal Operations and MCRA Operations During in-MCR, Normal Operations During MCRA Operations (typical plant) (typical plant)

Control room team, acting as a single centralized Control room team distributed in plant areas cognitive entity

• Supervisor may be alone at RSDP, but typically also has

• Coordination with fire brigade and some plant area staff one RO

• Coordination with fire brigade and plant areas Shared visual cues Single views of plant information by individuals Well-rehearsed and tested plans and actions MCRA phase III is one of the most frequently

• Resources anticipated & available exercised fire PRA scenarios, however it is not sure if

• Limited need for flexibility in response C&C is exercised. The concern is that C&C portions

• Recognition-primed decision-making of the plans and actions occasionally rehearsed, rarely tested (but this is plant dependent).

• Some resources anticipated & available but complete range untested

• Potential need for flexibility in response Communications (mostly) face-to-face, voice Communications mostly audio via radios, phones; so few confirmatory indications such as annunciators &

alarms Restricted interruptions during response period Unknown potential for interruptions ACRS PRA Subcommittee - April 4, 2018 A Collaboration of U.S. NRC Office of Nuclear Regulatory 33 Supplement 2, NUREG-1921 Research (RES) & Electric Power Research Institute (EPRI)

Phase III: Expert Input on Key Issues/Concerns for C&C Because of how the MCRA safe shutdown strategy is implemented, C&C is different for MCRA operations, such as:

- While the role of shift supervisor (SS) or shift manager (SM) who has the primary responsibility for C&C in main control room (MCR) is the same following MCRA, the capabilities for indications and controls at the remote shutdown panel (RSDP) are different

- Communications are different, and impacts time required for response

- Most likely no alarms, and few indications at the RSDP so need to closely monitor parameters and may be more susceptible to distractions

- Coordination of actions may be required & is complicated by operators being in different locations & any associated communications issues Focus on C&C failures to coordinate actions, especially irreversible damage ACRS PRA Subcommittee - April 4, 2018 A Collaboration of U.S. NRC Office of Nuclear Regulatory 34 Supplement 2, NUREG-1921 Research (RES) & Electric Power Research Institute (EPRI)

Changes in Role of Supervisor and Communications Role of SS/SM in MCRA operations SS/SM continues to drive operations but for most actions cannot directly observe the action

- Monitors some actions, and coordinates some actions Operators have different roles Allocation of resources mostly done via assignments of various procedure attachments to operator (ROs & field operators)

Communications are different for MCRA operations Typically NOT face-to-face May be only reports that actions are completed Communications equipment & associated problems can be a concern ACRS PRA Subcommittee - April 4, 2018 A Collaboration of U.S. NRC Office of Nuclear Regulatory 35 Supplement 2, NUREG-1921 Research (RES) & Electric Power Research Institute (EPRI)

Phase III: Screening Rules for When to Model a C&C Failure Screening Description of Screening Step If No If Yes Step C&C coordination is required for placing equipment Screened Go to into service, e.g., successful pump operation requires

1. from screening adequate suction head from supporting consideration step 2.

equipment/system.

Failure to properly sequence operator actions for placing equipment into service would result in an Screened Go to

2. irreversible failure of the equipment, such as either from screening condition below leading to SSC failure within 15 consideration step 3.

minutes or less.

Operators can not immediately detect improper functioning of equipment (in order to immediately shut down equipment), due to, for example, Screened Go to

i. A lack of local indications (including a lack of

3. from screening equipment or flow noises that are recognizable from consideration step 4.

training or experience); or ii. The field operator moving to a different location without checking for proper functioning ACRS PRA Subcommittee - April 4, 2018 A Collaboration of U.S. NRC Office of Nuclear Regulatory 36 Supplement 2, NUREG-1921 Research (RES) & Electric Power Research Institute (EPRI)

Phase III: Screening Rules for When to Model a C&C Failure Screening Description of Screening Step If No If Yes Step Supervisor in C&C role has responsibility for all (or the bulk) of communications to/from field operators, e.g.,

i. No one else is providing significant help to take or make calls to field operators implementing MCRA safe shutdown strategy and call from other plant staff (e.g., fire brigade, health physics), or ii. C&C is NOT solely (or mostly) focused on the communications associated with the equipment of concern Screened Go to

4. and its supporting equipment/systems such as due to lack of from screening help from other staff in taking/making these communications. consideration step 4.

iii. Communications are "segregated" such that supervisor and multiple field operators whose actions must be coordinated are NOT on a common loop such that all parties hear all communications (e.g., operator controlling cooling water to a pump does not hear the command to start the front-line system pump and therefore cannot alert the supervisor that there is no cooling water in service).

ACRS PRA Subcommittee - April 4, 2018 A Collaboration of U.S. NRC Office of Nuclear Regulatory 37 Supplement 2, NUREG-1921 Research (RES) & Electric Power Research Institute (EPRI)

Phase III: Screening Rules for When to Model a C&C Failure Screening Description of Screening Step If No If Yes Step There are NO compensatory measures to assist the supervisor with coordination. Example cases where compensatory measures are NOT present are:

i. The MCRA procedure does NOT include a written step, or Hold Point, or Warning (Caution) that prerequisite SSC alignment is needed prior to Include Screened operation. For example, if an MCRA procedure C&C-related

5. from coordination includes a caution about putting in supporting consideration failure.

equipment/system into service before putting into service the equipment in question.

ii. The MCRA procedure does NOT include place-keeping aids such that the supervisor can record when support systems are in service, allowing the start of front-line systems.

ACRS PRA Subcommittee - April 4, 2018 A Collaboration of U.S. NRC Office of Nuclear Regulatory 38 Supplement 2, NUREG-1921 Research (RES) & Electric Power Research Institute (EPRI)

Phase III: HEP to Assign to C&C Failure If screening rules result in C&C failure should be modeled, then an HEP of 5E-2 is recommended Basis for HEP:

- Focused on sequencing failures and how they might occur

- Review of literature for other technologies turned our focus to distractions and interruptions Focused on interruptions as a good description of many types of communications that could compete for supervisors attention

- Search for HRA methods that addressed similar issues:

NUREG-2114 considers workload and multi-tasking NUREG-2199 (IDHEAS at-power) has workload decision tree that includes distractions/interruptions NARA has a communications generic task type with error producing conditions such as information overload THERP, Table 20-8 addresses errors in recalling oral communications

- Range of HEPs from these sources: 1E-2 to 0.1 ACRS PRA Subcommittee - April 4, 2018 A Collaboration of U.S. NRC Office of Nuclear Regulatory 39 Supplement 2, NUREG-1921 Research (RES) & Electric Power Research Institute (EPRI)

Status and Future Work ACRS PRA Subcommittee - April 4, 2018 A Collaboration of U.S. NRC Office of Nuclear Regulatory 40 Supplement 2, NUREG-1921 Research (RES) & Electric Power Research Institute (EPRI)

Current Status Supplement 2, Quantitative HRA Guidance for MCRA scenarios:

- Draft report provided to ACRS PRA SC will be starting point for peer review

- Peer Review:

Scheduled to begin April 2018

- Final publication is planned for end of 2018 ACRS PRA Subcommittee - April 4, 2018 A Collaboration of U.S. NRC Office of Nuclear Regulatory 41 Supplement 2, NUREG-1921 Research (RES) & Electric Power Research Institute (EPRI)

Closing Remarks from EPRI and NRC/RES management Questions?

ACRS PRA Subcommittee - April 4, 2018 A Collaboration of U.S. NRC Office of Nuclear Regulatory 42 Supplement 2, NUREG-1921 Research (RES) & Electric Power Research Institute (EPRI)

ACRS PRA Subcommittee - April 4, 2018 A Collaboration of U.S. NRC Office of Nuclear Regulatory 43 Supplement 2, NUREG-1921 Research (RES) & Electric Power Research Institute (EPRI)

Supplement 2 Backup Slides ACRS PRA Subcommittee - April 4, 2018 A Collaboration of U.S. NRC Office of Nuclear Regulatory 44 Supplement 2, NUREG-1921 Research (RES) & Electric Power Research Institute (EPRI)

Phase I - Time Period Before Abandonment Decision Operator actions directed from MCR, similar to other fire scenario

- Tripping reactor

- Starting emergency diesel generator (EDG)

- Starting a system that failed to auto start Quantification guidance:

- Use NUREG-1921 to evaluate and quantify Phase I actions ACRS PRA Subcommittee - April 4, 2018 A Collaboration of U.S. NRC Office of Nuclear Regulatory 45 Supplement 2, NUREG-1921 Research (RES) & Electric Power Research Institute (EPRI)

ATHEANA expert elicitation process (NUREG-1800) used in expert panel Facilitator guided feedback from experts (& controlled bias)

Experts were already familiar with MCRA context & concept of probabilities Important details of contexts were discussed:

- Decision to abandon on LOC Discussed HRA teams list of key issues Confirmed, clarified, & focused list of issues Pruned initial three decision trees => one decision tree with fewer branches

- Role of C&C after MCRA Discussed HRA teams candidate issues Confirmed & focused issues (e.g., priority of C&C sequencing that results in catastrophic equipment failure)

Facilitator guided experts through quantification process (decision to abandon only)

- Best case, worst case, & intermediate case for each expert, then consensus reached

- Pair-wise comparisons used to determine other branch points ACRS PRA Subcommittee - April 4, 2018 A Collaboration of U.S. NRC Office of Nuclear Regulatory 46 Supplement 2, NUREG-1921 Research (RES) & Electric Power Research Institute (EPRI)

Supplement 1 Backup Slides ACRS PRA Subcommittee - April 4, 2018 A Collaboration of U.S. NRC Office of Nuclear Regulatory 47 Supplement 2, NUREG-1921 Research (RES) & Electric Power Research Institute (EPRI)

Qualitative Analysis (NUREG-1921 Supplement 1 / EPRI 3002009215)

Provides foundational material required to support quantification Content:

- Mindset shift from internal events and fire HRA to MCRA HRA

- PRA modeling of MCRA scenarios

- Analysis of the decision to abandon

- Identification and definition of MCRA HFEs

- Feasibility assessment for MCRA scenarios

- Time and timelines for MCRA scenarios

- Performance shaping factors

- Recovery/dependency/uncertainty

- Appendices (Historical events review of MCRA scenarios, Command and control, and Guidance and tips for MCRA information collections)

ACRS PRA Subcommittee - April 4, 2018 A Collaboration of U.S. NRC Office of Nuclear Regulatory 48 Supplement 2, NUREG-1921 Research (RES) & Electric Power Research Institute (EPRI)

Qualitative Analysis ACRS Organizational Comments PRA and HRA people should read everything, not just pertinent sections

- Removed Section 1.2, Intended Audience, which encouraged selective reading of material

- Toned down degree of interface between HRA/PRA throughout Add more in Appendix A about capabilities of RSDP

- Added new Section A.3, Alternative and Remote Shutdown Panel Variations ACRS PRA Subcommittee - April 4, 2018 A Collaboration of U.S. NRC Office of Nuclear Regulatory 49 Supplement 2, NUREG-1921 Research (RES) & Electric Power Research Institute (EPRI)

Qualitative Analysis ACRS Reference Comments References to IDHEAS (specifically Appendix B / PSF)

- Report acknowledges IDHEAS-G in Section 2

- IDHEAS also referenced in Supplement 2 (both for decision to abandon and command and control)

Discuss NUREG-2114 influence on PSF / C&C

- More explicated references in Section 8 (PSF) & Appendix B (C&C)

ACRS PRA Subcommittee - April 4, 2018 A Collaboration of U.S. NRC Office of Nuclear Regulatory 50 Supplement 2, NUREG-1921 Research (RES) & Electric Power Research Institute (EPRI)

Qualitative Analysis ACRS Approach/Scope Comments SISBO strategy should be addressed in the report

- ACRS version: SISBO not addressed

- Final: Added Section 3.5.6 Self-Induced Station Blackout (SISBO) and Other Recoverable Pre-Emptive Actions Why is LOH fundamentally different from LOC?

- Greatly expanded on discussion on LOH (Section 4.1), including revisiting the technical basis

- Silent on discussion between too early and too late.

ACRS PRA Subcommittee - April 4, 2018 A Collaboration of U.S. NRC Office of Nuclear Regulatory 51 Supplement 2, NUREG-1921 Research (RES) & Electric Power Research Institute (EPRI)

Qualitative Analysis ACRS Timing Comments Timing

- Time = 0 Clarified T=0 assumption in Section 2.4.3, General Assumptions

- Recovery actions called out in Phase III as separate items (10 minute recovery artificially constraining timeline)

Removed recovery actions in timeline discussion

- Time uncertainty Time remains source of uncertainty In interviews generally look for a range Use point estimate in development of timeline If key source of uncertainty, should do sensitivity study ACRS PRA Subcommittee - April 4, 2018 A Collaboration of U.S. NRC Office of Nuclear Regulatory 52 Supplement 2, NUREG-1921 Research (RES) & Electric Power Research Institute (EPRI)