ML18303A086

Transcript of the 657th Advisory Committee on Reactor Safeguards Full Committee Meeting - October 4-6, 2018

ML18303A086
Person / Time
Issue date:	10/04/2018
From:	Advisory Committee on Reactor Safeguards
To:
Banks M
References
NRC-3917
Download: ML18303A086 (327)

Text

Official Transcript of Proceedings NUCLEAR REGULATORY COMMISSION

Title:

Advisory Committee on Reactor Safeguards Docket Number: (n/a)

Location: Rockville, Maryland Date: Thursday, October 4, 2018 Work Order No.: NRC-3917 Pages 1-327 NEAL R. GROSS AND CO., INC.

Court Reporters and Transcribers 1323 Rhode Island Avenue, N.W.

Washington, D.C. 20005 (202) 234-4433

1 1

2 3

4 DISCLAIMER 5

6 7 UNITED STATES NUCLEAR REGULATORY COMMISSIONS 8 ADVISORY COMMITTEE ON REACTOR SAFEGUARDS 9

10 11 The contents of this transcript of the 12 proceeding of the United States Nuclear Regulatory 13 Commission Advisory Committee on Reactor Safeguards, 14 as reported herein, is a record of the discussions 15 recorded at the meeting.

16 17 This transcript has not been reviewed, 18 corrected, and edited, and it may contain 19 inaccuracies.

20 21 22 23 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 www.nealrgross.com

1 1 UNITED STATES OF AMERICA 2 NUCLEAR REGULATORY COMMISSION 3 + + + + +

4 657TH MEETING 5 ADVISORY COMMITTEE ON REACTOR SAFEGUARDS 6 (ACRS) 7 + + + + +

8 THURSDAY 9 OCTOBER 4, 2018 10 + + + + +

11 ROCKVILLE, MARYLAND 12 + + + + +

13 The Advisory Committee met at the Nuclear 14 Regulatory Commission, Two White Flint North, Room 15 T2B1, 11545 Rockville Pike, at 8:30 a.m., Michael L.

16 Corradini, Chairman, presiding.

17 COMMITTEE MEMBERS:

18 MICHAEL L. CORRADINI, Chairman 19 PETER RICCARDELLA, Vice Chairman 20 RONALD G. BALLINGER, Member 21 DENNIS C. BLEY, Member 22 CHARLES H. BROWN, JR. Member 23 MARGARET SZE-TAI Y. CHU, Member 24 VESNA B. DIMITRIJEVIC, Member 25 JOSE MARCH-LEUBA, Member NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

2 1 DANA A. POWERS, Member 2 HAROLD B. RAY, Member 3 JOY L. REMPE, Member 4 GORDON R. SKILLMAN, Member 5 MATTHEW SUNSERI, Member 6

7 DESIGNATED FEDERAL OFFICIAL:

8 DEREK WIDMAYER 9

10 *Present via telephone 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

3 1 CONTENTS 2 Opening Remarks . . . . . . . . . . . . . . . . . 4 3 Draft Rule on Emergency Preparedness for Small 4 Modular Reactors . . . . . . . . . . . . . 6 5 Annual Operating Reactor Experience Briefing . . 63 6 Assessment of the Quality of Selected NRC 7 Research Projects . . . . . . . . . . . . 131 8 Adjourn . . . . . . . . . . . . . . . . . . . . 199 9

10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

4 1 P R O C E E D I N G S 2 (8:28 a.m.)

3 CHAIRMAN CORRADINI: Okay, the meeting 4 will now come to order. This is the first day of the 5 657th Meeting of the Advisory Committee on Reactor 6 Safeguards.

7 In today's Meeting, the Committee will 8 consider the following, draft rule on emergency 9 preparedness for small modular reactors and other 10 nuclear technologies, annual operating reactor 11 experience, assessment of the quality of selected NRC 12 research projects, and preparation of ACRS reports.

13 The ACRS was established by statute and is 14 governed by the Federal Advisory Committee Act, or 15 FACA. As such, this Meeting is being conducted in 16 accordance with the provisions of FACA. That means 17 the Committee can only speak through its published 18 letter reports.

19 We hold Meetings to gathering information 20 to support our deliberations. Interested parties who 21 wish to provide comments can contact our Offices 22 requesting time after the Federal Register notice 23 describing the Meeting is published.

24 That said, we also set aside 10 minutes 25 for extemporaneous comments from members of the public NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

5 1 attending or listening to our Meetings. Written 2 comments are also welcome. Today Mr. Derek Widmayer 3 is the designated Federal official for the initial 4 portion of the Meeting.

5 The ACRS Section of the U.S. NRC public 6 website provides our charter bylaws, letter reports, 7 and full transcripts of all full and Subcommittee 8 Meetings including all slides presented at those 9 Meetings.

10 We're received no written comments or 11 requests to make oral statements from members of the 12 public regarding today's sessions. There will be a 13 phone bridge line. To preclude interruption of the 14 Meeting, the phone will be placed in a listen-only 15 mode during the presentations and Committee 16 discussions.

17 Also, today's Meeting is being webcast. A 18 transcript of portions of the Meeting is being kept 19 and it is requested that speakers use one of the 20 microphones to identify themselves and speak with 21 sufficient clarity and volume so they can be really 22 heard.

23 I'll also remind everybody to please turn 24 off your devices or mute them so we don't have 25 buzzing, ringing, beeping during the Meeting. With NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

6 1 that, our first topic will be talking about the draft 2 rule on emergency preparedness for small modular 3 reactors and Mike Scott will begin the session. Mr.

4 Scott?

5 MR. SCOTT: Good morning, Mr. Chairman, 6 thank you very much. Yes, my name is Mike Scott, I am 7 the Director of the Division of Preparedness and 8 response in the Office of Nuclear Security and Instant 9 Response at the NRC.

10 I am here speaking on behalf of the 11 Steering Committee for Emergency Preparedness for 12 small modular reactors and other new technologies 13 since Dr. Holahan, who would normally be sitting here 14 is on travel.

15 So I'd like to take this opportunity to 16 thank the ACRS Committee for allowing us the 17 opportunity to discuss with you the emergency 18 preparedness for small modular reactors and other new 19 technologies, or EPSMR ONT, proposed rulemaking. It's 20 a lot to say.

21 Seated at the table with me are Kenny 22 Thomas from NSIR, Andrew Carrera, the rulemaking PM 23 from the Office of Nuclear Material, Safety, and 24 Safeguards, and Arlon Costa from the Office of New 25 Reactors.

NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

7 1 We have Working Group Members in 2 management from various Offices here at the NRC who 3 are also here this morning to support the 4 presentation.

5 Key Staff Members include Steve Lynch from 6 NRR, Ed Roach from NSIR, Howard Benowitz, and Marcia 7 Carpentier from OGC, Michelle Hart from NRO, and Keith 8 Compton from the Office of Research, all in attendance 9 to assist in addressing questions the Committee might 10 have.

11 As most of you are aware, on August 22nd, 12 the Staff met with the ACRS Joint Subcommittee to 13 discuss the draft proposed rule and proposed guidance.

14 At the Meeting the Staff discussed the background and 15 regulatory basis associated with this rulemaking, as 16 well as the approaches the Staff has taken to the 17 rulemaking.

18 Staff appreciates the comments and 19 feedback that we have received from the Subcommittee 20 regarding the proposed rulemaking.

21 The Staff has considered the 22 Subcommittee's comments in preparing for today's 23 Meeting, and the goal for today's Meeting primarily 24 for us is to address the ACRS Subcommittee's comments.

25 We understand that it was the NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

8 1 Subcommittee's preference that we focus on these 2 subjects and so that's primarily what you'll hear 3 today. Of course, if there are additional questions 4 that come up, we will of course answer those.

5 We will also provide an overview of the 6 rule and the philosophy behind it. As previously 7 presented in the Subcommittee Meeting, the Staff is 8 proposing new EP requirements and implementing 9 guidance to adopt a consequence-oriented, 10 risk-informed, and performance-based approach to 11 emergency planning, as well as one that is technology 12 inclusive.

13 The proposed rule would provide all 14 existing and future SMR and non-light water reactor 15 licensees applicants and future utilization facility 16 licensees that would be licensed after the effective 17 date of the rule, an alternative to develop and to 18 comply with a performance-based EP rule.

19 The proposed rule would be an alternative 20 to the existing deterministic EP requirements that are 21 in 10 CFR Part 50. Because it is optional, it is not 22 a backfit and backfit considerations do not apply.

23 One of the aspects of the rule that is 24 most transformational I'd like to highlight is the 25 concept of making EPZ sizing proportional to the risk NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

9 1 posed by the facility. The NRC has a long history of 2 doing that in a more case-specific manner.

3 If you think back to the days of Fort 4 Saint Vrain and Big Rock Point, those were examples of 5 reactors that operate at power reactors that operated 6 with a smaller EPC size than the 10 miles that is 7 normally specified for large, light-water reactors and 8 that is, in fact, the case for all of the existing 9 fleet because those small reactors with lower risks 10 are no longer in operation.

11 So that has been the case for a 12 substantial period of time. We also do not require 13 offsite Federally-mandated licensee-funded 14 radiological emergency preparedness plans for research 15 and test reactors.

16 So this is not new to us, the concept is 17 not new, but the opportunity for an Applicant or 18 licensee to demonstrate the appropriate EPZ size based 19 on risk posed by credible accidents regardless of the 20 technology involved is new.

21 So to take that to its full extent, there 22 is the potential for a site boundary EPZ, that is to 23 say for a facility that has a particularly low risk, 24 to have the EPZ end at the site boundary such that 25 there would be no offsite Federally-mandated EPZ.

NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

10 1 So that is a transformational process for 2 the NRC. The staff has developed an associated draft 3 implementing guidance document, DG1350 for use by 4 licensees, applicants, and the NRC Staff.

5 The draft proposed rule is on track to be 6 submitted to the Commission in October of this year 7 for a vote prior to issuance for public comment.

8 Later in the presentation, Andy will provide you will 9 further details regarding the rulemaking deliverables 10 and scheduling.

11 I'd like to especially acknowledge and 12 express my appreciation for the Working Group Members 13 for all of the outstanding work that they have done 14 associated with this rulemaking effort. It's been a 15 substantial rule and we believe it's come out in a 16 good place.

17 We look forward to an informative 18 interaction with the ACRS today, and with that, I'll 19 turn the presentation over to Kenny Thomas.

20 CHAIRMAN CORRADINI: Dennis, go ahead.

21 MEMBER BLEY: The words credible accident 22 has come up many times in the discussion. We're more 23 than 40 years since WASH1400. We speak of risk as 24 consequences and likelihood. What does credible mean 25 and why is it in here if this is --

NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

11 1 MR. SCOTT: What does incredible mean?

2 MEMBER REMPE: Credible.

3 CHAIRMAN CORRADINI: I think, Dennis, you 4 meant to say what does credible mean? Either one.

5 MEMBER BLEY: Our risk concept is based on 6 credibility and consequences and credible seems to 7 have no point unless you put the number to it and you 8 want people never to think of anything out there.

9 It's about risk-informing.

10 MR. THOMAS: Thank you, Dr. Bley. I'm 11 Kenny Thomas and I will be leading the presentation 12 this morning but I would like to call on a couple of 13 our key Staff Members to be able to at least mention 14 their analysis or what we have at the NRC that 15 addresses credibles.

16 First up would be Michelle Hart from the 17 Office of New Reactors. Michelle?

18 MS. HART: I'm Michelle Hart from the 19 Office of New Reactors.

20 I work in the Radiation Protection and 21 Accident Consequences Branch and, yes, the term 22 credible in the regulation is intended to mean we're 23 talking about if you look at a range of accidents but 24 we don't want you to have to do anything that is very 25 unlikely.

NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

12 1 It's not intended to be just a design 2 basis accident or something like that. We don't have 3 a specific definition of credible in this document or 4 in the regulation, nor do we have a definition of 5 credible anywhere else that I'm aware of.

6 CHAIRMAN CORRADINI: So let me follow 7 Dennis. So what I hear is that it gives the Staff 8 wiggle room to determine what sequence, frequency is 9 above the line or below the line.

10 MS. HART: Right, we're looking and this 11 is something that as we review, looking at the 12 specific design and looking at the PRA information and 13 other information about the plant design, we can 14 determine the universe of scenarios that should be 15 included in this range of accidents.

16 And that's what credible is intended to 17 imply.

18 CHAIRMAN CORRADINI: Dennis? Can you 19 repeat that? The universe of what?

20 MS. HART: Accident scenarios. Universe 21 of accident scenarios.

22 MEMBER REMPE: Well, I'll ask, no one has 23 said the frequency word and cutoff frequency word at 24 this time, and that's the question that's kind of 25 being danced around here, right?

NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

13 1 MR. SCOTT: That plays into what credible 2 is.

3 MEMBER REMPE: It should, yes. That plus 4 uncertainties, yes, it definitely should. But nobody 5 wants to bite the bullet and say this is what the cut-6 off frequency should be.

7 MS. HART: This is Michelle Hart again.

8 I think that's something to be determined based on 9 specific analysis that we're looking at for the 10 specific design.

11 We're not pre-defining what credible is 12 based on a CDF or something like that, especially 13 since some future reactor design, non-light-water 14 reactor designs, or even non-light-water reactors, 15 other types of facilities may not have things such as 16 core damage.

17 So core damage frequency, it may be more 18 related to a release frequency and so we don't know 19 all these designs at this time and so we're not making 20 a pre-decision. We'll look at the total amount of 21 information about that plan to help make that 22 decision.

23 CHAIRMAN CORRADINI: So, I'm going to 24 pretend to be the company. So I'm coming in blind as 25 to what's expected? I'm putting you on the spot but NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

14 1 on the other hand, that's what I sense.

2 In other words, you could say go look at 3 0396 and supposedly, there's some sort of logic and 4 methodology in 0396 to years ago and at least you 5 ought to --

6 MS. HART: That will give you some idea of 7 the information that you would have and certainly, and 8 we can talk about this some more, there's an 9 expectation in the regulation that you took out a 10 probabilistic risk assessment that you describe in 11 your final safety analysis report and described the 12 results.

13 So, there's an expectation that you will 14 have looked at the risk of your plant in the first 15 place so you should have sufficient information to at 16 least get started on this.

17 CHAIRMAN CORRADINI: Dennis is back.

18 Dennis?

19 MEMBER BLEY: Yes, sir?

20 CHAIRMAN CORRADINI: Did you hear all of 21 this or part of it?

22 MEMBER BLEY: I've heard part of it. I 23 got knocked off the line for a little bit.

24 CHAIRMAN CORRADINI: Do you have a follow-25 up?

NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

15 1 MEMBER BLEY: I'm still where I was at the 2 beginning. The words incredible really have no 3 business in anything that's risk-informed. If the 4 kinds of answers are big enough, you want the 5 likelihood very, very small.

6 If the consequences are low, you don't 7 care so much. And undefined terms that we use when we 8 just did deterministic looking are not --

9 MEMBER MARCH-LEUBA: Is it the 10 understanding of the Staff that credible means 11 frequency or adverse consequences? That's I think what 12 Dennis is asking.

13 MS. HART: The Staff intends that for each 14 design they will be able to say what kind of releases 15 are coming from that plan and what are credible for 16 their specific design, what are likely to happen for 17 their design.

18 And so, yes, frequency can be used and is 19 likely to be used because they will have PRAs to be 20 able to define the likelihood of these events and the 21 consequences.

22 So, yes, the intent is to use both 23 aspects, the frequency and the consequence.

24 MEMBER BLEY: If I could jump in again, 25 credible as you just said is the result of the PRA NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

16 1 where you've looked at the whole range of 2 consequences, rather than something going in, the way 3 it's stated before that discussion sounds like there's 4 a whole class you don't even think about.

5 You draw a line somehow and that one just 6 doesn't fit the content.

7 MR. KAHLER: If I can, this is Bob Kahler, 8 I'm Branch Chief with NSIR DPR overseeing the 9 technical aspects of the rulemaking.

10 With regards to the EP rulemaking, the use 11 of the term for credible accidents, where we are 12 identifying, that is something that would be 13 predetermined in order to be used, in order to be part 14 of the risk-informing of the EPZ size and of the 15 regulation.

16 So, this is something that we looked to 17 the other Offices to be able to make that 18 determination. As it goes through the design review 19 process, that suite of accidents would be identified 20 by the design.

21 And 0396 had the luxury of having those 22 accidents already available to them from the current 23 designs that they were looking at for the ten-mile 24 EPZ.

25 We have that same approach that we're NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

17 1 utilizing here for the credible accidents that were 2 identified through WASH 1400 and such, were part of 3 the analysis done in 0396 as to what credible 4 accidents would be utilized to determine the 10-mile 5 EPZ size or to determine the EPZ size.

6 So, when you look at it, we are just 7 saying that the term credible is something that we are 8 utilizing within the EP rulemaking as the designs have 9 already been approved and predetermined with that 10 suite of accidents.

11 I just want to make sure we understand 12 that the EP rulemaking piece of this is -- we're not 13 defining the term credible within the EP rulemaking.

14 It is something that is going to be provided for the 15 designers and for those that will be determining and 16 proposing EPZ sizes.

17 CHAIRMAN CORRADINI: So let me quote from 18 0396. There was a sentence in here that, to me, said 19 it and it sounds to be novice, risk-informed. To 20 state this is on Appendix 1, Page 10.

21 To restate this, there was about a one 22 percent chance of emergency plans being activated in 23 the U.S. beyond the recommended EPZs within the next 24 few years or within the next time period. It seems to 25 me in an uninitiated fashion that's close to a NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

18 1 risk-informed criteria.

2 Now, I could add onto it that I want a 95 3 percent confidence relative to uncertainties but it 4 seems to me some sort of guidance like this is going 5 to have to be there whether it's in the rule or it's 6 going to be in a reg guide connected to the rule.

7 Otherwise, the incoming industries are 8 going to be -- we need a rock and you're going to tell 9 them not that rock, go find me another rock. And I 10 just don't find that to be an acceptable path forward.

11 That's what I sense Dennis is asking.

12 MEMBER RAY: Maybe adding to what Michael 13 just said, this discussion suggests that maybe there's 14 room for some elaboration such as we just heard about 15 the use of the word credible.

16 It's going to be determined in the 17 following manner or something like that.

18 MEMBER REMPE: So, if we are done with 19 that discussion, because I have a different question 20 for you.

21 CHAIRMAN CORRADINI: Just a second.

22 MR. COMPTON: This is Keith Compton from 23 the Office of Resources.

24 Just to get back to your point, I'm 25 familiar with that statement about the likelihood and NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

19 1 I think one of the key things that NUREG 0396 did, and 2 it's all through the document, is they were saying 3 that you should have considered not simply of design 4 basis accidents but beyond design basis accidents.

5 And what we are trying to do is keep that 6 philosophy and that's true there, that in the 7 methodology credible is not limited to design basis 8 documents. It would include your consideration beyond 9 design basis.

10 What NUREG 0396 did you is that for beyond 11 design basis accidents, it included the consideration 12 of their frequency. It did it in a certain way that 13 used the frequencies and the consequences from WASH 14 1400. So we're keeping that idea.

15 The question of credible is something 16 that, as I think Michelle just pointed to that, is 17 another layer. Is there something that is simply so 18 either low frequency or for which you make an argument 19 that it just simply is not credible?

20 That is something that would have to be 21 assessed by the Staff. It would be very challenging 22 for me right now to give a single prescriptive 23 guidance that would cover all possible circumstances.

24 So right now we're simply putting this information 25 out.

NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

20 1 MEMBER RAY: But I don't think we would 2 suggest that kind of detail.

3 Just some elaboration that indicates 4 because this goes out to a community of people that 5 aren't invested in this the way we all are, indicating 6 that what defines the credible or what is the credible 7 accident will be determined in whatever manner you 8 say. A sentence or two would be ample in my judgment.

9 MEMBER BLEY: This is Dennis again. The 10 way it was just phrased is pretty good.

11 I have a strong bias against language 12 because of the way it's been used in the past but if 13 you anchor it to those concepts and that is really 14 coming from the risk analysis and it goes beyond 15 design basis, that would certainly help.

16 Right now, it's just there is something 17 for which everyone has their own definition.

18 CHAIRMAN CORRADINI: I'm sorry, Joy, go 19 ahead. I guess now we're on a different subject.

20 MEMBER REMPE: I would like to explore a 21 little bit more about what happened with Fort Saint 22 Vrain. It started off being authorized or licensed by 23 the Atomic Energy Commission.

24 When did 0396 come out? When was the 25 planning zone selected? Was NRC really the NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

21 1 organization that authorized the EPZ because of timing 2 when 0396 came out?

3 MR. SCOTT: That's even before my time, I 4 hate to admit it.

5 MEMBER REMPE: Yes, I was in high school 6 back then.

7 MR. SCOTT: Do we have somebody here who 8 can speak to the details of the way Fort Saint Vrain 9 transpired? Bob, surely you weren't around for that?

10 MR. THOMAS: Dr. Rempe, again, Fort Saint 11 Vrain, we were in the midst of the post-TMI 12 rulemaking.

13 We had published the proposed rule in 1980 14 and there were three plants that applied during the 15 proposed rule phase and said, hey, could we get 16 reconsidered instead implementing a ten-mile emergency 17 planning zone around our plant, specifically Fort 18 Saint Vrain, Big Rock Point, and La Crosse?

19 They came in during the proposed rule 20 phase and said we propose a five-mile plume exposure 21 pathway emergency planning zone and a 30-mile 22 ingestion pathway emergency planning zone around our 23 plants.

24 They provided that technical 25 justification, the Staff reviewed that technical NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

22 1 justification and agreed to impose a 5-mile, 30-mile 2 emergency planning zone around those three facilities.

3 And that all happened prior to the 4 issuance of the 1980 rule. So if you were going to 5 look back at the rule in Section 50.33, you'll see 6 that specific exclusion is in 50.33G, Paragraph --

7 MEMBER REMPE: Out of curiosity, what kind 8 of justification? Did you go to that level of detail 9 to figure out what they provided that gave you that 10 confidence? Did they give you some risk assessment?

11 MR. THOMAS: I do have those documents, I 12 did not bring those with me today but they are 13 available. Mr. Ed Roach is approaching the 14 microphone, I can turn it over to him.

15 MR. ROACH: Hi, I'm Ed Roach, I work in 16 NSIR. I do have the Big Rock Point document that we 17 took off with the microfiche, and basically, they did 18 an analysis of the releases and where they would 19 exceed the EPA tags at that point.

20 And that's how they came in and requested 21 via the NRC to set the EPZ at five miles for Saint 22 Vrain and Lacrosse, and also provide a document like 23 that.

24 And that was prior to the final FRN being 25 issued to the final rule for the EP in 1980. It all NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

23 1 reflected in that final rule but it happened as a 2 result of the first TMI actions.

3 MEMBER REMPE: So when you said they 4 provided the releases, for what type of events beyond 5 design basis events?

6 MR. ROACH: An example, I can put this to 7 the Committee if you want but it's in our records, 8 what the fission yield for activity and then the 9 analysis with meteorology for that site, why they 10 justified setting it at 55.

11 MEMBER REMPE: I'm just curious if they 12 went beyond it to the beyond design basis events is 13 what I'm trying to get to.

14 MEMBER BLEY: This is Dennis, may I sneak 15 in?

16 MEMBER REMPE: Sure.

17 MEMBER BLEY: I kind of hate to admit it 18 but I was around then.

19 MEMBER BROWN: You're not the only one.

20 MEMBER BLEY: Two things happened at the 21 same time. WASH 1400 in about 1973 then got revised a 22 little bit. The shift from AEC to NRC happened in 23 about 1974, as I remember 1975.

24 The reactor safety study came out as WASH 25 1400 under the AEC and later was published under a NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

24 1 NUREG number under the NRC. But the Agency didn't 2 really change on the regulatory side.

3 What the process did was put up the 4 promotional side in the IRDA and later DOE. So the 5 regulatory side is pretty consistent. I think we'd be 6 smart to just consider AEC through NRC as the one 7 Agency that changed names on the regulatory side.

8 MEMBER BALLINGER: By the way, 50.33G 9 does not say 5 and 30. It says the size of EPZ may 10 also determine on a case-by-case basis for gas 11 pollutant reactors and for reactors with an authorized 12 power level of less than 250 megawatts, thermo. So it 13 didn't say, it didn't give a number.

14 MR. THOMAS: No, it didn't and it was that 15 case-by-case analysis that this rule really does 16 intend to say for small modular reactors, less than 17 1000 megawatts thermal, and other new technologies 18 based on the design we're going to take a look at that 19 on a case-by-case basis.

20 MEMBER BALLINGER: So this is a little bit 21 ambiguous if you talk about NuScale because one 22 NuScale module is X, Y modules is Y.

23 MR. THOMAS: Thermal?

24 MEMBER BALLINGER: Yes, sir.

25 MR. COSTA: This is Arlon Costa but the NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

25 1 definition that we have in the rule, we also define 2 small modular reactor that's anything less than 1000 3 megawatts thermal and so NuScale falls into that 4 category.

5 MR. SCOTT: I believe what he's referring 6 to is you get a 12-pack of NuScales, more than 1.

7 That's the distinction.

8 CHAIRMAN CORRADINI: I think we can go 9 ahead now. We've excavated this as far as we can so 10 far.

11 MR. THOMAS: Sounds good.

12 CHAIRMAN CORRADINI: On your first line.

13 MR. THOMAS: We are still on track believe 14 it or not. We can finish this.

15 MEMBER BALLINGER: Do you want any 16 morphine for the root canal?

17 MR. THOMAS: No, I've got plenty of water.

18 So, anyway, thank you again for these great comments 19 that really set the stage for this presentation, a 20 lively discussion and your comments.

21 I took notes through this so we'll 22 continue to do that. In this presentation we will 23 discuss the proposed rule and the thinking that went 24 into this rule. So Andy, if I can get the next slide, 25 please.

NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

26 1 Okay, the proposed rule would be 2 technology-inclusive.

3 It would provide an option to existing 4 small modular reactors or SMRs, we've been saying this 5 morning, and non-light-water reactor Applicants as 6 well as all future small modular reactor and other new 7 technology facilities licensed after the effective 8 date of the rule.

9 The Staff was tasked to develop a rule 10 that is technology-inclusive, which means that it will 11 apply to a wide variety of designs, most of which the 12 Staff has yet to see. The Staff does not have, as we 13 pointed out already, or need the information about the 14 source terms in order to develop the emergency 15 preparedness rule.

16 Source term information is developed and 17 reviewed under other processes. Therefore, the 18 requirements for developing the various source terms 19 and their other uses are not addressed by this rule.

20 CHAIRMAN CORRADINI: So let's stop there 21 for a minute. What are the other processes?

22 MR. THOMAS: That's a great question.

23 Andy, can we go to the last slide? We added a slide.

24 MR. CARRERA: This slide was developed 25 really late last night. Dr. Corradini developed some NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

27 1 of the responses in answer and the Staff developed 2 these slides and it's included in the back of the last 3 slide of the package.

4 And it really speaks about the Accident 5 Source Terms. Unfortunately, it was late so it was 6 not included.

7 CHAIRMAN CORRADINI: But where I'm going 8 with this is this is just one Member speaking but the 9 fact that I have one paragraph in Appendix A and one 10 sentence in the rule strikes me as a bit on the 11 minimal side, where one needs to go with this because 12 it leaves it wide open.

13 And I connect it back to Page 10 of 14 Appendix 1 of 0396 is that the reason I picked that 15 sentence is they then connected, as I think Ken noted, 16 to a set of both design basis accidents and severe 17 accident release source terms and by that combination 18 determined what they thought was the proposed 10-mile.

19 So I'm still back to the same thing that 20 it's leaving the reader of the rule very little 21 guidance at this early stage. And it strikes me as 22 not very helpful if you're really trying to help the 23 future industry.

24 MR. THOMAS: Thank you, Dr. Corradini.

25 Again, Michelle Hart is at the microphone.

NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

28 1 MS. HART: So I understand what you're 2 saying about the guidance doesn't say specifically 3 look at DBAs and look at severe accidents. Is that 4 what you're comment really is?

5 CHAIRMAN CORRADINI: At minimum, I think 6 that's kind of really where Harold was suggesting at 7 a minimum you want to provide some sort of discussion 8 because the sense of it is that's what happened back 9 40 years ago back in the '78 NUREG.

10 MS. HART: and that is our intent. So 11 there are requirements that they've develop the design 12 basis accidents to look at the siting and safety 13 analysis, look at control room compatibilities.

14 They're already doing that so that would be something 15 that's there. And of course, those have a defined --

16 and I'm talking purely about reactors here and I'm 17 talking about light water reactors especially, we do 18 have guidance on how to do that assessment. And so 19 it's something that they're already aware of how to 20 do. For severe accidents as I had mentioned earlier, 21 there is a requirement for reactors to provide 22 probabilistic risk assessment for their designs, and 23 so the development of the source terms for that, they 24 need that to look at the large release frequency that 25 they are required to look at the risk of the plant, to NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

29 1 look at the risk characteristics. So you use those 2 same source terms to develop those source terms, those 3 releases to the environment, you can use the PRA 4 standards. And there's one in development for the 5 non-light-water reactors as well. And so we're 6 relying on the fact that these other assessments are 7 already required to license a facility and that you 8 would have this information. And if I understand your 9 comment, it's that it's not clear in our guidance 10 that's what we're relying on. There's other processes 11 --

12 CHAIRMAN CORRADINI: I think it's so brief 13 as to the well-informed would probably know where to 14 go. You have a spectrum of individual companies that 15 think they're going to proceed with various designs.

16 I'm not sure all of them are all well-informed.

17 MS. HART: Right, and this discussion is 18 very reactor-focused and so this may not be as 19 applicable to other facilities as well.

20 CHAIRMAN CORRADINI: I've made my point.

21 Thank you.

22 MS. HART: Does that answer your concerns?

23 CHAIRMAN CORRADINI: Yes, thank you very 24 much.

25 MR. THOMAS: We've been using the term NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

30 1 other new technologies in this presentation as well as 2 in some of the associated documents to refer to non-3 light water reactors in medical radioisotope 4 facilities. However, in the rule we don't refer to 5 other new technologies. Rather, we use and define in 6 the regulations non-light water reactors and non-7 power- production utilization facilities. In the 8 context of this proposed rule, medical radioisotope 9 facilities to be licensed under 10 CFR Part 50 but 10 also be included within the use of non-power-11 production or utilization facilities. This rule 12 proposes to apply the Commission's expectation that 13 advanced reactors would provide enhanced margin of 14 safety and/or use simplified inherent passive or other 15 innovative means to accomplish their safety and 16 security functions.

17 Next slide, please. May the provisions of 18 this proposed rule and guidance would provide a new 19 performance-based emergency preparedness framework, 20 which is an alternative to the current regulations.

21 The framework contains requirements for demonstrating 22 effective response and drills and exercises for 23 emergency and accident conditions, a hazard analysis 24 of any NRC license or non-licensed facility contiguous 25 to a small modular reactor or other new technology NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

31 1 facility to identify hazards that could adversely 2 impact the implementation of the emergency plans. A 3 scalable approach for determining the size of the 4 plume exposure pathway emergency planning zone, 5 referred to as an EPZ, and a requirement for licensees 6 to describe ingestion response planning in the 7 facilities emergency plan, including the capabilities 8 and resources available to protect against 9 contaminated food and water from entering the 10 ingestion pathway. These requirements would apply to 11 those small modular reactor and other new technology 12 facilities that elect to use the rule in Section 13 50.160.

14 Next slide, please. During the 15 Subcommittee Meeting, the Staff received several 16 comments concerning source terms. The typical source 17 terms for credible accidents for each facilities will 18 be evaluated in conjunction with the remainder of the 19 review for the specific licensing application for the 20 facility. The source terms will vary by design and 21 each location will have a meteorology that will be a 22 factor in the consequence analysis. Therefore, the 23 consequences from an accident will depend on the 24 design and specific location for the facility. To 25 avoid confusion with the purpose of this rule, the NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

32 1 Staff made no changes to the rule or other document to 2 address source term.

3 MR. KAHLER: If I can, Kenny, Bob Kahler 4 again from NRC DPR, we've heard your concern and your 5 comment and we'll give the due consideration for the 6 guidance document on the elaboration on source term 7 usage and how it is related to the EP rule.

8 MR. THOMAS: During the Subcommittee 9 Meeting, the Staff received several comments 10 concerning the technical reasons for the selection of 11 1000 megawatts thermal as the upper bound for the 12 reactor power for light water, small modular reactors.

13 The Staff did not perform an analysis as to whether 14 there exists a characteristic of 1000 megawatts 15 thermal about which reactor phenomena occur. The 16 Staff selected the number based on the existing use by 17 several key stakeholders such as the Department of 18 Energy, facility designers, the international 19 community, and even the NRC and the fee rule in parts 20 170 and 171 of Title 10 of the Code of Federal 21 Regulations.

22 CHAIRMAN CORRADINI: So, can I restate 23 what you just said?

24 MR. THOMAS: Yes, sir.

25 MR. SCOTT: So there is no technical NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

33 1 analysis that limits the rule? It's more by mythology 2 and others?

3 MR. THOMAS: I don't think we'd say it was 4 mythology. Use that to extract the comment from you 5 but what I didn't hear is a technical analysis that 6 says 1000 is a breakpoint.

7 CHAIRMAN CORRADINI: And as a matter of 8 fact, you heard just the opposite. It is not a 9 technical analysis. The decision was made based on, 10 for example, assumptions as to the level of interest 11 among the current large light-water reactor fleet in 12 this rule and in this process. One of the things that 13 we're going to propose to the Commission that we do is 14 ask the question in the draft rule as to whether 15 there's a view on this matter.

16 MEMBER MARCH-LEUBA: Yes, but I don't 17 think this applies as much to the existing reactor.

18 Nobody is going to go to the effort of read through 19 the EPZ, they already have one. It's more if I want 20 to build an ABWR or AP1000, why doesn't it apply to 21 me?

22 MR. SCOTT: Again, it comes back to a 23 decision and an assumption about the level of 24 interest. So if we get a comment that comes in from 25 potential interested party in that indicates the NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

34 1 assumption was made was not a good one, they will 2 revisit it.

3 MEMBER MARCH-LEUBA: Okay, let's let Kenny 4 say what he wants to say.

5 MR. THOMAS: Thank you, sir.

6 Additionally, in issues related to the selection of 7 1000 megawatts thermal, the NRC did receive a comment 8 on the draft regulatory basis in 2017 that recommended 9 the NRC expand the scope of the rule to include large 10 light water reactors. Large light water reactors were 11 not included by the NRC in the scope of this proposed 12 rule because an emergency preparedness licensing 13 framework already exists for these reactors and 14 licensees for those plants have not presented a clear 15 interest in changing that framework. Nonetheless, in 16 light of the public comment on the draft regulatory 17 basis and although this proposed rule is written for 18 small modular reactors and other new technologies, the 19 Staff has included a question for public input in the 20 specific request for comment section of the Federal 21 Register notice. The present-end stakeholders whether 22 the NRC should consider a performance-based 23 consequence-oriented approach to emergency 24 preparedness for large light water reactor sites in 25 fuel cycle facilities and currently operating non-NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

35 1 power production or utilization facilities.

2 MEMBER MARCH-LEUBA: And that included 3 future light power reactor sites? It's always 4 existing ones unless there is something unusual, then 5 I wouldn't go to the effort of doing it.

6 MR. THOMAS: That question would go to 7 future large light water reactors.

8 MEMBER MARCH-LEUBA: Did you see any 9 technical reason why it wouldn't apply?

10 MR. THOMAS: Did I see any technical 11 reasons why it would not apply?

12 MEMBER MARCH-LEUBA: To a 3000 megawatt 13 thermal reactor.

14 MR. THOMAS: I'm an AP guy. I can turn 15 that over to the consequence analysis and the other 16 folks and ask them for their technical opinion.

17 MEMBER MARCH-LEUBA: If the consequences 18 are too large, then they won't be able to have a --

19 MR. SCOTT: I'm going to give you a 20 management opinion. If we believe that we should have 21 a risk-informed process then we should apply the 22 risk-informed process and see where it comes out.

23 But that does pose potential that you 24 could have either 10 miles or a different number for 25 large light water reactors.

NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

36 1 MEMBER MARCH-LEUBA: 25 miles if you 2 analyze this properly. Maybe we made a mistake. Most 3 likely it will be only five but it could while they're 4 away.

5 MR. SCOTT: Again, since we are focused on 6 being risk-informed and performance-based, then you 7 can take that to its logical conclusion.

8 MEMBER MARCH-LEUBA: My comment is it 9 looked like an arbitrary point.

10 MR. SCOTT: It is not based on technical 11 criteria and I would say it's a little more than --

12 MEMBER BROWN: I have an information 13 question. Where are the existing fleet's power 14 reactors that are 1000 megawatts or less?

15 Some of the early plants that were 16 developed, Shipping Port I thought was a much smaller 17 reactor in the early days so they're all over 1000 18 megawatts.

19 MR. SCOTT: Thermal.

20 MR. THOMAS: During the Subcommittee 21 Meeting, the Staff received several comments 22 concerning the hazard analysis updating requirements.

23 An update to the hazard analysis is already part of 24 the rule.

25 The requirement for licensees to update NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

37 1 the hazard analysis is part of the requirements in 2 Section 50.54 Q2 where the licensee must follow and 3 maintain the effectiveness of its emergency plan.

4 Based on the requirements within 50.54 Q2, 5 the Staff did not add any additional requirement to 6 the hazard analysis itself to be updated on any set 7 periodicity.

8 Next slide, please. During the 9 Subcommittee Meeting, the Staff received several 10 comments concerning the selection of 96 hours0.00111 days <br />0.0267 hours <br />1.587302e-4 weeks <br />3.6528e-5 months <br /> for a 11 dose criterion. The use of 96 hours0.00111 days <br />0.0267 hours <br />1.587302e-4 weeks <br />3.6528e-5 months <br /> is part of the 12 Environmental Protection Agency's protective action 13 guide manual.

14 Specifically in Section 2.2.2, the 2017 15 protection action guide manual, it states the first 96 16 hours1.851852e-4 days <br />0.00444 hours <br />2.645503e-5 weeks <br />6.088e-6 months <br />, specifically on Page 16.

17 Where dose projections are at levels less 18 than one rem, ten millisieverts, over the first four 19 days, evacuation is not recommended due to the 20 associated risk of moving large numbers of people.

21 The decision-makers may consider 22 implementing sheltering in place when projected doses 23 are below one rem, ten millisieverts over the first 24 four days.

25 MEMBER REMPE: So I think I was the one NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

38 1 who brought this up. I can recall years ago that 10 2 CFR 100 was based on the first two hours and the gas 3 reactor folks said, oh, this is good news because we 4 don't have stuff come out until later and you changed 5 it to the worst two hours.

6 NUREG 0396 is based on a bunch of releases 7 from light water reactors and so my question was, 8 well, you might have some sort of a reactor where they 9 might have circulating release early and then it takes 10 a long time to heat up and it might be 99 hours0.00115 days <br />0.0275 hours <br />1.636905e-4 weeks <br />3.76695e-5 months <br /> before 11 you get the bad release.

12 And I just was curious why are you 13 sticking with the 96 hours0.00111 days <br />0.0267 hours <br />1.587302e-4 weeks <br />3.6528e-5 months <br />?

14 MR. THOMAS: Well, the first part is the 15 Environmental Protection Agency's protective action 16 guides specifically calls out the 96 hours0.00111 days <br />0.0267 hours <br />1.587302e-4 weeks <br />3.6528e-5 months <br />. But also 17 in practice, the 96 hours0.00111 days <br />0.0267 hours <br />1.587302e-4 weeks <br />3.6528e-5 months <br /> is also looked at when at 18 the facility they start doing dose projections.

19 During the response, the facility performs 20 dose projections that forecast potential doses over 21 the next four days or 96 hours for a four-day 22 integrated dose to determine the emergency 23 classification levels and any protective action 24 recommendations.

25 This ongoing assessment during the NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

39 1 response looks forward 96 hours0.00111 days <br />0.0267 hours <br />1.587302e-4 weeks <br />3.6528e-5 months <br /> each time that dose 2 assessment is conducted to determine whether 3 protective action recommendations or an increase in 4 the emergency classification level is warranted.

5 MEMBER REMPE: So what I think you're 6 telling me is that it's because they continuously look 7 96 hours0.00111 days <br />0.0267 hours <br />1.587302e-4 weeks <br />3.6528e-5 months <br /> ahead so it's not just the first 96 hours0.00111 days <br />0.0267 hours <br />1.587302e-4 weeks <br />3.6528e-5 months <br />.

8 Is that what you're saying and what you're reading 9 there?

10 MR. THOMAS: The 96 hours0.00111 days <br />0.0267 hours <br />1.587302e-4 weeks <br />3.6528e-5 months <br /> that we used 11 were specifically in the rule where we're looking at 12 the first 96 hours because that's tied to the 13 Environmental Protection Agency's protective action 14 guide.

15 MEMBER REMPE: So it's based on LWR 16 evaluations?

17 MR. THOMAS: Yes, ma'am.

18 CHAIRMAN CORRADINI: Did you have more to 19 say?

20 MR. THOMAS: No, sir.

21 MR. KAHLER: Again, Bob Kahler from NSIR 22 DPR Branch Chief. It's not based upon large light 23 water, it's based upon the 1 rem of 96 hours0.00111 days <br />0.0267 hours <br />1.587302e-4 weeks <br />3.6528e-5 months <br /> for 24 determination of immediate protective actions for 25 public health and safety.

NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

40 1 So, what we're looking at is setting the 2 EPZ size based upon what immediate protective actions 3 need to be taken possibly within the first 96 hours0.00111 days <br />0.0267 hours <br />1.587302e-4 weeks <br />3.6528e-5 months <br /> to 4 set that EPZ size for the implementation of the EPA 5 PAGs.

6 So, when Kenny is talking about the dose 7 projections that would go out from the beginning of 8 the accident for the first 96 hours0.00111 days <br />0.0267 hours <br />1.587302e-4 weeks <br />3.6528e-5 months <br />.

9 And if that is what EPA says you need to 10 take immediate protective actions that exceed one rem, 11 then we need to establish an easy size to accommodate 12 how far out those projections would take that.

13 So that would be setting the EPZ size.

14 But that's how that 96 hours0.00111 days <br />0.0267 hours <br />1.587302e-4 weeks <br />3.6528e-5 months <br /> comes in play with the 15 determination of the EPZ size.

16 CHAIRMAN CORRADINI: But I think what Dr.

17 Rempe is getting at, though, is depending on 18 technology, you may have to reevaluate that going 19 forward and that's why I'm still trying to get a clear 20 picture from the NRC standpoint that is still going to 21 be looked at beyond the first order.

22 And I think that's what her question --

23 MR. KAHLER: That is still a requirement 24 of the regulation, is to continually do the dose 25 projections. Even if you do not have an EPZ size but NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

41 1 you have them on the site boundary.

2 You're still going to have to continually 3 assess that and if at any time during that time period 4 you would believe then that the offsites would be able 5 to implement some sort of offsite protective action 6 without the need to have the formal offsite 7 radiological emergency preparedness program in place 8 because that 96 hours0.00111 days <br />0.0267 hours <br />1.587302e-4 weeks <br />3.6528e-5 months <br /> would give you that leeway for 9 them to take that protective action.

10 MEMBER REMPE: So this helps a lot. It 11 would be nice if it was somewhere in the 12 documentation. I didn't see it there but maybe I've 13 missed it.

14 MR. KAHLER: To further clarify using of 15 the 96 hours0.00111 days <br />0.0267 hours <br />1.587302e-4 weeks <br />3.6528e-5 months <br />, thank you and we're going to watch for 16 that comment.

17 MEMBER BROWN: Could I ask a question 18 before you flip? You talked about the multiple 19 modular considerations and you talked about your 20 regulatory position, and that's the hazard analysis of 21 nearby adjacent or contiguous facilities.

22 And it just says your analysis should 23 identify the specific hazards proposed by multimodular 24 nuclear units or multimodular units. I didn't see any 25 criteria by which you might expect them to evaluate NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

42 1 those.

2 And I'm just thinking about the NuScale 3 situation where you've got 12 crammed into a building 4 but there's no guidance put forth at all in terms of 5 how you should assess the relevant power levels and 6 how they should be addressed for the emergency 7 planning zone considerations.

8 It just says look at it and identify but 9 where's your breakpoint? Do you have any? If you 10 look at the specific example which we're all well 11 aware of, that's a pretty big, large, power unit when 12 you combine them all together.

13 And that's different from having 10 units 14 on a site spread around half a mile or a mile 15 separating them all.

16 I'm just taking some type of other example 17 where you could have a big site with multiple regular 18 light water reactors and there's just no guidance, no 19 identification of anything in terms of how they should 20 just be performance analysis and tell us what you've 21 got, and they'll figure out whether we accept it or 22 not, I guess.

23 MR. THOMAS: You're right, there is very 24 little guidance, it's not a rule. You're right, there 25 is very little guidance in that except for what we had NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

43 1 in the Regulatory Position Number 8. The reason for 2 that is it's technology-inclusive.

3 We don't know which technologies would be 4 brought to us, which designs would be brought to us, 5 or how these would actually be sited.

6 MEMBER BROWN: Why does that matter?

7 Damage to a plant or an accident in a plant creates a 8 hazard situation where you need an EPZ.

9 Why does that matter relative to the fact 10 that the criteria should be able to be assessed based 11 on regardless of the technology?

12 Radiation is high levels or particulates 13 or spread of contamination is a function of 14 contamination and not necessarily what produces it.

15 MR. THOMAS: Yes, sir, what we were 16 looking at was the potential external hazards unless 17 the facility itself. That's why we looked at non-NRC 18 license.

19 For example, military installations, 20 transportations, so it's more focused on the licensee 21 or the Applicant should be aware of where they're 22 putting this.

23 MEMBER BROWN: You're talking about 24 contiguous facilities as opposed to the facility?

25 Isn't that somewhat short-sighted?

NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

44 1 MR. THOMAS: Well, we also talk about the 2 internal multi-modular considerations specifically.

3 MEMBER BROWN: But the only words say 4 multi-modular. There's still no criteria.

5 CHAIRMAN CORRADINI: There's no criteria 6 on the source term either.

7 MEMBER BROWN: I didn't mention that 8 earlier. I was going to.

9 CHAIRMAN CORRADINI: Again, I'm going to 10 break in.

11 I do think there are current plants that 12 have shared ultimate heat sinks that would have to 13 then -- that are being considered or should be 14 considered from a probabilistic standpoint just as we 15 would have hear in a shared open heat sink.

16 MEMBER BROWN: I understand that. It's an 17 interesting question the way this is done. The 18 differentiation between other nuclear technologies as 19 opposed to light water reactors, et cetera, I think it 20 produces a problem accident-wise and it doesn't matter 21 what the technology is.

22 It's a problem. There are to be criteria.

23 You made a statement earlier and I can't remember 24 which of you all made it but this performance-based 25 risk-informed, the magic words that we're using now, NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

45 1 could result in no EPZ at all.

2 I didn't say anything at the time, it just 3 took me back and that from a public health standpoint, 4 how can you not have any EPZ at all established other 5 than there's a boundary going around the plant that's 6 got a fence around it and you can't get bad guys to 7 get in?

8 That's a different issue. From an 9 accident standpoint or a criteria standpoint, I have 10 a hard time coming across no EPZ at all and that's the 11 way I would have read this to see that you could end 12 up with -- and you made that statement.

13 MR. SCOTT: I made the statement so allow 14 me to clarify.

15 MEMBER BROWN: You're all part of the 16 family.

17 MR. SCOTT: But let me clarify it. So I 18 didn't intend to say no --

19 MEMBER BROWN: But that's what you could 20 get out of this.

21 MR. SCOTT: What you can get, certainly, 22 there will always be an onsite radiological emergency 23 preparedness program.

24 MEMBER BROWN: That's different, that's 25 not what this is, though.

NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

46 1 MR. SCOTT: Right, so the delta there and 2 the concept of the site boundary EPZ is that the 3 hazard is low enough that an offsite formal Federally-4 mandated, licensee-funded emergency preparedness 5 program, focus radiological emergency preparedness 6 program would not apply.

7 That hazard would be in the range of other 8 hazards that the community needs to contend with and 9 that would be covered by the Federal all hazards 10 emergency response program.

11 And state and local all have those 12 emergency response programs so that's not the same 13 thing as say no emergency planning and saying no full 14 up Federally-mandated offsite program would be 15 required because the hazard wouldn't suggest that it's 16 necessary.

17 MEMBER BROWN: I have a hard time walking 18 that one past, that's my own personal opinion. Thank 19 you.

20 MEMBER SKILLMAN: I'd like to ask a 21 question, please. To the concern Charlie Brown raised 22 about contiguous facilities, it appears to me that 23 you've addressed this very thoroughly in your draft 24 guide 1350.

25 And as long as what you put in 1350 is NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

47 1 obeyed when this is finally -- then I think what 2 Charlie is pointing to has been thoroughly and fairly 3 taken care of.

4 The second comment, in this last 5 discussion as Charlie says, a logical extension of 6 this discussion is that the facility and the 7 technology is such that there really isn't any 8 release, there can't be any release. That could be 9 one conclusion.

10 I'm a qualified and experienced Emergency 11 Director and Emergency Support Director. I think in 12 terms of an unusual event, alert side area and 13 general.

14 And for me, general has always been that 15 point when you make the notifications, your 96-hour 16 clock begins or whatever the clocks are, and then you 17 stay with the scenario and you stay with the event 18 until you have to adjust what you've communicated in 19 terms of key hold or shelf room place or evacuate.

20 That's the moving target based on what the 21 radiological events offer that aren't naturally 22 occurring.

23 Here's my question. To Charlie's point, 24 do you envision a facility with other technologies 25 where you have no general emergency?

NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

48 1 MR. KAHLER: If I can, sir? Bob Kahler, 2 Branch Chief for NSIR DPR. To go back to your terms 3 of UE alert site area and GE, the definitions of those 4 classifications are based upon the release and the 5 magnitude of the release, and where that release is 6 occurring and the type of response that is needed.

7 MEMBER SKILLMAN: That's why I'm asking 8 the question. I'm well aware of that.

9 MR. KAHLER: So I would envision that if 10 you have an EPZ and it's very plausible to have an 11 emergency preparedness zone of where that one rem 12 exceeds 96 hours0.00111 days <br />0.0267 hours <br />1.587302e-4 weeks <br />3.6528e-5 months <br /> be contained within the site boundary 13 but there is no credible accident, if we can go back 14 to that, I'm sorry, that would provide for that one 15 rem within 96 hours0.00111 days <br />0.0267 hours <br />1.587302e-4 weeks <br />3.6528e-5 months <br /> beyond the site boundary.

16 That would then trigger the difference 17 between the site area emergency and the general 18 emergency. Site area emergency would require planning 19 immediate protective actions for those within the site 20 boundary and that is a requirement of the regulation 21 to protect those people.

22 Even members of the public would be 23 contained within that boundary but it would be the 24 licensees emergency plan that would provide for their 25 safety.

NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

49 1 If it extends beyond the site boundary, 2 that is when the offsite formal rep program is needed 3 in order for the offsite entities who have the 4 authority to protect and the responsibility to protect 5 the offsite public, that's when the formal rep would 6 occur.

7 So in response, if you have a facility 8 that provides less emergency planning zone that was 9 within the site boundary, that is saying that they 10 have no credible accident that would exceed one rem 11 within 96 hours0.00111 days <br />0.0267 hours <br />1.587302e-4 weeks <br />3.6528e-5 months <br /> beyond that site boundary.

12 Hence, you would not exceed a site area 13 emergency and no general emergency classification 14 would be plausible, which is the same as we're doing 15 right now with decommissioning, rulemaking as we go 16 into the different level two and three and so on and 17 so forth. So, it would follow suit with that.

18 If you determine that you have one rem 19 within 96 hours0.00111 days <br />0.0267 hours <br />1.587302e-4 weeks <br />3.6528e-5 months <br /> and you have a site boundary and the 20 licensee can determine how big that site boundary is, 21 that's how we tied it to the area in which the 22 licensee is responsible, that's how it is today.

23 Then if you exceed that, you would have 24 general emergency classifications.

25 MEMBER SKILLMAN: Okay, so the answer is NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

50 1 yes, you can have the site where you actually do not 2 get to a general emergency.

3 MR. KAHLER: That's correct.

4 MEMBER SKILLMAN: And if I'm a vendor with 5 real deep pockets, I might say, great, I'm going to 6 have a shield building and I'm going to have one of 7 these containments, and then because I'm a safety kind 8 of person, I'm going to have the second containment 9 inside that shield.

10 And because I've got lots and lots of 11 bucks I'm going to put in a third containment. That's 12 what you're saying?

13 MR. KAHLER: I am saying if they come up 14 with a technology and a design of the plant site that 15 says that as I go through my accident sequences, I 16 have such the design would render 1 rem within 96 17 hours1.967593e-4 days <br />0.00472 hours <br />2.810847e-5 weeks <br />6.4685e-6 months <br /> within the site boundary because of my design 18 because I've added on to these containments, yes.

19 MEMBER SKILLMAN: Thank you, that's the 20 point I wanted to make.

21 Under the right circumstances, the right 22 financial circumstances, one could considerably have 23 site's radiological protection if there is no -- I'm 24 going to use Dr. Bley's term -- there's no numerically 25 defendable basis for release.

NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

51 1 I made the release so improbably that I 2 cannot get to that 1 rem in 96 hours0.00111 days <br />0.0267 hours <br />1.587302e-4 weeks <br />3.6528e-5 months <br /> and even if the 3 accident continues, I still cannot get that 1 rem in 4 96 hours0.00111 days <br />0.0267 hours <br />1.587302e-4 weeks <br />3.6528e-5 months <br />.

5 MR. KAHLER: And quite frankly, we do 6 anticipate those kinds of designs being submitted to 7 us.

8 MEMBER SKILLMAN: Thank you.

9 MEMBER BROWN: Just to clarify, I really 10 wasn't -- I was more focused on the onsite multiple 11 units as opposed to contiguous facilities.

12 The location of contiguous facilities 13 initially as well as what might come down 10 or 15 14 years later all pose a thing that you just have to 15 address on kind of an ad hoc basis.

16 But for the initial plant development and 17 stuff, multiple reactors and their configuration of 18 how they're stuffed in poses a different thought 19 process.

20 And that's what I was really referring to 21 in terms of how much power can you stuff inside these 22 boundary conditions on the site and still consider 23 that there's no criteria that we have to worry about.

24 It's just we're going to evaluate it later at some 25 point.

NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

52 1 That just seems to be an open-ended, we're 2 not providing any guidance and it gives the impression 3 that a plant that stuffs 12 plants within 1 building 4 and they're all sitting side by side is 3000 or 4000 5 megawatts thermal and that's just happy because each 6 one of them is only 200 or 300 or whatever it may be.

7 So that's the point I was trying to make.

8 It seems to me we ought to have some criteria 9 somewhere. I'm not in favor of everything 10 risk-informed, if you hadn't figured that out by now, 11 or performance-based.

12 I think that's kind of a caveat and there 13 should be some bottom line that there's a line you 14 don't want to cross, that's all.

15 MR. SCOTT: I don't think it would be 16 accurate to say and I don't think you were implying 17 that we're not going to consider the potential for 18 accidents at one or more of the --

19 MEMBER BROWN: I don't know how the other 20 one is going to go right now. I forgot how many 21 plants are on this.

22 CHAIRMAN CORRADINI: I don't think we want 23 to focus on that.

24 MEMBER BROWN: I'm not, but it's an 25 obvious example of a multi-unit that's got a fairly NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

53 1 interesting configuration. I'm not accusing you of 2 saying you're not going to look at it.

3 MR. SCOTT: And we are going to look at 4 that. I think the comment is that we haven't put any 5 detail on that and there's no flesh on that right now.

6 MEMBER BROWN: But this does provide a 7 preview of your thinking if nothing else.

8 MR. THOMAS: This also goes back to what 9 are the credible accidents and source terms for that 10 extraneous or external to the specific emergency 11 preparedness. The accident happens, what are you 12 doing?

13 Publicly available documents, Dr. Rempe 14 made a notice that several or a couple of the 15 documents were not made publicly available at the time 16 of the Steering Committee Meeting -- I'm sorry, the 17 Subcommittee Meeting.

18 The two documents generalized dose 19 assessment methodology for informing emergency 20 planning zone size determinations, ADAMS accession 21 number ML18064A317 and the required analyses were 22 informing emergency planning zone size determinations, 23 ADAMS accession number ML18114A176 were made publicly 24 available on September 21st.

25 Use of other new technology, during the NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

54 1 Subcommittee Meeting, the Subcommittee stated that the 2 Staff missed an opportunity to define other new 3 technology in the rule and to use it more in the draft 4 regulatory guidance.

5 The use of the term other new technology 6 is used in the Federal Register notice, their draft 7 regulatory guide, and in other documents.

8 Next slide, please. The Staff is 9 proposing that Applicants who select to comply with 10 the new rule provide an analysis that supports the 11 requested emergency planning zone size.

12 The requirements would be in Sections 13 50.33 and 50.34. For the EPZ size determinations, the 14 size of the emergency planning zone should encompass 15 an area where prompt protective actions such as 16 evacuation or sheltering may be needed to minimize the 17 exposure to individuals.

18 If the Applicant or licensee demonstrates 19 that the environmental protection Agencies protective 20 action guides are not exceeded at the site boundary, 21 then there will be no NRC requirement for offsite 22 radiological emergency preparedness programs.

23 If the proposed emergency planning zone 24 size exceeds the site boundary, then for the distance 25 supported by the analysis there will be NRC NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

55 1 requirements for offsite radiological emergency 2 preparedness program.

3 The exact shape of the emergency planning 4 zone would need to be determined in relation to local 5 emergency response needs as they were affected by such 6 conditions as population, land characteristics, and 7 jurisdictional boundaries. The NRC would engage FEMA 8 in accordance with the 2015 NRC FEMA Memorandum of 9 Understanding.

10 Next slide, please. This diagram provides 11 the overall structure of the rule and its relationship 12 to the existing emergency preparedness regulations.

13 If the Applicant opts to use the proposed regulations 14 in 50.160, then the Applicant would have to provide an 15 analysis to support the specific emergency planning 16 zone size.

17 If the Applicant demonstrates that the 18 emergency planning zone is within or at the site 19 boundary, then it's shown in the lower right corner of 20 the diagram that the regulations in the proposed 21 section 50.160, Paragraph C1IVP would not apply to the 22 licensee.

23 If the emergency planning zone would 24 extend beyond the site boundary, then the Applicant 25 would need to address their requirements in proposed NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

56 1 Section 50.160 Paragraph C1IVA and B.

2 The Staff would then need to engage with 3 FEMA for a review of the offsite plans submitted as 4 part of the license application or a permit 5 application as applicable. There is guidance in the 6 draft regulatory guide that support the implementation 7 of the performance-based regulations.

8 Next slide, please. This slide describes 9 the Office of Research Support for the development of 10 the regulatory guidance for the emergency planning 11 zone size analyses.

12 The approach for the scaling and emergency 13 planning zone sizes for small modular reactors and 14 other new technologies was developed to be consistent 15 with the consequences or framework described in NUREG 16 0396 and the scaled approach use for operating 17 research and test reactors, fuel cycle facilities, and 18 independent spent fuel storage installations.

19 The Office of Research was asked to review 20 the rationale documented in NUREG 0396 to determine 21 whether the technical analyses described in that 22 document could be generalized to identify a 23 methodology for us by small modular reactors and other 24 new technologies.

25 The Office of Research found that the NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

57 1 selection of the current planning zone sizes appears 2 to have been based on judgment informed by a variety 3 of lines of evidence rather than being based on a 4 single specific prescriptive dose assessment 5 methodology.

6 The information from both scaling and 7 reanalysis of the information on potential doses from 8 design basis accidents as well as information on the 9 likelihood and consequences of beyond design basis 10 accidents derive from probabilistic risk analysis were 11 considered in developing the rationale.

12 Hence, the proposed consequence-oriented 13 approach would provide the same level of protection to 14 the public health and safety as afforded to other 15 currently operating facilities.

16 CHAIRMAN CORRADINI: So can I stop you 17 right there? The 0396 methodology did not just 18 include NRC, it included EPA and others.

19 Again, this is personal opinion, so it may 20 not appear in our report but it just seems to me it 21 would be appropriate, I'm trying to look for a better 22 word, but appropriate that the same sort of multi-23 agency guidance is going to be needed for these 24 advanced technologies.

25 Otherwise, you're going to get yourself in NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

58 1 a potential iterative loop of cycling through 2 proposals, another proposal, another proposal. And it 3 would seem to me FEMA, EPA, and NRC together are going 4 to have to come up with some sort of guidance on this.

5 If in the current rule you don't think 6 that's appropriate and you simply want to give 7 qualitative considerations, okay, but eventually, it's 8 going to have come to pass that you're going to have 9 to give better guidance because if you look at the 10 authors of 0396, it wasn't just the NRC.

11 MR. SCOTT: Appreciate that comment, thank 12 you. We'll consider it.

13 MR. THOMAS: The Office of Research 14 concluded there was sufficient information in NUREG 15 0396 and its references to identify several key 16 assumptions and elements of generalized methodology 17 for informing the emergency planning zone sizes.

18 The Office of Research used these key 19 assumptions in the elements to recommend a list of 20 analyses that a licensee would need to submit to 21 justify their selected emergency planning zone size.

22 Next slide, please. The NRC is proposing 23 ingestion response planning requirements instead of a 24 pre-determined fixed distance as part of the 25 performance-based framework for the ingestion pathway.

NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

59 1 The proposed rule would require licensees 2 who comply with Section 50.160 to describe in their 3 emergency plan the licensee's local, tribal, and 4 Federal resources for emergency response capabilities 5 to protect against contaminated food and water for 6 entering the ingestion pathway.

7 A successful quarantine and removal from 8 public access of contaminated food and water products 9 in response to biological contamination demonstrates 10 that a response to protect against ingestion of 11 contaminated foods and water can be performed in a 12 rapid manner without a predetermined planning zone.

13 Unlike biological contamination that 14 causes widespread illnesses and only discovered days 15 after infection, a reactor accident would be a leading 16 indicator that long-term actions to protect against 17 ingestion should be considered.

18 Next slide, please. Now Dr. Carrera will 19 discuss the status in our path forward.

20 Andy?

21 MR. CARRERA: Thank you, Kenny, and thank 22 you, Mr. Chairman and Members of ACRS for the 23 opportunity to be here. For the purpose of cumulative 24 effects of regulations, I'd like to provide a quick 25 status and path forward for this rule.

NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

60 1 The draft proposal package was submitted 2 to our senior management EDO for review last week.

3 It's in the process of reviewing. As Mike has 4 previously conveyed, we are on track to deliver the 5 cultural to the Commission valve for 12.

6 The Commissioners would review the packing 7 involved on it so if the Commission approves the 8 publication of the proposed rule, Staff will make any 9 changes to the proposed rule as directed by the 10 Commission and publish the proposed rule in the 11 Federal Register for public comment.

12 Now, there would be a 75-day public 13 comment period, however, a specific amount of time 14 will be determined by the Commission.

15 Staff also plans to conduct public 16 meetings during the comment period for the proposed 17 rule to promote full understanding of the proposed 18 rule and guidance, and to inform or facilitate written 19 public comments.

20 Our staff would consider all public 21 comments received on the proposed rule in the 22 development of a draft final rule.

23 The Staff will also conduct an 24 implementation period public meeting as part of the 25 cumulative effects of the regulation initiative during NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

61 1 the development of the final rule.

2 Thank you.

3 MR. SCOTT: Can I just add one more 4 comment?

5 So I made a comment earlier about a view 6 that we should, and I think the NRC does, encourage 7 risk-informing everything we do to the extent we can 8 but I may have left an inadvertently incorrect 9 impression about that regarding the applicability of 10 the approach in the new rule to large light water 11 reactors.

12 As we pointed out and Mr. Kenny mentioned, 13 we're putting that out open for comment. However, we 14 believe that the 10-mile EPZ is fully protective of 15 public health and safety for the existing fleet of 16 large light water reactors so we have no information 17 that suggests that we should backfit this new rule or 18 this process or approach on the existing fleet.

19 So I didn't want to leave a mis-impression 20 of that.

21 MEMBER MARCH-LEUBA: I'd like to comment 22 on that. My comments on that one were not backfitting 23 the existing reactors but I want to buy an APR1400 and 24 place it in Tennessee. This rule doesn't apply to me.

25 MR. SCOTT: As I understood it, your NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

62 1 question is whether it should be available to them.

2 I guess backfit wouldn't apply but there's no 3 intention if somebody comes in with a new ABWR large 4 light water reactor design to require that this rule 5 will be imposed on them.

6 Your question as I heard it is could it be 7 available? Yes, which is a different question.

8 MEMBER MARCH-LEUBA: That's correct. As 9 written it's not available.

10 MR. SCOTT: That's correct but open for 11 comment as we go forward.

12 MEMBER MARCH-LEUBA: I wonder who the 13 stakeholder will be because I want to build an ABWR in 14 Tennessee but I don't have the funding yet.

15 CHAIRMAN CORRADINI: That's a conflict.

16 MEMBER MARCH-LEUBA: So who will be the 17 stakeholder that will give you that conflict? A 18 vendor?

19 MR. SCOTT: We shall see.

20 MEMBER MARCH-LEUBA: Okay.

21 CHAIRMAN CORRADINI: I'm going to turn to 22 Dennis. Do you have other questions for the Staff?

23 MEMBER BLEY: No, I don't, thank you. It 24 was a good discussion today.

25 CHAIRMAN CORRADINI: Okay, let me go NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

63 1 around and then we'll go to the public. Vesna, any 2 comments? I'll just look at you if any of the Members 3 want to make further comments.

4 With that, let us turn to the audience 5 here and see if anybody in the room would like to make 6 a comment and then ask if the public line could be 7 open.

8 No comments from the gallery here? Is 9 there anybody on the public line? If you could please 10 speak up? Okay, the public line is open. With that, 11 I think we're done and I'll thank the staff.

12 Could you close the public line? Thank 13 you very much and we'll take a break. We're back at 14 10:15 a.m., to stay to the published schedule.

15 (Whereupon, the above-entitled matter 16 went off the record at 9:42 a.m. and 17 resumed at 10:15 a.m.)

18 CHAIRMAN CORRADINI: Okay. So let us come 19 back together here for our second topic, which is 20 summary of reactor operating experience. I'll turn --

21 excuse me. We'll start again. We'll begin with our 22 session on summary of reactor operating experience.

23 And we'll turn it over to Member Skillman to lead us 24 through this. Dick.

25 MEMBER SKILLMAN: Thank you, Mike.

NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

64 1 Colleagues, what we're going to do here is two things.

2 We're going to bring you up to speed on 2017 and mid-3 year 2018 fleet performance. And I want to leave some 4 time, about 20 minutes at the end of the session, for 5 the topic of leading indicators.

6 If you recall, Matt and I have been 7 kicking around this idea, perhaps others, of how do 8 you know when a licensee is entering into degrading 9 performance. And we want to talk about that. And so 10 we've got really two presentations back to back. And 11 we expect to finish by 1200.

12 Okay. So last year or last December we 13 caught up with 2015 and 2016 data. The purpose for 14 today is to talk about 2017 and half of 2018 for which 15 data is available. Next slide, please.

16 Just to remind everybody, the reactor 17 oversight framework really focuses on seven 18 cornerstones.

19 So, for those of you who have been out on 20 the fleet and those of you who watch from a distance, 21 the cornerstones are the initiating events, that is 22 what gets the thing, the event started, what are the 23 mitigating systems, in other words, what are you 24 depending upon to keep those systems functioning so 25 that they perform their appropriate function, issue of NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

65 1 barrier integrity. Emergency preparedness is a 2 standalone of a cornerstone. Public radiation and 3 occupational radiation are key for radiation safety, 4 and finally, security.

5 And so we're going to talk about all of 6 those in the next approximately 45 minutes. Next 7 slide, please.

8 What is the reactor oversight process?

9 You see the NRC inspection block on the upper left.

10 What that is doing is it's feeding the information 11 from the site.

12 And those actions include just regular 13 inspections that are part of the normal day-to-day 14 activity by the residents. If there's a finding, 15 there can be a supplemental inspection. If there is 16 an event at the site, the NRC may send in an advanced 17 team for event response.

18 There are the generic safety inspections 19 that are the day-to-day cadence of the site. And then 20 there are other inspections that may be called up if 21 there is an event or circumstance that requires 22 further review.

23 All that information finds its way up 24 through a significance determination process. And if 25 this significance determination process reaches a NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

66 1 certain threshold, there may be enforcement.

2 All that information goes up into the 3 assessment process. There's an action matrix. And 4 this is a living process. This isn't something that's 5 done once every Tuesday or once a year or once every 6 five years. This is an around-the-clock, continuing 7 performance.

8 You've got residents onsite finding 9 information, licensees performing. And the process 10 operates at the site as long as the site has a license 11 to operate. Next slide, please.

12 How many plants are involved in this? 101 13 is the answer. You can see the breakdown by region.

14 And those are all Part 50 licenses. And they are all 15 susceptible to or, if you will, under the magnifying 16 glass of the ROP. Next slide.

17 So something happens at the site. What 18 does that event mean in terms of safety significance?

19 What does that event mean in terms of safety 20 significance?

21 Well, you see at the bottom green. Those 22 are events that happen or findings that are discovered 23 whose impact in terms of core damage frequency are 24 less than or equal to 10 to the minus 6, one in a 25 million CDF, or LERF less than 10 to the minus 7.

NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

67 1 If the significance of that event reaches 2 a decade greater, that event will become a white 3 finding, and if it's a decade greater, a yellow 4 finding, and if a decade greater than that, a red 5 finding.

6 And I can tell you from years of 7 experience, licensees wish to be at green all the 8 time. Occasionally, there will be a white. When a 9 licensee has found himself/herself in a yellow or red, 10 that is extremely significant. And we'll see more of 11 that as we proceed.

12 MEMBER MARCH-LEUBA: Is there a difference 13 if it is self-reported or if it's found by the 14 inspectors?

15 MEMBER SKILLMAN: Well, the application of 16 the violation or the penalty may be affected by how 17 the licensee responded to --

18 MEMBER MARCH-LEUBA: So you want to see 19 the white, yellow, or red, but you will get the lower 20 finding if you're self-reported.

21 MEMBER SKILLMAN: Well, actually, to 22 Derek's credit, we've got a slide coming up for that.

23 But it really has to do with what is the Severity 24 Level IV.

25 But on the slide to instruct what is a NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

68 1 Severity Level IV, you can see how a Severity Level I, 2 II, and III, which would get us into yellow, white, 3 red or red, yellow, white, would be handled. And that 4 will answer your question if you just give us a second 5 here. Next slide, please.

6 What's a green finding? That's maybe 7 someone failed to sign off on the right line of a 8 procedure.

9 What's a greater-than-green? Greater-10 than-green only applies to security. So hold that 11 thought for a second.

12 White, yellow, and red. Now, white is a 13 moderate safety significance. Yellow is substantial.

14 And red is a high safety significance or a security 15 significance.

16 Back to greater-than-green. So let's say 17 in the security area the inspection program discovered 18 a major flaw in plant security. It's certainly 19 greater-than-green. It might be white, yellow, or 20 red. But that information is SUNSI.

21 And so for all security findings that are 22 greater-than-green, they are simply identified as 23 greater-than-green. And those more significant 24 security issues are handled SUNSI. That's why you 25 just see GTG, greater-than-green.

NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

69 1 Does that make sense to everybody? In 2 other words, if it's a bad security item, you don't 3 want to paint it in the paper. You just simply say 4 it's SUNSI.

5 Now, security will also have or can have 6 white, yellow, or red. They're just identified as 7 greater-than-green. Next slide.

8 This is a busy slide, but in a way it's 9 not. If you look at a Security Level IV violation on 10 top, this is to answer the question what is a Security 11 Level IV or what's beyond greater-than-green. If you 12 see the first block is -- I've got to read my own 13 slide here. I've got to increase the magnification.

14 The first block is fail to restore 15 compliance, yes or no. This is a Security Level IV.

16 And if the answer is, yes, they failed to restore 17 compliance, that may lead to a violation, notice of 18 violation. So that is the blue line under the green 19 blocks on top. And if you see the D, that is 20 discretion. The NRC reserves the right to make a 21 determination of the consequences or the circumstances 22 of that particular failure.

23 If the licensee did not fail to restore 24 compliance, in other words restored compliance 25 promptly, so the answer would be no on that first NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

70 1 diamond, then did that licensee fail to place that 2 item in the corrective action program, yes or no?

3 So, if the licensee identified the item, 4 put it in CAP, restored the issue, you keep moving to 5 the right. Is it repetitive, yes or no? If it's not 6 repetitive, it's a first time finding, the answer is 7 no. Was it willful? Did someone actually do this on 8 purpose? No. In that case, if you go all the way to 9 the right on that first set of diamonds, that's a non-10 cited violation.

11 And that will be true for every non-cited 12 violation that is a very low threshold level. Those 13 might be just NCVs. And you see those all the time.

14 That's an NCV, a non-cited violation.

15 And if you note also on the left-hand side 16 of this slide, the first line is for power reactor 17 licensees. The second line item is for all other 18 licensees. Those could be for research reactors.

19 Those could be for medical facilities, things of that 20 nature, for all of the other licenses that are not 21 Part 50 power licenses.

22 Also, notice at the bottom of the slide is 23 the escalating process. This is for type I, II, and 24 II, severity I, II, and III items.

25 And in that process, you see that the NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

71 1 inspection protocol is essentially the same. But as 2 you get more deeply into the escalated process, you 3 determine whether or not it's a first willful, was it 4 identified, is there credit for corrective action and 5 so on.

6 And then, you know, in the grimmest day, 7 you can end up in the lower right-hand corner where 8 you've got a notice of violation and a very 9 significant two times the base penalty.

10 What I also want to point out here, and 11 it's subtle, but if you understand how the regulations 12 function, this idea of credit for identification. And 13 it's in the second diamond on top. It has to do with 14 placing the issue in CAP.

15 What is CAP? That is the corrective 16 action program. And that is Criterion XVI of Appendix 17 B to 10 CFR 50. That's the QA program. That's the 18 thing Harold and I keep asking about.

19 Where is leadership in understanding 20 Appendix B to 10 CFR 50, because sites that are really 21 tuned in on Appendix B and have a very healthy 22 corrective action program and have a culture that 23 feeds the corrective action program, you normally find 24 those sites with some non-cited violations? But 25 they're normally running along that top line. Next NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

72 1 slide, please.

2 Okay. 2017, and what you're seeing here 3 is the four previous quarters in 2017. And what 4 you're seeing is four yellow findings. And if you 5 recall, those are just below red. That's in calendar 6 year 2017. And you see eight white findings. You 7 also see six greater-than-green.

8 So you can get an idea of what the 9 population is. And notice that the yellows in 2017 10 are initiating events in mitigating systems. The 11 whites are initiating events, mitigating systems in 12 emergency preparedness. And I'm going to just tap on 13 those just for a second.

14 There are inspection reports to back up 15 the information that I will indicate. But I'm just 16 going to go over them quite quickly. Next slide.

17 2018, there is only one white, and there 18 are five mitigating systems. And there are three 19 greater-than-green. Now, this is looking back four 20 quarters. So those three greater-than-greens are 21 either all in 2018 or could be part of the last two 22 quarters of 2017. If you check the prior slide, there 23 were six and eight in 2017. It looks like those three 24 are only in the first two quarters of 2018. Next 25 slide.

NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

73 1 So, going back to 2017 for those four 2 yellows, those four yellows came from Arkansas Nuclear 3 One and Two, Arkansas Nuclear One, which are One and 4 Two. Those came from calendar year 2015 and '16.

5 They affect initiating events and 6 cornerstones. And they carried over into 2017. In 7 other words, they had not been closed out.

8 So, even though the events were a year or 9 two earlier and then had to do with the dropping of 10 the rotor and the compromise of some flooding barriers 11 in the unit, those carried over into '17 until the CAL 12 had been issued and closed out. The CAL is the 13 confirmatory action letter. Those have been resolved 14 so they don't show in 2018. But those four yellows 15 showed in 2017.

16 So what were the events in 2017 or that 17 were carried in 2017? It was the failure to follow 18 the material handling program when they dropped the 19 Unit One stator. And that resulted in the loss of 20 off-site power in Unit One and Unit Two. And it 21 demonstrated -- if you recall, it severed a fire main 22 and they had some major flooding in lower levels at 23 the plant.

24 And as a consequence of this event at 25 Arkansas, they were placed in column 4 of the matrix.

NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

74 1 And they were going to do a supplemental, a 95003.

2 The 95 series of inspection procedures are 3 the ones where there is significantly enhanced NRC 4 inspection. There is a graded performance from 95001 5 to about 95002, then 95003.

6 And if a licensee were to find himself or 7 herself in 95003 and still be wanting, then the NRC 8 could take the keys for that unit and put the unit in 9 0350. And that has happened. That's happened to 10 several plants. Finally, next slide, please.

11 ANO also was in a situation with the, with 12 unplanned scrams. The site was finally inspected this 13 past May, a couple months ago. All of the actions for 14 ANO have been completed. The CAL was closed. And 15 they've been moved out of that action matrix column.

16 And that is reflected in the 2018 data from a few 17 slides earlier. Next slide.

18 There were the eight whites that showed 19 for 2017 and '18. Here are the plants that are 20 identified for those whites. Two plants had 21 initiating event cornerstones. They were Grand Gulf 22 and St. Lucie.

23 There were five in mitigating system 24 cornerstones, Catawba, Clinton, Oyster Creek, Perry, 25 and Pilgrim, and we'll get each of those in just a NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

75 1 minute, and one plant with emergency preparedness 2 cornerstone, and that was Fermi.

3 So here's the data for these eight events.

4 At Grand Gulf, this was scrams. They had a number of 5 scrams. And the real issue there was the poor root 6 cause analyses.

7 At St. Lucie, it was configuration 8 control. That's Criterion III of Appendix B to 10 CFR 9 50. It resulted in the reactor trip and loss of 10 power, a major change in core damage frequency. Next 11 slide.

12 Catawba, it had to do with the excitation 13 system for emergency diesel generator. At Clinton, it 14 had to do with the drop out voltages for replacement 15 relays with an EDG room vent fan. As a consequence at 16 Clinton, the EDG became inoperable. Next slide.

17 At Oyster Creek, it was the failure to 18 follow manufacturer's instructions on reassembling the 19 electromatic relief valve. And at Perry, it had to do 20 with failure to evaluate the effects of a suppression 21 diode.

22 Now, what's interesting is when you read 23 the inspection reports and you read the licensee's 24 response, there's a very healthy amount of dialogue 25 between the two where the licensee says, well, that's NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

76 1 not exactly what happened, here's what happened. The 2 NRC comes back and says here's the basis for our 3 findings.

4 And so there's a negotiated agreement for 5 how in this case the white findings are finally 6 dispositioned. So this isn't just a cop coming in 7 from the NRC and saying, bingo, you've got a white.

8 There is back and forth to finally arrive at the color 9 finding.

10 And you got to realize for a site to get 11 a white or a red, that has major implications, perhaps 12 in insurance, in the in-post standing of that plant, 13 and other similar things. So these are very serious 14 events for the licensee. Next slide.

15 Pilgrim was a failure to correct issues 16 pertaining to a safety relief valve. And this event 17 went on for a fairly significant time. That's 18 Pilgrim. Next one, next slide.

19 And at Fermi, it had to do with a 20 background radiation monitor. And as obscure as that 21 radiation monitor issue might have been, that was a 22 monitor that was depended upon for making a call 23 regarding a PAR, a protective action recommendation, 24 out of the EAL.

25 So here's the case where a relatively NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

77 1 obscure instrument plays a major role in the site 2 leadership's ability to create an accurate emergency 3 action level classification and develop the protective 4 action recommendation for the emergency, resulting in 5 a white at Fermi. Next slide.

6 For 2018, no more yellow findings. That 7 means that the ANO yellows have dropped off in 2018.

8 And there is just the one white coming over in 2018.

9 And that is a mitigating systems cornerstone finding 10 at Davis-Besse.

11 In that particular case, it was an 12 auxiliary feedwater turbine bearing gauge glass oil 13 issue. And they failed their feedwater pump. And as 14 a consequence, there were maintenance issues. But the 15 main thing is that the pump was inoperable for greater 16 than the tech spec period. That is the white at 17 Davis-Besse for 2018.

18 Now we're going to talk about the security 19 cornerstone. If you go back to one of the earlier 20 slides, you saw security as a cornerstone, number 7, 21 sitting far on the right.

22 Derek, can you expand this? Can you jump 23 out of presentation and just put it on, hit escape and 24 that will go to --

25 (Off mic comments.)

NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

78 1 MEMBER SKILLMAN: And in the lower right-2 hand corner you can move the vernier and it will 3 expand this.

4 What I wanted to show up and down is you 5 can see the extent of the baseline inspections for 6 security. And I would offer that if you haven't lived 7 at a site or spent time in a site, you probably 8 wouldn't appreciate how much work is done on this 9 cornerstone.

10 And I will tell you the inspections are 11 thorough. The site leadership is tuned in to all of 12 these pieces. And you will find, as you go over each 13 one of these, it has to do with who can get in, who 14 can get out, what information there is, how the 15 security force is trained, where their weapons are 16 stored, how the information is protected, how they 17 prepare for their force-on-force exercises, the 18 fitness-for-duty program.

19 Second from the right on the lower corner 20 there, the target sets, this information in the last 21 40 years has begun or has become basically as large as 22 operating the plant itself.

23 And if you were to go site to site, you 24 would be quite impressed at how large the security 25 teams are to protect the core and to protect the NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

79 1 people who are working at the site. The security 2 cornerstone is really a significant piece. Next 3 slide, please. Derek, thank you for doing that.

4 Okay. So plants greater-than-green, there 5 were four in 2016, in calendar year '16. There were 6 three in '17. And data for mid-year '18 is three.

7 Greater-than-green is not a new category.

8 It just means the finding was either white, yellow, or 9 red. And the details are SUNSI. So they're not 10 publicly available. Next slide.

11 This will give you an idea of how many 12 inspections were completed in 2017. You can see that 13 the number is not trivial. This is a major effort by 14 the Agency. And it's a large effort by the licensees 15 to support these activities.

16 So, of 250 inspections, there were 3 17 findings greater-than-green in 2017 in the power 18 reactors. In the fuel cycle facilities, there were 15 19 security inspections. And there were 4 Security Level 20 IVs. And there were no greater-than-greens. Next 21 slide, please.

22 And this is just force-on-force. Within 23 the security cornerstone, these are the, if you will, 24 the inspections that really tell the tale. This is 25 where you determine the effectiveness of your security NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

80 1 team. And that will give you an idea of how many 2 inspections were conducted in 2018. Next slide, 3 please.

4 And this aligns with the prior slide. One 5 exercise was ineffective. There were three that were 6 marginal. And all of the licensees took appropriate 7 corrective actions. And those corrective actions 8 include procedure changes, policy changes, updates, 9 technology improvements, and personnel or security 10 force enhancements, all kinds of things. Next slide.

11 The NRC reviews cross-cutting issues. If 12 you go back to the second slide, cross-cutting issues 13 cut across all of the seven cornerstones. And the NRC 14 evaluates whether there are cross-cutting issues. And 15 they do that twice a year at mid-cycle and end-of-16 cycle. There were no new items in 2017. And the data 17 has not been reported yet in 2018.

18 This is an important feature of the ROP 19 because it prevents group think. It kind of takes the 20 inspection to maybe a 100,000-foot level to look at 21 the entire available information to make sure there 22 isn't something that is emerging that is evading or 23 failing to be recognized. Next slide, please.

24 Safety cornerstones, how you think about 25 these things. You can see initiating events. The NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

81 1 performance indicators are as you see on the right.

2 I'm not going to read all of these. But you can see 3 those cornerstones and what it is that is being 4 measured to assess the robustness of that cornerstone.

5 If no questions, next slide, please.

6 Performance indicators. Green is where 7 the performance is within an expected level and where 8 all of the objectives are met. And every licensee has 9 a whole set of performance measures and indicators.

10 And those are, if you will, observed or watched on a 11 daily and continuing basis for all the cornerstones.

12 White means the performance, it's outside 13 the range where you want to be, but the basic 14 objectives are met. Yellow is there is a reduction, 15 a minimal reduction in safety margin. And red is 16 where there's a significant reduction in safety 17 margin. Next slide.

18 For 2017, there were no plants with red or 19 yellow. And there were two with white. And we'll 20 show those in a minute here. Next slide.

21 At Columbia, the trigger was unplanned 22 scrams with complications, not scrams, but scrams with 23 complications. A scram is acceptable, in fact, 24 required within the, if you will, the operating 25 experience of the plant. There are times when you've NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

82 1 got a scram and the plant goes down.

2 The issue is was there a complication.

3 And that complication could be equipment failure to 4 operate, the operator's failure to --

5 MEMBER MARCH-LEUBA: Is the complication 6 after the scram or the complication results in the 7 scram?

8 MEMBER SKILLMAN: After the scram. So you 9 get the buzz. The rods drop or insert. And now 10 things go south very quickly. TMI-2 was a perfect 11 scram with complication. Down it went, bam, floor 12 opened. No one saw it.

13 So this is scram with complications. And 14 at Columbia, it happened twice. It happened in the 15 fourth quarter of '16 and third quarter of '17.

16 Hence, this became a white. Next slide, please.

17 And at Watts Bar 2, now Watts Bar went 18 online in late 2016. So we haven't accrued enough 19 hours2.199074e-4 days <br />0.00528 hours <br />3.141534e-5 weeks <br />7.2295e-6 months <br />. But they were struggling. And you can see 20 that they were in a situation. And what you do is you 21 do the average for the four quarters. So they were 22 scramming I guess quite regularly coming up from their 23 extended, if you will, construction and come back 24 online as a power reactor. Next slide.

25 Performance indicators for 2018, there are NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

83 1 no red or yellow or white through January to June of 2 2018. Next slide.

3 So, in the action matrix, you take the 4 inspection findings and the performance indicators, 5 and that gives a plant assessment. Next slide.

6 All right. So every once in a while you 7 read a news release that such and such a plant has 8 been moved from column 1 to column 2 or it's moved 9 from column 3 to column 2. This is what those columns 10 are. Column 5 is basically you're an 0350. You've --

11 the NRC has taken your keys. Davis-Besse was one of 12 those.

13 Column 4, you've got multiple/repetitive 14 degraded cornerstones. We'll talk about two plants 15 that found themselves in column 4.

16 Column 3 is degraded cornerstone. Column 17 2 is regulatory response. And you can move, if you 18 will, from 4 to 3 to 2 to 1 depending on licensee 19 performance and how the licensee resolves the issues 20 that put the unit in the cornerstone category where it 21 once was.

22 And, of course, the licensees want to be 23 in 1. That is where the licensee is responding. And 24 there is no supplemental NRC regulatory oversight at 25 that site. Next slide.

NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

84 1 And you can see on this slide, column 1, 2 lower left-hand corner, all performance indicators 3 green, column 2, no more than two white, column 3, one 4 degraded cornerstone. It could be three whites or one 5 yellow and three whites in any strategic area. Four 6 is multiple/repetitive degraded. And 5 is 7 unacceptable performance.

8 And if you look in the right-hand column 9 for licensee response at the bottom column 1 on the 10 right-hand side, that's your normal and routine 11 inspection. You've got your inspectors. There is a 12 baseline inspection program. There's annual 13 assessments. And there's public meetings.

14 If you're in regulatory response column 2, 15 you've got a meeting with NRC management. And I would 16 tell you, for those of us who've had to go to those 17 meetings, they are very uncomfortable. It's not 18 because it's pejorative. But it's because there's an 19 issue that the site has just not been able to get a 20 hold of. And it's very objective. But it's clear the 21 burden is on the licensee to make the changes that are 22 necessary to bring the plant back to column 1.

23 And as you escalate on that right-hand 24 corner from bottom to top, I would say there are two 25 things going on. The culture at the site is put to NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

85 1 the test. And the amount of resources that are 2 necessary to bring the plant back to column 1, at 3 least I would say, increase exponentially.

4 Once you get into a situation where you're 5 in column 2 or column 3, you are facing some very 6 significant costs to bring that plant back to where 7 you want to be. And I'm not talking a couple hundred 8 K. I'm talking millions and sometimes many millions 9 of dollars to bring that plant back.

10 And so the consequences are very 11 significant for not keeping the plant in that all 12 green category. Next slide.

13 So, if you look at 2017, if you do the 14 arithmetic, you'll find there are 101 plants 15 represented there. And first quarter of 2017, it was 16 82 in licensee response, and that's green. Sixteen 17 are regulatory response, and three in 18 multiple/repetitive degraded cornerstones. The three 19 plants are Arkansas One and Two and Pilgrim. And that 20 cadence continued through all four quarters of 2017.

21 Next slide.

22 In 2018 for two quarters, 95 in licensee 23 response, all green, 5 in regulatory response, and 1 24 in multiple degraded cornerstones. That is Pilgrim.

25 And we don't have the second quarter data yet. So NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

86 1 let's talk about this. Next slide.

2 So the highlight here is Arkansas Nuclear 3 One and Two are back in column 1. Why? They 4 completed their CAL actions, their confirmatory action 5 letter requirements, and they brought the plant back.

6 And they went into a 95003, a huge amount of effort to 7 bring that plant back to where it's in column 1, the 8 first time since 2015.

9 Now, the next slide is the lowlights.

10 This is where you've got to be thinking what's going 11 on. Pilgrim is six consecutive quarters in 3, 13 12 consecutive quarters in column 4.

13 And if you take a look at the inspection 14 reports and get just an understanding of what the 15 cadence has been at that plant, it's kind of a poster 16 child for how things should have been handled and 17 weren't in terms of corrective action, in terms of 18 work management, in terms of material condition.

19 And Derek offered that NRC muted response 20 considers Pilgrim's plan to shut the plant down. That 21 is a major, that's a major item for the industry right 22 now.

23 And Grand Gulf and Clinton and Columbia, 24 consecutive quarters in column 2, believe me, the 25 owners of those plants are eager to bring those plants NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

87 1 back into column 1. Next slide.

2 Fitness-for-duty, I don't know if this was 3 done, if this presentation occurred back when Jack 4 Seaver (phonetic) was providing these presentations.

5 I recall the first time I did it I said I'd like to 6 present the fitness-for-duty information because of 7 what I witnessed in terms of the change in the culture 8 in my years.

9 But I find this interesting, because what 10 is happening at the plants is in a way indicative of 11 what's happening in our culture. It doesn't make any 12 difference whether it's in a power plant, police 13 department, hospital, large employer with high tech.

14 There are things going on in society that 15 are driving this data. And to me, it's maybe a 16 thermostat of the culture at least in terms as we view 17 safety, in terms of the high tech business, could be 18 driving a bus, flying an airplane, operating a tank, 19 flying a fighter aircraft, being in a control room.

20 This is our culture. And so that's why I 21 was so eager for this to be part of this discussion, 22 because, you know, I live ten miles from a nuclear 23 power plant. I know the people who are working down 24 there. I -- firsthand stories of what was going on.

25 But the data shows up here. And our neighbors and NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

88 1 friends know this is going on.

2 So our knowing about it I think is 3 valuable. And that's why it's here. Next slide, 4 please.

5 There are five types of tests. There's 6 pre-access. You know, you apply for a job. You fill 7 out the form, say, no, no, no, no, no. And you get 8 admitted as an employee.

9 Then the day comes you have a random.

10 And, oh, yeah, yeah, yes, I had some of that last 11 weekend and I probably shouldn't have.

12 And then there's the unfortunate event 13 where you're working and your colleague is not 14 functioning on all eight cylinders. And as a manager 15 or a director, you're put in a position to have to 16 make a decision to call security and go and visit that 17 person. And I've done that. It's very uncomfortable.

18 It's even more uncomfortable when the discovery is a 19 positive.

20 And then there is the we had a scram, and 21 four people were involved, and we're not quite sure, 22 and someone was injured. Then you might do a post-23 event, because there's been human error, to determine 24 whether or not drugs or alcohol were involved.

25 Then there's the follow-up if you had a NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

89 1 positive. And they have a follow-up after the 2 positive to see if you're clean or not.

3 That is alive and well at every nuclear 4 power plant in the country. It's alive and well in 5 this building by the way. So those of us who have 6 been called, that's where we are. Next slide.

7 So there were data for three years. There 8 were subversion attempts. Do you know what that is?

9 That's where someone tries to either alter their 10 specimen or give somebody else's specimen or do 11 something like that. It happens all the time.

12 Amphetamines have been increasing. And 13 construction sites have the highest positive rate, 14 particularly in pre-access. This is where you got 200 15 laborers coming on board. And you're required under 16 Part 26 to do some screening. And lo and behold, you 17 screen out 10 or 15 of these people because they, for 18 doing drugs from their last construction site. And 19 they've had a big weekend. And they came in on a 20 Monday morning. And there they are. They get caught.

21 Next slide.

22 The NRC submitted a proposed rule for 23 consideration that would address three multi-year 24 trends by lowering the cutoff levels for amphetamine 25 and methamphetamine and expanding testing measures NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

90 1 related to subversion as an attempt to not be 2 detected.

3 There's no Commission response yet. But 4 that SECI is up there for consideration. In other 5 words, it's a trimming of the rule. Next slide.

6 So the data tells a story. So there is an 7 increase in subversion attempts. And, you know, you 8 stop and say what does that mean. Well, it means --

9 does it mean there are more people doing drugs?

10 Probably. It means more people doing drugs or trying 11 to hide doing drugs.

12 And I would just sense that this data is 13 probably applicable almost anywhere you might travel 14 in a high tech industry.

15 And at the bottom there, 45 facilities 16 with at least 1 subversion attempt. How many 17 facilities are there? What is it, 67, 68 facilities 18 in the country? So three-quarters of the sites, 19 facilities have at least one subversion attempt.

20 Sixty-seven are pre-access. And 98 percent of those 21 are contractors or vendors. Next slide.

22 There's an overall positive rate around 23 .75, .77, .76 percent. It's low, but it continues 24 upward. There was a slight reduction in calendar year 25 '17 total tests. There's a continuing downward trend.

NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

91 1 About 64 percent of the calendar year '17's positives 2 and refusals occur pre-access.

3 For those of you who don't know, if a 4 contractor comes in and you're asked to go give us a 5 specimen, there will be some who will say no thanks.

6 And they'll take their lunch bucket and their backpack 7 and go back to the parking lot and leave the site.

8 They'll just say, no, I'm not taking that test. That 9 happens.

10 And the randoms identified about 22.7 11 percent, one-fifth of the abusers. And that next to 12 the last caret identifies more employees using 13 substances than pre-access. That is what is alarming 14 to me.

15 We have a culture, not only in nuclear but 16 throughout our society, where there are a lot of 17 people using substances, these substances that are 18 banned. Next slide.

19 For-cause testing continues to have the 20 highest positive rate. That is where a supervisor or 21 someone points out an employee that needs to be 22 escorted to the fitness-for-duty station. And the 23 substances that account for about 85 percent of the 24 positives are marijuana, alcohol, and cocaine.

25 I would just say back in 1966 when I NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

92 1 started that we were concerned about people coming on 2 the watch that had a couple beers. I don't know. I 3 don't even remember these words. Marijuana was 4 something that happened in California. Those of us 5 from the east didn't, at least I didn't know about it 6 in '67. I got my RO in '67, so never even thought 7 about it. And here --

8 (Off mic comments.)

9 MEMBER SKILLMAN: Yeah, that's, I mean, 10 that's a culture I wasn't part of.

11 (Laughter.)

12 MEMBER SKILLMAN: Yeah, I'm just saying 13 the reason I wanted to show this is because I think 14 it's something that we as serious, engaged, active 15 professionals just need to keep in the back of our 16 minds. It's a shame that the culture is here. But 17 this is where it is. And this is where we live. We 18 need to be aware of it.

19 MEMBER SUNSERI: Hey, Dick, does the data 20 suggest that since the legalization of marijuana in 21 several states is contributing to the trend in 22 marijuana use, or is there, is it indifferent to that?

23 MEMBER SKILLMAN: Matt, that's a great 24 question. I think it's too -- this is Dick Skillman's 25 opinion. I think it's too soon to tell, because, you NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

93 1 know, it's just beginning to be legitimized as a 2 medical treatment. I don't think we've seen the data 3 yet that would be the result of those changes.

4 I know I've been kind of watching Exelon.

5 And they're saying we're not changing our standards.

6 You know, if you're taking medical marijuana, 7 recognize you can be called to fitness-for-duty on 8 Monday morning. So medical or not, if you show up 9 positive, you're going to, you're either not going to 10 have a license or you're not going to be working here.

11 MEMBER SUNSERI: Yeah, I understand that 12 about the medical. But I was thinking, you know, so 13 you got Colorado, Oregon, you know, California.

14 PARTICIPANT: Massachusetts.

15 MEMBER SKILLMAN: So the reason I wanted 16 to present this information is so we can kind of have 17 this conversation and say what's going on here. You 18 legalize marijuana. You go to a football game at 19 State College and you smoke up all Saturday afternoon.

20 And you come back and take a security watch on Monday 21 morning. I'm not sure. That's the issue.

22 VICE CHAIRMAN RICCARDELLA: You know, the 23 other issue is, you know, evidence of marijuana can 24 stay in your bloodstream for 30 days, you know. And 25 it doesn't really, you know -- if you're at that NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

94 1 football game 30 days ago, it's not clear to me that 2 it affects your performance. But it is what it is, I 3 mean.

4 MEMBER SKILLMAN: Next slide, please.

5 Fitness-for-duty data, understand that if you're at a 6 site, if you're a director or manager of a site, 7 you've got some very clear guidelines for how you 8 handle this information.

9 Once the employee has been sent to the 10 fitness-for-duty station, if there's a positive, there 11 are all kinds of reportings that do occur. And so 12 this will issue a fitness-for-duty as I think kind of 13 taken over at least a large portion of how the site 14 thinks about its health.

15 What is important is that for 2017 there 16 were 33 24-hour reportable events and half of those 17 are supervisors and managers. That's alarming. At 18 least I find that alarming.

19 And so, you know, what do you do if you've 20 got a very key manager upon whom you depend for key 21 decisions and you find that individual positive? Then 22 you enter your program, fitness-for-duty, that might 23 take that person off site for 30 days. There are 24 major issues associated with this whole topic.

25 So I just wanted to present this so that NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

95 1 the members have a sense of perhaps what the site 2 leadership and the site executives have to deal with, 3 but also what the NRC staff has to deal with. Our 4 culture has changed. Next slide.

5 Annual abnormal occurrence report to 6 Congress, there was one, one item. This was an 7 Americium-241 ampule, probably as big as your little 8 finger, that leaked on a lab table up the street here 9 up at NIST in Gaithersburg. And that resulted in an 10 overexposure. That is the only abnormal occurrence 11 that has been reported. Next slide.

12 Colleagues, any questions? Okay. Don 13 Helton, we're ready for you to come forward, please.

14 This is the second piece that I wanted to 15 talk about. And this is the topic of leading 16 indicators. Matt and I were kind of working our way 17 into this. I've got some strong feelings about what 18 is a leading indicator for future performance.

19 When I look at the identification of the 20 problems that ANO and at Pilgrim, you know, my own 21 involvement with 95002 and 95003 plants and 10350 22 plant, my view is that there are some items in the 23 culture at the site that will point to degrading 24 performance. And the question is at what magnitude.

25 And that is what kind of triggered this NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

96 1 discussion. Could we actually make a rubric or some 2 kind of a formula where you would say if you take a 3 look at these data over a reasonable time period, 4 could you predict that that licensee is going to go 5 into a ditch? I believe the answer is yes. But 6 that's what I wanted to talk about here.

7 MR. BOWMAN: Yes, so neither of us are Don 8 Helton in case you haven't realized that. I'm Greg 9 Bowman.

10 MEMBER SKILLMAN: I thought Don was going 11 to make the presentation. So --

12 MR. BOWMAN: So I'm Greg Bowman. I'm the 13 Branch Chief for the ROP Assessment Branch --

14 MEMBER SKILLMAN: Okay, Greg.

15 MR. BOWMAN: Tom Hipschman is the Branch 16 Chief for our Inspection Branch. And so Tom is going 17 to go through the first part of our presentation 18 talking about, you know, how our inspection program 19 picks up on degrading performance, whether it's due to 20 financial challenges or whatever.

21 And then I wanted to give a brief 22 discussion at the end about some recent developments 23 on ROP enhancements. We've gotten a lot of -- we're 24 in the process of dealing with a lot of proposals from 25 our stakeholders, both internal and external, for ways NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

97 1 we can make the ROP better.

2 And so we really just wanted to give you 3 a, you know, heads-up on something that's coming down 4 the road that you'll likely be involved in, you know, 5 helping review our, whatever we come up with. So --

6 MR. HIPSCHMAN: Thank you.

7 MEMBER SKILLMAN: Please proceed.

8 MR. HIPSCHMAN: Hi, I'm Tom Hipschman.

9 I'm the Chief of the Inspection Branch. This is my 10 first opportunity to present to the ACRS.

11 I took over the branch back in February of 12 this year. Previously, I've been a senior resident 13 inspector at Indian Point, Diablo Canyon, the resident 14 inspector at Oyster Creek, a regional inspector and 15 also a regional branch chief and served on the 16 Commission as a TA.

17 I'll just kind of fill in a little bit of 18 my perspective. It was a very good overview of the 19 ROP. So I'm not going to reiterate a lot of that.

20 But the ROP is designed as a process that 21 we use to identify declining licensee performance 22 prior to the loss of reasonable assurance or adequate 23 protection.

24 In part we do that by plants that will 25 progress through the action matrix, so columns 2, 3, NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

98 1 4, or possibly even 5. So we have increasing levels 2 of Agency response and oversight and management 3 oversight.

4 We also apply additional inspection 5 resources in response to greater-than-green findings.

6 And roughly our 9500X procedure, such as 95001, would 7 be used for a white finding, 02 for yellow, red for 3 8 roughly.

9 And throughout those inspections, every 10 quarter following completion of the 95003 there will 11 typically be a confirmatory action letter. The 12 regions will send out several inspectors on a 13 quarterly basis to do CAL closeout. Additionally, 14 there will be probably annual problem identification 15 and resolution inspections.

16 So there's quite a bit of additional 17 resources applied to plants that increase in the 18 action matrix.

19 MEMBER SKILLMAN: Tom, how many plants 20 have been subjected to 95003? I'm thinking 5, 6.

21 MR. HIPSCHMAN: I don't have a number off 22 the top of my head.

23 (Simultaneous speaking.)

24 MEMBER SKILLMAN: It's not 20. It's not 25 ten.

NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

99 1 MR. HIPSCHMAN: No.

2 MR. BOWMAN: I'm thinking probably five to 3 ten.

4 MR. HIPSCHMAN: Just off the top of my 5 head, Pilgrim, ANO --

6 MR. BOWMAN: Cooper.

7 MR. HIPSCHMAN: -- Indian Point.

8 MR. BOWMAN: Browns Ferry.

9 MR. HIPSCHMAN: -- Cooper, Browns Ferry, 10 Fort Calhoun. Those are the ones that come to mind.

11 MEMBER SKILLMAN: So maybe it's 6, 7 out 12 of 100.

13 MR. BOWMAN: Yes.

14 MEMBER SKILLMAN: 6 or 7.

15 MR. BOWMAN: Since the beginning of the 16 ROP in 2000.

17 MEMBER SKILLMAN: Yeah, okay. Thank you.

18 MR. HIPSCHMAN: So, notwithstanding, 19 there's been always interest in leading indicators.

20 The Commission in various forms has encouraged the 21 staff thinking about this. I know my former boss was 22 also very interested in leading indicators.

23 We do routinely assess the effectiveness 24 of the ROP through annual self-assessments, biannual 25 procedure self-assessments where we look at NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

100 1 procedures, inspection procedures and findings.

2 Additionally, as you mentioned, there is 3 an annual assessment meeting that all the regions 4 perform where they review each plant's performance.

5 And they have various discussions about certain areas 6 of concern. Next slide.

7 There have been -- various discussions 8 have highlighted ideas as, you know, what can we do.

9 Can we look at the trend and the number of green 10 findings? While there are some plants that tend to 11 have greater number of green findings or challenges, 12 we haven't historically used that as any kind of 13 trigger for increased oversight.

14 However, again, it is an assessment piece 15 that the regions will look at during end of cycle 16 meetings.

17 One aspect that sometimes has been 18 suggested is looking at cross-cutting aspects. With 19 those findings you will tend to accumulate in certain 20 areas, such as human performance or problem 21 identification, a number of cross-cutting aspects that 22 could give you insights into licensees' performance in 23 a particular area.

24 Other areas of findings that might be of 25 interest are QA, Appendix B related findings, NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

101 1 maintenance rules. Other areas that the NRC is 2 interested is plant operations.

3 (Background noise.)

4 CHAIRMAN CORRADINI: Just so you know 5 what's going on is that the wonderful system here cuts 6 out if there's no communication. So we lost a member 7 out in the far west. So we're going to try to 8 reconnect. So I apologize.

9 MR. HIPSCHMAN: All right. So proceed or 10 --

11 CHAIRMAN CORRADINI: Keep on going.

12 MEMBER SKILLMAN: Please proceed, yes.

13 MR. HIPSCHMAN: Plant operations, you 14 know, we'll look at --

15 PARTICIPANT: Do you have a question?

16 MEMBER REMPE: I do, but I thought you 17 were going on to the next slide. When you finish this 18 slide, I have a -- oh, I'll just say it now.

19 On some of these things where you're 20 talking about discussions of using the data, is there 21 something that could be done? Like right now a hot 22 research area is big data, right, and using some sort 23 of system to help you analyze it.

24 Has that discussion ever gone that way 25 that maybe there's something that research could do to NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

102 1 help you use this data more effectively?

2 MR. BOWMAN: So, when we get to the 3 transformation discussion or the enhancement 4 discussion at the end, some of those suggestions have 5 come in to us. We haven't made a lot of progress on, 6 you know, shipping that work over to research to help 7 us with. But we do need to assess that as one of our 8 ROP enhancement proposals.

9 MEMBER REMPE: So --

10 MR. BOWMAN: Most of the things that Tom 11 is discussing on this slide are already looked at as 12 part of our inspection program, so, but not from a big 13 data standpoint, not, you know --

14 MEMBER REMPE: So that's something that's 15 coming up in the DOE arena with industry. And so 16 there might be, and we can talk about it later, but 17 with some collaborative efforts --

18 MR. BOWMAN: Yeah.

19 MEMBER REMPE: -- with the Department of 20 Energy.

21 MR. HIPSCHMAN: With regards to plant 22 operations, inspectors will keep tabs on various 23 things such as work management. They'll look at the 24 number of corrective actions or corrective maintenance 25 in the backlog. They'll -- some plants tend to have NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

103 1 higher backlogs than others. You know, anecdotally, 2 that could be sort of an indication.

3 Other things that occur are to help the 4 inspectors relate to operation of the plants and 5 material condition of the plant. Some plants are in 6 obviously better condition and better upkeep than 7 other plants.

8 And one way that inspectors can be aware 9 of that is that they will do annual objectivity visits 10 to other sites so that they can get comparisons on how 11 both residents and also other plant managements are 12 performing.

13 Also, there are frequent regional 14 management visits that, you know, are also very 15 instructive for the resident inspectors. Just on the 16 list, you know, there's other things that the 17 inspectors can trend.

18 Interestingly, you know, staffing 19 resources such as, you know, what kind of license 20 operator pipeline is there is kind of also an 21 indicator of how well or, you know, of what's going on 22 at the plant. If there's a lot of license operators 23 in the pipeline or if there's a healthy number of 24 senior reactor operators, you know, that typically 25 seems to be a good indicator.

NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

104 1 You know, trend and conditions, 5072, 5073 2 reports, again, those are sort of, those are things 3 that inspectors follow up on. They also will do an 4 inspection of those after the fact. And that is also 5 reviewed during end of cycle meeting.

6 One of the outcome of end of cycle 7 meetings are senior management key messages that the 8 regions will develop. And they're sort of talking 9 points for when the regional senior managers go out to 10 the site. They're areas of performance that they 11 would like to emphasizes during their site visits.

12 MEMBER SKILLMAN: Tom, let me ask this, 13 please. When Matt and I were trying to develop some 14 energy around leading indicators, the question that 15 was foremost in my mind is, has the staff ever taken 16 all the 95003 plants, just remove the name, just take 17 the data that were the initiating events that got the 18 plants in the 95003 and compared them and asked the 19 question are the conditions that brought the plant to 20 95003 similar enough that those specific conditions 21 would themselves become a leading indicator.

22 For example, at least one man's opinion, 23 the robustness of the system health reports, the 24 degree of connection between the corrective action 25 program and the work management program and the NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

105 1 expressed beliefs by senior leadership and their 2 knowledge of the Appendix B to 10 CFR 50, I keep 3 coming back to that, those plants where senior 4 leadership understands that Appendix B is 18 pieces.

5 If you pull them together right, if you've got a 6 strong corrective action program and a strong work 7 management program, you might get a lot of greens, but 8 they're in the grass because people are taking care of 9 stuff, little things that happen.

10 But you're also moving the bigger items 11 into strong root cause evaluations. And you're 12 actually fixing the underlying problem as opposed to 13 repair and move on.

14 So my real, the question I would ask is, 15 have you ever or has the staff ever considered looking 16 at the six or eight or ten 95003 plants and the one 17 0350 plant -- I don't know how many 0350s, but at 18 least one -- and asked is there any common set of 19 findings, which if given even a thicker magnifying 20 glass, might produce a fairly short list of leading 21 indicators.

22 My view is the answer would be yes. But, 23 and I would sure like to have the time and energy to 24 do that study, but I don't. But it seems like that's 25 a gold mine.

NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

106 1 MR. BOWMAN: Yes, so -- I'm not aware of 2 us having done a study like that. It's a great 3 question.

4 When we get to the -- and I hate to kick 5 everything to the enhancement discussion we're going 6 to have shortly. But we've got a number of 7 suggestions both from internal stakeholders and from 8 external stakeholders about our performance indicator 9 program and whether that's where it should be, whether 10 it's giving us meaningful information.

11 And one of the things that I -- this is 12 just -- we haven't actually kicked off the review of 13 that yet. But I think doing something like that is, 14 would be very valuable in determining whether we can 15 identify different performance indicators or better 16 performance indicators to help us in that area.

17 I think that's something we had planned.

18 We haven't really done that yet. I'm not aware of us 19 having done that historically.

20 But as we look at the performance 21 indicator program in the context of our opening 22 enhancement, I think that will be a very fruitful 23 exercise, going through and seeing, you know, what 24 95003 plants might have been experiencing before they 25 got there. That's a good, very good suggestion.

NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

107 1 MR. HIPSCHMAN: Yes, I'm not --

2 MEMBER BLEY: Dick, can I cut in sometime?

3 MEMBER SKILLMAN: Yes, sir. Dennis, good 4 to hear from you. Please.

5 MEMBER BLEY: Yeah, well, I've been locked 6 out for a while. I finally found out why. I'm back.

7 This business of leading indicators has 8 been a hot topic since at least the mid-70s. And the 9 NRC sponsored a lot of research back in the 80s on 10 this, so have others.

11 The unfortunate thing that's happened is 12 when each of us thought we had it figured out, because 13 it kind of made sense, when you tracked it, you found 14 that you kept getting surprised. The things the 15 leading indicators pointed to didn't turn bad and 16 other things turned bad that you weren't expecting.

17 A couple of years ago we had INPO up here.

18 And they claimed, at least the gentleman who was 19 representing them, that they had developed a new 20 leading indicator model that really was working and 21 gave short term and longer term predictions that were 22 coming out true.

23 I wonder if the staff is aware of what was 24 going on there and if that's actually come to fruition 25 at all. And I also like the idea, as Dick put out, of NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

108 1 things you might look at, because I think some of 2 those nobody has really looked at yet. That's all.

3 MR. BOWMAN: Yeah, and this is Greg 4 Bowman. I'm not aware of the INPO, what INPO's, what 5 they've been working on or whether it's come to 6 fruition or not.

7 We do try to keep a healthy amount of 8 separation between our oversight program and what INPO 9 does just to avoid -- you know, they obviously have 10 different objectives than we do. And we try to keep 11 them separate. But, yeah, so I'm not aware of where 12 INPO is with their process.

13 CHAIRMAN CORRADINI: Dennis, can I just 14 follow up? Dennis, was it an open discussion or was 15 it a closed discussion? I remember it. But I can't 16 remember much about it.

17 MEMBER BLEY: I think our whole meeting 18 with INPO was closed.

19 CHAIRMAN CORRADINI: Okay.

20 MEMBER BLEY: And it followed on the heels 21 of Fukushima.

22 CHAIRMAN CORRADINI: Okay. Thank you.

23 MR. HIPSCHMAN: One comment regarding 24 INPO, and I'll finish my answer on the other one, is 25 with respect to INPO evaluations, we do have a NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

109 1 memorandum of understanding with them.

2 And one of the things that the NRC 3 inspectors have access to is their evaluation report.

4 And inspectors will typically read that just to make 5 sure that they haven't missed anything, verify, you 6 know, their conclusions as far as licensee 7 performance. And if there's any new insights that 8 inspectors need to follow up, they will leverage those 9 reports as needed.

10 With regard to the 95003s, I am not aware 11 of anything as Greg mentioned. However, we do have 12 sort of a lagging indicator, to some respects, is that 13 we do look at the 95003s after they're completed.

14 We do a lessons learned report, which 15 tends to be rather extensive. There are several 16 corrective actions that are generated from each 17 lessons learned report that the staff follows up on to 18 look to see how inspection procedures, manual chapters 19 can be changed. But I don't think we've done anything 20 that kind of looks at all of them in the whole.

21 MEMBER SKILLMAN: I would sure like to 22 encourage finding a way perhaps to take the 95003 23 conclusions and seeing if there are not commonalities 24 among all of those 95003 inspection, the completed 25 reports, because I have a pretty strong sense that NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

110 1 there are diamonds down in there.

2 MR. BOWMAN: That's the first suggestion 3 we'll --

4 MR. HIPSCHMAN: Yeah.

5 MR. BOWMAN: -- think about.

6 MR. HIPSCHMAN: Yeah, I think it's 7 interesting --

8 MEMBER SKILLMAN: Thank you. Please 9 proceed.

10 MR. HIPSCHMAN: Next slide, please. A 11 related point is other areas where we can adjust our 12 inspection program for plants is for plants that are 13 experiencing financial issues. We do have guidance 14 for that for various things to, inspector to look at 15 such as challenges to material conditions. And you'll 16 see four plants that have been in that. And the 17 inspectors will comment in the inspection report.

18 Also, we do have guidance for plants that 19 are nearing cessation of permanent operations. We'll 20 adjust, the inspectors will adjust their inspection 21 areas such as, you know, they'll look for is there an 22 increased attrition of, say perhaps, licensed 23 operator, are there any changes in material condition.

24 And so they'll look at those kind of 25 things. And as appropriate, they'll follow up on the NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

111 1 baseline inspection program for areas that might 2 warrant that.

3 And the other things that we have as tools 4 are safety culture initiatives. As part of our 5 biannual problem identification and resolution 6 inspections, we have a part of that that looks at 7 safety culture. And also in our 95002 and 95003 we 8 have much more extensive guidance regarding looking at 9 safety culture.

10 Let's see. Also, we do have other 11 inspection procedures in manual chapters such that 12 during plant walkdowns or senior manager site visits 13 we do look at material conditions. We do look at 14 long-standing issues. We do highlight those, bring 15 those up with licensee management.

16 And again, you know, overall the ROP is a 17 very flexible inspection program. Inspectors have a 18 lot of flexibility within the program to look at 19 various things, use modules to look at degraded 20 material condition, look at human performance, look at 21 problem identification. And they can review and 22 report those as appropriate.

23 Again, in the action matrix, one of the 24 aspects of that is increased management presence, as 25 well as a site we'll go through the action matrix or NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

112 1 increase in the action matrix. Not only is there 2 increased management presence, but there's also 3 increased management review, such as with the 4 inspection reports and things like that and public 5 meetings.

6 With that, I'll -- if there's not any more 7 questions, I'll turn it over to Greg.

8 MR. BOWMAN: Okay. So, on the next slide, 9 so, at the very beginning of his presentation, Tom 10 talked about the self-assessment process.

11 So the ROP is a mature program. It's been 12 in place for about 20 years. One of the key 13 components of the ROP is self-reflection, figuring out 14 things, you know, on a periodic basis that we could 15 have done differently, could have done better to 16 improve the program.

17 So the beginning of this slide, the first 18 main bullet and the sub-bullets on this slide talk 19 about some of the ongoing initiatives we have to make 20 the ROP better.

21 You know, we've had a lot of work on the 22 inspection report development process. We've 23 streamlined our inspection reports. And we're nearing 24 the end of an initiative to automate the production of 25 our inspection reports to make it easier for our NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

113 1 inspectors so they can be focusing on inspection and 2 not inspection report writing.

3 Dick, when you presented, you talked about 4 the four levels of violations. There's a fifth down 5 there that we, that gets, that results in a lot of 6 work but doesn't, you know, see often. Those are 7 minor violations, right.

8 So, if you have a -- you don't fill out a 9 -- or you're working through a procedure and you don't 10 follow every step, but there's no impact from failing 11 to follow that step, we'd often characterize those as 12 minor violations.

13 And Tom and I being inspectors can tell 14 you that we spend a lot of time with our management 15 discussing, and I'm using discussing gently, that's a 16 gentle term for what we wind up doing, with our 17 managers about whether an issue is minor or more than 18 minor.

19 So we have work going on to sort of help 20 give better guidance to our inspectors so they don't 21 have to deal with the, all the back and forth. Things 22 are more clear to them on what's minor and what's not.

23 MEMBER SKILLMAN: Greg, thank you for that 24 clarification. That was an oversight --

25 MEMBER BLEY: Can I --

NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

114 1 MEMBER SKILLMAN: -- on my part. Thank 2 you.

3 MEMBER BLEY: May I sneak in a question 4 here?

5 MEMBER SKILLMAN: Please, go ahead, 6 Dennis.

7 MEMBER BLEY: You know, from an overall 8 plant safety point of view, that seems to make sense.

9 From a human performance point of view, if it's the 10 same action that could have created a serious problem, 11 you really ought to look at it.

12 MR. BOWMAN: Yeah, and that's a function 13 of how our program is set up. We look at, we 14 essentially look at what the consequence was of an 15 issue.

16 There's often times cases where, if a 17 circumstance was slightly different, then the 18 violation or the finding would be of greater 19 significance. And that's sort of a fundamental part 20 of how our program works.

21 I think one way we get to that issue is 22 through our cross-cutting issue program where we, you 23 know, when we go through, we have a green finding. We 24 figure out what the causal factors are. And we look 25 to aggregate, you know, see if there's a trend in an NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

115 1 area.

2 But that's a fair point. And it's sort of 3 just a fundamental part of how our program works.

4 One thing I would add is that, you know, 5 on a biannual basis we do a problem identification and 6 resolution inspection. And that inspection procedure 7 allows us to dig in a little more deeply on issues 8 that are less, that are minor.

9 So we can do -- we do some trending. We 10 document observations that normally wouldn't be 11 documented because of the low significance of the 12 issue.

13 MR. HIPSCHMAN: You know, notwithstanding 14 that, in my experience at the sites that I have been 15 at is that when the inspectors provide comments that 16 aren't more than minor or their observations, the 17 licensees treat those very seriously. They put those 18 in the corrective action program. And they do take 19 action.

20 And also, the inspectors have the 21 opportunity to follow up those that -- you know, 22 they'll keep those in mind. And if they start 23 aggregating and they look for a bigger programmatic 24 type deficiency or trend, the opportunity is also to 25 have an opportunity to make a finding or violation NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

116 1 from that.

2 MEMBER SKILLMAN: Yeah, Dennis, to your 3 point, my experience, my firsthand experience is very 4 commonly the minors or the debate about more than 5 minor normally falls to the director of operations or 6 whoever is running engineering, because it's either a 7 procedure failure, a failure to perform procedure 8 properly or it's a Criterion III design issue.

9 And the amount -- debate is the right 10 word, but perhaps not politically correct. But the 11 amount of energy in discussing whether it's minor or 12 more than minor, at least in my view, addresses, 13 Dennis, your comment. Could there be something that 14 is very subtle whose impact is not fully appreciated?

15 And in the multiple events I've been 16 involved in, the extent of discussion and the tone of 17 the discussion has been very effective in sorting out 18 whether or not the minor issue was truly minor or 19 whether it was more than minor and could have been 20 leading to a more subtle or more serious outcome. And 21 those debates --

22 MEMBER BLEY: Well, what --

23 MEMBER SKILLMAN: -- have been very, very 24 --

25 MEMBER BLEY: What --

NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

117 1 MEMBER SKILLMAN: -- very spirited.

2 MEMBER BLEY: What you say and what the 3 other fellows just said is encouraging. But the idea 4 that Joe just got lucky and Harry gets nailed to the 5 wall because he wasn't lucky is a significant thing to 6 worry about.

7 MR. BOWMAN: And one thing I just wanted 8 -- this is Greg again. One thing I wanted to just 9 amplify something Tom said. So, regardless of whether 10 a violation is minor or more than minor, the licensee 11 still needs to fix it. They need to put it in a 12 corrective action program.

13 And if you have a case where there was a 14 near miss, we would expect the licensee's corrective 15 action program to include a robust causal analysis, 16 you know, in-depth corrective actions. And as Tom 17 mentioned, we can go back and inspect those even if 18 they were minor, even if it was a minor issue.

19 MR. HIPSCHMAN: You know, and one of the 20 things --

21 MEMBER BLEY: Okay.

22 MR. HIPSCHMAN: -- in assessing whether 23 it's minor or not, there are several questions that 24 the inspectors have at their use. And one of those is 25 if that issue was left uncorrected, could it be a NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

118 1 precursor to a more significant event. And if it is, 2 then it would be kicked up into more than minor.

3 MEMBER BLEY: Okay. And since you used 4 the word precursor, the precursor program looks at 5 events and then says from this point on could this 6 have become serious. And it seems like that kind of 7 thinking, from what you're saying, is getting picked 8 up. So I think that's very important.

9 MR. HIPSCHMAN: Thanks.

10 MR. BOWMAN: So then the next, you know, 11 sort of sub-bullet on this slide is the work we've 12 been doing over the past couple years to improve the 13 engineering inspection program. I believe we briefed 14 you on that. And we're getting close to the point of 15 sending a paper up to the Commission with 16 recommendations to make the program better.

17 So I mentioned at the beginning, you know, 18 we established this transformation team at the NRC 19 probably I guess it's been about a year ago now. That 20 sort of kicked off a lot of input coming in our 21 direction.

22 The transformation team was largely 23 focused on making the NRC more agile, more able to 24 deal with new technologies, that kind of thing. But 25 they went out and solicited stakeholder input on NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

119 1 pretty much anything the NRC could do to be better.

2 A number of those recommendations were 3 related to the ROP. Because it wasn't really within 4 the transformation team's charter, they just 5 essentially passed those recommendations on to my 6 group to review.

7 You know, sort of coincident with that or 8 in conjunction with that, we got feedback from the 9 industry, from NEI, and from NRU, the National -- I'm 10 going to forget the acronym.

11 MR. HIPSCHMAN: Regional Utility --

12 MR. BOWMAN: Group.

13 MR. HIPSCHMAN: -- or Users Group.

14 MR. BOWMAN: With additional proposals 15 from them on how to enhance the ROP.

16 And then we just recently got a follow-up 17 letter from NEI with sort of a consolidated list of 18 recommendations from the industry on what we could, 19 what they believe we should change in the ROP.

20 So, on the next couple slides, I'll go 21 through just at a high level what some of that 22 feedback was.

23 So the input from the transformation team, 24 I mentioned there were about 70 recommendations, fit 25 into generally these categories. And it was, you NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

120 1 know, it was raw, sort of unfiltered feedback from the 2 staff. So, you know, often there were recommendations 3 that conflicted with one another or, you know, some of 4 them were not very well developed.

5 But, you know, probably the most common 6 recommendation had to do with the structure of the 7 regions, you know, whether the resident inspector 8 program, whether that should be expanded or done away 9 with, whether our Division of Reactor Safety and 10 Division of Reactor Projects should be organized 11 differently. We got a lot of suggestions in that 12 area.

13 And then again, you know, pretty much 14 every component of the ROP inspections, PI, 15 performance indicators assessment, and SDP, we got 16 some suggestions.

17 And so we have all those recommendations 18 in. We've done some binning, you know, to group them 19 together. But that's the extent of what we've done to 20 date. Next slide, please.

21 So the -- we got a letter from the 22 industry, from NEI, on the 19th of September, so just 23 a week ago, a little over a week ago. And then we had 24 a public meeting with NEI the following day for them 25 to brief us on their recommendations.

NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

121 1 The one thing that I think we've heard 2 from both industry and from other stakeholders is that 3 the ROP is a sound oversight program. You know, it's 4 one that countries around the world model. And it's 5 not one that I think any of us argue should be 6 substantially changed. So, however, I think we all 7 acknowledge, both the staff, industry, other external 8 stakeholders, that there are areas we can improve.

9 So this slide provides sort of a synopsis 10 of what the industry's recommendations were. And they 11 really fit into four areas.

12 Impact of white findings, and so that 13 would be things like do we issue a press release for 14 a white finding or do we not issue a press release, 15 how do we communicate the relative risk of a finding.

16 You know, a white finding is low to, we 17 characterize it as a low to moderate risk. Is that 18 really what a white finding is? And when we're in the 19 minus 6 range for a finding like that, is that an 20 appropriate characterization? Does the public 21 understand what we're saying when we say low to 22 moderate? So that type of thing.

23 And then we got feedback also on the type 24 of inspection we do to follow up on a white finding.

25 As, Dick, you mentioned, we do a 95001 inspection, NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

122 1 which, you know, it's the lowest of the 9500X 2 inspections, but it's a fairly intensive effort. And 3 so does that make sense given the low safety 4 significance of a white finding?

5 We also got feedback from NEI on the 6 baseline inspection program, whether it's where it 7 should be. You know, one of their suggestions was 8 take a look at licensee performance over the last 9 several years. If a plant has been in column 1 or has 10 been doing well based on whatever indicators we come 11 up with, could they get something less than the 12 baseline inspection program?

13 MEMBER SKILLMAN: I would like to just 14 jump in here because --

15 MR. BOWMAN: Go ahead.

16 MEMBER SKILLMAN: -- I think that is 17 where, if one were to take a 95003 lessons learned and 18 refine them into action statement and then look at the 19 baseline inspection program, one might say, one might 20 conclude or assert that licensees that are performing 21 excellently against those 95003 --

22 MR. BOWMAN: Criteria.

23 MEMBER SKILLMAN: -- items could almost be 24 exonerated from some level of inspection because their 25 behavior is so opposite to what the findings are in NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

123 1 the 95003 category.

2 In other words, the work management 3 program is bulletproof. The connection from the work 4 management program to the root causes is remarkable.

5 I mean, they really do root causes. They go right 6 down to the last nut and bolt. And by and large, they 7 are on target. Their recordkeeping, their MT&E is 8 right where it needs to be.

9 You could almost say a plant that has that 10 type of performance and also has a safety culture that 11 is thoroughly positive, one might say we can probably 12 back off. So what I'm suggesting is there's a 13 connection between that question and lessons learned 14 out of those 95003 inspections.

15 MR. BOWMAN: Yeah, that's exactly what I 16 meant when I said we would, this would be something we 17 would look at as part of this.

18 I think, you know, the inspection program 19 and the performance indicator program are meant to be 20 complementary, right. You have performance 21 indicators, and then you have the things that can't be 22 readily measured by a performance indicator we go out 23 and inspect.

24 If we were theoretically able to identify 25 a better set of performance indicators that could NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

124 1 result in need for less inspection, but, you know, we 2 just started with this so I don't want to --

3 MEMBER SKILLMAN: Yep.

4 MR. BOWMAN: -- you know, predict where 5 we're going to end with it.

6 MEMBER SKILLMAN: I'm agreeing, I guess in 7 a way agreeing or maybe even championing a real 8 serious look at this because it might be to everyone's 9 benefit. What it really does is it might make 10 resources available to do other really important 11 stuff.

12 MR. BOWMAN: Right, right. Similarly, 13 this doesn't really fit into the inspection program 14 cleanly, but industry had some suggestions on the 15 mitigating systems performance indicator and 16 specifically whether that indicator is really needed 17 anymore, whether it's -- you know, we don't have very 18 many -- I think it's very rare for us to have a 19 greater-than-green MSPI.

20 And most licensees, you know, when the 21 program was put in place, many of them took actions to 22 make those systems less risk significant. You know, 23 they made modifications to the plant.

24 So the value, one could argue that the 25 value of the indicator has gone away somewhat. It's NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

125 1 very complicated to calculate. It's not easy to 2 understand necessarily for the public. So the 3 feedback was, you know, what's -- I think whether we 4 could do things better with that indicator.

5 They had suggestions on improving the SDP, 6 some specific SDPs that they suggested we go back and 7 look at, and then some sort of infrastructure work to 8 -- you know, we have a lot of interaction with 9 licensees as we're completing a significance 10 determination process for potentially greater-than-11 green finding.

12 A lot of that is around the assumptions 13 that go into the SDP, things like recovery credit, 14 common cause failure. Those things result in a lot of 15 discussion. And so the recommendation was really can 16 we establish better ground rules up front so that NRC 17 and industry both know what sort of assumptions are 18 going to go into an analysis.

19 And then they had, the last, you know, 20 sub-bullet there, some feedback on resolution of 21 inspection issues. So, and some of this is addressed 22 by our backfit initiative.

23 But is there a way we can more easily 24 resolve low risk compliance issues, things that -- you 25 know, we have an inspection finding. We know based on NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

126 1 a PRA that the finding is very low risk. But 2 sometimes we can spend a lot of effort trying to get 3 that resolved. Sometimes there's action statements 4 that require the plant to shut down. And is that 5 really where we need to be based on the risk of the 6 finding? So that's sort of the fourth item. So next 7 slide.

8 So we are in the very early stages of 9 working through all these recommendations. As I 10 mentioned, the letter from NEI just came in a week and 11 a half ago. We've done some work to sort of bin all 12 the recommendations between what we got from internal 13 stakeholders and from the industry.

14 And what we're planning on doing is 15 setting up some working groups internal to the NRC to 16 go through and evaluate the proposals. We had a 17 kickoff meeting with our team. October 3rd was 18 yesterday. So it was just yesterday.

19 And then we already -- we meet with the 20 industry on a monthly basis to go over items of 21 interest. We'll be using those meetings to sort of 22 engage with both the industry and stakeholders on 23 their proposals, talk through them with our next 24 meeting scheduled for the 18th of October.

25 We do -- some of the recommendations we NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

127 1 got, many of them actually, are fairly significant, 2 you know, either fairly significant from an evaluation 3 standpoint or fairly significant when it comes to 4 implementation. And many of them would likely require 5 Commission approval if we were to choose to adopt 6 them. And that process is laid out in that management 7 directive, the criteria for when we go to the 8 Commission versus when we can make changes on our own.

9 And then I guess I didn't put this on the 10 slide. But obviously, the big changes, we would need 11 to engage with ACRS to get your feedback before we 12 went to the Commission with any sort of proposals.

13 MR. HIPSCHMAN: You know, Greg, also with 14 a Commission meeting, recently operating reactor 15 business line --

16 MR. BOWMAN: Right.

17 MR. HIPSCHMAN: -- it was a topic during 18 the Commission meeting. And the Commission pretty 19 much encouraged us, you know, take a look at these, 20 consider your path forward.

21 MR. BOWMAN: So that was all we had.

22 Let's open it up for questions now if you --

23 MEMBER SKILLMAN: Greg and Tom, thank you 24 very much. Colleagues, what Greg and Tom are showing 25 here is, first of all, it's a heck of a lot better NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

128 1 than SOLP.

2 PARTICIPANT: I can agree with that.

3 MEMBER SKILLMAN: For those who weren't 4 around, that was the Systematic Assessment of Licensee 5 Performance, and the ROP replaced that. And this is 6 much more objective and much more thorough. I would 7 say much more flexible, and it's not personal. So 8 this has been a huge enhancement at least from my 9 perspective.

10 And I would just like to keep thinking 11 about are there some tools readily available that will 12 help our industry colleagues and help the staff maybe 13 shape forward-looking activities that might result in 14 optimization of resources for everybody.

15 So thank you. Colleagues, any questions 16 for Greg or for Tom?

17 (Off mic comments.)

18 MEMBER SKILLMAN: On the phone line, is 19 there anybody out there that would like to simply say 20 hello so we know you're there? Anybody in the room?

21 MR. BOWMAN: I think there's a comment.

22 MEMBER SKILLMAN: Great. Sir, good 23 morning.

24 MR. THORPE: Good morning. John Thorpe 25 with Office of the Inspector General. I'm their NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

129 1 Senior Technical Adviser. I'm a former Chief of the 2 Operating Experience Branch.

3 I've heard some discussion earlier 4 regarding the initiatives that INPO might be taking 5 relative to leading indicators and evaluating what 6 they can do to get a better pulse on what's happening 7 with their plants.

8 And I would recommend that you guys talk 9 to the Operating Experience Branch. They have a 10 routine. It's once or twice a year. I don't think 11 they've stopped it since I was the Branch Chief. They 12 meet with INPO. They do a compare notes kind of 13 session.

14 And I know for sure that in one of the 15 meetings that I sat in on we had INPO staff talk to us 16 about the leading performance indicators, the leading 17 indicators that they were trying to develop from their 18 perspective as INPO.

19 Now, obviously, they're shooting for 20 different goals and thresholds. But they're also 21 really trying hard to try to find out what it is that 22 they can sense from all the data that comes in, from 23 EPIX, from all these other things. How can we sense 24 whether somebody's on the decline or are they 25 improving or remaining the same?

NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

130 1 So I think there's some valuable 2 information that may be available to you from the 3 Operating Experience Branch and just wanted to offer 4 that to you.

5 MEMBER SKILLMAN: Mr. Thorpe, thank you.

6 Thank you.

7 Before we come to close here, I want to 8 thank Derek for his effort to put this together.

9 About 24 sets of reports that I've been studying and 10 that Derek's been studying are the basis for the first 11 presentation. So, Derek, thank you for your effort.

12 Colleagues, any questions for me or for 13 Greg or for Tom, please? Hearing none, Mr. Chairman, 14 back to you, sir.

15 CHAIRMAN CORRADINI: Okay. Thank you very 16 much to the staff. That clock does not work. Just so 17 we're all clear, it's still not -- we haven't -- we're 18 not like an airplane that arrived five years later or 19 whatever the hell that is.

20 PARTICIPANT: It's an hour behind right 21 now.

22 PARTICIPANT: It's actually adjusting 23 itself.

24 PARTICIPANT: It stopped.

25 CHAIRMAN CORRADINI: Yeah, I think it's NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

131 1 dead. It's dead.

2 (Off mic comments.)

3 CHAIRMAN CORRADINI: So thanks to the 4 staff. Appreciate it. And we're going to go into a 5 lunch break and be back at 1:00.

6 At 1:00 we're going to do the quality 7 review for the research topics of interest. And Dr.

8 Rempe will, Member Rempe will lead us through. Okay?

9 Thank you.

10 (Whereupon, the above-entitled matter 11 went off the record at 11:47 a.m. and 12 resumed at 1:00 p.m.)

13 CHAIRMAN CORRADINI: So we'll start again 14 with the quality review. And, Dennis, I think you're 15 the first one up as chairman of the team and Joy will 16 take over.

17 MEMBER REMPE: Okay. Just before you 18 start though I want to remind everyone we're going to 19 have two of these.

20 And as we go through this, think about to 21 the two chairs, Margaret and Dennis, how soon you can 22 turn around the input for the actual document because 23 that's a question we need to think about in when we're 24 going to issue the letter. So go for it, Dennis.

25 MEMBER BLEY: Okay, thank you. On the NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

132 1 title slide I remind everybody our team was myself and 2 Walt and Dick. Are we still on the record or is that 3 done?

4 CHAIRMAN CORRADINI: We're still on the 5 record.

6 MEMBER BLEY: Okay. So we'll have notes 7 from this. That's good because I don't think I can 8 take them. Okay, second slide is the PIRT project 9 description high energy arc fault.

10 And it's been on the NRC's table for a 11 while. But some things happened in the last few years 12 that have changed their thinking a bit. These arc 13 faults seem to be, not seem to be, have been more 14 prevalent than people thought in the past.

15 Very severe arc events often involve 16 unanticipated break or coordination failure. Things 17 that you wouldn't normally know weren't properly 18 aligned if it was very high past current then they 19 don't work the way people expect them and that causes 20 multiple problems.

21 The existing models they have developed 22 did not comport with experiments that the staff 23 sponsored. And you all saw the aluminum bus fire and 24 enclosure fire, hello.

25 CHAIRMAN CORRADINI: Yes, we're still NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

133 1 here, Dennis. We can hear you.

2 MEMBER REMPE: If anyone is out on the 3 webcast please put your phones on mute, okay.

4 MEMBER BLEY: I am hearing a reprise of 5 Mike's introduction that's very loud.

6 CHAIRMAN CORRADINI: I think, Dennis, 7 there seems to be a delay with the webcast so if you 8 --

9 MEMBER BLEY: Well there always is. But 10 before I did not hear it over my phone and now it's 11 really, I'm hearing 50 seconds behind and it's really 12 confusing.

13 CHAIRMAN CORRADINI: I understand.

14 MEMBER BLEY: So if they can cut down that 15 crosstalk it would be great. But I'll go ahead.

16 CHAIRMAN CORRADINI: Okay.

17 MEMBER BLEY: And try to put that out of 18 my mind.

19 CHAIRMAN CORRADINI: I can even hear 20 myself.

21 MEMBER BLEY: Well once they ran into that 22 problem with the aluminum and other issues that kind 23 of surprised them they thought maybe doing a PIRT that 24 helped them set priorities for further research.

25 MEMBER REMPE: Dennis.

NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

134 1 MEMBER BLEY: The report --

2 MEMBER REMPE: The webcast is coming 3 through perhaps your system. Can you put your system 4 of the webcast on mute because we are hearing you 5 through the phone line, okay.

6 MEMBER BLEY: I didn't hear it because I 7 had the headset on.

8 MR. BROWN: Dennis, you can't have the 9 webcast going at the same time you're speaking because 10 it's a ten minute, ten second delay.

11 MEMBER BLEY: It's actually a 50 second 12 delay.

13 MR. BROWN: It's a delay so you can't have 14 --

15 MEMBER BLEY: I didn't know my speaker --

16 MR. BROWN: You can't have both on.

17 MEMBER BLEY: I didn't know my speaker was 18 on because I had these headphones on. This should be 19 better now.

20 MEMBER REMPE: Thank you.

21 MEMBER BLEY: We found that the PIRT 22 exercise was conducted in a reasonably satisfactory 23 way to frame the financial risk contribution in 24 nuclear power plants from these events. And they also 25 tried to evaluate their own state of knowledge when NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

135 1 they did the PIRT which is a smart thing to do.

2 Next slide, documentation, clarity of 3 presentation. Our consensus score was a five. I'm 4 going to take a little diversion because of the 5 conversation that went on in the last couple weeks.

6 We did this kind of the way I've been on 7 three or four of these, kind of the way we've, I've 8 always done it. But others have said, gee, the way we 9 did this if one person is an outlier we just make 10 them, talk them into changing their score so they're 11 all about the same.

12 Well we didn't do that. We followed what 13 you do on elicitation. And if we were diverse and on 14 a few things we had scores as wide as, whoever is 15 moving the paper is driving me nuts.

16 We had things as diverse as an eight and 17 a four or a three. So instead of saying let's come 18 together we talked about why each of us had the scores 19 we had.

20 And when you do that you find out we were 21 looking at slightly different things. And some 22 things, I don't know if you've played with this scale 23 much, some you could put in multiple places.

24 And some of us would put one issue under 25 clarity of presentation where the other person might NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

136 1 put it under uncertainty. So we reached agreement on 2 where everything belonged and exactly what questions 3 we were trying to answer.

4 And then independently still, we 5 reevaluated, boy, that sound keeps coming in. We 6 reevaluated our individual scores and you'll see those 7 at the end of the presentation.

8 After that point we had a final discussion 9 following the ideas that are in the Shack report 10 saying let's come to a consensus on the score that we 11 think represents kind of the best knowledge of 12 technical people in this area. So we treated our 13 scoring like a properly done elicitation that was 14 facilitated and I did the facilitation. Let's go 15 ahead then.

16 MEMBER REMPE: Dennis, do you want to 17 point out to everybody on Slide 9 that you do have the 18 individual scores too just as they go through this?

19 MEMBER BLEY: I was going to do that when 20 I got to Slide 9, but you pointed it out.

21 MEMBER REMPE: Okay, thanks.

22 MEMBER BLEY: We had slightly different 23 scores on this to begin with. When you start reading 24 the report it's very good. But then it turns out that 25 really three of the chapters, 1, 2 and 4 are really NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

137 1 well written, clear and precise.

2 Chapter 2 provides a really good 3 description of the PIRT process and how it ought to 4 work and including the things that would affect 5 uncertainty and helping identify important factors.

6 Chapter 4 was pretty tight and ordered on 7 the presentation of results, but at a high level.

8 They didn't dig into the details of the ordering out 9 at that point.

10 If you read Chapter 3 which is kind of the 11 results section and you don't know what you're 12 reading. After you read the appendices you know 13 what's in Chapter 3 but you can't really tell it just 14 from Chapter 3.

15 It's a collection of results and tables 16 with not much to tell you what you're looking at and 17 why it is the way it is. So overall we thought it was 18 pretty good and you'll see later that we think there's 19 enough information you can figure out what's in 20 Chapter 3.

21 Our individual scores on this one after we 22 had that reconciliation of what we were looking for 23 was a five, six and a seven. But when we talked about 24 it, especially because of the way Chapter 3 is put 25 together, we all agreed we would give it a five.

NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

138 1 Next slide which is identification of 2 major assumptions. This is a tough one and Joy 3 pointed out, you know, there are some things we 4 complain about that maybe it shouldn't be a five.

5 But one of the key things they had to do 6 was figure out what to look at. And they found three 7 generic fault scenarios that were very well documented 8 and provided a reasonable baseline for the PIRT.

9 The report I thought, well we all thought, 10 clearly and objectively described the assignment of a 11 hierarchy of phenomena and employment of the 12 methodology. They did very good, better than 13 satisfactory on description of the bases of the 14 phenomenon and defending their importance.

15 There were, however, some unstated 16 assumptions that took this away from being a lot 17 better than good, normal work, good, professional 18 work. They give you the rankings they use but they 19 don't really have, what you ought to do is have a 20 plain English scale that describes what they mean so 21 you're all using them in the same way.

22 They had an unknown ranking and although 23 it's never explicitly stated the way they put the 24 scores together they essentially treated unknown as if 25 this particular phenomena has no value and that's not NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

139 1 right. If it's unknown it could be really important 2 or it might not be.

3 To treat it as absolutely unimportant is 4 putting a pretty strong bias in the report. This is 5 linked to something you'll see later on facilitating 6 the process.

7 They have a rank equation and it kind of 8 makes sense. It goes the right way when you put the 9 scores together. But there's really no justification 10 of why it's especially, why it's the right way to put 11 them together.

12 Excuse me a second. Wow, pardon me. And 13 finally they assume that their three scenarios span 14 the space of high energy arc faults.

15 Some place later they note that the people 16 on the Panel could have looked to see if there were 17 other things they hadn't looked at. But that should 18 have been, we think, explicitly a part.

19 So there were some unstated assumptions.

20 But they documented what they did very well and we 21 came out with a five on that one. Our own scores were 22 five, five and seven.

23 I had a couple notes here. I've already 24 covered those, very good. Next slide is justification 25 of major assumptions. I also had a consensus score of NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

140 1 five on this one.

2 The overall assessment was reasonably 3 good. The authors provided appropriate and useful 4 justification of the assumptions they identified as 5 assumptions.

6 The identification of the phenomena that 7 they were investigating created a little confusion 8 between cause and effect for us as readers especially 9 one of us. And that's a reasonable thing.

10 Later they give enough story that you see 11 that they're evaluating both cause and effect issues 12 and it's a reasonable thing to do. But they didn't 13 introduce that as well as they should have.

14 I already noted the treating of the 15 unknown ranking as having no value is never justified.

16 And it apparently is not recognized. If they're going 17 to get facilitation that should have popped out at us.

18 The rank equation, so we didn't really hit 19 them too hard on the previous one on these things that 20 were assumptions. But over here under the assumptions 21 expanded it a little bit.

22 And the other one is one I talked about 23 before, little explanation of why the three scenarios 24 were picked and why that, you know, we ought to think 25 that stands in place. But on the main issues they did NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

141 1 and on the assumptions they announced they did a 2 really good job of explaining them.

3 So there are some things that push well 4 above a five, some things that push below a five. Our 5 scores here were four, five and six with one of us 6 thinking they really did a pretty good job and one 7 thinking not so good and one thinking down the middle.

8 After we discussed it given where we 9 scored other things related to this we agreed that a 10 five was a reasonable score for this issue. We'll 11 come back to those things that were hidden assumptions 12 later.

13 Next slide is soundness of technical 14 approach. Here they come out a little better. Our 15 final scores were, this was after we readjusted our 16 individual scores.

17 We were a four, a six and an eight. So 18 two of us above five, one of us below it. After we 19 had our discussion we kind of centered back on it, 20 they really did a pretty good job but you've got to 21 dig around to find it.

22 So the first bullet is talking about that.

23 If you take the whole report and you mentally 24 integrate the description of the methodology which is 25 very well done in Chapter 2, the pretty cryptic NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

142 1 summary in Chapter 3, the detailed results in the 2 appendices and the conclusion of Chapter 4 you can 3 evaluate the overall soundness of the report.

4 And we think it's a little better than a 5 good, professional job. The approach used in 6 performing the elicitation from the six experts was 7 sound, well documented and produced a useful product 8 for informing a road map moving forward with HEAF.

9 We have some caveats on that, that will 10 show up on a later score. There are a number of 11 issues associated with the proper role of the 12 facilitator. We have chosen to evaluate all of these 13 under the following section.

14 They could fit in other places like here.

15 But we didn't evaluate them here. We grouped them all 16 in that one place and think that was a reasonable 17 thing to do.

18 I'll take an aside. If you read other 19 PIRTs where you can find some guidance on doing PIRTs 20 you'll find they don't talk much about the facilitator 21 and why that's important.

22 There's vast literature on how you 23 facilitate things like this, things like expert 24 elicitation. If anybody is interested I can provide 25 you a paper or two or just a nice short summary that NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

143 1 links you to the literature.

2 Much of this goes back to the mid-70s when 3 a lot of these ideas were developed. Even the Shack 4 system when you get into that were very sparse on how 5 to do this facilitation well. And if you don't do it 6 well you open yourself up to problems which we'll 7 discuss in a minute.

8 The last one results the objectives on 9 uncertainties and sensitivities addressed. Our scores 10 on this after our readjustment that show the 11 individual score was a three, a five and a five.

12 After we had a discussion and thought where this fits 13 within the scope of everything we came up with a 14 consensus score of a three, not as good as it ought to 15 be.

16 The first one is that they really had a 17 diverse background of people on the Panel who were 18 volunteers from international organizations. Some 19 sent project managers. Some sent technical experts, 20 but it was a mix.

21 But it's got a nice diverse background for 22 gaining different perspectives and addressing and 23 ranking important aspects of the three scenarios they 24 evaluated. This could have enabled uncertainties and 25 sensitivity to be well identified as Chapter 2 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

144 1 indicated they ought to do and addressed quite 2 objectively.

3 On the other hand, the diversity of the 4 experts working essentially independently with the 5 same data, the good thing about that is you don't get 6 a group think and this form of elicitation is 7 effective because it enables objective assessment that 8 accounts for uncertainties and sensitivity.

9 When you get to how things are facilitated 10 however, you really want to bring the group back 11 together and understand why independently they came up 12 with what they did and if there are places where 13 sharing information might be very helpful. In fact, 14 there are a lot of clues in the results that indicate 15 places they really should have delved into.

16 We, our little group, questioned the 17 completeness of the three scenarios which were derived 18 from actual events and say are there other 19 possibilities that are not covered here. And there 20 could be.

21 You can invent some if you think through 22 this thing. There's a little bit on the other side.

23 There were, I'll save that until we get to the next 24 one.

25 This is a continuation. This one NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

145 1 especially bothered me. The evidence of facilitation 2 that, is that could be is pretty strong. Chapter 2 I 3 think fairly thoroughly addresses this.

4 We all agreed on this. But it appears 5 that no one forced the experts to identify the 6 uncertainty in their own evaluations. It was also not 7 clear how the experts were advised and they were 8 advised to base their importance ranking on risk.

9 But if you're not a risk an analyst and 10 haven't done a lot of study they should have given you 11 some guidance on how to do that well. They may have 12 done it informally. But not even a hint of it shows 13 up in the report.

14 Chapter 2 gives a hint about how to seek 15 consensus. But there's no discussion of consensus 16 building. In cases where rankings span the full range 17 and this happened quite often, I forget what their 18 scale is.

19 But say it was one to five. On the same 20 elicitation one person has a five, another had a one.

21 They just leave it that way and use it. But that's a 22 real hint, as I said, we found out in our own that 23 they're evaluating different things.

24 The facilitator should have taken those 25 and said, okay, each of you explain why you give it NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

146 1 the score you did and when you go through that process 2 you see you're doing things differently. You're 3 looking at different aspects of this issue.

4 Sometimes the right thing to do is say 5 there are two different things here. Let's break this 6 into two separate elicitations. And you usually find 7 that then you come much closer together.

8 Sometimes you would find that one person 9 had access to information that not everyone had access 10 to and when you share that you do better. The 11 facilitator ought to dig into this stuff when it looks 12 funny and help seek resolution.

13 There was another area where they, on a 14 few issues said we don't know enough and didn't make 15 the recommendation. The facilitator could really help 16 with that pushing on what you do know, what you don't 17 know, do we need some outside expertise to come in and 18 help you with it.

19 And that sort of thing wasn't done.

20 Here's our final summary. Now remember, the scores 21 you see under the three of us were after we had first 22 decided how we needed to evaluate each of the points 23 that we judged and these scores are much closer 24 together than our original scores were because we were 25 evaluating the same thing.

NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

147 1 We didn't just do arithmetic on these 2 because there is a serious problem with just doing 3 arithmetic on these kind of scores. And these are 4 fairly close together so it's not as big a problem.

5 But if you have a very high outlier and 6 you do an average, a mean value the high outlier 7 drives the answer. You can play with some numbers and 8 see that.

9 And then you say well instead of that 10 let's do a geometric average. Well, if you did the 11 geometric average that kind of takes care of the high 12 outlier but now the low outlier drives the show.

13 And if you play with that you'll see 14 that's true too. We come together and discussed these 15 and say we think a reasonable place the community 16 would come together on this is the clarity of 17 presentation of five.

18 After we get all our scores we go through 19 and do the weighting as indicated in the methodology 20 and we come up with a final score. So that's our 21 whole story.

22 We have to write text around it. I am 23 still a little slow and bleary so I am not sure I will 24 get that done in October. There's a good chance I 25 will.

NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

148 1 If I don't we'll certainly be able to 2 resolve this in November and pull it together. My 3 expectation is I'll be with you folks in person in 4 November. But that still remains to be seen.

5 I've got a lot of things to go through 6 before they say, yes, hop on an airplane, go back 7 there. That's all I have to say.

8 MEMBER REMPE: Thank you, Dennis. Do any 9 of the Members have any comments or, about the ranking 10 and the information that Dennis has presented here?

11 So actually since, did you want to say 12 anything to add to what Dennis has presented?

13 MEMBER SKILLMAN: No. I support Dennis.

14 I looked at the slides, you know, he sent them out for 15 comments and Walt and I got back to Dennis and I'm 16 comfortable with Dennis' explanation of what we've 17 done.

18 MEMBER BLEY: Walt agreed with that as 19 well. I mean the slides come from our discussion, 20 pretty much straight from the discussion onto the 21 paper.

22 MEMBER REMPE: So I have one comment. And 23 again, it's just, when I looked at the report and what 24 I was thinking about. And it's not, some of your 25 comments are actually just general guidance that I NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

149 1 think is useful for PIRTs in general.

2 And so I guess that's a comment, why I'm 3 giving this comment is that with respect to unstated 4 assumptions about the usefulness for the PIRT for 5 future research because the report actually, although 6 maybe it wasn't a stated assumption, they did go 7 through and talk about some of the limitations of the 8 information that they extracted from this PIRT, like 9 country specific influences.

10 And I think that again they took care of 11 it in this document. But it might be something worth 12 highlighting in your text or something along that line 13 is why I'm bringing up that discussion because right 14 now we're hearing that the accident tolerant fuel 15 program will be soliciting and performing PIRTs to 16 guide that research.

17 And so I think any thoughts we have that 18 are generally good thoughts or guidance on PIRTs could 19 be documented here too. Any thoughts about that, 20 Dennis or Dick?

21 MEMBER BLEY: Well it's, I kind of like 22 the idea of the country specific stuff. And I will 23 try to read some of that in. I'm a little unsure of 24 us giving guidance in a review of the research report.

25 If we want to maybe we ought to do a NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

150 1 separate letter that says, you know, there are some 2 things that are missing in general in this PIRT 3 process as implemented by some people. Now I think if 4 you read their Chapter 2 all of these things that I'm 5 talking about should have been done.

6 But it wasn't explicitly covered on some 7 things. So it, we can put a little outline of that.

8 We can put a little outline of what the facilitation 9 would be.

10 It seems like the wrong place. But I'm 11 not sure of that.

12 MEMBER REMPE: Well actually another point 13 they raised is sometimes when they do a PIRT the cost 14 of getting the data influences how the experts weigh 15 the importance of certain data. If they know it costs 16 a lot they basically didn't rank it very highly even 17 though it might be important to have.

18 And so some of those insights I thought 19 were good insights. And so I'd like to see it 20 somewhere in your write-up. But it's up to you on 21 what you guys want to do. Any other comments?

22 MEMBER BLEY: I don't quite remember that 23 one. I have to go back and try to find that.

24 MEMBER REMPE: Yes, I thought it was kind 25 of an interesting nugget.

NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

151 1 MEMBER BLEY: I don't remember they said 2 it cost too much so we shouldn't think about it.

3 MEMBER REMPE: They said it was like an 4 underlying influence on how the experts, I'm pretty 5 sure I saw that somewhere in the report because I 6 think that might be a true statement just in general 7 about person, it's a limitation that they noticed.

8 MEMBER BLEY: I'm going to have to go find 9 that because that's a, that's really, you know, this 10 shouldn't be a cost thing. This ought to be saying 11 for what's important here's what you ought to do then.

12 Then a manager says well, that costs too 13 much, I can't do that even though it would be very 14 important to us. But it shouldn't affect the scores 15 they give them. And I missed that if that's in there.

16 MEMBER REMPE: Well I don't think they 17 encouraged people to do that. They just observed that 18 it was an underlying factor that influences experts.

19 And so I'll try to find that, the actual quote and 20 send it to you. But I thought it was --

21 MEMBER BLEY: That would be helpful. And 22 it might show up in the place I was talking about 23 where they said we didn't have the expertise to 24 address this issue.

25 And I think that falls, can be included NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

152 1 under discussion of what good facilitation for this 2 kind of process would be which is missing from many of 3 the lists of advice for doing this, for doing 4 elicitation, et cetera.

5 MEMBER REMPE: So my notes indicate --

6 MEMBER BLEY: If you can find that I'd 7 like to highlight it and I can, I know I can link it 8 to the, we didn't have the right experts here issue.

9 MEMBER REMPE: My notes indicate it was in 10 Chapter 4. But I'll find the exact location for you.

11 MEMBER BLEY: Thanks.

12 MEMBER REMPE: If there's no other 13 comments from anybody.

14 MEMBER SUNSERI: I had one question really 15 for the group. Can someone remind me what the scoring 16 scale is? I mean a five represents good, sound, 17 technical work. Is that right?

18 MEMBER BLEY: That is correct.

19 MEMBER SUNSERI: So the overall assessment 20 of this Working Group is that this product is a sound 21 technical product but has some room for improvement 22 based on the recommendations you're making.

23 PARTICIPANT: That is correct.

24 MEMBER BLEY: It would be, it's useful for 25 going ahead and prioritizing the research. But you NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

153 1 could do better, yes.

2 And, Matt, if you look on the second page 3 of Chapter 2 of our report where our methodology is 4 laid out there's a table that tells you how the scores 5 go from outstanding to unacceptable.

6 MEMBER REMPE: And Margaret actually 7 included in her --

8 MEMBER CHU: In my presentation I have a 9 slide on that.

10 MEMBER REMPE: Page 7.

11 MEMBER CHU: Yes.

12 VICE CHAIRMAN RICCARDELLA: This is Pete.

13 I hate to --

14 MEMBER BLEY: In our report it shows up in 15 Chapter 2. We didn't replicate it.

16 VICE CHAIRMAN RICCARDELLA: This is Pete.

17 I hate to show my ignorance. But what does PIRT stand 18 for?

19 MEMBER REMPE: Phenomenon importance 20 ranking.

21 MEMBER SKILLMAN: Phenomenon 22 identification and ranking table. I was going to just 23 make one comment, Joy, and perhaps this will support 24 what Dennis is doing, has done.

25 This was not our review of the research.

NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

154 1 It was our review of the PIRT process. And so we 2 really had two different bundles of information. One 3 is the data that research has provided including that 4 excellent video of the aluminum bus exploding.

5 But then we had the documentation of that.

6 And this was a review of the documentation of the 7 PIRT.

8 MEMBER REMPE: That's what it should be, 9 yes.

10 VICE CHAIRMAN RICCARDELLA: So we were, if 11 you will, swapping between glasses and binoculars 12 making sure we were keeping clear in our mind which 13 one we were evaluating. And it is the PIRT that we 14 were evaluating.

15 Now there was abundant, excellent, 16 remarkable data. But as stunning as the data was we 17 were trying to evaluate the PIRT of that data. And at 18 least I found that to be kind of a wrestling contest 19 in my mind.

20 The other thing that, about what would be 21 in our letter relative to the PIRT, I think Dennis has 22 identified the three or four items regarding 23 uncertainty which when captured will help the next 24 effort for those who would evaluate a PIRT.

25 But that will be in the record so we can NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

155 1 capture it for our letter. So I think the main things 2 that Dennis was trying to capture will be in the 3 transcript.

4 VICE CHAIRMAN RICCARDELLA: Are the 5 uncertainties relative to the data or to the PIRT?

6 MEMBER SKILLMAN: To the PIRT, how they 7 handled it.

8 VICE CHAIRMAN RICCARDELLA: All right.

9 MEMBER REMPE: So if there's no other 10 comments or questions should we move on to Margaret's 11 group?

12 MEMBER CHU: Thank you.

13 MEMBER REMPE: Thank you, Dennis, and to 14 his team and to Walt and Dick also.

15 MEMBER CHU: We'll move on to the second 16 review project. We have a review panel of three; 17 Jose, Vesna and myself. And then the research project 18 title is correlation of seismic performance in similar 19 structures, systems and components.

20 It's NUREG CR-7237, next please. I'll 21 give you a little background. When an earthquake 22 occurs near a nuclear power plant it subjects all the 23 SSCs to ground motions and it depends on the level of 24 the ground motion one or more failure of SSCs could 25 occur.

NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

156 1 So the Seismic Probabilistic Risk Analysis 2 requires an estimation of possible correlations among 3 seismic failures of similar components. This 4 correlation is a large area of uncertainty.

5 And frequently, actually all the time 6 almost, very simple assumptions are made that 7 component failures are either fully coupled or 8 completely uncoupled depending on the design and the 9 relative locations of the SSC in the plant.

10 So basically as you have two equipments or 11 components that are co-located to each other you 12 assume they are 100 percent dependent on each other.

13 They fail, either they don't fail or fail 14 simultaneously otherwise they are totally independent.

15 Now this project was actually part of the 16 research office 2010 to 2014 Seismic and Structural 17 Research Plan. And then the project was initiated to 18 determine the impact or correlation assumptions of 19 risk estimate and to recommend a better approach that 20 may reduce uncertainty.

21 Next please. Now this project consists of 22 four required tasks from the Research Office. The 23 first is the review of SPRA, seismic PRA in literature 24 to understand how people treat correlation and the 25 impact of correlation assumptions on risk estimates.

NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

157 1 The second task is review of existing 2 literature on seismic correlation analysis 3 methodology. The third is the review of existing data 4 from earthquake experience and shake table test for 5 their usefulness in the correlation issues.

6 The fourth one is with the help of experts 7 in a series of workshops, actually there were two 8 workshops, recommend methodology that better addresses 9 correlation issues. Next please.

10 Before I go to the scoring I just want to 11 briefly summarize what are the results of this 12 project. First one from review of existing seismic 13 PRAs seven categories of SSCs are judged to be more 14 sensitive to correlation assumptions.

15 I'm going to have another view graph just 16 after this. Now the earthquake data shake table test 17 are found not to be useful for this project.

18 And from literature reviews of 19 methodologies to treat seismic correlation, the 20 research team identified the Reed-McCann methodology.

21 That's from, I think 1985, also called separation of 22 independent and common variables methodology.

23 And that is recommended for further 24 research. Next one please. When I say they identify 25 seven categories of SSC that are judged to be NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

158 1 important to correlation assumptions what the team did 2 was upon reviewing the seismic PRAs these seven 3 categories were found to first dominate seismic risk 4 contributors.

5 And secondly, they were judged to have a 6 high degree of a potential correlation because of 7 their numbers within the plant and their typical 8 locations within the plant.

9 And this is the list of seven things:

10 masonry walls; electrical tanks; mechanical batteries 11 or racks and so on. Next please.

12 VICE CHAIRMAN RICCARDELLA: Question, 13 Margaret.

14 MEMBER CHU: Sure.

15 VICE CHAIRMAN RICCARDELLA: Did they 16 actually look in those PRAs where they had considered 17 full correlation and then did the same analysis with 18 no correlation and, I mean we saw numbers yesterday 19 like 30 percent to 70 percent potential difference?

20 MEMBER CHU: I think they did.

21 MEMBER DIMITRIJEVIC: I think my 22 impression is this is not easy to do because I think 23 PRAs are already complex. And just to do sensitivity 24 study like that it would require a lot of 25 manipulation.

NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

159 1 So I think whatever they did has to be 2 some simplified way of looking, you know, between the, 3 how was correlation treated. I didn't get the 4 impression how exactly they did.

5 But they say it's complex and a not easy 6 project. That would be one research project of itself 7 to do sensitivity studies like that especially --

8 VICE CHAIRMAN RICCARDELLA: Do the people 9 who do PRAs normally do sensitivity studies?

10 MEMBER DIMITRIJEVIC: The couple I am 11 familiar with tried to do that by doing the scale, you 12 know, the scale correlation factor which is between 13 zero and one. So they can then easily do the zero and 14 one.

15 It wasn't this basic assumption. That's 16 a little different method. If then they discover 17 strange results they actually, and then spend two 18 weeks trying to understand and didn't really come to, 19 the result was actually you will assume the full 20 correlation with a reduced, the highest CDF it 21 actually produced a lower CDF.

22 VICE CHAIRMAN RICCARDELLA: Really.

23 MEMBER DIMITRIJEVIC: And it came out 24 because sometimes in the model when you separate this 25 occur you go into different branches. And when you NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

160 1 put this together in correlation it was a very 2 complicated case.

3 But this can also, it is not necessary 4 that one is more conservative than the other. It 5 often depends on the model of the PRA. But that's my 6 personal experience.

7 This was not discussed in the report in 8 the task. That's one of our comments you will see 9 that we were not sure there was proof importance of 10 that.

11 VICE CHAIRMAN RICCARDELLA: So it was 12 basically judgment the factors that they said could 13 affect CDF by this much was basically from judgment 14 not from action?

15 MEMBER DIMITRIJEVIC: Well looking in that 16 in sequences and concepts, yes. I don't think that 17 was detached sensitivity study. But I'm sure there 18 was technically some. That's my --

19 MEMBER MARCH-LEUBA: I thought they told 20 us yesterday that the state of the practice was to 21 send it to see the one and see the difference and 22 that's how they got to that 30 to 60 percent. But 23 you're the expert. That's what I thought I heard.

24 MEMBER DIMITRIJEVIC: Yes. When we had 25 discussion with them.

NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

161 1 MEMBER MARCH-LEUBA: That's what I thought 2 I heard, but I'm biased.

3 MEMBER CHU: I have a feeling there's a 4 lot of judgment in there because they went to, with 5 the experts in the workshop on these. So that made me 6 think it's sort of a, judgmental in many ways, yes.

7 VICE CHAIRMAN RICCARDELLA: They didn't 8 present any hard data results in the report.

9 MEMBER CHU: No.

10 MEMBER DIMITRIJEVIC: And it's not an easy 11 study to do. The model is complex.

12 MEMBER CHU: Okay. And then the Reed-13 McCann methodology is the recommended separation of 14 independent and common variables methodology. In this 15 methodology they develop a procedure to estimate 16 dependency between component failures by searching for 17 common sources of variability in the response and 18 strength calculations.

19 And then the analyst needs to carefully 20 examine the component design and qualification 21 documents, material properties, installation methods, 22 et cetera to make judgment on what are the common 23 variables and then decide what's not.

24 And so we as a panel feel the methodology 25 could be difficult to implement and it could vary from NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

162 1 analyst to analyst because it's such a judgmental 2 cause.

3 Next one please. Now just to refresh 4 everybody's memory on what those scores mean as Matt 5 requested, five is really the baseline. It's a 6 satisfactory score. It means you have satisfied the 7 research objectives.

8 And then it goes up and down, you know, 9 from zero to ten and a zero being unacceptable, ten is 10 outstanding. But five is basically a satisfactory 11 score. Next please.

12 And then these are the individual scores 13 from the three of us. I am number one. Jose is 14 number two. Vesna is number three. You can see that 15 mostly that we're pretty even, okay.

16 So what we did was basically take an 17 average of the three scores and then everybody agrees 18 is this about right. Sometimes we would round up a 19 little bit, sometimes we round down a little bit, 20 okay.

21 Next please. And then the five areas of 22 measure, performance measures is, Dennis already went 23 through this. Clarity of presentation, identification 24 of major assumptions, justification of major 25 assumptions, soundness of technical approach, NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

163 1 treatment of uncertainty sensitivities.

2 And then there are predetermined weighting 3 factors for each. You notice that the soundness of 4 technical approach and the results basically is 50 5 percent of the overall score.

6 And then with this our consensus final 7 score is five, satisfactory. Next please. I'm going 8 to go through a little bit on each of the performance 9 measures, yes.

10 The first one, clarity of presentation it 11 gets a six. And we feel the report is very well 12 written and easy to understand. It clearly 13 communicates the purpose, scope and technical approach 14 of the project.

15 Existing methodologies that deal with 16 correlations clearly presented the rationale for 17 selected methodology is clearly articulated. We did 18 find a few deficiencies in the report writing.

19 For example, the organization didn't 20 follow the task sequence that was dictated. And then 21 there are also minor problems with the definition of 22 captions in tables or figures.

23 But overall, we felt it was a good report.

24 So we gave it a six. Next please. Now identification 25 of major assumptions. We kind of struggled a little NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

164 1 bit with this one because basically the project, the 2 whole project is a literature review and literature 3 assessment.

4 So we, you know, so we felt the major 5 assumptions are not clearly identifiable. So we 6 looked through examples of the assumptions discussed 7 in the report, okay.

8 These are the three big ones. The first 9 one is the correlation assumption is important to risk 10 results and risk insights from seismic PRA. And then 11 the second one is that thumb rule of independence, 100 12 percent or zero percent that is being used right now 13 is unsatisfactory and needs to be improved.

14 And the third one is the recommended Reed-15 McCann methodology would require assumptions on design 16 qualification, installation and so on of SSCs. Next 17 one please.

18 VICE CHAIRMAN RICCARDELLA: Those were 19 stated as assumptions in the report?

20 MEMBER CHU: No, they were kind of buried 21 all over. And we kind of identified them.

22 MEMBER REMPE: It's simpler --

23 MEMBER MARCH-LEUBA: There is no chapter 24 that says assumptions.

25 MEMBER REMPE: And I note that's similar NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

165 1 to what Dennis' group had to do too. Both of these 2 projects were a bit difficult and this is the way the 3 group decided to do this rather than eliminate the 4 metrics.

5 MEMBER CHU: Yes, because it is not the 6 usual research project. You make assumptions and you 7 do this and you do that. So that's the best we could 8 do.

9 Now as to justification of major 10 assumptions remember the three listed before. We felt 11 there was one that is not rigorously justified in the 12 report which is the correlation assumption is 13 important for the seismic PRA results.

14 This is a very major assumption. You say 15 is it important or is it not. The report says it is 16 important but we felt it's not rigorously justified.

17 The following bullets are examples of the statement.

18 These are the statement from the report.

19 It says, for example, correlation assumptions may not 20 significantly impact the seismic PRA results but it 21 could impact the risk insights. But it didn't quite 22 go into what that meant.

23 And second one for some seismic PRAs the 24 difference in seismic CDF could be as much as almost 25 a factor of two. Typically it was a difference of 30 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

166 1 percent to 60 percent.

2 And then it says in some SPRAs the 3 dependent failure issues is not as important when the 4 overall seismic CDF is dominated by an accident 5 sequence that itself dominated by a PRA singleton, 6 that means a single failure.

7 And then it says for some key accident 8 sequences the difference could be as much as a factor 9 of two to four in the frequency of, notice it's the 10 sequence, the frequency of the sequence. So with sort 11 of these vague and all over kind of statements our 12 panel wonders are these factors or percentage changes 13 in the PRA number numerical results significant enough 14 to justify to the time consuming and costly new 15 methodology.

16 VICE CHAIRMAN RICCARDELLA: Is a factor of 17 two on CDF considered significant? I mean it seems to 18 me that's probably in the uncertainty band.

19 MEMBER REMPE: But, Pete, they didn't say 20 it was, typically it was more 30 to 60 --

21 VICE CHAIRMAN RICCARDELLA: Thirty to 60.

22 But even if it were two.

23 MEMBER DIMITRIJEVIC: It's upper bound for 24 significant.

25 VICE CHAIRMAN RICCARDELLA: You know, if NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

167 1 it's two it's not.

2 MEMBER BALLINGER: I read the document 3 like four times and could for the life of me not 4 figure out what the actual uncertainty would be. But 5 I knew it was large.

6 MEMBER REMPE: Yes.

7 MEMBER BALLINGER: And so a factor of two 8 compared to PRA results which are sort of like fatigue 9 results is a magnitude one way or the other.

10 MEMBER CHU: So therefore you notice this 11 one got the lowest score, four, okay. And the next 12 one is the soundness of technical approach and 13 results. And then we feel that the tasks accomplished 14 were defined and then it was done competently.

15 And then these other tasks that are done 16 as required by the Office of Research and then they 17 did every single one of them, okay. And so overall 18 it's done well with the exception of the previous 19 slide, okay.

20 Next please, treatment of uncertainties 21 and sensitivities. It gets a five score. We believe 22 the project team has appropriately considered 23 uncertainty and sensitivity.

24 Some sensitivity analysis were performed.

25 The report addresses how to obtain the uncertainty NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

168 1 distribution for the results. In the Reed-McCann 2 methodology the analysts have to deal directly with 3 common variables and their epistemic uncertainty and 4 random variability.

5 I don't know how to pronounce that word, 6 yes. And since expert judgment is needed in the 7 partition in between the independent and dependent 8 parts the proposed methodology are introduced new 9 model uncertainty, okay.

10 Next please. Now we have some concluding 11 remarks. I'm going to turn this most important slide 12 to Jose and Vesna.

13 MEMBER MARCH-LEUBA: Vesna gave me the 14 permission of starting. So there are three 15 conclusions that we arrived at only on the scope.

16 On the first bullet the name that sticks 17 out is competence. When you read this NUREG you feel 18 the competence of the people that were doing it. And 19 when we interviewed the staff and we tried to obtain 20 more information you really felt that they know what 21 they're doing.

22 So that has to be stated. This was a 23 competent effort. I'm sure it was the only good thing 24 we can say about the report because it's like point 25 number two says that even though it is a good NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

169 1 mathematical method and it will work it's almost 2 impossible to obtain data to apply it because we have 3 difficulty getting the data for the known correlated 4 seismic PRA.

5 You have such uncertainty. Now you bring 6 uncertainty too into two, one that is correlated and 7 one that's uncorrelated and it's almost impossible to 8 get because all the shaker table experience that, 9 doesn't tell you anything at all because you never 10 shake it to failure.

11 So they never fail. So you never know how 12 they correlate together. And experience from real 13 life earthquakes has so much variability that you 14 really cannot tell.

15 So we find that this would be very 16 difficult to implement with accuracy. And actually 17 you would increase the uncertainty of the results.

18 On point number three is what we've 19 already been discussing over the whole presentation.

20 We questioned whether it is worth, given all the 21 uncertainty that we're going to increase and at best 22 we're going to get a factor of two, more likely 30 to 23 60 percent.

24 Is it worth implementing it? And Vesna, 25 you probably can tell us what a factor of two buys you NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

170 1 in this one.

2 MEMBER DIMITRIJEVIC: These factors, 3 numerical values it's, they, I mean obviously the 4 factors of uncertainties and the importance measures 5 in the PRA are not so significant request. But our 6 main impression was when the goals for this project 7 were assessed this report was a pleasure to read.

8 And I'm very thankful to doubters because 9 they have the good skills. They do a lot. They put 10 a lot in the report.

11 However, when they defined the goals in 12 the beginning that's exactly what I would want to 13 know, you know, if I was concerned about correlation.

14 Is it important?

15 And you have a feeling that it should be 16 important, right just like assumptions the components 17 failing the same time or, you know, that it's in the 18 patterns between them. And then to say they're going 19 to look in the data to see can they justify something 20 and then they propose the method.

21 And if they said in the beginning we're 22 going to look in the different method, give you an 23 overview of that and tell you what we see in the 24 industry and don't set this goal this report would get 25 much higher value because it was our feeling that they NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

171 1 did not introduce goal in the way, as Margaret is 2 showing that slide, they didn't really make a 3 statement of how important it is.

4 When you read what they did you realize 5 yourself but it's not an easy task and it will require 6 much more effort. So they could not really tell us 7 what's the importance of using one co-relationship, 8 whether it's other or not using it at all or, and what 9 would be benefit for this which is very complex and 10 costly method which has to be done with a person who 11 knows fragility very well.

12 And there is not too many people we bring 13 in to do these tasks. And to be, what is really true 14 that doubters are honest. They didn't overstate it.

15 They did not tell us, hey, this is 16 important, you have to do it. They did not tell us 17 these assumptions are bad or simple assumptions. They 18 were basically honest.

19 But they didn't reach their goal because 20 they set this goal very high. If they say listen we 21 look in the literature. We cannot conclude because of 22 blah, blah, blah.

23 We look in data. We couldn't find the 24 data to support. They said that in the project from 25 the data or from because you always ask, Joy, if there NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

172 1 is enough data in industry so that we can conclude 2 something about the co-relationship.

3 And they say but we present to you a lot 4 of, you know, not a lot but some selection of method 5 and we think that this method has promise. It would 6 be different type of evaluation if that was stated as 7 the goal.

8 So we couldn't really basically conclude 9 that the data, that anybody would really, you know, 10 want to invest that this report, basically they have 11 a problem that some correlated components dominate the 12 risk. They may want to look into this methodology.

13 PARTICIPANT: You know, I think the key 14 words are on one of your slides. Slide 12 is the 15 impact risk insights.

16 And I think that's maybe my opinion about 17 PRA in general is it's not so much, I mean you don't 18 design things based on you've got to get 10-7. But 19 you get some insights, some general insights.

20 MEMBER DIMITRIJEVIC: Yes, and that's and 21 we were hoping they would provide some examples. I 22 can think myself of some examples of this risk insight 23 would be that you conclude then that, let's say you 24 were like you could measure intake structure.

25 The intake structure is extremely NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

173 1 important for, you know, the Westinghouse plants. But 2 however, if you have a service for the pumps which are 3 all on the same rotation and the same design that co-4 relationship between those pumps may be more important 5 than intake structure.

6 That could be one sort of insight because 7 insights from PRA in identifying important sequences 8 has important components, human actions and things 9 like that. But they never, the only way they mention 10 that is in that one sentence.

11 It was never explored further in the 12 report.

13 MEMBER MARCH-LEUBA: I think, Peter, your 14 recommendation, your comment is very wise. And even 15 though when we write the report which Margaret has 16 volunteered to do we maybe, our job is to grade the 17 report.

18 But maybe we should make a recommendation 19 that it, this methodology would be more valuable 20 applied for insights than for quantitative sharpening 21 of the pencil. We don't think the sharpening of the 22 pencil by itself is worth it.

23 But the analysis tells you what components 24 you can improve especially for new reactors with 25 multiple modules.

NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

174 1 MEMBER SKILLMAN: I want to ask a question 2 here. Jose, you used the, you expressed the comment 3 because the shaker table doesn't take the device to 4 failure we end up, I think, not knowing or a comment 5 like that.

6 And that is a comment that was also made 7 yesterday afternoon.

8 MEMBER MARCH-LEUBA: That is correct.

9 MEMBER SKILLMAN: I'd like to try to flip 10 that at least in challenge. When general design 11 criteria were set out, the revised general design 12 criteria was 1969 and 1970.

13 In '70, '71 and '72, I bet you would 14 remember this, we were doing seismics. And we were 15 doing square root some of the squares three 16 dimensionally, two horizontal, one vertical.

17 And Reg. Guide 1.48 was produced and that 18 was active seismic testing. And then came Reg. Guides 19 1.26 and 1.29 that identified quality classification 20 levels one, two and three and what was seismic one, 21 what was seismic two and seismic three.

22 And I know for a fact because I was part 23 of it the team at VNW, the team at Westinghouse, the 24 team at combustion and the team at GE we were all the 25 plumbers and we were buying, we were designing and NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

175 1 buying basically ECCS and all the auxiliaries.

2 And scramble became wood shaker table 3 tests are available. And there was one set available 4 at the University of Alabama in Birmingham. There was 5 one set of all of them.

6 There was a lot of competition to get on 7 those shaker tables. And we shook HPI pumps, 13 stage 8 and 12 stage pumps and we shook heat pumps. And I 9 know Bar W was doing the same thing, combustion was 10 doing the same thing.

11 And when those shaker table tests 12 concluded for the ground floor response that we put 13 into the shaker tables that if those devices did not 14 fail that was a success. So now and we said fine, so 15 we installed this equipment all over the product line.

16 Now to say that because it didn't fail on 17 the shaker table sounds to me like fails in 18 correlation, sounds to me it's almost the opposite 19 because it didn't fail on the shaker table what should 20 conclude that remains operable if the earthquake 21 envelope has not been exceeded at that location for 22 that amount.

23 MEMBER MARCH-LEUBA: The failure of one 24 and all components would be single.

25 MEMBER SKILLMAN: Let me go further. I NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

176 1 think that same argument is the argument that wins on 2 passive components. And that is your piping and the 3 other robust components that also do not fail on that 4 seismic envelope.

5 Now so I'm struggling with because it 6 didn't fail on shaker table it's somehow not 7 compliant.

8 VICE CHAIRMAN RICCARDELLA: No, no, not 9 that it's not compliant. It just didn't give you any 10 information about seismic correlation.

11 MEMBER SKILLMAN: It gave me a whole lot 12 of information about that component.

13 VICE CHAIRMAN RICCARDELLA: About that 14 component, yes. But it doesn't tell you whether if 15 you have three other components.

16 MEMBER RAY: Dick, listen, the issue isn't 17 will it meet the design basis level of shaking. The 18 question is at what point above that does it fail.

19 And then I would tell you as somebody who has run more 20 shaker table tests than everybody here put together 21 that failure isn't a single defined term.

22 You're talking about the dam foundation 23 failing, you're talking about a breaker within the box 24 failing. What are you talking about failing?

25 So trying to gather the data and define NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

177 1 what failure you're talking about when there are 20 2 different ways that something can fail, it's a complex 3 device and we had to do all of our own because we had 4 a two-thirds GE plant, we had to do all of our switch 5 gear, everything is, I just agree with Jose and I'll 6 shut up.

7 VICE CHAIRMAN RICCARDELLA: You agree?

8 MEMBER RAY: I agree that trying to get 9 the data that you need through testing, I mean it's 10 different to talk about how the flow and response 11 spectrum is different at different elevations in the 12 building and things like that.

13 But to talk about using a PRA in which you 14 have failure data to project the likelihood of they're 15 not being a failure at some level of shaking above the 16 design earthquake, which is what you need for a PRA, 17 is to me it's a hopeless enterprise.

18 MEMBER SKILLMAN: You know, I agree with 19 that. If you're trying to project failure there's 20 never enough data. I agree with that.

21 VICE CHAIRMAN RICCARDELLA: If, okay, go 22 ahead. You know, you and I have both done a lot of 23 fatigue analysis, right. We computer the cumulative 24 fatigue usage. And so we have a fatigue usage curve 25 or a fatigue design curve data.

NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

178 1 MEMBER SKILLMAN: We've got a cuff and we 2 know how to use it.

3 VICE CHAIRMAN RICCARDELLA: Imagine if 4 when you were setting up this fatigue analysis that 5 you said well my component needs to take, my pipe has 6 to take 400 cycles of this and 50,000 cycles of that.

7 And so I take my test specimen and I sampled it and I 8 tested it for 400 cycles of this and 50,000 cycles of 9 that and said it's fine, you know.

10 You're not getting any information versus 11 when you take these specimens and you fail them and 12 you see that, wow, you get a huge degree of scatter in 13 the actual failure data from, you know, half a million 14 cycles to five million cycles.

15 That is information in testing to failure 16 that you don't get by that proof test.

17 MEMBER SKILLMAN: I agree with that. I 18 certainly agree with that.

19 VICE CHAIRMAN RICCARDELLA: That's all 20 we're talking about.

21 MEMBER SKILLMAN: Okay. So we're talking 22 about testing to failure is about, as opposed to 23 testing for competency.

24 MEMBER DIMITRIJEVIC: Yes, this is a very 25 good moment actually because we can, Harold introduced NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

179 1 something which shows seismic PRA complexity when it 2 comes to this. The seismic PRA there is a failure 3 probability for let's say ten.

4 Usually the seismic intensity is picked 5 in ten intervals. And it is tightened for different 6 ground acceleration rate, the ground acceleration.

7 In these ten intervals, the last intervals 8 that the components are actually failing are least 9 important from correlation because all the failure 10 probabilities are very high already, .9 or something.

11 And therefore, the correlation which is 12 not as significant and the frequencies of occurrence 13 of that initial event are very low. And they say this 14 report, the co-relationship is most important in the 15 low or medium range of the seismic where the 16 components are not failing but they may fail.

17 Probability of that failure can be 10-2 or 18 something. And it's not clear what it is. How do you 19 measure for that failure probability because you can 20 only say component didn't fail or it doesn't fail on 21 the shaking table?

22 The same thing when you have a Fukushima 23 you can go around and check how many components 24 actually failed. But that doesn't tell you anything 25 about failure probability.

NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

180 1 Then you have a different failure mode.

2 Will it fail to start? Will it fail to run? It's 3 incredible. You have a ground acceleration, but it's 4 a million combinations. It's very difficult to get 5 from data.

6 VICE CHAIRMAN RICCARDELLA: If anybody is 7 interested in delving into this in more detail Dennis 8 sent me an old 1988 report and it's kind of like a 9 fundamental, you know, it's a paper on seismic, 10 probabilistic seismic risk analysis.

11 It's very, very interesting. It's Kennedy 12 and Cornell and some of these guys that were, you 13 know, I could distribute that if people are 14 interested. And it was a lot easier to read than that 15 NUREG. It didn't get into Boolean algebra.

16 MEMBER SUNSERI: I would like to add some 17 perspective here or a perspective. So those of you 18 who know me I'm not much of a curmudgeon outside of 19 the meeting.

20 So I just want you to recognize what chair 21 I'm sitting in and think back to past meetings and so 22 my curmudgeon remarks might make more sense. So 23 research, why do we do research?

24 And the purpose of this review is to 25 provide feedback to the research organization to help NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

181 1 them do better research, right? So when I listen to 2 this conversation and I read the report and I think 3 about well research is going to be done to explore an 4 area that we have some unclarity or uncertainty of 5 we're going to isolate.

6 We're going to research. We're going to 7 review and we're going to elevate the level of 8 understanding of a certain topic so that decision 9 makers can apply that in a way and make better 10 decisions going forward.

11 That's fundamentally why we do research.

12 So when I see the conclusion that, you know, maybe we 13 have correlated or uncorrelated, it takes a lot of 14 effort, what do we do, is it going to make any 15 difference it causes me to have pause about the value 16 of this research or whether it was really actually 17 research or is this a report on a various topic.

18 So taking this a little further --

19 CHAIRMAN CORRADINI: You mean a report in 20 progress versus some strong conclusion?

21 MEMBER SUNSERI: So I'm going to go back 22 to the statement justification of assumptions which 23 got the lowest score here which it seems to me the 24 whole purpose of this research would have been to 25 identify your assumptions, isolate them and then run NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

182 1 a process so that we get a sound technical result that 2 decision makers can use to make better decisions.

3 So I think the fact that this got high 4 marks for sound technical approach but low marks for 5 justification measures assumptions is a big 6 disconnect. And when we look at the final results I 7 don't think that this "research" is much use to 8 anybody because whether you correlate it or not 9 correlate it, it doesn't make that much difference as 10 the report says.

11 So I would have given it an overall much 12 lower score, maybe marginal because what, I mean what 13 did it really do from a research perspective? I think 14 that techniques that have been outlined here about 15 starting from a fundamental saying we've got to, you 16 know, there's too much variability in just looking at 17 literature or random earthquakes.

18 Let's define a specific systematic 19 assessment way to look at correlation, put stuff on 20 shaker tables, use different configurations, use 21 configurations that we know would be fully correlated 22 and shake it.

23 Use situations that are not correlated, 24 shake it. See what the results are and draw some 25 conclusions like that. So that wasn't done. So NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

183 1 therefore I'm not sure, one opinion, what research 2 this accomplished.

3 MEMBER CHU: But, Matt, you know this has 4 given to them by the Office of Research.

5 MEMBER SUNSERI: So I'm just telling you 6 --

7 MEMBER CHU: We kind of --

8 MEMBER SUNSERI: I don't disagree with 9 that. I'm not going to, and that is a fact. But my 10 point is though that's a bad start right, you know.

11 So if we're going to give feedback to the 12 research department on how to do better research maybe 13 there needs to be some comment around clarity of that.

14 MEMBER DIMITRIJEVIC: But, I can see how 15 you come to your conclusion and that could be maybe 16 our failure of how we presented this in the general 17 because we didn't have a chance to talk about it but 18 there are things that we just presented.

19 We were looking. But I can completely see 20 how you came to this conclusion. But I would disagree 21 with this because of the one point. First, they did 22 not have many seismic PRAs.

23 Seismic PRAs are just coming with 2.1.

24 And then you suddenly look now in importance of 25 correlation has money and therefore to analyze NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

184 1 correlation I am sure they will be able to conclude 2 that co-relationship is important.

3 Also they used such a primitive actually 4 assumptions which may be good enough or not but we 5 don't know. We just want to say that this report 6 didn't have enough material to conclude for this that, 7 you know, in the general, everybody who is doing that 8 has a feeling there should be something better out of 9 there to do this more scientifically.

10 As a matter of fact and theoretically 11 pleasing, and it seems to me we'll have a sense. It's 12 simply complex and I am not a fragility expert so I 13 cannot say.

14 So they are proposing, they do show 15 different methods and they say, industry, that's what 16 is variable. If you have a problem with co-17 relationship because your dominant card set is 18 correlated you can consider analyzing these methods.

19 They didn't do any shaker table experiment 20 as that wasn't their job. Their job was to look in 21 results for shaker table results.

22 CHAIRMAN CORRADINI: But if I might, since 23 I didn't read the report so I'm totally able to do 24 this, I think where he's going with it, it would be I 25 expect this report to say okay, we've now developed a NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

185 1 methodology.

2 This methodology has certain pieces to it.

3 Now of those pieces this is a key piece. Now go do an 4 experiment or go do something because, in other words, 5 what are they proposing to advance it further. I 6 thought that's where --

7 MEMBER REMPE: Well, okay, so hold on for 8 a minute. A long time ago when we were discussing 9 this because I just review what they presented, the 10 very beginning of the report says they were supposed 11 to find a strong technical basis for a new method is 12 one of their objectives.

13 And I think as Margaret and the Panel have 14 said today they didn't do that. They didn't give a 15 strong technical basis for what they are suggesting is 16 an approved methodology because they acknowledge there 17 aren't enough data and as Vesna has emphasized it's 18 not clear it's going to make much of an impact.

19 Now the authors of the report were 20 contractors. They were not the Office of Research, 21 right. And they basically, I think were a little 22 optimistic because they get paid for doing research.

23 But yesterday, which is not part of our 24 review yesterday and I asked Jose about it. He said, 25 yes, it's something we'll look at if something else NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

186 1 comes in. But we've done due diligence in evaluating 2 this possible methodology.

3 And so I think again, I think maybe 4 research was doing due diligence that this methodology 5 might be better and, yes, okay, the contractors were 6 maybe a little more optimistic. They didn't fully 7 maybe go as far as they could have gone on how, you 8 know, whether this approach should even be considered.

9 But they're contractors and I don't know 10 if the Office of Research can tell them to change the 11 words in your report and say there's a lot more 12 uncertainty and all that. And that's another issue 13 with when you're doing research as a contractor for 14 the Office of Research. But --

15 MEMBER SUNSERI: That's fine too and I 16 understand that. But so maybe the, and let me back up 17 a little bit and say not all research has to be, for 18 it to be successful research doesn't have to result in 19 a positive outcome.

20 You can find it says that the research we 21 did in this area and there are no better ways to do 22 this, right. That could have been the fundamental 23 conclusion.

24 MEMBER REMPE: It might have been nice.

25 MEMBER SUNSERI: And that would have been NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

187 1 a great outcome, right.

2 MEMBER REMPE: If the project at the end 3 had said this might be useful. But it does have some 4 caveats in there if you look. It's not in the 5 conclusions as much as I would like to have seen or up 6 front in the abstract.

7 But they did say, hey, the data are 8 difficult. It's difficult with experts. They did say 9 that. Again, that's a problem about you're dealing 10 with a laboratory where I used to work were doing 11 something that way we would have had trouble if we 12 knew our future bucks hung on that document.

13 So I mean that's just an issue with the 14 system. But I understand where you're coming from.

15 But I think that, they did do a competent job, a good 16 professional job.

17 How they presented it, you know, it and I 18 think Margaret, Vesna and Jose are well aware of some 19 of the limitations of the report in how they write it 20 up.

21 MEMBER SUNSERI: Right. But if you asked 22 your research, you know, if you were, if Ron asked one 23 of his research students to do or they made a proposal 24 to do some research in his lab and they wrote up a 25 great report but it had nothing to do with the topic NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

188 1 would he --

2 MEMBER REMPE: But that's not what they 3 did. They --

4 MEMBER MARCH-LEUBA: That's not what they 5 did. Let me give you a completely different approach 6 for the same problem. You know, my brain works 7 differently than anyone else's.

8 The research was given a problem. Let me 9 start by saying first that I agree with you 100 10 percent. I wanted to give them a two on every single 11 item because I absolutely hate the results of the 12 report.

13 And that's why I don't know if you 14 remember I spent with you a whole afternoon and Pete 15 was here. We spent six hours just going through the 16 whole methodology through how it is because I even 17 wrote my own Monte Carlo simulation of the problem to 18 get the feeling of how things work.

19 Then I was convinced that I was looking at 20 the report that they were given a problem and that 21 problem was go do a search of the literature. Don't 22 reinvent the wheel. See what has been done and pick 23 the best that's available.

24 And that's the way I understood this 25 report to be. I will give the guys that wrote the NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

189 1 scope an F. I would give a two to the guys that wrote 2 the scope of the problem.

3 But once you hear the scope research did 4 the best they could and came up with some bad product 5 because they were given a bad question.

6 MEMBER SUNSERI: So let me understand.

7 And there was a user need and that was the scope.

8 MEMBER MARCH-LEUBA: It wasn't user need.

9 It was a research program.

10 MEMBER REMPE: Part of their seismic.

11 MEMBER CHU: My perspective is I think 12 Office of Research, they probably had a vague idea, 13 okay, and then they say the correlations should be 14 important. And they say if you do literature search, 15 you do this and that I think they are very optimistic 16 something would come out.

17 But they knew it was hard. Therefore, 18 they say get experts, field experts in this area and 19 then have a few workshops and ask them to help you.

20 So if you look at the scope it's an optimistic scope, 21 okay.

22 And then these contractors got the scope 23 and then went, diligently went through this. This is 24 the best they could do. So we, I think we all 25 understood that. I know we discussed many, many times NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

190 1 about what does it mean, yes.

2 And then this is what we came up with 3 because they did everything they were told to do. And 4 then they tried to find all the information. They 5 tried to see data, they could get data, okay. So this 6 is what we got.

7 MEMBER SUNSERI: And I understand. So let 8 me just respond. And I appreciate the fact that the 9 research organization is going to be asked to do some 10 things that is not purely research, okay.

11 So the PIRT was another example. I mean 12 a PIRT isn't research, right, it's a process. It's 13 just like okay, you know, go take a given process and 14 apply it to this phenomena and tell me what the 15 outcome is, right?

16 That's in my judgment not research. But 17 nonetheless.

18 MEMBER MARCH-LEUBA: And that should be 19 reflected in the report. I think that's what we need 20 to do.

21 MEMBER SUNSERI: And so maybe in 22 reflection of my feedback here it's back on us that we 23 should be more selective of what projects we select to 24 do our research quality review on to make them more 25 research oriented projects.

NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

191 1 CHAIRMAN CORRADINI: I don't think we have 2 that luxury. The Research Office here is an applied 3 research office. And a lot of the things they do are 4 relatively narrow in scope and incremental.

5 And what I heard Margaret, what I thought 6 I heard Margaret say is this is an incremental work 7 scope that they thought they knew the answer before 8 they were going to do it. They said go forth and do 9 it.

10 And they come back and it's kind of like, 11 a result. So if you're going to write --

12 MEMBER BLEY: Can I get a word in?

13 VICE CHAIRMAN RICCARDELLA: Sure. We 14 didn't even know you were there, Dennis.

15 MEMBER BLEY: Well I've been yelling for 16 the last 45 minutes. So Ron must have had me cut off.

17 But he's now cut me back in.

18 I'll try to be very quick and brief. On 19 this last go around I really agree with the idea that 20 these, research that is applied and basic research and 21 the things we looked at today are to me in the applied 22 research area.

23 And I think they're reasonable. Now to go 24 back to the earlier discussion about how much this 25 correlation can affect the risk assessment, I'm going NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

192 1 to take you back to the late 1970s.

2 One of the references in the report is a 3 Seismic Safety Margins Research Program in Livermore.

4 And those guys did a tremendous amount of work and 5 modeled all sorts of correlation.

6 And it was quite interesting. But then 7 over the next couple of years PRAs were being done.

8 And some of the things that drove this kind of 9 disappeared.

10 You know, we were very worried back then 11 that you have all of these segments of piping and if 12 they are 100 percent correlated, you know, we could 13 have some real problems. Well it turns out that all 14 the piping is, I think that's me causing trouble 15 again.

16 All the piping is very stout. Also the 17 valves we were worried about. So that the issue kind 18 of disappears because you never get up to the levels 19 that take those out.

20 And on the other hand, some things are 21 very weak. And you go over their threshold whether 22 they're correlated or not.

23 In the early 80s Kennedy and Cornell wrote 24 a piece that was kind of a sensitivity study on 25 varying the, looking at the correlation in some detail NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

193 1 and also using these rules of thumb. And it showed 2 the rules of thumb work pretty well.

3 Vesna gave a real good discussion of the 4 idea that the real design of the plant is what affects 5 whether correlation is important or not, the real 6 design of the plant and the absolute level of 7 fragility.

8 For the very low fragility it doesn't 9 matter very much. For the very high fragility you 10 never challenge them whether they're correlated or 11 not. For the stuff in the middle it matters but these 12 rules of thumb have worked pretty well.

13 So I kind of agree with all of your 14 discussions. And I just wanted to get that historical 15 note in there. We were worried about things that 16 would dramatically swing the results that impact the 17 correlation matter.

18 And they turned out not to matter because 19 of the details of the fragilities and the design.

20 That's all.

21 MEMBER REMPE: So we are supposed to be 22 done at 2:30 and I need to let the audience, which is 23 the staff who has come and we've not let them have an 24 opportunity to talk about either project.

25 And then I would like Margaret to also NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

194 1 weigh in about timing to get her draft done because I 2 assume based on what Dennis has said we're not going 3 to see the draft report until December full committee 4 and I know you're looking forward to another item to 5 put on the agenda in December, right.

6 CHAIRMAN CORRADINI: It doesn't have to be 7 done in December.

8 MEMBER REMPE: Okay. I just wanted to 9 make sure because then we might be February. Is 10 February okay?

11 CHAIRMAN CORRADINI: When is it due, 12 March? We're not in a rush.

13 MEMBER REMPE: So I wanted to make sure 14 that was okay because I didn't want to --

15 MR. NOURBAKAHSH: We are not in a rush.

16 We could send it any time.

17 MEMBER REMPE: Okay. So then if that's 18 the case then let's plan for February and everybody 19 won't worry about it. Staff, you have listened so 20 kindly without jumping up and down.

21 Do you have any comments about what you've 22 heard today for accuracy that we need to be 23 considering? And tell us which project too just to 24 make sure we understand.

25 MR. HAMBURGER: Good afternoon. I'm Ken NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

195 1 Hamburger. I'm the author of the PIRT report, NUREG 2 2218. Briefly just want to say thank you for your 3 comments and your feedback.

4 This was my first NUREG so I will 5 certainly use this as an opportunity improve the 6 quality of our research products. Two clarifications 7 that I think are worth making.

8 The first is where a value of unknown or 9 uncertain was given and I said that has no value 10 that's not to say that the phenomenon was not of 11 value, simply that it wasn't given a numerical 12 ranking.

13 It wasn't including in the arithmetic mean 14 that we calculated. I was not saying that the 15 phenomenon is not valuable. And the second thing that 16 I --

17 MEMBER BLEY: I understand. That's what 18 I thought you meant.

19 MR. HAMBURGER: Okay. The second thing 20 that I wanted to mention was although this was not 21 documented in the report we did have more than three 22 scenarios made up at the time of the PIRT.

23 And after we got through the first 24 scenario the group chose the remaining two scenarios 25 based on how much time we had left and how much time NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

196 1 we thought we would have to get through those 2 scenarios.

3 So there was some group judgment in 4 deciding which scenario we were going to use as part 5 of the PIRT though it wasn't documented and your 6 comments about the scope of the issue are certainly 7 well taken. Thank you.

8 MEMBER REMPE: Thank you. Do we need to 9 also open the line and we'll let the public, if 10 there's anyone out there can someone, Theron, can we 11 assume the lines are open for the public now?

12 CHAIRMAN CORRADINI: I hear noise.

13 MEMBER REMPE: Usually they'll come on and 14 say line open.

15 CHAIRMAN CORRADINI: Can somebody 16 acknowledge that they're out there on the public line 17 please if anybody is out there?

18 MEMBER BLEY: This is Dennis. I'm on a 19 separate line.

20 CHAIRMAN CORRADINI: Yes, we figured that.

21 We've heard you enough.

22 MEMBER REMPE: Okay, well since we're not 23 hearing anyone else I'm going to assume that there's 24 no one out there that wants to speak. And we will 25 plan that the draft will be provided some time in NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

197 1 December right now if that's okay with Margaret and 2 Dennis.

3 MEMBER CHU: I think we can probably give 4 it to you in November.

5 MEMBER REMPE: Okay, well the sooner the 6 better and then we'll get a document together.

7 MEMBER CHU: We would also get it.

8 MEMBER REMPE: Yes, well before February 9 and plan to try to have this on the agenda in February 10 then. Great.

11 CHAIRMAN CORRADINI: I just would 12 emphasize that it's important that you kind of get it 13 in a sense that you're happy with the result in terms 14 of the writing and then you go with Joy. But we're 15 not in a, from a rush standpoint, we're not in a rush.

16 We'll fit it in where we can. If we can 17 fit it in November because you're done so quickly, 18 fine. If we have to fit it in, in March, fine. It's 19 not as if we're on some sort of enormous schedule.

20 MEMBER MARCH-LEUBA: Do we need to do 21 another full committee on it?

22 MEMBER REMPE: No. Typically what we do 23 is we give it to the whole committee about a month 24 before and if they have any burning comments they will 25 send them to Jose and the two chairs and myself and NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

198 1 we'll get them resolved.

2 And then there's just like a single one 3 page letter. So --

4 CHAIRMAN CORRADINI: There's the 5 transmittal letter and that's it.

6 MEMBER MARCH-LEUBA: It will probably have 7 to be involved on P&P.

8 MEMBER REMPE: Yes. Well it will be, 9 there will be time on the agenda and we'll go through 10 the letter. And if someone didn't like the way their 11 comment was reserved that's the time to bring it up.

12 But usually it's more of a review offline.

13 Thank you. And thank you again, to the two chairs and 14 all the Members of the panels. Good discussion today.

15 CHAIRMAN CORRADINI: Okay. Dennis, are 16 you going to stay with us for the letter or are we 17 going to say goodbye to you?

18 MEMBER BLEY: As I said earlier, to you 19 privately, if you want me around for the first reading 20 okay. But I don't intend to stay for the letter. You 21 ought to run that because I'm here I can't really 22 participate except being an annoyance.

23 CHAIRMAN CORRADINI: Okay, that's fine.

24 MEMBER BLEY: I'm happy to let you guys go 25 ahead with it.

NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

199 1 CHAIRMAN CORRADINI: That's fine. All 2 right. So we're going to take a break, Dennis, and 3 then if you want to come back and listen to the 4 reading that's fine otherwise that's fine too. But 5 we'll come back at quarter to three.

6 (Whereupon, the above-entitled matter 7 went off the record at 2:30 p.m.)

8 9

10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433

ACRS October 4, 2018 Emergency Preparedness for Small Modular Reactors and Other New Technologies Proposed Rulemaking 10 CFR Parts 50 and 52 NRC-2015-0225 RIN 3150-AJ68

• Project Manager: Andy Carrera (NMSS)

• Technical Leads: Kenneth Thomas (NSIR)

Arlon Costa (NRO) 1

Emergency Preparedness for Small Modular Reactors and Other New Technologies Proposed Rulemaking

• Purpose of Rulemaking

- Amend regulations for new alternative, performance-based EP requirements for SMRs and ONTs.

- Address one of the policy, licensing, and technical issues identified in SECY-10-0034

• Source term, security, and siting criteria are not affected by this rule.

2

Emergency Preparedness for Small Modular Reactors and Other New Technologies Proposed Rulemaking

• Major provisions of this proposed rule:

- technology-inclusive for future SMRs and ONTs, including medical radioisotope facilities and non-light-water reactors

- performance-based EP framework, including demonstration of effective response in drills and exercises

- hazard analysis for contiguous facilities

- scalable approach for plume exposure pathway EPZ

- ingestion response planning for SMRs and ONTs 3

Emergency Preparedness for Small Modular Reactors and Other New Technologies Proposed Rulemaking

• Comments from ACRS Subcommittee Meeting August 22:

- Source terms

• Governed by separate rules and guidance

• Those rules and guidance are not within the scope of this rulemaking

- 1000 MWt

• No technical analysis

• Existing large light-water reactors

- Hazard analysis

• Updated by the provisions in § 50.54(q)(2) 4

Emergency Preparedness for Small Modular Reactors and Other New Technologies Proposed Rulemaking

• Comments from ACRS Subcommittee Meeting August 22:

- 96 hours0.00111 days <br />0.0267 hours <br />1.587302e-4 weeks <br />3.6528e-5 months <br />

• Used by the EPA in the PAG manual, section 2.2

• Dose projections measure integrated 4-day dose

- Multiple modular considerations

• Details are in DG, Staff Regulatory Position 8

- Documents are publicly available

• Made public in September

- Use of Other New Technologies

• Used in FRN, SECY, and DG 5

Emergency Preparedness for Small Modular Reactors and Other New Technologies Proposed Rulemaking

• EPZ size technical analysis

- The plume exposure pathway EPZ should encompass an area where prompt protective measures, such as evacuation and sheltering, may be needed to minimize the exposure to individuals.

- The analysis should consider radiological releases from credible accidents for the facility.

- Scalable EPZ

• If EPZ site boundary No NRC requirement for offsite radiological emergency preparedness programs

• If EPZ > site boundary NRC requirements for offsite radiological emergency preparedness programs 6

Emergency Preparedness for Small Modular Reactors and Other New Technologies Proposed Rulemaking Existing EP for Nuclear Power EP for SMRs and ONTs Reactors Onsite only Onsite and Offsite 7

Emergency Preparedness for Small Modular Reactors and Other New Technologies Proposed Rulemaking

• Scalable approach for plume exposure pathway EPZ

- Consistent with the analyses documented in NUREG-0396

- Consistent with the existing graded-approach afforded to:

• Research and test reactors

• Fuel cycle facilities

• Independent spent fuel storage installations

• Same level of protection afforded to public health and safety

• Development of guidance supported by Office of Nuclear Regulatory Research 8

Emergency Preparedness for Small Modular Reactors and Other New Technologies Proposed Rulemaking

• Ingestion response planning

- Early phase of the response

• Precautionary protective actions

- Washing garden products and food

- Placing livestock on stored feeds

- Longer term actions

• Leading indicator drives response

- Biological contamination similarities 9

Status and Path Forward

• Draft proposed rule due to Commission on October 12, 2018.

- Draft regulatory guidance is planned for issuance with proposed rule in early 2019 (pending Commissions approval).

- Public meeting during public comment period of proposed rule.

• Draft final rule due to Commission in early 2020.

10

Abbreviations ACRS - Advisory Committee on Reactor Safeguards CFR - Code of Federal Regulations OEDO - Office of the Executive Director of Operations EP - emergency preparedness EPZ - emergency planning zone FEMA - Federal Emergency Management Agency FRN - Federal Register Notice MWt - Megawatts thermal (units for reactor power) 11

Accident Source Terms for EPZ Size Considerations

• Evaluate a range of accidents credible for the facility

- Use DBA and severe accident source terms developed to support other required safety and environmental assessments

- Multi-module is considered

• Guidance exists for LWRs

- DBA releases (new reactors)

• SRP 15.0.3, RG 1.183 - AST (based on NUREG-1465)

• SRP 11.1, 12.2 - coolant activity, radiation sources

• SRP 3.11, RG 1.89 - EQ

- Severe accident releases

• SRP 19.0, 19.2

• PRA standards 1

ACRS 657th Full Committee Meeting Summary of Operating Experience for CY2017 and MY2018 Gordon Skillman, ACRS October 4, 2018 1

ROP Framework 2

ROP Process 3

CY2017 and MY18 ROP Total Number of Plants in ROP Evaluation:

Region I = 25 Region II = 34 Region III = 23 Region IV = 19 TOTAL = 101 4

CY2017 and MY18 ROP Inspection Findings - Quantitative Thresholds Red CDF greater than 10-4CDF or LERF greater than 10-5 LERF Yellow CDF greater than 10-5 and less than or equal to 10-4 CDF or LERF greater than 10-6 and less than or equal to 10-5 LERF White CDF greater than 10-6 and less than or equal to 10-5 CDF or LERF greater than 10-7 and less than or equal to 10-6 LERF Green CDF less than or equal to 10-6 CDF or LERF less than or equal to 10-7 LERF 5

CY2017 and MY18 ROP Inspection Findings (Qualitative Definitions)

Green: a finding of very low safety or security significance Greater-Than-Green: a finding of more than very low significance - security cornerstone only White: a finding of low to moderate safety or security significance Yellow: a finding of substantial safety or security significance Red: a finding of high safety or security significance 6

CY2017 and MY18 ROP 7

CY2017 ROP CY 2017* Inspection Findings Number of Plants Insp. Safety Cornerstone Find. Initiating Mitigating Barrier Emerg. Occ. Rad Pub. Rad. Security Events Systems Integrity Prep. Safety Safety Red 0 0 0 0 0 0  ?

Yellow 2 2 0 0 0 0  ?

White 2 5 0 1 0 0  ?

GTG - - - - - - 6 Green 27 81 38 7 9 5 26

• Snapshot at end of 4th QTR 17 that shows all findings from previous 4 QTRs

? = Red, Yellow, or White designation is SUNSI, reported as GTG.

8

MY2018 ROP MY 2018* Inspection Findings Number of Plants Insp. Safety Cornerstone Find. Initiating Mitigating Barrier Emerg. Occ. Rad Pub. Rad. Security Events Systems Integrity Prep. Safety Safety Red 0 0 0 0 0 0  ?

Yellow 0 0 0 0 0 0  ?

White 1 5 0 0 0 0  ?

GTG - - - - - - 3 Green 43 86 43 9 20 10 42

• Snapshot at end of 2nd QTR 18 that shows all findings from previous 4 QTRs Only new (1st or 2nd QTR 2018) presented in following slides 9

CY2017 ROP Inspection Findings Safety and Rad Protection Cornerstones Four Yellows (2017)

- Two Each at Arkansas 1 and 2 (Both CY2015 and 2016):

• Initiating Events Cornerstone

• Mitigating Systems Cornerstone

- Resolved and No Longer show (see MY2018 ROP) 10

CY2017 ROP Inspection Findings (S&RP)

Four Yellows - Arkansas Nuclear 1 and 2 - Historic:

• Initiating Events Cornerstone:

- Failure to Follow the Materials Handling Program during the Unit 1 Generator Stator Drop - Initially in 1st QTR 2014

- LOOP of 6 days Unit 1; Partial LOOP Unit 2

• Mitigating Systems Cornerstone:

- Inadequate Flood Protection for Auxiliary and Emergency Diesel Fuel Storage Buildings Concurrent with Stator Drop Event

- Over 100 Flood Barriers Inadequate

• Placed in Col 4 of Matrix Assessment

• Supplemental Inspection Procedure (IP) 95003 planned 11

CY2017 ROP Inspection Findings (S&RP)

Four Yellows - ANO 1 and 2 - Historic (continued):

2nd & 3rd QTRs 2014 - Unplanned Scrams/7000 Hrs White PI Subsequent decision to also conduct Supplemental Inspections 95001 and 95002 3 Supplemental Inspections Conducted Jan & Feb 2016 Entergy submitted ANO Comprehensive Recovery Plan Area Action Plans, May 2016 to respond to recover from Col 4 NRC Issued Confirmatory Action Letter (CAL) June 2016 Finally -

Inspection May 2018 - All actions completed, CAL closed, and assessment of performance of ANO 1 & 2 updated (reflected in 2018) 12

CY2017 ROP Inspection Findings (S&RP)

Whites - 8 Plants

• 2 Plants in Initiating Events Cornerstone:

- Grand Gulf 1 - Parallel Performance Indicator White Finding

- St. Lucie 1 - Loss of Configuration Control

• 5 Plants in Mitigating Systems Cornerstone:

- Catawba; Clinton; Oyster Creek; Perry 1; Pilgrim

- Various Systems / Causes

• 1 Plant in Emergency Preparedness Cornerstone:

- Fermi - Failure to Maintain Effectiveness of Site EP 13

CY2017 ROP Inspection Findings (S&RP)

• White at Grand Gulf 1 (IE)

• Parallel Performance Indicator White Inspection Finding.

• PI = Unplanned scrams per 7000 hours0.081 days <br />1.944 hours <br />0.0116 weeks <br />0.00266 months <br />

• Inadequate and Weaknesses in response to White Performance Indicator reported by licensee in 3rd QTR 16.

• Poor Root Cause Analysis

• White at Saint Lucie 1 (IE)

• Failure to maintain configuration control of the Unit 1 main generator inadvertent energization lockout relay circuitry.

• Resulted in reactor trip and loss of offsite power (LOOP) on August 21, 2016.

• 2E-6 change in core damage frequency (CDF).

14

CY2017 ROP Inspection Findings (S&RP)

• White at Catawba (MS)

• Failure to adequately develop and adjust preventive maintenance activities in accordance with procedure.

• Failed to develop preventive maintenance strategies which considered operating experience for the emergency diesel generator (EDG) excitation system.

• White at Clinton (MS)

• Failure to evaluate the change in the dropout voltages for replacement relays associated with the Div. 1 EDG Room Vent Fan.

• Change in dropout voltages prevented the fan from operating during an under voltage condition,

• Division 1 EDG became inoperable.

15

CY2017 ROP Inspection Findings (S&RP)

• White at Oyster Creek (MS)

• Tech Spec violation - failure to follow the electromatic relief valve (EMRV) reassembly instructions, which caused the E EMRV to be incorrectly reassembled.

• E EMRV was determined to be inoperable for greater than the technical specification allowed outage time.

• Supplemental 95001 Inspection completed Sept 17

• White at Perry 1 (MS)

• Failure to evaluate the effects of voltage suppression diode failure on the Standby Diesel Generator (SDG) control circuit.

• Introduction of new components (diodes) into the control circuitry resulted in the eventual failure of the SDG control circuit.

• SDG inoperable and unable to start.

16

CY2017 ROP Inspection Findings (S&RP)

• White at Pilgrim (MS)

• Failure to identify, evaluate, and correct a significant condition adverse to quality associated with the A safety/relief valve (SRV).

• Failed to correct the A SRVs failure to open upon manual actuation during a plant cool-down on 2/9/13, following a LOOP event.

• Subsequent 1/27/15 failure of C SRV could have been avoided.

• Supplemental 95001 - collective issues associated with the root cause evaluation represented a significant weakness.

• Supplemental 95003 (May 17) for different purpose extended to cover this finding.

17

CY2017 ROP Inspection Findings (S&RP)

• White at Fermi (EP)

• Failure to maintain the effectiveness of the EP and use adequate methods, systems, and equipment for assessing and monitoring actual or potential offsite consequences of a radiological emergency.

• Failure to accurately analyze the effect of increasing background radiation on the sites Standby Gas Treatment System (SGTS) accident range radiation monitor (AXM)

• As configured, the AXM would provide inaccurate indications of radioactive releases that are used as the licensees basis for determining EAL classification and development of PARs.

18

MY2018 ROP Inspection Findings(S&RP)

• No More Yellow Findings

• Whites - Reduced to 4 Plants

• Only 1 New Finding since 4th QTR 2017

• Mitigating Systems Cornerstone:

- Davis-Besse 19

MY2018 ROP Inspection Findings (S&RP)

• White at Davis-Besse (MS)

• Failure to provide appropriate instructions to maintain adequate oil in auxiliary feedwater turbine bearing oil sumps.

• Led to failure of auxiliary feedwater pump.

• Pump found to be inoperable for > tech spec limit.

20

CY2017 and MY18 ROP Security Cornerstone 21

CY2017 and MY18 ROP Inspection Findings - Security Cornerstone Plants Greater-Than-Green; 4 in CY16; 6 in CY17; 3 in MY18 (not including Force-on-Force)

GTG is not a new category, it means the finding was either white, yellow, or red Details are SUNSI and not publically available But NUREG-1885 - Annual non-SUNSI report provided to Congress 22

CY2017 ROP Security Cornerstone Baseline Inspection Findings 23

CY2017 ROP Security Cornerstone - Inspection Findings Force - on - Force Exercises 24

CY2017 ROP Force-On-Force Exercises

• No Greater-than-Green Findings

• 1 Ineffective Exercise:

- Licensees inability to demonstrate an effective implementation of its protective strategy to defend designated target set components

• 3 Marginal Exercises:

- In all three cases, licensees neutralized the adversary at a location, or in preparation to enter a location, that contained a single element target set.

• Licensees all took appropriate corrective actions.

- Policy and procedure changes

- Physical security upgrades and technology improvements

- Personnel or security force enhancements 25

CY2017 and MY18 ROP Cross-Cutting Issues

• NRC evaluates whether cross-cutting issues exist at NPPs 2x/year: at mid-cycle and end-of-cycle assessments:

• No New Open Issues in 2017

• No Data reported yet for mid-cycle 2018 assessments 26

CY2017 and MY18 ROP 27

CY2017 and MY18 ROP Performance Indicators Green: performance within an expected level where all cornerstone objectives are met White: performance outside an expected range of nominal utility performance but related cornerstone objectives are met Yellow: related cornerstone objectives are met, but with a minimal reduction in safety margin Red: significant reduction in safety margin in area measured by the PI 28

CY2017 ROP Performance Indicators

• NO Plants with Red or Yellow

• Two Plants with White 29

CY2017 ROP Performance Indicators IE04 - Unplanned Scrams with Complications

• Columbia Generating Station White Threshold > 1.0 1Q 2Q 3Q 4Q 1.0 1.0 2.0 1.0

• Unplanned scrams with complications occurred in 4th QTR 16 and 3rd QTR 17 30

CY2017 ROP Performance Indicators IE01 - Unplanned Scrams per 7000 Critical Hours

• Watts Bar 2 White Threshold > 3.0 1Q 2Q 3Q 4Q NR NR 1.4 3.1 NR = Watts Bar 2 went online October 2016. Significant critical hours to require reporting not reached until 3Q17.

31

MY2018 ROP Performance Indicators No Red, Yellow or White Thru first two quarters Of 2018 32

CY2017 and MY18 ROP ROP Action Matrix Inspection Findings

+

Performance Indicators

=

Plant Assessment 33

CY2017 and MY18 ROP ROP Action Matrix Assessment Column 5: Unacceptable Performance Column 4: Multiple/Repetitive Degraded Cornerstone Column 3 Degraded Cornerstone Column 2: Regulatory Response Column 1: Licensee Response 34

CY2017 and MY18 ROP 35

CY2017 ROP ROP Action Matrix Assessment Number of Plants in Each Column (per QTR) 2017 Col 1 Col 2 Col 3 Col 4 Col 5 QTR Licensee

Response

Regulatory

Response

Degraded Cornerstone Multiple/Repetitive Degraded Cornerstone Unacceptable 1 82 16 0 3* 0 2 84 14 0 3* 0 3 86 12 0 3* 0 4 89 9 0 3* 0

• Arkansas 1 & 2, and Pilgrim 36

MY2018 ROP ROP Action Matrix Assessment Number of Plants in Each Column (per QTR) 2018 Col 1 Col 2 Col 3 Col 4 Col 5 QTR Licensee

Response

Regulatory

Response

Degraded Cornerstone Multiple/Repetitive Degraded Cornerstone Unacceptable 1 95 5 0 1* 0 2** x x x x x 3 x x x x x 4 x x x x x

• Pilgrim

• • 2nd QTR 2018 Action Matrix Summary data not posted on website yet, most Mid-Year Assessment letters are available.

37

CY2017 and MY18 ROP Action Matrix Summary Highlights

• Arkansas Nuclear 1 & 2 - Back in Column 1 in 2018 after being in Column 4 since 1st QTR 2015 38

CY2017 and MY18 ROP Action Matrix Summary Lowlights

• Pilgrim 1 - 6 consecutive QTRs in Col. 3 (starting 4th QTR 2013) followed by 13 consecutive QTRs in Col. 4 since then.

- NRC muted response considers Pilgrim plans to shutdown.

• Grand Gulf 1 - 8 consecutive QTRs in Col. 2 (started 3rd QTR16 - back in Col 1)

• Clinton - 4 consecutive QTRs in Col. 2 (starting 3rd QTR17 - remains in Col. 2) *

• Columbia - 4 consecutive QTRs in Col. 2 (all 4 QTRs of 2017 - back in Col 1)

• Supplemental Inspection conducted September 18, 2018.

39

CY2016 & 17 FFD Fitness for Duty

• Electronic Reporting since 2009

• Summary Reports available from some licensees

• No Industry-Wide Summaries being prepared as in the past

• NRC staff provides annual briefing to Drug Testing Advisory Panel

• Tests conducted on licensee and C/V employees 40

CY2016 & 17 FFD Fitness for Duty

• 5 Types of Tests Conducted:

• Pre-Access (applicants for employment)

• Random (unscheduled/unannounced for employees)

• For Cause (behavior of, or information received about employee)

• Post-Event (after an event involving human error)

• Follow-up (after a positive test) 41

CY2016 & 17 FFD Fitness for Duty Data (2008-2014)

• Three Multi-year Trends were being tracked:

• Subversion attempts prevalent since CY2011 (18 to 21% of violations: 143 to 187 events per year) with 54 to 66% of sites reporting at least one.

• Amphetamine positive results increasing since CY2008 (from 3.8%

(in 2008) to 10.6% (in 2014) of drug & alcohol positives.

• Reactor construction sites have higher positive rates, primarily in pre-access and random tests, and have higher incidence of subversion attempts than operating reactor sites.

42

CY2016 & 17 FFD Fitness for Duty Data (2008-2014) (cont)

• In February 2017, the NRC staff submitted a proposed rule to the Commission (SECY-17-0027) for consideration.

• The proposed rule, in part, would address those three multi-year trends by lowering the testing cutoff levels for amphetamine and methamphetamine, and expanding testing measures related to subversion attempt detection.

• No Commission Response yet.

43

CY2016 & 17 FFD

• MEANWHILE - Subversion attempts prevalent since CY2011 continue to rise

• Subversion Attempt Trends:

• 2012 - 177 of 1,114 violations (15.8% subversions)

• 2013 - 148 of 1,007 violations (14.7% subversions)

• 2014 - 187 of 1,133 violations (16.5% subversions)

• 2015 - 232 of 1,200 violations (19.3% subversions)

• 2016 - 304 of 1,164 violations (26.1% subversions)

• 2017 - 298 of 1,143 violations (26.1% subversions)

• Subversion Attempts in 2017:

• 45 facilities with at least 1 subversion attempt

• 67% identified at Pre-Access testing (200 of 298)

• 98% by contractor/vendors 44

CY2016 & 17 FFD Fitness for Duty Data (cont.)

• Overall industry positive rate CY17 = 0.77% (0.76% in CY16)

Low, but continues upward trend since CY12 and 13 (both were 0.62%)

• Total tests in CY2017 decreased by 3.6%

Tests in CY2016 decreased by 5.9% from CY2015 Continuing downward trend

• Approx 64% of CY17 positives and refusals occur at pre-access Prevents access, directly protecting public health and safety Almost constant percentage (65%) for several years Random testing CY17 identified 22.7% of substance users Identifies more employees using substances than pre-access 22.3% in CY2016; Continues upward trend since 2014 45

CY2016 & 17 FFD Fitness for Duty Data (cont.)

• For cause testing continues to have highest positive rate (9.9%)

(But is trending down 13.40% in CY13, 12%in CY14; 6% in CY16)

• Three substances continue to account for 85% of positives Marijuana ( 50%), Alcohol ( 23%), Cocaine ( 12%)

Amphetamine positives continuing upward trend, now nearly same percentage as cocaine.

Cocaine positives way down from high in 2006, but trending up again.

Opiate positives way down (only 1% in CY 2017))

46

CY2016 & 17 FFD Fitness for Duty Data (cont.)

• Events concerning individual employee violations must be reported to NRC within 24 hours2.777778e-4 days <br />0.00667 hours <br />3.968254e-5 weeks <br />9.132e-6 months <br />

• 33 24-hour reportable events in 2017 15 involved Supervisors/Managers 47

2017 Abnormal Occurrence Report Annual Abnormal Occurrence Report to Congress

• No AOs at NPPs

• 11 AOs at Medical Facilities

• 1 Significant Event that does not meet definition of AO:

• Exposure at National Institute of Standards and Technology (NIST), Gaithersburg MD 48

Questions ?

49

Reactor Oversight Process Updates Anticipating Degrading Licensee Performance Reactor Oversight Process Enhancement Proposals ACRS Full Committee Greg Bowman and Tom Hipschman, NRR/DIRS October 4, 2018 1

Background

• The Reactor Oversight Process (ROP) is designed to identify declining licensee performance prior to loss of reasonable assurance of adequate protection

- Columns 2-4 of the Action Matrix provide this buffer

- 9500x procedures guide the associated agency response

• That notwithstanding:

- ACRS and others have wondered about leading indicators

- The Commission has, in various forums, encouraged the staff to continue thinking about this topic

- We routinely evaluate the effectiveness of the ROP (e.g., the annual self assessment) and seek to continually improve 2

Previous Discussions

• Various discussions have highlighted ideas such as:

- Trend in the # of findings

• Green findings

• Quality assurance / Appendix B-related findings

• Maintenance Rule-related enforcement actions

- Plant operations:

• Work management

• Material condition of the plant

• Trend in unplanned outage rate relative to industry average

• # of off-normal procedure entries

• # of unplanned Limiting Condition for Operation (LCO) entries

• Staffing resources

- Trend in # of conditions requiring NRC notification (50.72/50.73)

• Nature of these conditions (i.e., atypical?)

- Industry (e.g., INPO) evaluations

- Leadership performance 3

Examples of Tailoring the Existing ROP

• A related point is how NRC adjusts the inspection program for plants that are:

- Experiencing financial issues

- Nearing cessation of permanent operation

• Inspection Manual Chapter (IMC) 2515, Appendix G

- Provides inspection guidance for inspecting plants that are approaching shutdown and decommissioning

• Staff also utilizes a handful of means for addressing these situations:

- Safety culture initiatives

- Supplemental guidance memos to address specific cases

- IMC governing resident activities calls out material condition and long-standing issues

- Sample selection for inspection procedures that cover areas where financial difficulties could translate to degradation of performance (e.g., P&IR, Maintenance Effectiveness, Equipment Alignment)

- Increased NRC management presence 4

Enhancing the Reactor Oversight Process

• NRC has a number of initiatives ongoing to make the ROP more effective and efficient. Examples include:

- Improvements to the inspection report development process

- Initiative to improve the minor/more-than-minor component of the issue screening process

- Work to improve the NRCs engineering inspection program

• DIRS received input from stakeholders as part of the ongoing agency initiative associated with transformation:

- ~70 recommendations from the internal stakeholders related to potential enhancements to the ROP

- NEI publication and NRUG letter with initial industry proposals to improve the ROP

- Follow-up letter from NEI with consolidated industry proposals 5

Internal Stakeholder Feedback

• The Transformation Team provided DIRS with consolidated stakeholder feedback related to ROP transformation.

• The feedback generally fit into several categories:

- Changes to organization and staffing supporting oversight, primarily regional changes

- Changes to frequency of inspections and focus of inspection procedures

- Changes to make performance indicators more effective

- Changes to the various components of the assessment process

- Changes to make the enforcement program more risk-informed

- Changes to streamline or eliminate inspection reports

• No assessment was done of the proposals; the input to consisted of direct feedback provide by NRC staff.

6

Recent Industry Feedback

• NEI letter submitted on September 19 (ML18262A322) and discussed at September 20 ROP public meeting

• Acknowledgment that the ROP remains a sound and effective oversight program.

• Provided some recommendations on how the ROP could be enhanced:

- Impact of White findings (e.g., press releases, labeling and communication, type of inspection follow-up)

- Reassess baseline inspection program (e.g., focused evaluation of inspection areas, reduce burden of MSPI)

- Improvements to the SDP (e.g., improve specific SDPs, address differences in NRC and licensee models)

- Resolution of inspection issues (e.g., communication of potential issues, resolution of low risk compliance issues) 7

Next Steps

• Establish working group(s) to obtain stakeholder input, evaluate proposals, develop recommendations.

- Initial kick-off meeting took place on October 3

• Establish recurring discussion at monthly ROP public meetings, with separate focused public meetings on specific ROP enhancements.

- Planning public meetings on October 18 and November 15 (tentative)

• Commission approval will be needed for any significant changes to the ROP, per Management Directive 8.13.

8

ACRS 657th Full Committee Meeting Summary of Operating Experience for CY2017 and MY2018 Gordon Skillman, ACRS October 4, 2018 1

ROP Framework 2

ROP Process 3

CY2017 and MY18 ROP Total Number of Plants in ROP Evaluation:

Region I = 25 Region II = 34 Region III = 23 Region IV = 19 TOTAL = 101 4

CY2017 and MY18 ROP Inspection Findings - Quantitative Thresholds Red CDF greater than 10-4CDF or LERF greater than 10-5 LERF Yellow CDF greater than 10-5 and less than or equal to 10-4 CDF or LERF greater than 10-6 and less than or equal to 10-5 LERF White CDF greater than 10-6 and less than or equal to 10-5 CDF or LERF greater than 10-7 and less than or equal to 10-6 LERF Green CDF less than or equal to 10-6 CDF or LERF less than or equal to 10-7 LERF 5

CY2017 and MY18 ROP Inspection Findings (Qualitative Definitions)

Green: a finding of very low safety or security significance Greater-Than-Green: a finding of more than very low significance - security cornerstone only White: a finding of low to moderate safety or security significance Yellow: a finding of substantial safety or security significance Red: a finding of high safety or security significance 6

CY2017 and MY18 ROP 7

CY2017 ROP CY 2017* Inspection Findings Number of Plants Insp. Safety Cornerstone Find. Initiating Mitigating Barrier Emerg. Occ. Rad Pub. Rad. Security Events Systems Integrity Prep. Safety Safety Red 0 0 0 0 0 0  ?

Yellow 2 2 0 0 0 0  ?

White 2 5 0 1 0 0  ?

GTG - - - - - - 6 Green 27 81 38 7 9 5 26

• Snapshot at end of 4th QTR 17 that shows all findings from previous 4 QTRs

? = Red, Yellow, or White designation is SUNSI, reported as GTG.

8

MY2018 ROP MY 2018* Inspection Findings Number of Plants Insp. Safety Cornerstone Find. Initiating Mitigating Barrier Emerg. Occ. Rad Pub. Rad. Security Events Systems Integrity Prep. Safety Safety Red 0 0 0 0 0 0  ?

Yellow 0 0 0 0 0 0  ?

White 1 5 0 0 0 0  ?

GTG - - - - - - 3 Green 43 86 43 9 20 10 42

• Snapshot at end of 2nd QTR 18 that shows all findings from previous 4 QTRs Only new (1st or 2nd QTR 2018) presented in following slides 9

CY2017 ROP Inspection Findings Safety and Rad Protection Cornerstones Four Yellows (2017)

- Two Each at Arkansas 1 and 2 (Both CY2015 and 2016):

• Initiating Events Cornerstone

• Mitigating Systems Cornerstone

- Resolved and No Longer show (see MY2018 ROP) 10

CY2017 ROP Inspection Findings (S&RP)

Four Yellows - Arkansas Nuclear 1 and 2 - Historic:

• Initiating Events Cornerstone:

- Failure to Follow the Materials Handling Program during the Unit 1 Generator Stator Drop - Initially in 1st QTR 2014

- LOOP of 6 days Unit 1; Partial LOOP Unit 2

• Mitigating Systems Cornerstone:

- Inadequate Flood Protection for Auxiliary and Emergency Diesel Fuel Storage Buildings Concurrent with Stator Drop Event

- Over 100 Flood Barriers Inadequate

• Placed in Col 4 of Matrix Assessment

• Supplemental Inspection Procedure (IP) 95003 planned 11

CY2017 ROP Inspection Findings (S&RP)

Four Yellows - ANO 1 and 2 - Historic (continued):

2nd & 3rd QTRs 2014 - Unplanned Scrams/7000 Hrs White PI Subsequent decision to also conduct Supplemental Inspections 95001 and 95002 3 Supplemental Inspections Conducted Jan & Feb 2016 Entergy submitted ANO Comprehensive Recovery Plan Area Action Plans, May 2016 to respond to recover from Col 4 NRC Issued Confirmatory Action Letter (CAL) June 2016 Finally -

Inspection May 2018 - All actions completed, CAL closed, and assessment of performance of ANO 1 & 2 updated (reflected in 2018) 12

CY2017 ROP Inspection Findings (S&RP)

Whites - 8 Plants

• 2 Plants in Initiating Events Cornerstone:

- Grand Gulf 1 - Parallel Performance Indicator White Finding

- St. Lucie 1 - Loss of Configuration Control

• 5 Plants in Mitigating Systems Cornerstone:

- Catawba; Clinton; Oyster Creek; Perry 1; Pilgrim

- Various Systems / Causes

• 1 Plant in Emergency Preparedness Cornerstone:

- Fermi - Failure to Maintain Effectiveness of Site EP 13

CY2017 ROP Inspection Findings (S&RP)

• White at Grand Gulf 1 (IE)

• Parallel Performance Indicator White Inspection Finding.

• PI = Unplanned scrams per 7000 hours0.081 days <br />1.944 hours <br />0.0116 weeks <br />0.00266 months <br />

• Inadequate and Weaknesses in response to White Performance Indicator reported by licensee in 3rd QTR 16.

• Poor Root Cause Analysis

• White at Saint Lucie 1 (IE)

• Failure to maintain configuration control of the Unit 1 main generator inadvertent energization lockout relay circuitry.

• Resulted in reactor trip and loss of offsite power (LOOP) on August 21, 2016.

• 2E-6 change in core damage frequency (CDF).

14

CY2017 ROP Inspection Findings (S&RP)

• White at Catawba (MS)

• Failure to adequately develop and adjust preventive maintenance activities in accordance with procedure.

• Failed to develop preventive maintenance strategies which considered operating experience for the emergency diesel generator (EDG) excitation system.

• White at Clinton (MS)

• Failure to evaluate the change in the dropout voltages for replacement relays associated with the Div. 1 EDG Room Vent Fan.

• Change in dropout voltages prevented the fan from operating during an under voltage condition,

• Division 1 EDG became inoperable.

15

CY2017 ROP Inspection Findings (S&RP)

• White at Oyster Creek (MS)

• Tech Spec violation - failure to follow the electromatic relief valve (EMRV) reassembly instructions, which caused the E EMRV to be incorrectly reassembled.

• E EMRV was determined to be inoperable for greater than the technical specification allowed outage time.

• Supplemental 95001 Inspection completed Sept 17

• White at Perry 1 (MS)

• Failure to evaluate the effects of voltage suppression diode failure on the Standby Diesel Generator (SDG) control circuit.

• Introduction of new components (diodes) into the control circuitry resulted in the eventual failure of the SDG control circuit.

• SDG inoperable and unable to start.

16

CY2017 ROP Inspection Findings (S&RP)

• White at Pilgrim (MS)

• Failure to identify, evaluate, and correct a significant condition adverse to quality associated with the A safety/relief valve (SRV).

• Failed to correct the A SRVs failure to open upon manual actuation during a plant cool-down on 2/9/13, following a LOOP event.

• Subsequent 1/27/15 failure of C SRV could have been avoided.

• Supplemental 95001 - collective issues associated with the root cause evaluation represented a significant weakness.

• Supplemental 95003 (May 17) for different purpose extended to cover this finding.

17

CY2017 ROP Inspection Findings (S&RP)

• White at Fermi (EP)

• Failure to maintain the effectiveness of the EP and use adequate methods, systems, and equipment for assessing and monitoring actual or potential offsite consequences of a radiological emergency.

• Failure to accurately analyze the effect of increasing background radiation on the sites Standby Gas Treatment System (SGTS) accident range radiation monitor (AXM)

• As configured, the AXM would provide inaccurate indications of radioactive releases that are used as the licensees basis for determining EAL classification and development of PARs.

18

MY2018 ROP Inspection Findings(S&RP)

• No More Yellow Findings

• Whites - Reduced to 4 Plants

• Only 1 New Finding since 4th QTR 2017

• Mitigating Systems Cornerstone:

- Davis-Besse 19

MY2018 ROP Inspection Findings (S&RP)

• White at Davis-Besse (MS)

• Failure to provide appropriate instructions to maintain adequate oil in auxiliary feedwater turbine bearing oil sumps.

• Led to failure of auxiliary feedwater pump.

• Pump found to be inoperable for > tech spec limit.

20

CY2017 and MY18 ROP Security Cornerstone 21

CY2017 and MY18 ROP Inspection Findings - Security Cornerstone Plants Greater-Than-Green; 4 in CY16; 6 in CY17; 3 in MY18 (not including Force-on-Force)

GTG is not a new category, it means the finding was either white, yellow, or red Details are SUNSI and not publically available But NUREG-1885 - Annual non-SUNSI report provided to Congress 22

CY2017 ROP Security Cornerstone Baseline Inspection Findings 23

CY2017 ROP Security Cornerstone - Inspection Findings Force - on - Force Exercises 24

CY2017 ROP Force-On-Force Exercises

• No Greater-than-Green Findings

• 1 Ineffective Exercise:

- Licensees inability to demonstrate an effective implementation of its protective strategy to defend designated target set components

• 3 Marginal Exercises:

- In all three cases, licensees neutralized the adversary at a location, or in preparation to enter a location, that contained a single element target set.

• Licensees all took appropriate corrective actions.

- Policy and procedure changes

- Physical security upgrades and technology improvements

- Personnel or security force enhancements 25

CY2017 and MY18 ROP Cross-Cutting Issues

• NRC evaluates whether cross-cutting issues exist at NPPs 2x/year: at mid-cycle and end-of-cycle assessments:

• No New Open Issues in 2017

• No Data reported yet for mid-cycle 2018 assessments 26

CY2017 and MY18 ROP 27

CY2017 and MY18 ROP Performance Indicators Green: performance within an expected level where all cornerstone objectives are met White: performance outside an expected range of nominal utility performance but related cornerstone objectives are met Yellow: related cornerstone objectives are met, but with a minimal reduction in safety margin Red: significant reduction in safety margin in area measured by the PI 28

CY2017 ROP Performance Indicators

• NO Plants with Red or Yellow

• Two Plants with White 29

CY2017 ROP Performance Indicators IE04 - Unplanned Scrams with Complications

• Columbia Generating Station White Threshold > 1.0 1Q 2Q 3Q 4Q 1.0 1.0 2.0 1.0

• Unplanned scrams with complications occurred in 4th QTR 16 and 3rd QTR 17 30

CY2017 ROP Performance Indicators IE01 - Unplanned Scrams per 7000 Critical Hours

• Watts Bar 2 White Threshold > 3.0 1Q 2Q 3Q 4Q NR NR 1.4 3.1 NR = Watts Bar 2 went online October 2016. Significant critical hours to require reporting not reached until 3Q17.

31

MY2018 ROP Performance Indicators No Red, Yellow or White Thru first two quarters Of 2018 32

CY2017 and MY18 ROP ROP Action Matrix Inspection Findings

+

Performance Indicators

=

Plant Assessment 33

CY2017 and MY18 ROP ROP Action Matrix Assessment Column 5: Unacceptable Performance Column 4: Multiple/Repetitive Degraded Cornerstone Column 3 Degraded Cornerstone Column 2: Regulatory Response Column 1: Licensee Response 34

CY2017 and MY18 ROP 35

CY2017 ROP ROP Action Matrix Assessment Number of Plants in Each Column (per QTR) 2017 Col 1 Col 2 Col 3 Col 4 Col 5 QTR Licensee

Response

Regulatory

Response

Degraded Cornerstone Multiple/Repetitive Degraded Cornerstone Unacceptable 1 82 16 0 3* 0 2 84 14 0 3* 0 3 86 12 0 3* 0 4 89 9 0 3* 0

• Arkansas 1 & 2, and Pilgrim 36

MY2018 ROP ROP Action Matrix Assessment Number of Plants in Each Column (per QTR) 2018 Col 1 Col 2 Col 3 Col 4 Col 5 QTR Licensee

Response

Regulatory

Response

Degraded Cornerstone Multiple/Repetitive Degraded Cornerstone Unacceptable 1 95 5 0 1* 0 2** x x x x x 3 x x x x x 4 x x x x x

• Pilgrim

• • 2nd QTR 2018 Action Matrix Summary data not posted on website yet, most Mid-Year Assessment letters are available.

37

CY2017 and MY18 ROP Action Matrix Summary Highlights

• Arkansas Nuclear 1 & 2 - Back in Column 1 in 2018 after being in Column 4 since 1st QTR 2015 38

CY2017 and MY18 ROP Action Matrix Summary Lowlights

• Pilgrim 1 - 6 consecutive QTRs in Col. 3 (starting 4th QTR 2013) followed by 13 consecutive QTRs in Col. 4 since then.

- NRC muted response considers Pilgrim plans to shutdown.

• Grand Gulf 1 - 8 consecutive QTRs in Col. 2 (started 3rd QTR16 - back in Col 1)

• Clinton - 4 consecutive QTRs in Col. 2 (starting 3rd QTR17 - remains in Col. 2) *

• Columbia - 4 consecutive QTRs in Col. 2 (all 4 QTRs of 2017 - back in Col 1)

• Supplemental Inspection conducted September 18, 2018.

39

CY2016 & 17 FFD Fitness for Duty

• Electronic Reporting since 2009

• Summary Reports available from some licensees

• No Industry-Wide Summaries being prepared as in the past

• NRC staff provides annual briefing to Drug Testing Advisory Panel

• Tests conducted on licensee and C/V employees 40

CY2016 & 17 FFD Fitness for Duty

• 5 Types of Tests Conducted:

• Pre-Access (applicants for employment)

• Random (unscheduled/unannounced for employees)

• For Cause (behavior of, or information received about employee)

• Post-Event (after an event involving human error)

• Follow-up (after a positive test) 41

CY2016 & 17 FFD Fitness for Duty Data (2008-2014)

• Three Multi-year Trends were being tracked:

• Subversion attempts prevalent since CY2011 (18 to 21% of violations: 143 to 187 events per year) with 54 to 66% of sites reporting at least one.

• Amphetamine positive results increasing since CY2008 (from 3.8%

(in 2008) to 10.6% (in 2014) of drug & alcohol positives.

• Reactor construction sites have higher positive rates, primarily in pre-access and random tests, and have higher incidence of subversion attempts than operating reactor sites.

42

CY2016 & 17 FFD Fitness for Duty Data (2008-2014) (cont)

• In February 2017, the NRC staff submitted a proposed rule to the Commission (SECY-17-0027) for consideration.

• The proposed rule, in part, would address those three multi-year trends by lowering the testing cutoff levels for amphetamine and methamphetamine, and expanding testing measures related to subversion attempt detection.

• No Commission Response yet.

43

CY2016 & 17 FFD

• MEANWHILE - Subversion attempts prevalent since CY2011 continue to rise

• Subversion Attempt Trends:

• 2012 - 177 of 1,114 violations (15.8% subversions)

• 2013 - 148 of 1,007 violations (14.7% subversions)

• 2014 - 187 of 1,133 violations (16.5% subversions)

• 2015 - 232 of 1,200 violations (19.3% subversions)

• 2016 - 304 of 1,164 violations (26.1% subversions)

• 2017 - 298 of 1,143 violations (26.1% subversions)

• Subversion Attempts in 2017:

• 45 facilities with at least 1 subversion attempt

• 67% identified at Pre-Access testing (200 of 298)

• 98% by contractor/vendors 44

CY2016 & 17 FFD Fitness for Duty Data (cont.)

• Overall industry positive rate CY17 = 0.77% (0.76% in CY16)

Low, but continues upward trend since CY12 and 13 (both were 0.62%)

• Total tests in CY2017 decreased by 3.6%

Tests in CY2016 decreased by 5.9% from CY2015 Continuing downward trend

• Approx 64% of CY17 positives and refusals occur at pre-access Prevents access, directly protecting public health and safety Almost constant percentage (65%) for several years Random testing CY17 identified 22.7% of substance users Identifies more employees using substances than pre-access 22.3% in CY2016; Continues upward trend since 2014 45

CY2016 & 17 FFD Fitness for Duty Data (cont.)

• For cause testing continues to have highest positive rate (9.9%)

(But is trending down 13.40% in CY13, 12%in CY14; 6% in CY16)

• Three substances continue to account for 85% of positives Marijuana ( 50%), Alcohol ( 23%), Cocaine ( 12%)

Amphetamine positives continuing upward trend, now nearly same percentage as cocaine.

Cocaine positives way down from high in 2006, but trending up again.

Opiate positives way down (only 1% in CY 2017))

46

CY2016 & 17 FFD Fitness for Duty Data (cont.)

• Events concerning individual employee violations must be reported to NRC within 24 hours2.777778e-4 days <br />0.00667 hours <br />3.968254e-5 weeks <br />9.132e-6 months <br />

• 33 24-hour reportable events in 2017 15 involved Supervisors/Managers 47

2017 Abnormal Occurrence Report Annual Abnormal Occurrence Report to Congress

• No AOs at NPPs

• 11 AOs at Medical Facilities

• 1 Significant Event that does not meet definition of AO:

• Exposure at National Institute of Standards and Technology (NIST), Gaithersburg MD 48

Questions ?

49

Reactor Oversight Process Updates Anticipating Degrading Licensee Performance Reactor Oversight Process Enhancement Proposals ACRS Full Committee Greg Bowman and Tom Hipschman, NRR/DIRS October 4, 2018 1

Background

• The Reactor Oversight Process (ROP) is designed to identify declining licensee performance prior to loss of reasonable assurance of adequate protection

- Columns 2-4 of the Action Matrix provide this buffer

- 9500x procedures guide the associated agency response

• That notwithstanding:

- ACRS and others have wondered about leading indicators

- The Commission has, in various forums, encouraged the staff to continue thinking about this topic

- We routinely evaluate the effectiveness of the ROP (e.g., the annual self assessment) and seek to continually improve 2

Previous Discussions

• Various discussions have highlighted ideas such as:

- Trend in the # of findings

• Green findings

• Quality assurance / Appendix B-related findings

• Maintenance Rule-related enforcement actions

- Plant operations:

• Work management

• Material condition of the plant

• Trend in unplanned outage rate relative to industry average

• # of off-normal procedure entries

• # of unplanned Limiting Condition for Operation (LCO) entries

• Staffing resources

- Trend in # of conditions requiring NRC notification (50.72/50.73)

• Nature of these conditions (i.e., atypical?)

- Industry (e.g., INPO) evaluations

- Leadership performance 3

Examples of Tailoring the Existing ROP

• A related point is how NRC adjusts the inspection program for plants that are:

- Experiencing financial issues

- Nearing cessation of permanent operation

• Inspection Manual Chapter (IMC) 2515, Appendix G

- Provides inspection guidance for inspecting plants that are approaching shutdown and decommissioning

• Staff also utilizes a handful of means for addressing these situations:

- Safety culture initiatives

- Supplemental guidance memos to address specific cases

- IMC governing resident activities calls out material condition and long-standing issues

- Sample selection for inspection procedures that cover areas where financial difficulties could translate to degradation of performance (e.g., P&IR, Maintenance Effectiveness, Equipment Alignment)

- Increased NRC management presence 4

Enhancing the Reactor Oversight Process

• NRC has a number of initiatives ongoing to make the ROP more effective and efficient. Examples include:

- Improvements to the inspection report development process

- Initiative to improve the minor/more-than-minor component of the issue screening process

- Work to improve the NRCs engineering inspection program

• DIRS received input from stakeholders as part of the ongoing agency initiative associated with transformation:

- ~70 recommendations from the internal stakeholders related to potential enhancements to the ROP

- NEI publication and NRUG letter with initial industry proposals to improve the ROP

- Follow-up letter from NEI with consolidated industry proposals 5

Internal Stakeholder Feedback

• The Transformation Team provided DIRS with consolidated stakeholder feedback related to ROP transformation.

• The feedback generally fit into several categories:

- Changes to organization and staffing supporting oversight, primarily regional changes

- Changes to frequency of inspections and focus of inspection procedures

- Changes to make performance indicators more effective

- Changes to the various components of the assessment process

- Changes to make the enforcement program more risk-informed

- Changes to streamline or eliminate inspection reports

• No assessment was done of the proposals; the input to consisted of direct feedback provide by NRC staff.

6

Recent Industry Feedback

• NEI letter submitted on September 19 (ML18262A322) and discussed at September 20 ROP public meeting

• Acknowledgment that the ROP remains a sound and effective oversight program.

• Provided some recommendations on how the ROP could be enhanced:

- Impact of White findings (e.g., press releases, labeling and communication, type of inspection follow-up)

- Reassess baseline inspection program (e.g., focused evaluation of inspection areas, reduce burden of MSPI)

- Improvements to the SDP (e.g., improve specific SDPs, address differences in NRC and licensee models)

- Resolution of inspection issues (e.g., communication of potential issues, resolution of low risk compliance issues) 7

Next Steps

• Establish working group(s) to obtain stakeholder input, evaluate proposals, develop recommendations.

- Initial kick-off meeting took place on October 3

• Establish recurring discussion at monthly ROP public meetings, with separate focused public meetings on specific ROP enhancements.

- Planning public meetings on October 18 and November 15 (tentative)

• Commission approval will be needed for any significant changes to the ROP, per Management Directive 8.13.

8