Text

Transcript of the 657th Advisory Committee on Reactor Safeguards Full Committee Meeting - October 4-6, 2018
	ML18303A086
Person / Time
Issue date:	10/04/2018
From:	; Advisory Committee on Reactor Safeguards
To:	;
	Banks M A
References
	NRC-3917
	Download: ML18303A086 (327)
	v • d • e

Official Transcript of Proceedings NUCLEAR REGULATORY COMMISSION Title: Advisory Committee on Reactor SafeguardsDocket Number:(n/a)Location:Rockville, Maryland

Date:Thursday, October 4, 2018Work Order No.:NRC-3917 Pages 1-NEAL R. GROSS AND CO., INC.

Court Reporters and Transcribers 1323 Rhode Island Avenue, N.W.

Washington, D.C. 20005(202)234-4433 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.

(202) 234-4433 WASHINGTON, D.C. 20005

-3701 www.nealrgross.com 1 1 2 3 DISCLAIMER 4 5 6 UNITED STATES NUCLEAR REGULATORY COMMISSION'S 7 ADVISORY COMMITTE E ON REACTOR SAFEGUARDS 8 9 10 The contents of this transcript of the 11 proceeding of the United States Nuclear Regulatory 12 Commission Advisory Committee on Reactor Safeguards, 13 as reported herein, is a record of the discussions 14 recorded at the meeting.

15 16 This t ranscript has not been reviewed, 17 corrected, and edited, and it may contain 18 inaccuracies.

19 20 21 22 23 1 UNITED STATES OF AMERICA 1 NUCLEAR REGULATORY COMMISSION 2+ + + + +3 657TH MEETING 4 ADVISORY COMMITTEE ON REACTOR SAFEGUARDS 5 (ACRS)6+ + + + +7 THURSDAY 8 OCTOBER 4, 2018 9+ + + + +10 ROCKVILLE, MARYLAND 11+ + + + +12 The Advisory Committee met at the Nuclear 13 Regulatory Commission, Two White Flint North, Room 14 T2B1, 11545 Rockville Pike, at 8:30 a.m., Michael L.

15 Corradini, Chairman, presiding.

16 COMMITTEE MEMBERS:

17 MICHAEL L. CORRADINI, Chairman 18 PETER RICCARDELLA, Vice Chairman 19 RONALD G. BALLINGER, Member 20 DENNIS C. BLEY, Member 21 CHARLES H. BROWN, JR. Member 22 MARGARET SZE-TAI Y. CHU, Member 23 VESNA B. DIMITRIJEVIC, Member 24 JOSE MARCH-LEUBA, Member 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 2 DANA A. POWERS, Member 1 HAROLD B. RAY, Member 2 JOY L. REMPE, Member 3 GORDON R. SKILLMAN, Member 4 MATTHEW SUNSERI, Member 5 6 DESIGNATED FEDERAL OFFICIAL:

7 DEREK WIDMAYER 8 9*Present via telephone 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 3 CONTENTS 1 Opening Remarks.................4 2 Draft Rule on Emergency Preparedness for Small 3 Modular Reactors.............6 4Annual Operating Reactor Experience Briefing..63 5 Assessment of the Quality of Selected NRC 6 Research Projects............131 7 Adjourn....................199 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 4 P R O C E E D I N G S 1 (8:28 a.m.)

2CHAIRMAN CORRADINI: Okay, the meeting 3will now come to order. This is the first day of the 4657th Meeting of the Advisory Committee on Reactor 5 Safeguards.

6 In today's Meeting, the Committee will 7 consider the following, draft rule on emergency 8 preparedness for small modular reactors and other 9 nuclear technologies, annual operating reactor 10 experience, assessment of the quality of selected NRC 11 research projects, and preparation of ACRS reports.

12 The ACRS was established by statute and is 13 governed by the Federal Advisory Committee Act, or 14FACA. As such, this Meeting is being conducted in 15accordance with the provisions of FACA. That means 16the Committee can only speak through its published 17 letter reports.

18 We hold Meetings to gathering information 19to support our deliberations. Interested parties who 20 wish to provide comments can contact our Offices 21 requesting time after the Federal Register notice 22 describing the Meeting is published.

23 That said, we also set aside 10 minutes 24 for extemporaneous comments from members of the public 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 5attending or listening to our Meetings. Written 1 comments are also welcome. Today Mr. Derek Widmayer 2is the designated Federal official for the initial 3 portion of the Meeting.

4 The ACRS Section of the U.S. NRC public 5 website provides our charter bylaws, letter reports, 6and full transcripts of all full and Subcommittee 7 Meetings including all slides presented at those 8 Meetings.

9 We're received no written comments or 10 requests to make oral statements from members of the 11public regarding today's sessions. There will be a 12phone bridge line. To preclude interruption of the 13 Meeting, the phone will be placed in a listen-only 14 mode during the presentations and Committee 15 discussions.

16 Also, today's Meeting is being webcast. A 17 transcript of portions of the Meeting is being kept 18 and it is requested that speakers use one of the 19 microphones to identify themselves and speak with 20 sufficient clarity and volume so they can be really 21 heard. 22 I'll also remind everybody to please turn 23 off your devices or mute them so we don't have 24buzzing, ringing, beeping during the Meeting. With 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 6 that, our first topic will be talking about the draft 1 rule on emergency preparedness for small modular 2 reactors and Mike Scott will begin the session. Mr.

3 Scott?4MR. SCOTT: Good morning, Mr. Chairman, 5thank you very much. Yes, my name is Mike Scott, I am 6 the Director of the Division of Preparedness and 7 response in the Office of Nuclear Security and Instant 8 Response at the NRC.

9 I am here speaking on behalf of the 10 Steering Committee for Emergency Preparedness for 11 small modular reactors and other new technologies 12 since Dr. Holahan, who would normally be sitting here 13 is on travel.

14 So I'd like to take this opportunity to 15thank the ACRS Committee for allowing us the 16 opportunity to discuss with you the emergency 17 preparedness for small modular reactors and other new 18technologies, or EPSMR ONT, proposed rulemaking. It's 19 a lot to say.

20Seated at the table with me are Kenny 21 Thomas from NSIR, Andrew Carrera, the rulemaking PM 22 from the Office of Nuclear Material, Safety, and 23 Safeguards, and Arlon Costa from the Office of New 24 Reactors.

25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 7 We have Working Group Members in 1 management from various Offices here at the NRC who 2 are also here this morning to support the 3 presentation.

4 Key Staff Members include Steve Lynch from 5 NRR, Ed Roach from NSIR, Howard Benowitz, and Marcia 6 Carpentier from OGC, Michelle Hart from NRO, and Keith 7 Compton from the Office of Research, all in attendance 8 to assist in addressing questions the Committee might 9 have. 10 As most of you are aware, on August 22nd, 11 the Staff met with the ACRS Joint Subcommittee to 12 discuss the draft proposed rule and proposed guidance.

13 At the Meeting the Staff discussed the background and 14 regulatory basis associated with this rulemaking, as 15 well as the approaches the Staff has taken to the 16 rulemaking.

17 Staff appreciates the comments and 18 feedback that we have received from the Subcommittee 19 regarding the proposed rulemaking.

20 The Staff has considered the 21 Subcommittee's comments in preparing for today's 22 Meeting, and the goal for today's Meeting primarily 23 for us is to address the ACRS Subcommittee's comments.

24 We understand that it was the 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 8 Subcommittee's preference that we focus on these 1 subjects and so that's primarily what you'll hear 2 today. Of course, if there are additional questions 3 that come up, we will of course answer those.

4 We will also provide an overview of the 5rule and the philosophy behind it. As previously 6 presented in the Subcommittee Meeting, the Staff is 7 proposing new EP requirements and implementing 8 guidance to adopt a consequence-oriented, 9 risk-informed, and performance-based approach to 10 emergency planning, as well as one that is technology 11 inclusive.

12 The proposed rule would provide all 13 existing and future SMR and non-light water reactor 14 licensees applicants and future utilization facility 15 licensees that would be licensed after the effective 16 date of the rule, an alternative to develop and to 17 comply with a performance-based EP rule.

18 The proposed rule would be an alternative 19 to the existing deterministic EP requirements that are 20in 10 CFR Part 50. Because it is optional, it is not 21 a backfit and backfit considerations do not apply.

22 One of the aspects of the rule that is 23most transformational I'd like to highlight is the 24 concept of making EPZ sizing proportional to the risk 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 9posed by the facility. The NRC has a long history of 1 doing that in a more case-specific manner.

2 If you think back to the days of Fort 3 Saint Vrain and Big Rock Point, those were examples of 4 reactors that operate at power reactors that operated 5 with a smaller EPC size than the 10 miles that is 6 normally specified for large, light-water reactors and 7 that is, in fact, the case for all of the existing 8 fleet because those small reactors with lower risks 9 are no longer in operation.

10So that has been the case for a 11substantial period of time. We also do not require 12 offsite Federally-mandated licensee-funded 13 radiological emergency preparedness plans for research 14 and test reactors.

15 So this is not new to us, the concept is 16 not new, but the opportunity for an Applicant or 17 licensee to demonstrate the appropriate EPZ size based 18 on risk posed by credible accidents regardless of the 19 technology involved is new.

20 So to take that to its full extent, there 21 is the potential for a site boundary EPZ, that is to 22 say for a facility that has a particularly low risk, 23 to have the EPZ end at the site boundary such that 24 there would be no offsite Federally-mandated EPZ.

25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 10 So that is a transformational process for 1the NRC. The staff has developed an associated draft 2 implementing guidance document, DG1350 for use by 3 licensees, applicants, and the NRC Staff.

4 The draft proposed rule is on track to be 5 submitted to the Commission in October of this year 6 for a vote prior to issuance for public comment.

7 Later in the presentation, Andy will provide you will 8 further details regarding the rulemaking deliverables 9 and scheduling.

10 I'd like to especially acknowledge and 11 express my appreciation for the Working Group Members 12for all of the outstanding work that they have done 13 associated with this rulemaking effort. It's been a 14 substantial rule and we believe it's come out in a 15 good place.

16 We look forward to an informative 17 interaction with the ACRS today, and with that, I'll 18 turn the presentation over to Kenny Thomas.

19 CHAIRMAN CORRADINI: Dennis, go ahead.

20MEMBER BLEY: The words credible accident 21has come up many times in the discussion. We're more 22than 40 years since WASH1400. We speak of risk as 23consequences and likelihood. What does credible mean 24 and why is it in here if this is --

25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 11 MR. SCOTT: What does incredible mean?

1 MEMBER REMPE: Credible.

2CHAIRMAN CORRADINI: I think, Dennis, you 3 meant to say what does credible mean? Either one.

4MEMBER BLEY: Our risk concept is based on 5 credibility and consequences and credible seems to 6 have no point unless you put the number to it and you 7 want people never to think of anything out there.

8 It's about risk-informing.

9MR. THOMAS: Thank you, Dr. Bley. I'm 10 Kenny Thomas and I will be leading the presentation 11 this morning but I would like to call on a couple of 12 our key Staff Members to be able to at least mention 13 their analysis or what we have at the NRC that 14 addresses credibles.

15 First up would be Michelle Hart from the 16 Office of New Reactors. Michelle?

17MS. HART: I'm Michelle Hart from the 18 Office of New Reactors.

19 I work in the Radiation Protection and 20 Accident Consequences Branch and, yes, the term 21 credible in the regulation is intended to mean we're 22 talking about if you look at a range of accidents but 23 we don't want you to have to do anything that is very 24 unlikely.

25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 12It's not intended to be just a design 1basis accident or something like that. We don't have 2 a specific definition of credible in this document or 3 in the regulation, nor do we have a definition of 4 credible anywhere else that I'm aware of.

5CHAIRMAN CORRADINI: So let me follow 6Dennis. So what I hear is that it gives the Staff 7 wiggle room to determine what sequence, frequency is 8 above the line or below the line.

9 MS. HART: Right, we're looking and this 10 is something that as we review, looking at the 11 specific design and looking at the PRA information and 12 other information about the plant design, we can 13determine the universe of scenarios that should be 14 included in this range of accidents.

15 And that's what credible is intended to 16 imply. 17CHAIRMAN CORRADINI: Dennis? Can you 18 repeat that? The universe of what?

19 MS. HART: Accident scenarios. Universe 20 of accident scenarios.

21MEMBER REMPE: Well, I'll ask, no one has 22 said the frequency word and cutoff frequency word at 23 this time, and that's the question that's kind of 24 being danced around here, right?

25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 13MR. SCOTT: That plays into what credible 1 is.2MEMBER REMPE: It should, yes. That plus 3uncertainties, yes, it definitely should. But nobody 4 wants to bite the bullet and say this is what the cut-5 off frequency should be.

6MS. HART: This is Michelle Hart again.

7 I think that's something to be determined based on 8 specific analysis that we're looking at for the 9 specific design.

10 We're not pre-defining what credible is 11 based on a CDF or something like that, especially 12 since some future reactor design, non-light-water 13 reactor designs, or even non-light-water reactors, 14 other types of facilities may not have things such as 15 core damage.

16 So core damage frequency, it may be more 17 related to a release frequency and so we don't know 18 all these designs at this time and so we're not making 19 a pre-decision. We'll look at the total amount of 20 information about that plan to help make that 21 decision.

22CHAIRMAN CORRADINI: So, I'm going to 23pretend to be the company. So I'm coming in blind as 24 to what's expected? I'm putting you on the spot but 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 14 on the other hand, that's what I sense.

1 In other words, you could say go look at 2 0396 and supposedly, there's some sort of logic and 3 methodology in 0396 to years ago and at least you 4 ought to --

5MS. HART: That will give you some idea of 6 the information that you would have and certainly, and 7 we can talk about this some more, there's an 8 expectation in the regulation that you took out a 9 probabilistic risk assessment that you describe in 10your final safety analysis report and described the 11 results.

12 So, there's an expectation that you will 13 have looked at the risk of your plant in the first 14 place so you should have sufficient information to at 15 least get started on this.

16CHAIRMAN CORRADINI: Dennis is back.

17 Dennis?18 MEMBER BLEY: Yes, sir?

19 CHAIRMAN CORRADINI: Did you hear all of 20 this or part of it?

21MEMBER BLEY: I've heard part of it. I 22 got knocked off the line for a little bit.

23CHAIRMAN CORRADINI: Do you have a follow-24 up?25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 15MEMBER BLEY: I'm still where I was at the 1beginning. The words incredible really have no 2business in anything that's risk-informed. If the 3 kinds of answers are big enough, you want the 4 likelihood very, very small.

5 If the consequences are low, you don't 6care so much. And undefined terms that we use when we 7 just did deterministic looking are not --

8MEMBER MARCH-LEUBA: Is it the 9 understanding of the Staff that credible means 10 frequency or adverse consequences? That's I think what 11 Dennis is asking.

12MS. HART: The Staff intends that for each 13 design they will be able to say what kind of releases 14are coming from that plan and what are credible for 15 their specific design, what are likely to happen for 16 their design.

17 And so, yes, frequency can be used and is 18 likely to be used because they will have PRAs to be 19 able to define the likelihood of these events and the 20 consequences.

21 So, yes, the intent is to use both 22 aspects, the frequency and the consequence.

23MEMBER BLEY: If I could jump in again, 24 credible as you just said is the result of the PRA 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 16 where you've looked at the whole range of 1 consequences, rather than something going in, the way 2 it's stated before that discussion sounds like there's 3 a whole class you don't even think about.

4 You draw a line somehow and that one just 5 doesn't fit the content.

6MR. KAHLER: If I can, this is Bob Kahler, 7 I'm Branch Chief with NSIR DPR overseeing the 8 technical aspects of the rulemaking.

9 With regards to the EP rulemaking, the use 10 of the term for credible accidents, where we are 11 identifying, that is something that would be 12 predetermined in order to be used, in order to be part 13 of the risk-informing of the EPZ size and of the 14 regulation.

15 So, this is something that we looked to 16 the other Offices to be able to make that 17 determination. As it goes through the design review 18 process, that suite of accidents would be identified 19 by the design.

20 And 0396 had the luxury of having those 21 accidents already available to them from the current 22 designs that they were looking at for the ten-mile 23 EPZ. 24 We have that same approach that we're 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 17 utilizing here for the credible accidents that were 1 identified through WASH 1400 and such, were part of 2 the analysis done in 0396 as to what credible 3 accidents would be utilized to determine the 10-mile 4 EPZ size or to determine the EPZ size.

5 So, when you look at it, we are just 6 saying that the term credible is something that we are 7 utilizing within the EP rulemaking as the designs have 8 already been approved and predetermined with that 9 suite of accidents.

10 I just want to make sure we understand 11 that the EP rulemaking piece of this is -- we're not 12 defining the term credible within the EP rulemaking.

13 It is something that is going to be provided for the 14 designers and for those that will be determining and 15 proposing EPZ sizes.

16CHAIRMAN CORRADINI: So let me quote from 170396. There was a sentence in here that, to me, said 18it and it sounds to be novice, risk-informed. To 19 state this is on Appendix 1, Page 10.

20To restate this, there was about a one 21 percent chance of emergency plans being activated in 22 the U.S. beyond the recommended EPZs within the next 23few years or within the next time period. It seems to 24 me in an uninitiated fashion that's close to a 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 18 risk-informed criteria.

1 Now, I could add onto it that I want a 95 2 percent confidence relative to uncertainties but it 3 seems to me some sort of guidance like this is going 4 to have to be there whether it's in the rule or it's 5 going to be in a reg guide connected to the rule.

6 Otherwise, the incoming industries are 7 going to be -- we need a rock and you're going to tell 8them not that rock, go find me another rock. And I 9 just don't find that to be an acceptable path forward.

10 That's what I sense Dennis is asking.

11MEMBER RAY: Maybe adding to what Michael 12 just said, this discussion suggests that maybe there's 13 room for some elaboration such as we just heard about 14 the use of the word credible.

15 It's going to be determined in the 16 following manner or something like that.

17MEMBER REMPE: So, if we are done with 18 that discussion, because I have a different question 19 for you. 20 CHAIRMAN CORRADINI: Just a second.

21 MR. COMPTON: This is Keith Compton from 22 the Office of Resources.

23 Just to get back to your point, I'm 24 familiar with that statement about the likelihood and 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 19 I think one of the key things that NUREG 0396 did, and 1 it's all through the document, is they were saying 2 that you should have considered not simply of design 3 basis accidents but beyond design basis accidents.

4 And what we are trying to do is keep that 5 philosophy and that's true there, that in the 6 methodology credible is not limited to design basis 7documents. It would include your consideration beyond 8 design basis.

9 What NUREG 0396 did you is that for beyond 10 design basis accidents, it included the consideration 11 of their frequency. It did it in a certain way that 12used the frequencies and the consequences from WASH 13 1400. So we're keeping that idea.

14 The question of credible is something 15 that, as I think Michelle just pointed to that, is 16 another layer. Is there something that is simply so 17 either low frequency or for which you make an argument 18 that it just simply is not credible?

19 That is something that would have to be 20 assessed by the Staff. It would be very challenging 21 for me right now to give a single prescriptive 22 guidance that would cover all possible circumstances.

23 So right now we're simply putting this information 24 out. 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 20MEMBER RAY: But I don't think we would 1 suggest that kind of detail.

2 Just some elaboration that indicates 3because this goes out to a community of people that 4 aren't invested in this the way we all are, indicating 5 that what defines the credible or what is the credible 6accident will be determined in whatever manner you 7say. A sentence or two would be ample in my judgment.

8 MEMBER BLEY: This is Dennis again. The 9 way it was just phrased is pretty good.

10 I have a strong bias against language 11 because of the way it's been used in the past but if 12 you anchor it to those concepts and that is really 13 coming from the risk analysis and it goes beyond 14 design basis, that would certainly help.

15 Right now, it's just there is something 16 for which everyone has their own definition.

17CHAIRMAN CORRADINI: I'm sorry, Joy, go 18 ahead. I guess now we're on a different subject.

19 MEMBER REMPE: I would like to explore a 20 little bit more about what happened with Fort Saint 21Vrain. It started off being authorized or licensed by 22 the Atomic Energy Commission.

23When did 0396 come out? When was the 24planning zone selected? Was NRC really the 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 21 organization that authorized the EPZ because of timing 1 when 0396 came out?

2MR. SCOTT: That's even before my time, I 3 hate to admit it.

4 MEMBER REMPE: Yes, I was in high school 5 back then.

6 MR. SCOTT: Do we have somebody here who 7 can speak to the details of the way Fort Saint Vrain 8transpired? Bob, surely you weren't around for that?

9MR. THOMAS: Dr. Rempe, again, Fort Saint 10 Vrain, we were in the midst of the post-TMI 11 rulemaking.

12 We had published the proposed rule in 1980 13 and there were three plants that applied during the 14 proposed rule phase and said, hey, could we get 15 reconsidered instead implementing a ten-mile emergency 16 planning zone around our plant, specifically Fort 17 Saint Vrain, Big Rock Point, and La Crosse?

18 They came in during the proposed rule 19 phase and said we propose a five-mile plume exposure 20 pathway emergency planning zone and a 30-mile 21 ingestion pathway emergency planning zone around our 22 plants. 23 They provided that technical 24 justification, the Staff reviewed that technical 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 22 justification and agreed to impose a 5-mile, 30-mile 1 emergency planning zone around those three facilities.

2 And that all happened prior to the 3issuance of the 1980 rule.

So if you were going to 4look back at the rule in Section 50.33, you'll see 5 that specific exclusion is in 50.33G, Paragraph --

6MEMBER REMPE: Out of curiosity, what kind 7of justification? Did you go to that level of detail 8to figure out what they provided that gave you that 9 confidence? Did they give you some risk assessment?

10MR. THOMAS: I do have those documents, I 11 did not bring those with me today but they are 12available. Mr. Ed Roach is approaching the 13 microphone, I can turn it over to him.

14MR. ROACH: Hi, I'm Ed Roach, I work in 15 NSIR. I do have the Big Rock Point document that we 16 took off with the microfiche, and basically, they did 17 an analysis of the releases and where they would 18 exceed the EPA tags at that point.

19 And that's how they came in and requested 20 via the NRC to set the EPZ at five miles for Saint 21 Vrain and Lacrosse, and also provide a document like 22 that. 23 And that was prior to the final FRN being 24 issued to the final rule for the EP in 1980. It all 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 23 reflected in that final rule but it happened as a 1 result of the first TMI actions.

2MEMBER REMPE: So when you said they 3 provided the releases, for what type of events beyond 4 design basis events?

5MR. ROACH: An example, I can put this to 6the Committee if you want but it's in our records, 7 what the fission yield for activity and then the 8 analysis with meteorology for that site, why they 9 justified setting it at 55.

10MEMBER REMPE: I'm just curious if they 11 went beyond it to the beyond design basis events is 12 what I'm trying to get to.

13MEMBER BLEY: This is Dennis, may I sneak 14 in? 15 MEMBER REMPE: Sure.

16 MEMBER BLEY: I kind of hate to admit it 17 but I was around then.

18 MEMBER BROWN: You're not the only one.

19 MEMBER BLEY: Two things happened at the 20 same time. WASH 1400 in about 1973 then got revised a 21 little bit. The shift from AEC to NRC happened in 22 about 1974, as I remember 1975.

23 The reactor safety study came out as WASH 24 1400 under the AEC and later was published under a 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 24NUREG number under the NRC. But the Agency didn't 1 really change on the regulatory side.

2 What the process did was put up the 3promotional side in the IRDA and later DOE. So the 4regulatory side is pretty consistent. I think we'd be 5 smart to just consider AEC through NRC as the one 6 Agency that changed names on the regulatory side.

7 MEMBER BALLINGER: By the way, 50.33G 8does not say 5 and 30. It says the size of EPZ may 9 also determine on a case-by-case basis for gas 10 pollutant reactors and for reactors with an authorized 11power level of less than 250 megawatts, thermo. So it 12 didn't say, it didn't give a number.

13MR. THOMAS: No, it didn't and it was that 14 case-by-case analysis that this rule really does 15 intend to say for small modular reactors, less than 16 1000 megawatts thermal, and other new technologies 17 based on the design we're going to take a look at that 18 on a case-by-case basis.

19MEMBER BALLINGER: So this is a little bit 20 ambiguous if you talk about NuScale because one 21 NuScale module is X, Y modules is Y.

22 MR. THOMAS: Thermal?

23 MEMBER BALLINGER: Yes, sir.

24MR. COSTA: This is Arlon Costa but the 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 25definition that we have in the rule, we also define 1 small modular reactor that's anything less than 1000 2megawatts thermal and so NuScale falls into that 3 category.

4MR. SCOTT: I believe what he's referring 5 to is you get a 12-pack of NuScales, more than 1.

6 That's the distinction.

7CHAIRMAN CORRADINI: I think we can go 8 ahead now. We've excavated this as far as we can so 9 far.10 MR. THOMAS: Sounds good.

11 CHAIRMAN CORRADINI: On your first line.

12MR. THOMAS: We are still on track believe 13 it or not. We can finish this.

14MEMBER BALLINGER: Do you want any 15 morphine for the root canal?

16MR. THOMAS: No, I've got plenty of water.

17 So, anyway, thank you again for these great comments 18 that really set the stage for this presentation, a 19 lively discussion and your comments.

20 I took notes through this so we'll 21continue to do that. In this presentation we will 22 discuss the proposed rule and the thinking that went 23into this rule. So Andy, if I can get the next slide, 24 please.25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 26 Okay, the proposed rule would be 1 technology-inclusive.

2 It would provide an option to existing 3 small modular reactors or SMRs, we've been saying this 4 morning, and non-light-water reactor Applicants as 5 well as all future small modular reactor and other new 6 technology facilities licensed after the effective 7 date of the rule.

8 The Staff was tasked to develop a rule 9 that is technology-inclusive, which means that it will 10 apply to a wide variety of designs, most of which the 11Staff has yet to see. The Staff does not have, as we 12 pointed out already, or need the information about the 13 source terms in order to develop the emergency 14 preparedness rule.

15 Source term information is developed and 16reviewed under other processes. Therefore, the 17 requirements for developing the various source terms 18 and their other uses are not addressed by this rule.

19 CHAIRMAN CORRADINI: So let's stop there 20 for a minute. What are the other processes?

21MR. THOMAS: That's a great question.

22Andy, can we go to the last slide? We added a slide.

23MR. CARRERA: This slide was developed 24really late last night. Dr. Corradini developed some 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 27 of the responses in answer and the Staff developed 1 these slides and it's included in the back of the last 2 slide of the package.

3 And it really speaks about the Accident 4Source Terms. Unfortunately, it was late so it was 5 not included.

6 CHAIRMAN CORRADINI: But where I'm going 7 with this is this is just one Member speaking but the 8 fact that I have one paragraph in Appendix A and one 9 sentence in the rule strikes me as a bit on the 10 minimal side, where one needs to go with this because 11 it leaves it wide open.

12 And I connect it back to Page 10 of 13 Appendix 1 of 0396 is that the reason I picked that 14 sentence is they then connected, as I think Ken noted, 15 to a set of both design basis acciden ts and severe 16 accident release source terms and by that combination 17 determined what they thought was the proposed 10-mile.

18 So I'm still back to the same thing that 19 it's leaving the reader of the rule very little 20guidance at this early stage. And it strikes me as 21 not very helpful if you're really trying to help the 22 future industry.

23MR. THOMAS: Thank you, Dr. Corradini.

24 Again, Michelle Hart is at the microphone.

25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 28MS. HART: So I understand what you're 1 saying about the guidance doesn't say specifically 2look at DBAs and look at severe accidents. Is that 3 what you're comment really is?

4 CHAIRMAN CORRADINI: At minimum, I think 5 that's kind of really where Harold was suggesting at 6 a minimum you want to provide some sort of discussion 7 because the sense of it is that's what happened back 8 40 years ago back in the '78 NUREG.

9MS. HART: and that is our intent. So 10 there are requirements that they've develop the design 11 basis accidents to look at the siting and safety 12 analysis, look at control room compatibilities.

13 They're already doing that so that would be something 14that's there. And of course, those have a defined --

15and I'm talking purely about reactors here and I'm 16 talking about light water reactors especially, we do 17have guidance on how to do that assessment. And so 18 it's something that they're already aware of how to 19do. For severe accidents as I had mentioned earlier, 20 there is a requirement for reactors to provide 21probabilistic risk assessment for their designs, and 22 so the development of the source terms for that, they 23 need that to look at the large release frequency that 24 they are required to look at the risk of the plant, to 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 29look at the risk characteristics. So you use those 1 same source terms to develop those source terms, those 2 releases to the environment, you can use the PRA 3standards. And there's one in development for the 4non-light-water reactors as well. And so we're 5 relying on the fact that these other assessments are 6 already required to license a facility and that you 7would have this information. And if I understand your 8 comment, it's that it's not clear in our guidance 9that's what we're relying on. There's other processes 10--11CHAIRMAN CORRADINI: I think it's so brief 12 as to the well-informed would probably know where to 13go. You have a spectrum of individual companies that 14 think they're going to proceed with various designs.

15 I'm not sure all of them are all well-informed.

16 MS. HART: Right, and this discussion is 17 very reactor-focused and so this may not be as 18 applicable to other facilities as well.

19 CHAIRMAN CORRADINI: I've made my point.

20 Thank you.

21MS. HART: Does that answer your concerns?

22 CHAIRMAN CORRADINI: Yes, thank you very 23 much. 24MR. THOMAS: We've been using the term 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 30 other new technologies in this presentation as well as 1 in some of the associated documents to refer to non-2 light water reactors in medical radioisotope 3facilities. However, in the rule we don't refer to 4other new technologies. Rather, we use and define in 5 the regulations non-light water reactors and non-6power- production utilization facilities. In the 7 context of this proposed rule, medical radioisotope 8 facilities to be licensed under 10 CFR Part 50 but 9 also be included within the use of non-power-10production or utilization facilities. This rule 11 proposes to apply the Commission's expectation that 12 advanced reactors would provide enhanced margin of 13 safety and/or use simplified inherent passive or other 14 innovative means to accomplish their safety and 15 security functions.

16 Next slide, please. May the provisions of 17 this proposed rule and guidance would provide a new 18 performance-based emergency preparedness framework, 19 which is an alternative to the current regulations.

20 The framework contains requirements for demonstrating 21 effective response and drills and exercises for 22 emergency and accident conditions, a hazard analysis 23 of any NRC license or non-licensed facility contiguous 24 to a small modular reactor or other new technology 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 31 facility to identify hazards that could adversely 1 impact the implementation of the emergency plans. A 2 scalable approach for determining the size of the 3 plume exposure pathway emergency planning zone, 4 referred to as an EPZ, and a requirement for licensees 5 to describe ingestion response planning in the 6 facilities emergency plan, including the capabilities 7 and resources available to protect against 8 contaminated food and water from entering the 9ingestion pathway. These requirements would apply to 10 those small modular reactor and other new technology 11 facilities that elect to use the rule in Section 12 50.160. 13Next slide, please. During the 14 Subcommittee Meeting, the Staff received several 15comments concerning source terms. The typical source 16 terms for credible accidents for each facilities will 17 be evaluated in conjunction with the remainder of the 18 review for the specific licensing application for the 19facility. The source terms will vary by design and 20 each location will have a meteorology that will be a 21factor in the consequence analysis. Therefore, the 22 consequences from an accident will depend on the 23design and specific location for the facility. To 24 avoid confusion with the purpose of this rule, the 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 32 Staff made no changes to the rule or other document to 1 address source term.

2 MR. KAHLER: If I can, Kenny, Bob Kahler 3 again from NRC DPR, we've heard your concern and your 4 comment and we'll give the due consideration for the 5 guidance document on the elaboration on source term 6 usage and how it is related to the EP rule.

7MR. THOMAS: During the Subcommittee 8 Meeting, the Staff received several comments 9 concerning the technical reasons for the selection of 10 1000 megawatts thermal as the upper bound for the 11 reactor power for light water, small modular reactors.

12 The Staff did not perform an analysis as to whether 13 there exists a characteristic of 1000 megawatts 14thermal about which reactor phenomena occur. The 15 Staff selected the number based on the existing use by 16 several key stakeholders such as the Department of 17 Energy, facility designers, the international 18 community, and even the NRC and the fee rule in parts 19 170 and 171 of Title 10 of the Code of Federal 20 Regulations.

21CHAIRMAN CORRADINI: So, can I restate 22 what you just said?

23 MR. THOMAS: Yes, sir.

24MR. SCOTT: So there is no technical 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 33analysis that limits the rule? It's more by mythology 1 and others?

2MR. THOMAS: I don't think we'd say it was 3 mythology. Use that to extract the comment from you 4but what I didn't hear is a technical analysis that 5 says 1000 is a breakpoint.

6CHAIRMAN CORRADINI: And as a matter of 7fact, you heard just the opposite. It is not a 8 technical analysis. The decision was made based on, 9 for example, assumptions as to the level of interest 10 among the current large light-water reactor fleet in 11this rule and in this process. One of the things that 12 we're going to propose to the Commission that we do is 13 ask the question in the draft rule as to whether 14 there's a view on this matter.

15MEMBER MARCH-LEUBA: Yes, but I don't 16 think this applies as much to the existing reactor.

17 Nobody is going to go to the effort of read through 18 the EPZ, they already have one. It's more if I want 19 to build an ABWR or AP1000, why doesn't it apply to 20 me? 21MR. SCOTT: Again, it comes back to a 22 decision and an assumption about the level of 23 interest. So if we get a comment that comes in from 24 potential interested party in that indicates the 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 34 assumption was made was not a good one, they will 1 revisit it.

2MEMBER MARCH-LEUBA: Okay, let's let Kenny 3 say what he wants to say.

4MR. THOMAS: Thank you, sir.

5 Additionally, in issues related to the selection of 6 1000 megawatts thermal, the NRC did receive a comment 7 on the draft regulatory basis in 2017 that recommended 8 the NRC expand the scope of the rule to include large 9light water reactors. Large light water reactors were 10 not included by the NRC in the scope of this proposed 11 rule because an emergency preparedness licensing 12 framework already exists for these reactors and 13 licensees for those plants have not presented a clear 14interest in changing that framework. Nonetheless, in 15 light of the public comment on the draft regulatory 16 basis and although this proposed rule is written for 17 small modular reactors and other new technologies, the 18 Staff has included a question for public input in the 19specific request for comment section of the Federal 20Register notice. The present-end stakeholders whether 21 the NRC should consider a performance-based 22 consequence-oriented approach to emergency 23 preparedness for large light water reactor sites in 24 fuel cycle facilities and currently operating non-25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 35 power production or utilization facilities.

1MEMBER MARCH-LEUBA: And that included 2future light power reactor sites? It's always 3 existing ones unless there is something unusual, then 4 I wouldn't go to the effort of doing it.

5MR. THOMAS: That question would go to 6 future large light water reactors.

7MEMBER MARCH-LEUBA: Did you see any 8 technical reason why it wouldn't apply?

9MR. THOMAS: Did I see any technical 10 reasons why it would not apply?

11MEMBER MARCH-LEUBA: To a 3000 megawatt 12 thermal reactor.

13MR. THOMAS: I'm an AP guy. I can turn 14 that over to the consequence analysis and the other 15 folks and ask them for their technical opinion.

16 MEMBER MARCH-LEUBA: If the consequences 17 are too large, then they won't be able to have a --

18MR. SCOTT: I'm going to give you a 19management opinion. If we believe that we should have 20 a risk-informed process then we should apply the 21 risk-informed process and see where it comes out.

22 But that does pose potential that you 23 could have either 10 miles or a different number for 24 large light water reactors.

25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 36MEMBER MARCH-LEUBA: 25 miles if you 1analyze this properly. Maybe we made a mistake. Most 2 likely it will be only five but it could while they're 3 away.4MR. SCOTT: Again, since we are focused on 5 being risk-informed and performance-based, then you 6 can take that to its logical conclusion.

7MEMBER MARCH-LEUBA: My comment is it 8 looked like an arbitrary point.

9 MR. SCOTT: It is not based on technical 10 criteria and I would say it's a little more than --

11MEMBER BROWN: I have an information 12question. Where are the existing fleet's power 13 reactors that are 1000 megawatts or less?

14 Some of the early plants that were 15 developed, Shipping Port I thought was a much smaller 16 reactor in the early days so they're all over 1000 17 megawatts.

18 MR. SCOTT: Thermal.

19MR. THOMAS: During the Subcommittee 20 Meeting, the Staff received several comments 21 concerning the hazard analysis updating requirements.

22 An update to the hazard analysis is already part of 23 the rule.

24 The requirement for licensees to update 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 37 the hazard analysis is part of the requirements in 1 Section 50.54 Q2 where the licensee must follow and 2 maintain the effectiveness of its emergency plan.

3 Based on the requirements within 50.54 Q2, 4 the Staff did not add any additional requirement to 5 the hazard analysis itself to be updated on any set 6 periodicity.

7Next slide, please. During the 8 Subcommittee Meeting, the Staff received several 9 comments concerning the selection of 96 hours0.00111 days 0.0267 hours 1.587302e-4 weeks 3.6528e-5 months for a 10dose criterion. The use of 96 hours0.00111 days 0.0267 hours 1.587302e-4 weeks 3.6528e-5 months is part of the 11 Environmental Protection Agency's protective action 12 guide manual.

13 Specifically in Section 2.2.2, the 2017 14 protection action guide manual, it states the first 96 15 hours1.736111e-4 days 0.00417 hours 2.480159e-5 weeks 5.7075e-6 months , specifically on Page 16.

16 Where dose projections are at levels less 17 than one rem, ten millisieverts, over the first four 18 days, evacuation is not recommended due to the 19 associated risk of moving large numbers of people.

20 The decision-makers may consider 21 implementing sheltering in place when projected doses 22 are below one rem, ten millisieverts over the first 23 four days.

24MEMBER REMPE: So I think I was the one 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 38 who brought this up. I can recall years ago that 10 1 CFR 100 was based on the first two hours and the gas 2 reactor folks said, oh, this is good news because we 3 don't have stuff come out until later and you changed 4 it to the worst two hours.

5 NUREG 0396 is based on a bunch of releases 6from light water reactors and so my question was, 7 well, you might have some sort of a reactor where they 8 might have circulating release early and then it takes 9 a long time to heat up and it might be 99 hours0.00115 days 0.0275 hours 1.636905e-4 weeks 3.76695e-5 months before 10 you get the bad release.

11And I just was curious why are you 12 sticking with the 96 hours0.00111 days 0.0267 hours 1.587302e-4 weeks 3.6528e-5 months ?

13 MR. THOMAS: Well, the first part is the 14 Environmental Protection Agency's protective action 15guides specifically calls out the 96 hours0.00111 days 0.0267 hours 1.587302e-4 weeks 3.6528e-5 months . But also 16 in practice, the 96 hours0.00111 days 0.0267 hours 1.587302e-4 weeks 3.6528e-5 months is also looked at when at 17the facility they start doing dose projections.

18 During the response, the facility performs 19 dose projections that forecast potential doses over 20 the next four days or 96 hours0.00111 days 0.0267 hours 1.587302e-4 weeks 3.6528e-5 months for a four-day 21 integrated dose to determine the emergency 22 classification levels and any protective action 23 recommendations.

24 This ongoing assessment during the 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 39 response looks forward 96 hours0.00111 days 0.0267 hours 1.587302e-4 weeks 3.6528e-5 months each time that dose 1 assessment is conducted to determine whether 2 protective action recommendations or an increase in 3 the emergency classification level is warranted.

4MEMBER REMPE: So what I think you're 5 telling me is that it's because they continuously look 6 96 hours0.00111 days 0.0267 hours 1.587302e-4 weeks 3.6528e-5 months ahead so it's not just the first 96 hours0.00111 days 0.0267 hours 1.587302e-4 weeks 3.6528e-5 months .

7 Is that what you're saying and what you're reading 8 there?9MR. THOMAS: The 96 hours0.00111 days 0.0267 hours 1.587302e-4 weeks 3.6528e-5 months that we used 10 were specifically in the rule where we're looking at 11 the first 96 hours0.00111 days 0.0267 hours 1.587302e-4 weeks 3.6528e-5 months because that's tied to the 12 Environmental Protection Agency's protective action 13 guide. 14MEMBER REMPE: So it's based on LWR 15 evaluations?

16 MR. THOMAS: Yes, ma'am.

17CHAIRMAN CORRADINI: Did you have more to 18 say?19 MR. THOMAS: No, sir.

20 MR. KAHLER: Again, Bob Kahler from NSIR 21DPR Branch Chief. It's not based upon large light 22 water, it's based upon the 1 rem of 96 hours0.00111 days 0.0267 hours 1.587302e-4 weeks 3.6528e-5 months for 23 determination of immediate protective actions for 24 public health and safety.

25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 40 So, what we're looking at is setting the 1 EPZ size based upon what immediate protective actions 2 need to be taken possibly within the first 96 hours0.00111 days 0.0267 hours 1.587302e-4 weeks 3.6528e-5 months to 3set that EPZ size for the implementation of the EPA 4 PAGs. 5 So, when Kenny is talking about the dose 6 projections that would go out from the beginning of 7 the accident for the first 96 hours0.00111 days 0.0267 hours 1.587302e-4 weeks 3.6528e-5 months .

8 And if that is what EPA says you need to 9 take immediate protective actions that exceed one rem, 10 then we need to establish an easy size to accommodate 11 how far out those projections would take that.

12 So that would be setting the EPZ size.

13 But that's how that 96 hours0.00111 days 0.0267 hours 1.587302e-4 weeks 3.6528e-5 months comes in play with the 14 determination of the EPZ size.

15CHAIRMAN CORRADINI: But I think what Dr.

16 Rempe is getting at, though, is depending on 17technology, you may have to reevaluate that going 18 forward and that's why I'm still trying to get a clear 19 picture from the NRC standpoint that is still going to 20 be looked at beyond the first order.

21 And I think that's what her question --

22 MR. KAHLER: That is still a requirement 23 of the regulation, is to continually do the dose 24 projections. Even if you do not have an EPZ size but 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 41 you have them on the site boundary.

1 You're still going to have to continually 2 assess that and if at any time during that time period 3 you would believe then that the offsites would be able 4 to implement some sort of offsite protective action 5 without the need to have the formal offsite 6 radiological emergency preparedness program in place 7 because that 96 hours0.00111 days 0.0267 hours 1.587302e-4 weeks 3.6528e-5 months would give you that leeway for 8 them to take that protective action.

9MEMBER REMPE: So this helps a lot. It 10 would be nice if it was somewhere in the 11 documentation. I didn't see it there but maybe I've 12 missed it.

13 MR. KAHLER: To further clarify using of 14 the 96 hours0.00111 days 0.0267 hours 1.587302e-4 weeks 3.6528e-5 months , thank you and we're going to watch for 15 that comment.

16MEMBER BROWN: Could I ask a question 17before you flip? You talked about the multiple 18 modular considerations and you talked about your 19 regulatory position, and that's the hazard analysis of 20 nearby adjacent or contiguous facilities.

21 And it just says your analysis should 22 identify the specific hazards proposed by multimodular 23nuclear units or multimodular units. I didn't see any 24 criteria by which you might expect them to evaluate 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 42 those. 1 And I'm just thinking about the NuScale 2 situation where you've got 12 crammed into a building 3 but there's no guidance put forth at all in terms of 4 how you should assess the relevant power levels and 5 how they should be addressed for the emergency 6 planning zone considerations.

7 It just says look at it and identify but 8where's your breakpoint? Do you have any? If you 9 look at the specific example which we're all well 10 aware of, that's a pretty big, large, power unit when 11 you combine them all together.

12 And that's different from having 10 units 13 on a site spread around half a mile or a mile 14 separating them all.

15 I'm just taking some type of other example 16 where you could have a big site with multiple regular 17 light water reactors and there's just no guidance, no 18 identification of anything in terms of how they should 19 just be performance analysis and tell us what you've 20 got, and they'll figure out whether we accept it or 21 not, I guess.

22 MR. THOMAS: You're right, there is very 23little guidance, it's not a rule. You're right, there 24 is very little guidance in that except for what we had 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 43 in the Regulatory Position Number 8. The reason for 1 that is it's technology-inclusive.

2 We don't know which technologies would be 3 brought to us, which designs would be brought to us, 4 or how these would actually be sited.

5MEMBER BROWN: Why does that matter?

6 Damage to a plant or an accident in a plant creates a 7 hazard situation where you need an EPZ.

8 Why does that matter relative to the fact 9 that the criteria should be able to be assessed based 10 on regardless of the technology?

11 Radiation is high levels or particulates 12 or spread of contamination is a function of 13 contamination and not necessarily what produces it.

14MR. THOMAS: Yes, sir, what we were 15looking at was the potential external hazards unless 16the facility itself. That's why we looked at non-NRC 17 license.

18 For example, military installations, 19 transportations, so it's more focused on the licensee 20 or the Applicant should be aware of where they're 21 putting this.

22MEMBER BROWN: You're talking about 23 contiguous facilities as opposed to the facility?

24 Isn't that somewhat short-sighted?

25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 44MR. THOMAS: Well, we also talk about the 1 internal multi-modular considerations specifically.

2MEMBER BROWN: But the only words say 3 multi-modular. There's still no criteria.

4 CHAIRMAN CORRADINI: There's no criteria 5 on the source term either.

6MEMBER BROWN: I didn't mention that 7 earlier. I was going to.

8 CHAIRMAN CORRADINI: Again, I'm going to 9 break in.

10 I do think there are current plants that 11 have shared ultimate heat sinks that would have to 12 then -- that are being considered or should be 13 considered from a probabilistic standpoint just as we 14 would have hear in a shared open heat sink.

15MEMBER BROWN: I understand that. It's an 16interesting question the way this is done. The 17 differentiation between other nuclear technologies as 18 opposed to light water reactors, et cetera, I think it 19 produces a problem accident-wise and it doesn't matter 20 what the technology is.

21It's a problem. There are to be criteria.

22 You made a statement earlier and I can't remember 23 which of you all made it but this performance-based 24 risk-informed, the magic words that we're using now, 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 45 could result in no EPZ at all.

1 I didn't say anything at the time, it just 2 took me back and that from a public health standpoint, 3 how can you not have any EPZ at all established other 4 than there's a boundary going around the plant that's 5got a fence around it and you can't get bad guys to 6 get in? 7That's a different issue. From an 8 accident standpoint or a criteria standpoint, I have 9 a hard time coming across no EPZ at all and that's the 10 way I would have read this to see that you could end 11 up with -- and you made that statement.

12MR. SCOTT: I made the statement so allow 13 me to clarify.

14MEMBER BROWN: You're all part of the 15 family. 16 MR. SCOTT: But let me clarify it. So I 17 didn't intend to say no --

18 MEMBER BROWN: But that's what you could 19 get out of this.

20 MR. SCOTT: What you can get, certainly, 21 there will always be an onsite radiological emergency 22 preparedness program.

23MEMBER BROWN: That's different, that's 24 not what this is, though.

25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 46MR. SCOTT: Right, so the delta there and 1the concept of the site boundary EPZ is that the 2 hazard is low enough that an offsite formal Federally-3 mandated, licensee-funded emergency preparedness 4 program, focus radiological emergency preparedness 5 program would not apply.

6 That hazard would be in the range of other 7 hazards that the community needs to contend with and 8that would be covered by the Federal all hazards 9 emergency response program.

10 And state and local all have those 11 emergency response programs so that's not the same 12 thing as say no emergency planning and saying no full 13 up Federally-mandated offsite program would be 14 required because the hazard wouldn't suggest that it's 15 necessary.

16MEMBER BROWN: I have a hard time walking 17that one past, that's my own personal opinion. Thank 18 you. 19MEMBER SKILLMAN: I'd like to ask a 20question, please. To the concern Charlie Brown raised 21 about contiguous facilities, it appears to me that 22 you've addressed this very thoroughly in your draft 23 guide 1350.

24 And as long as what you put in 1350 is 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 47obeyed when this is finally -- then I think what 1 Charlie is pointing to has been thoroughly and fairly 2 taken care of.

3 The second comment, in this last 4 discussion as Charlie says, a logical extension of 5this discussion is that the facility and the 6technology is such that there really isn't any 7release, there can't be any release. That could be 8 one conclusion.

9 I'm a qualified and experienced Emergency 10 Director and Emergency Support Director. I think in 11 terms of an unusual event, alert side area and 12 general.

13 And for me, general has always been that 14 point when you make the notifications, your 96-hour 15 clock begins or whatever the clocks are, and then you 16stay with the scenario and you stay with the event 17 until you have to adjust what you've communicated in 18 terms of key hold or shelf room place or evacuate.

19 That's the moving target based on what the 20 radiological events offer that aren't naturally 21 occurring.

22 Here's my question. To Charlie's point, 23 do you envision a facility with other technologies 24 where you have no general emergency?

25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 48 MR. KAHLER: If I can, sir? Bob Kahler, 1 Branch Chief for NSIR DPR. To go back to your terms 2 of UE alert site area and GE, the definitions of those 3 classifications are based upon the release and the 4 magnitude of the release, and where that release is 5 occurring and the type of response that is needed.

6MEMBER SKILLMAN: That's why I'm asking 7 the question. I'm well aware of that.

8 MR. KAHLER: So I would envision that if 9 you have an EPZ and it's very plausible to have an 10 emergency preparedness zone of where that one rem 11 exceeds 96 hours0.00111 days 0.0267 hours 1.587302e-4 weeks 3.6528e-5 months be contained within the site boundary 12 but there is no credible accident, if we can go back 13to that, I'm sorry, that would provide for that one 14 rem within 96 hours0.00111 days 0.0267 hours 1.587302e-4 weeks 3.6528e-5 months beyond the site boundary.

15 That would then trigger the difference 16 between the site area emergency and the general 17emergency. Site area emergency would require planning 18 immediate protective actions for those within the site 19 boundary and that is a requirement of the regulation 20 to protect those people.

21 Even members of the public would be 22 contained within that boundary but it would be the 23 licensees emergency plan that would provide for their 24 safety. 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 49 If it extends beyond the site boundary, 1 that is when the offsite formal rep program is needed 2 in order for the offsite entities who have the 3 authority to protect and the responsibility to protect 4 the offsite public, that's when the formal rep would 5 occur. 6 So in response, if you have a facility 7 that provides less emergency planning zone that was 8 within the site boundary, that is saying that they 9 have no credible accident that would exceed one rem 10 within 96 hours0.00111 days 0.0267 hours 1.587302e-4 weeks 3.6528e-5 months beyond that site boundary.

11 Hence, you would not exceed a site area 12 emergency and no general emergency classification 13 would be plausible, which is the same as we're doing 14 right now with decommissioning, rulemaking as we go 15 into the different level two and three and so on and 16 so forth. So, it would follow suit with that.

17 If you determine that you have one rem 18 within 96 hours0.00111 days 0.0267 hours 1.587302e-4 weeks 3.6528e-5 months and you have a site boundary and the 19 licensee can determine how big that site boundary is, 20 that's how we tied it to the area in which the 21 licensee is responsible, that's how it is today.

22 Then if you exceed that, you would have 23 general emergency classifications.

24 MEMBER SKILLMAN: Okay, so the answer is 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 50 yes, you can have the site where you actually do not 1 get to a general emergency.

2 MR. KAHLER: That's correct.

3MEMBER SKILLMAN: And if I'm a vendor with 4 real deep pockets, I might say, great, I'm going to 5 have a shield building and I'm going to have one of 6 these containments, and then because I'm a safety kind 7 of person, I'm going to have the second containment 8 inside that shield.

9And because I've got lots and lots of 10 bucks I'm going to put in a third containment. That's 11 what you're saying?

12 MR. KAHLER: I am saying if they come up 13 with a technology and a design of the plant site that 14 says that as I go through my accident sequences, I 15 have such the design would render 1 rem within 96 16 hours1.851852e-4 days 0.00444 hours 2.645503e-5 weeks 6.088e-6 months within the site boundary because of my design 17 because I've added on to these containments, yes.

18MEMBER SKILLMAN: Thank you, that's the 19 point I wanted to make.

20 Under the right circumstances, the right 21 financial circumstances, one could considerably have 22 site's radiological protection if there is no -- I'm 23 going to use Dr. Bley's term -- there's no numerically 24 defendable basis for release.

25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 51 I made the release so improbably that I 1 cannot get to that 1 rem in 96 hours0.00111 days 0.0267 hours 1.587302e-4 weeks 3.6528e-5 months and even if the 2 accident continues, I still cannot get that 1 rem in 3 96 hours0.00111 days 0.0267 hours 1.587302e-4 weeks 3.6528e-5 months .

4MR. KAHLER: And quite frankly, we do 5 anticipate those kinds of designs being submitted to 6 us. 7 MEMBER SKILLMAN: Thank you.

8 MEMBER BROWN: Just to clarify, I really 9 wasn't -- I was more focused on the onsite multiple 10 units as opposed to contiguous facilities.

11 The location of contiguous facilities 12 initially as well as what might come down 10 or 15 13 years later all pose a thing that you just have to 14 address on kind of an ad hoc basis.

15 But for the initial plant development and 16 stuff, multiple reactors and their configuration of 17 how they're stuffed in poses a different thought 18 process. 19 And that's what I was really referring to 20 in terms of how much power can you stuff inside these 21 boundary conditions on the site and still consider 22 that there's no criteria that we have to worry about.

23 It's just we're going to evaluate it later at some 24 point. 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 52 That just seems to be an open-ended, we're 1 not providing any guidance and it gives the impression 2 that a plant that stuffs 12 plants within 1 building 3 and they're all sitting side by side is 3000 or 4000 4 megawatts thermal and that's just happy because each 5 one of them is only 200 or 300 or whatever it may be.

6 So that's the point I was trying to make.

7It seems to me we ought to have some criteria 8somewhere. I'm not in favor of everything 9 risk-informed, if you hadn't figured that out by now, 10 or performance-based.

11 I think that's kind of a caveat and there 12should be some bottom line that there's a line you 13 don't want to cross, that's all.

14MR. SCOTT: I don't think it would be 15 accurate to say and I don't think you were implying 16 that we're not going to consider the potential for 17 accidents at one or more of the --

18MEMBER BROWN: I don't know how the other 19one is going to go right now. I forgot how many 20 plants are on this.

21CHAIRMAN CORRADINI: I don't think we want 22 to focus on that.

23MEMBER BROWN: I'm not, but it's an 24 obvious example of a multi-unit that's got a fairly 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 53interesting configuration. I'm not accusing you of 1 saying you're not going to look at it.

2MR. SCOTT: And we are going to look at 3that. I think the comment is that we haven't put any 4 detail on that and there's no flesh on that right now.

5MEMBER BROWN: But this does provide a 6 preview of your thinking if nothing else.

7 MR. THOMAS: This also goes back to what 8 are the credible accidents and source terms for that 9 extraneous or external to the specific emergency 10preparedness. The accident happens, what are you 11 doing? 12 Publicly available documents, Dr. Rempe 13 made a notice that several or a couple of the 14 documents were not made publicly available at the time 15 of the Steering Committee Meeting -- I'm sorry, the 16 Subcommittee Meeting.

17 The two documents generalized dose 18 assessment methodology for informing emergency 19 planning zone size determinations, ADAMS accession 20 number ML18064A317 and the required analyses were 21 informing emergency planning zone size determinations, 22 ADAMS accession number ML18114A176 were made publicly 23 available on September 21st.

24 Use of other new technology, during the 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 54 Subcommittee Meeting, the Subcommittee stated that the 1 Staff missed an opportunity to define other new 2 technology in the rule and to use it more in the draft 3 regulatory guidance.

4 The use of the term other new technology 5 is used in the Federal Register notice, their draft 6 regulatory guide, and in other documents.

7Next slide, please. The Staff is 8 proposing that Applicants who select to comply with 9 the new rule provide an analysis that supports the 10 requested emergency planning zone size.

11 The requirements would be in Sections 1250.33 and 50.34. For the EPZ size determinations, the 13 size of the emergency planning zone should encompass 14 an area where prompt protective actions such as 15 evacuation or sheltering may be needed to minimize the 16 exposure to individuals.

17 If the Applicant or licensee demonstrates 18 that the environmental protection Agencies protective 19 action guides are not exceeded at the site boundary, 20 then there will be no NRC requirement for offsite 21 radiological emergency preparedness programs.

22 If the proposed emergency planning zone 23 size exceeds the site boundary, then for the distance 24 supported by the analysis there will be NRC 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 55 requirements for offsite radiological emergency 1 preparedness program.

2 The exact shape of the emergency planning 3 zone would need to be determined in relation to local 4 emergency response needs as they were affected by such 5 conditions as population, land characteristics, and 6jurisdictional boundaries. The NRC would engage FEMA 7 in accordance with the 2015 NRC FEMA Memorandum of 8 Understanding.

9Next slide, please. This diagram provides 10 the overall structure of the rule and its relationship 11 to the existing emergency preparedness regulations.

12 If the Applicant opts to use the proposed regulations 13 in 50.160, then the Applicant would have to provide an 14 analysis to support the specific emergency planning 15 zone size.

16 If the Applicant demonstrates that the 17 emergency planning zone is within or at the site 18 boundary, then it's shown in the lower right corner of 19 the diagram that the regulations in the proposed 20 section 50.160, Paragraph C1IVP would not apply to the 21 licensee.

22 If the emergency planning zone would 23 extend beyond the site boundary, then the Applicant 24 would need to address their requirements in proposed 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 56 Section 50.160 Paragraph C1IVA and B.

1 The Staff would then need to engage with 2FEMA for a review of the offsite plans submitted as 3 part of the license application or a permit 4 application as applicable. There is guidance in the 5 draft regulatory guide that support the implementation 6 of the performance-based regulations.

7 Next slide, please. This slide describes 8 the Office of Research Support for the development of 9 the regulatory guidance for the emergency planning 10 zone size analyses.

11 The approach for the scaling and emergency 12 planning zone sizes for small modular reactors and 13 other new technologies was developed to be consistent 14 with the consequences or framework described in NUREG 15 0396 and the scaled appro ach use for operating 16 research and test reactors, fuel cycle facilities, and 17 independent spent fuel storage installations.

18 The Office of Research was asked to review 19 the rationale documented in NUREG 0396 to determine 20 whether the technical analyses described in that 21 document could be generalized to identify a 22 methodology for us by small modular reactors and other 23 new technologies.

24 The Office of Research found that the 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 57 selection of the current planning zone sizes appears 1 to have been based on judgment informed by a variety 2 of lines of evidence rather than being based on a 3 single specific prescriptive dose assessment 4 methodology.

5 The information from both scaling and 6 reanalysis of the information on potential doses from 7 design basis accidents as well as information on the 8 likelihood and consequences of beyond design basis 9 accidents derive from probabilistic risk analysis were 10 considered in developing the rationale.

11 Hence, the proposed consequence-oriented 12 approach would provide the same level of protection to 13 the public health and safety as afforded to other 14 currently operating facilities.

15CHAIRMAN CORRADINI: So can I stop you 16right there? The 0396 methodology did not just 17 include NRC, it included EPA and others.

18 Again, this is personal opinion, so it may 19 not appear in our report but it just seems to me it 20 would be appropriate, I'm trying to look for a better 21 word, but appropriate that the same sort of multi-22 agency guidance is going to be needed for these 23 advanced technologies.

24 Otherwise, you're going to get yourself in 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 58 a potential iterative loop of cycling through 1proposals, another proposal, another proposal. And it 2 would seem to me FEMA, EPA, and NRC together are going 3 to have to come up with some sort of guidance on this.

4 If in the current rule you don't think 5 that's approp riate and you simply want to give 6 qualitative considerations, okay, but eventually, it's 7 going to have come to pass that you're going to have 8 to give better guidance because if you look at the 9 authors of 0396, it wasn't just the NRC.

10MR. SCOTT: Appreciate that comment, thank 11 you. We'll consider it.

12MR. THOMAS: The Office of Research 13 concluded there was sufficient information in NUREG 14 0396 and its references to identify several key 15 assumptions and elements of generalized methodology 16 for informing the emergency planning zone sizes.

17 The Office of Research used these key 18 assumptions in the elements to recommend a list of 19 analyses that a licensee would need to submit to 20 justify their selected emergency planning zone size.

21Next slide, please. The NRC is proposing 22 ingestion response planning requirements instead of a 23 pre-determined fixed distance as part of the 24 performance-based framework for the ingestion pathway.

25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 59 The proposed rule would require licensees 1 who comply with Section 50.160 to describe in their 2 emergency plan the licensee's local, tribal, and 3 Federal resources for emergency response capabilities 4to protect against contaminated food and water for 5 entering the ingestion pathway.

6 A successful quarantine and removal from 7 public access of contaminated food and water products 8 in response to biological contamination demonstrates 9 that a response to pro tect against ingestion of 10 contaminated foods and water can be performed in a 11 rapid manner without a predetermined planning zone.

12 Unlike biological contamination that 13 causes widespread illnesses and only discovered days 14 after infection, a reactor accident would be a leading 15 indicator that long-term actions to protect against 16 ingestion should be considered.

17Next slide, please. Now Dr. Carrera will 18 discuss the status in our path forward.

19 Andy?20MR. CARRERA: Thank you, Kenny, and thank 21 you, Mr. Chairman and Members of ACRS for the 22opportunity to be here. For the purpose of cumulative 23 effects of regulations, I'd like to provide a quick 24 status and path forward for this rule.

25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 60 The draft proposal package was submitted 1 to our senior management EDO for review last week.

2 It's in the process of reviewing. As Mike has 3 previously conveyed, we are on track to deliver the 4 cultural to the Commission valve for 12.

5 The Commissioners would review the packing 6 involved on it so if the Commission approves the 7 publication of the proposed rule, Staff will make any 8 changes to the proposed rule as directed by the 9Commission and publish the proposed rule in the 10 Federal Register for public comment.

11 Now, there would be a 75-day public 12 comment period, however, a specific amount of time 13 will be determined by the Commission.

14 Staff also plans to conduct public 15 meetings during the comment period for the proposed 16 rule to promote full understanding of the proposed 17 rule and guidance, and to inform or facilitate written 18 public comments.

19 Our staff would consider all public 20 comments received on the proposed rule in the 21 development of a draft final rule.

22 The Staff will also conduct an 23 implementation period public meeting as part of the 24 cumulative effects of the regulation initiative during 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 61 the development of the final rule.

1 Thank you.

2MR. SCOTT: Can I just add one more 3 comment?

4 So I made a comment earlier about a view 5 that we should, and I think the NRC does, encourage 6 risk-informing everything we do to the extent we can 7 but I may have left an inadvertently incorrect 8 impression about that regarding the applicability of 9 the approach in the new rule to large light water 10 reactors.

11 As we pointed out and Mr. Kenny mentioned, 12we're putting that out open for comment. However, we 13 believe that the 10-mile EPZ is fully protective of 14 public health and safety for the existing fleet of 15 large light water reactors so we have no information 16 that suggests that we should backfit this new rule or 17 this process or approach on the existing fleet.

18 So I didn't want to leave a mis-impression 19 of that. 20 MEMBER MARCH-LEUBA: I'd like to comment 21on that. My comments on that one were not backfitting 22 the existing reactors but I want to buy an APR1400 and 23place it in Tennessee. This rule doesn't apply to me.

24MR. SCOTT: As I understood it, your 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 62 question is whether it should be available to them.

1 I guess backfit wouldn't apply but there's no 2 intention if somebody comes in with a new ABWR large 3 light water reactor design to require that this rule 4 will be imposed on them.

5 Your question as I heard it is could it be 6 available? Yes, which is a different question.

7 MEMBER MARCH-LEUBA: That's correct. As 8 written it's not available.

9MR. SCOTT: That's correct but open for 10 comment as we go forward.

11MEMBER MARCH-LEUBA: I wonder who the 12 stakeholder will be because I want to build an ABWR in 13 Tennessee but I don't have the funding yet.

14 CHAIRMAN CORRADINI: That's a conflict.

15MEMBER MARCH-LEUBA: So who will be the 16stakeholder that will give you that conflict? A 17 vendor?18 MR. SCOTT: We shall see.

19 MEMBER MARCH-LEUBA: Okay.

20CHAIRMAN CORRADINI: I'm going to turn to 21 Dennis. Do you have other questions for the Staff?

22MEMBER BLEY: No, I don't, thank you. It 23 was a good discussion today.

24CHAIRMAN CORRADINI: Okay, let me go 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 63around and then we'll go to the public. Vesna, any 1comments? I'll just look at you if any of the Members 2 want to make further comments.

3With that, let us turn to the audience 4 here and see if anybody in the room would like to make 5 a comment and then ask if the public line could be 6 open. 7No comments from the gallery here? Is 8there anybody on the public line? If you could please 9speak up? Okay, the public line is open. With that, 10 I think we're done and I'll thank the staff.

11Could you close the public line? Thank 12 you very much and we'll take a break. We're back at 13 10:15 a.m., to stay to the published schedule.

14 (Whereupon, the above-entitled matter 15went off the record at 9:42 a.m. and 16 resumed at 10:15 a.m.)

17CHAIRMAN CORRADINI: Okay. So let us come 18 back together here for our second topic, which is 19summary of reactor operating experience. I'll turn --

20 excuse me. We'll start again. We'll begin with our 21 session on summary of reactor operating experience.

22 And we'll turn it over to Member Skillman to lead us 23 through this. Dick.

24MEMBER SKILLMAN: Thank you, Mike.

25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 64 Colleagues, what we're going to do here is two things.

1 We're going to bring you up to speed on 2017 and mid-2year 2018 fleet performance. And I want to leave some 3 time, about 20 minutes at the end of the session, for 4 the topic of leading indicators.

5If you recall, Matt and I have been 6 kicking around this idea, perhaps others, of how do 7 you know when a licensee is entering into degrading 8performance. And we want to talk about that. And so 9we've got really two presentations back to back. And 10 we expect to finish by 1200.

11Okay. So last year or last December we 12caught up with 2015 and 2016 data. The purpose for 13 today is to talk about 2017 and half of 2018 for which 14 data is available. Next slide, please.

15 Just to remind everybody, the reactor 16 oversight framework really focuses on seven 17 cornerstones.

18 So, for those of you who have been out on 19 the fleet and those of you who watch from a distance, 20 the cornerstones are the initiating events, that is 21 what gets the thing, the event started, what are the 22 mitigating systems, in other wo rds, what are you 23 depending upon to keep those systems functioning so 24 that they perform their appropriate function, issue of 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 65barrier integrity. Emergency preparedness is a 1standalone of a cornerstone. Public radiation and 2 occupational radiation are key for radiation safety, 3 and finally, security.

4 And so we're going to talk about all of 5those in the next approximately 45 minutes. Next 6 slide, please.

7 What is the reactor oversight process?

8 You see the NRC inspection block on the upper left.

9 What that is doing is it's feeding the information 10 from the site.

11 And those actions include just regular 12inspections that are part of the normal day-to-day 13activity by the residents. If there's a finding, 14 there can be a supplemental inspection. If there is 15 an event at the site, the NRC may send in an advanced 16 team for event response.

17 There are the generic safety inspections 18that are the day-to-day cadence of the site. And then 19 there are other inspections that may be called up if 20 there is an event or circumstance that requires 21 further review.

22 All that information finds its way up 23through a significance determination process. And if 24 this significance determination process reaches a 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 66 certain threshold, there may be enforcement.

1 All that information goes up into the 2assessment process. There's an action matrix. And 3this is a living process. This isn't something that's 4 done once every Tuesday or once a year or once every 5 five years. This is an around-the-clock, continuing 6 performance.

7 You've got residents onsite finding 8information, licensees performing. And the process 9 operates at the site as long as the site has a license 10 to operate. Next slide, please.

11How many plants are involved in this? 101 12 is the answer. You can see the breakdown by region.

13And those are all Part 50 licenses. And they are all 14 susceptible to or, if you will, under the magnifying 15 glass of the ROP. Next slide.

16So something happens at the site. What 17 does that event mean in terms of safety significance?

18 What does that event mean in terms of safety 19 significance?

20Well, you see at the bottom green. Those 21 are events that happen or findings that are discovered 22 whose impact in terms of core damage frequency are 23 less than or equal to 10 to the minus 6, one in a 24 million CDF, or LERF less than 10 to the minus 7.

25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 67 If the significance of that event reaches 1 a decade greater, that event will become a white 2 finding, and if it's a decade greater, a yellow 3 finding, and if a decade greater than that, a red 4 finding.5 And I can tell you from years of 6 experience, licensees wish to be at green all the 7time. Occasionally, there will be a white. When a 8 licensee has found himself/herself in a yellow or red, 9that is extremely significant. And we'll see more of 10 that as we proceed.

11MEMBER MARCH-LEUBA: Is there a difference 12 if it is self-reported or if it's found by the 13 inspectors?

14MEMBER SKILLMAN: Well, the application of 15the violation or the penalty may be affected by how 16 the licensee responded to --

17MEMBER MARCH-LEUBA: So you want to see 18 the white, yellow, or red, but you will get the lower 19 finding if you're self-reported.

20MEMBER SKILLMAN: Well, actually, to 21 Derek's credit, we've got a slide coming up for that.

22 But it really has to do with what is the Severity 23 Level IV.24 But on the slide to instruct what is a 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 68 Severity Level IV, you can see how a Severity Level I, 1 II, and III, which would get us into yellow, white, 2red or red, yellow, white, would be handled. And that 3 will answer your question if you just give us a second 4 here. Next slide, please.

5What's a green finding? That's maybe 6 someone failed to sign off on the right line of a 7 procedure.

8What's a greater-than-green? Greater-9than-green only applies to security. So hold that 10 thought for a second.

11 White, yellow, and red. Now, white is a 12moderate safety significance. Yellow is substantial.

13 And red is a high safety significance or a security 14 significance.

15Back to greater-than-green. So let's say 16 in the security area the inspection program discovered 17a major flaw in plant security. It's certainly 18greater-than-green. It might be white, yellow, or 19 red. But that information is SUNSI.

20 And so for all security findings that are 21 greater-than-green, they are simply identified as 22greater-than-green. And those more significant 23security issues are handled SUNSI. That's why you 24 just see GTG, greater-than-green.

25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 69Does that make sense to everybody? In 1other words, if it's a bad security item, you don't 2 want to paint it in the paper. You just simply say 3 it's SUNSI.

4 Now, security will also have or can have 5white, yellow, or red. They're just identified as 6 greater-than-green. Next slide.

7 This is a busy slide, but in a way it's 8not. If you look at a Security Level IV violation on 9 top, this is to answer the question what is a Security 10Level IV or what's beyond greater-than-green. If you 11see the first block is -- I've got to read my own 12 slide here. I've got to increase the magnification.

13 The first block is fail to restore 14 compliance, yes or no. This is a Security Level IV.

15 And if the answer is, yes, they failed to restore 16 compliance, that may lead to a violation, notice of 17 violation. So that is the blue line under the green 18blocks on top. And if you see the D, that is 19discretion. The NRC reserves the right to make a 20 determination of the consequences or the circumstances 21 of that particular failure.

22 If the licensee did not fail to restore 23 compliance, in other words restored compliance 24 promptly, so the answer would be no on that first 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 70 diamond, then did that licensee fail to place that 1 item in the corrective action program, yes or no?

2 So, if the licensee identified the item, 3 put it in CAP, restored the issue, you keep moving to 4the right. Is it repetitive, yes or no? If it's not 5 repetitive, it's a first time finding, the answer is 6no. Was it willful? Did someone actually do this on 7purpose? No. In that case, if you go all the way to 8 the right on that first set of diamonds, that's a non-9 cited violation.

10 And that will be true for every non-cited 11 violation that is a very low threshold level. Those 12 might be just NCVs. And you see those all the time.

13 That's an NCV, a non-cited violation.

14 And if you note also on the left-hand side 15of this slide, the first line is for power reactor 16licensees. The second line item is for all other 17licensees. Those could be for research reactors.

18 Those could be for medical facilities, things of that 19 nature, for all of the other licenses that are not 20 Part 50 power licenses.

21 Also, notice at the bottom of the slide is 22 the escalating process. This is for type I, II, and 23 II, severity I, II, and III items.

24And in that process, you see that the 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 71 inspection protocol is essentially the same. But as 1 you get more deeply into the escalated process, you 2 determine whether or not it's a first willful, was it 3 identified, is there credit for corrective action and 4 so on.5 And then, you know, in the grimmest day, 6you can end up in the lower right-hand corner where 7 you've got a notice of violation and a very 8 significant two times the base penalty.

9 What I also want to point out here, and 10 it's subtle, but if you understand how the regulations 11function, this idea of credit for identification. And 12it's in the second diamond on top. It has to do with 13 placing the issue in CAP.

14What is CAP? That is the corrective 15action program. And that is Criterion XVI of Appendix 16B to 10 CFR 50. That's the QA program. That's the 17 thing Harold and I keep asking about.

18 Where is leadership in understanding 19 Appendix B to 10 CFR 50, because sites that are really 20 tuned in on Appendix B and have a very healthy 21 corrective action program and have a culture that 22 feeds the corrective action program, you normally find 23 those sites with some non-cited violations? But 24they're normally running along that top line. Next 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 72 slide, please.

1 Okay. 2017, and what you're seeing here 2is the four previous quarters in 2017. And what 3you're seeing is four yellow findings. And if you 4recall, those are just below red. That's in calendar 5year 2017. And you see eight white findings. You 6 also see six greater-than-green.

7 So you can get an idea of what the 8population is. And notice that the yellows in 2017 9are initiating events in mitigating systems. The 10 whites are initiating events, mitigating systems in 11emergency preparedness. And I'm going to just tap on 12 those just for a second.

13 There are inspection reports to back up 14the information that I will indicate. But I'm just 15 going to go over them quite quickly. Next slide.

16 2018, there is only one white, and there 17are five mitigating systems. And there are three 18greater-than-green. Now, this is looking back four 19quarters. So those three greater-than-greens are 20either all in 2018 or could be part of the last two 21quarters of 2017. If you check the prior slide, there 22were six and eight in 2017. It looks like those three 23are only in the first two quarters of 2018. Next 24 slide.25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 73 So, going back to 2017 for those four 1 yellows, those four yellows came from Arkansas Nuclear 2 One and Two, Arkansas Nuclear One, which are One and 3 Two. Those came from calendar year 2015 and '16.

4 They affect initiating events and 5cornerstones. And they carried over into 2017. In 6 other words, they had not been closed out.

7 So, even though the events were a year or 8 two earlier and then had to do with the dropping of 9 the rotor and the compromise of some flooding barriers 10 in the unit, those carried over into '17 until the CAL 11had been issued and closed out. The CAL is the 12confirmatory action letter. Those have been resolved 13so they don't show in 2018. But those four yellows 14 showed in 2017.

15 So what were the events in 2017 or that 16were carried in 2017? It was the failure to follow 17 the material handling program when they dropped the 18Unit One stator. And that resulted in the loss of 19off-site power in Unit One and Unit Two. And it 20 demonstrated -- if you recall, it severed a fire main 21and they had some major flooding in lower levels at 22 the plant.

23 And as a consequence of this event at 24 Arkansas, they were placed in column 4 of the matrix.

25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 74 And they were going to do a supplemental, a 95003.

1 The 95 series of inspection procedures are 2 the ones where there is significantly enhanced NRC 3inspection. There is a graded performance from 95001 4 to about 95002, then 95003.

5 And if a licensee were to find himself or 6 herself in 95003 and still be wanting, then the NRC 7 could take the keys for that unit and put the unit in 80350. And that has happened. That's happened to 9 several plants. Finally, next slide, please.

10 ANO also was in a situation with the, with 11unplanned scrams. The site was finally inspected this 12past May, a couple months ago. All of the actions for 13ANO have been completed. The CAL was closed. And 14 they've been moved out of that action matrix column.

15 And that is reflected in the 2018 data from a few 16 slides earlier. Next slide.

17 There were the eight whites that showed 18for 2017 and '18. Here are the plants that are 19identified for those whites. Two plants had 20 initiating event cornerstones. They were Grand Gulf 21 and St. Lucie.

22 There were five in mitigating system 23 cornerstones, Catawba, Clinton, Oyster Creek, Perry, 24 and Pilgrim, and we'll get each of those in just a 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 75 minute, and one plant with emergency preparedness 1 cornerstone, and that was Fermi.

2 So here's the data for these eight events.

3At Grand Gulf, this was scrams. They had a number of 4scrams. And the real issue there was the poor root 5 cause analyses.

6 At St. Lucie, it was configuration 7control. That's Criterion III of Appendix B to 10 CFR 850. It resulted in the reactor trip and loss of 9power, a major change in core damage frequency. Next 10 slide.11 Catawba, it had to do with the excitation 12system for emergency diesel generator. At Clinton, it 13 had to do with the drop out voltages for replacement 14relays with an EDG room vent fan. As a consequence at 15 Clinton, the EDG became inoperable. Next slide.

16 At Oyster Creek, it was the failure to 17 follow manufacturer's instructions on reassembling the 18electromatic relief valve. And at Perry, it had to do 19 with failure to evaluate the effects of a suppression 20 diode.21 Now, what's interesting is when you read 22the inspection reports and you read the licensee's 23 response, there's a very healthy amount of dialogue 24 between the two where the licensee says, well, that's 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 76not exactly what happened, here's what happened. The 1 NRC comes back and says here's the basis for our 2 findings.3 And so there's a negotiated agreement for 4 how in this case the white findings are finally 5dispositioned. So this isn't just a cop coming in 6from the NRC and saying, bingo, you've got a white.

7 There is back and forth to finally arrive at the color 8 finding.9 And you got to realize for a site to get 10 a white or a red, that has major implications, perhaps 11 in insurance, in the in-post standing of that plant, 12 and other similar things. So these are very serious 13 events for the licensee. Next slide.

14 Pilgrim was a failure to correct issues 15 pertaining to a safety relief valve. And this event 16went on for a fairly significant time. That's 17 Pilgrim. Next one, next slide.

18 And at Fermi, it had to do with a 19background radiation monitor. And as obscure as that 20 radiation monitor issue might have been, that was a 21 monitor that was depended upon for making a call 22 regarding a PAR, a protective action recommendation, 23 out of the EAL.

24 So here's the case where a relatively 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 77 obscure instrument plays a major role in the site 1 leadership's ability to create an accurate emergency 2 action level classification and develop the protective 3 action recommendation for the emergency, resulting in 4 a white at Fermi. Next slide.

5 For 2018, no more yellow findings. That 6 means that the ANO yellows have dropped off in 2018.

7 And there is just the one white coming over in 2018.

8 And that is a mitigating systems cornerstone finding 9 at Davis-Besse.

10 In that particular case, it was an 11 auxiliary feedwater turbine bearing gauge glass oil 12issue. And they failed their feedwater pump. And as 13a consequence, there were maintenance issues. But the 14 main thing is that the pump was inoperable for greater 15than the tech spec period. That is the white at 16 Davis-Besse for 2018.

17 Now we're going to talk about the security 18cornerstone. If you go back to one of the earlier 19 slides, you saw security as a cornerstone, number 7, 20 sitting far on the right.

21Derek, can you expand this? Can you jump 22 out of presentation and just put it on, hit escape and 23 that will go to --

24 (Off mic comments.)

25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 78MEMBER SKILLMAN: And in the lower right-1hand corner you can move the vernier and it will 2 expand this.

3 What I wanted to show up and down is you 4 can see the extent of the baseline inspections for 5security. And I would offer that if you haven't lived 6 at a site or spent time in a site, you probably 7 wouldn't appreciate how much work is done on this 8 cornerstone.

9 And I will tell you the inspections are 10 thorough. The site leadership is tuned in to all of 11these pieces. And you will find, as you go over each 12 one of these, it has to do with who can get in, who 13 can get out, what information there is, how the 14security force is trained, where their weapons are 15 stored, how the information is protected, how they 16 prepare for their force-on-force exercises, the 17 fitness-for-duty program.

18 Second from the right on the lower corner 19 there, the target sets, this information in the last 20 40 years has begun or has become basically as large as 21 operating the plant itself.

22 And if you were to go site to site, you 23 would be quite impressed at how large the security 24 teams are to protect the core and to protect the 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 79people who are working at the site. The security 1 cornerstone is really a significant piece. Next 2 slide, please. Derek, thank you for doing that.

3Okay. So plants greater-than-green, there 4 were four in 2016, in calendar year '16. There were 5 three in '17. And data for mid-year '18 is three.

6 Greater-than-green is not a new category.

7 It just means the finding was either white, yellow, or 8red. And the details are SUNSI. So they're not 9 publicly available. Next slide.

10 This will give you an idea of how many 11inspections were completed in 2017. You can see that 12the number is not trivial. This is a major effort by 13the Agency. And it's a large effort by the licensees 14 to support these activities.

15 So, of 250 inspections, there were 3 16 findings greater-than-green in 2017 in the power 17reactors. In the fuel cycle facilities, there were 15 18security inspections. And there were 4 Security Level 19IVs. And there were no greater-than-greens. Next 20 slide, please.

21 And this is just force-on-force. Within 22 the security cornerstone, these are the, if you will, 23the inspections that really tell the tale. This is 24 where you determine the effectiveness of your security 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 80team. And that will give you an idea of how many 1 inspections were conducted in 2018. Next slide, 2 please.3And this aligns with the prior slide. One 4exercise was ineffective. There were three that were 5 marginal. And all of the licensees took appropriate 6corrective actions. And those corrective actions 7 include procedure changes, policy changes, updates, 8 technology improvements, and personnel or security 9force enhancements, all kinds of things. Next slide.

10The NRC reviews cross-cutting issues. If 11 you go back to the second slide, cross-cutting issues 12cut across all of the seven cornerstones. And the NRC 13evaluates whether there are cross-cutting issues. And 14 they do that twice a year at mid-cycle and end-of-15cycle. There were no new items in 2017. And the data 16 has not been reported yet in 2018.

17 This is an important feature of the ROP 18because it prevents group think. It kind of takes the 19 inspection to maybe a 100,000-foot level to look at 20 the entire available information to make sure there 21 isn't something that is emerging that is evading or 22 failing to be recognized. Next slide, please.

23 Safety cornerstones, how you think about 24these things. You can see initiating events. The 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 81 performance indicators are as you see on the right.

1 I'm not going to read all of these. But you can see 2 those cornerstones and what it is that is being 3 measured to assess the robustness of that cornerstone.

4 If no questions, next slide, please.

5Performance indicators. Green is where 6 the performance is within an expected level and where 7all of the objectives are met. And every licensee has 8 a whole set of performance measures and indicators.

9 And those are, if you will, observed or watched on a 10 daily and continuing basis for all the cornerstones.

11 White means the performance, it's outside 12the range where you want to be, but the basic 13 objectives are met. Yellow is there is a reduction, 14a minimal reduction in safety margin. And red is 15 where there's a significant reduction in safety 16 margin. Next slide.

17 For 2017, there were no plants with red or 18yellow. And there were two with white. And we'll 19 show those in a minute here. Next slide.

20 At Columbia, the trigger was unplanned 21 scrams with complications, not scrams, but scrams with 22complications. A scram is acceptable, in fact, 23required within the, if you will, the operating 24experience of the plant. There are times when you've 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 82 got a scram and the plant goes down.

1 The issue is was there a complication.

2 And that complication could be equipment failure to 3 operate, the operator's failure to --

4 MEMBER MARCH-LEUBA: Is the complication 5 after the scram or the complication results in the 6 scram?7MEMBER SKILLMAN: After the scram. So you 8get the buzz. The rods drop or insert. And now 9 things go south very quickly. TMI-2 was a perfect 10scram with complication. Down it went, bam, floor 11 opened. No one saw it.

12So this is scram with complications. And 13at Columbia, it happened twice. It happened in the 14 fourth quarter of '16 and third quarter of '17.

15 Hence, this became a white. Next slide, please.

16 And at Watts Bar 2, now Watts Bar went 17online in late 2016.

So we haven't accrued enough 18hours. But they were struggling. And you can see 19that they were in a situation. And what you do is you 20do the average for the four quarters. So they were 21 scramming I guess quite regularly coming up from their 22 extended, if you will, construction and come back 23 online as a power reactor. Next slide.

24 Performance indicators for 2018, there are 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 83 no red or yellow or white through January to June of 1 2018. Next slide.

2So, in the action matrix, you take the 3 inspection findings and the performance indicators, 4 and that gives a plant assessment. Next slide.

5 All right. So every once in a while you 6 read a news release that such and such a plant has 7 been moved from column 1 to column 2 or it's moved 8from column 3 to column 2. This is what those columns 9are. Column 5 is basically you're an 0350. You've --

10 the NRC has taken your keys. Davis-Besse was one of 11 those.12 Column 4, you've got multiple/repetitive 13degraded cornerstones. We'll talk about two plants 14 that found themselves in column 4.

15Column 3 is degraded cornerstone. Column 162 is regulatory response. And you can move, if you 17 will, from 4 to 3 to 2 to 1 depending on licensee 18 performance and how the licensee resolves the issues 19 that put the unit in the cornerstone category where it 20 once was.21 And, of course, the licensees want to be 22in 1. That is where the licensee is responding. And 23 there is no supplemental NRC regulatory oversight at 24 that site. Next slide.

25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 84 And you can see on this slide, column 1, 1 lower left-hand corner, all performance indicators 2 green, column 2, no more than two white, column 3, one 3degraded cornerstone. It could be three whites or one 4 yellow and three whites in any strategic area. Four 5is multiple/repetitive degraded. And 5 is 6 unacceptable performance.

7 And if you look in the right-hand column 8 for licensee response at the bottom column 1 on the 9 right-hand side, that's your normal and routine 10 inspection. You've got your inspectors. There is a 11baseline inspection program. There's annual 12 assessments. And there's public meetings.

13 If you're in regulatory response column 2, 14you've got a meeting with NRC management. And I would 15tell you, for those of us who've had to go to those 16meetings, they are very uncomfortable. It's not 17because it's pejorative. But it's because there's an 18issue that the site has just not been able to get a 19hold of. And it's very objective. But it's clear the 20 burden is on the licensee to make the changes that are 21 necessary to bring the plant back to column 1.

22 And as you escalate on that right-hand 23 corner from bottom to top, I would say there are two 24things going on. The culture at the site is put to 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 85the test. And the amount of resources that are 1 necessary to bring the plant back to column 1, at 2 least I would say, increase exponentially.

3 Once you get into a situation where you're 4 in column 2 or column 3, you are facing some very 5 significant costs to bring that plant back to where 6you want to be. And I'm not talking a couple hundred 7 K. I'm talking millions and sometimes many millions 8 of dollars to bring that plant back.

9 And so the consequences are very 10 significant for not keeping the plant in that all 11 green category. Next slide.

12So, if you look at 2017, if you do the 13 arithmetic, you'll find there are 101 plants 14represented there. And first quarter of 2017, it was 1582 in licensee response, and that's green. Sixteen 16 are regulatory response, and three in 17multiple/repetitive degraded cornerstones. The three 18plants are Arkansas One and Two and Pilgrim. And that 19 cadence continued through all four quarters of 2017.

20 Next slide.

21 In 2018 for two quarters, 95 in licensee 22 response, all green, 5 in regulatory response, and 1 23 in multiple degraded cornerstones. That is Pilgrim.

24And we don't have the second quarter data yet. So 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 86 let's talk about this. Next slide.

1 So the highlight here is Arkansas Nuclear 2One and Two are back in column 1. Why? They 3 completed their CAL actions, their confirmatory action 4 letter requirements, and they brought the plant back.

5 And they went into a 95003, a huge amount of effort to 6 bring that plant back to where it's in column 1, the 7 first time since 2015.

8 Now, the next slide is the lowlights.

9 This is where you've got to be thinking what's going 10on. Pilgrim is six consecutive quarters in 3, 13 11 consecutive quarters in column 4.

12 And if you take a look at the inspection 13 reports and get just an understanding of what the 14 cadence has been at that plant, it's kind of a poster 15 child for how things should have been handled and 16 weren't in terms of corrective action, in terms of 17 work management, in terms of material condition.

18 And Derek offered that NRC muted response 19considers Pilgrim's plan to shut the plant down. That 20 is a major, that's a major item for the industry right 21 now.22 And Grand Gulf and Clinton and Columbia, 23 consecutive quarters in column 2, believe me, the 24 owners of those plants are eager to bring those plants 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 87 back into column 1. Next slide.

1 Fitness-for-duty, I don't know if this was 2 done, if this presentation occurred back when Jack 3 Seaver (phonetic) was providing these presentations.

4 I recall the first time I did it I said I'd like to 5 present the fitness-for-duty information because of 6 what I witnessed in terms of the change in the culture 7 in my years.

8 But I find this interesting, because what 9 is happening at the plants is in a way indicative of 10what's happening in our culture. It doesn't make any 11 difference whether it's in a power plant, police 12 department, hospital, large employer with high tech.

13 There are things going on in society that 14are driving this data. And to me, it's maybe a 15 thermostat of the culture at least in terms as we view 16 safety, in terms of the high tech business, could be 17 driving a bus, flying an airplane, operating a tank, 18 flying a fighter aircraft, being in a control room.

19This is our culture. And so that's why I 20 was so eager for this to be part of this discussion, 21because, you know, I live ten miles from a nuclear 22 power plant. I know the people who are working down 23 there. I -- firsthand stories of what was going on.

24But the data shows up here. And our neighbors and 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 88 friends know this is going on.

1So our knowing about it I think is 2valuable. And that's why it's here. Next slide, 3 please.4There are five types of tests. There's 5pre-access. You know, you apply for a job. You fill 6out the form, say, no, no, no, no, no. And you get 7 admitted as an employee.

8 Then the day comes you have a random.

9 And, oh, yeah, yeah, yes, I had some of that last 10 weekend and I probably shouldn't have.

11 And then there's the unfortunate event 12 where you're working and your colleague is not 13functioning on all eight cylinders. And as a manager 14 or a director, you're put in a position to have to 15 make a decision to call security and go and visit that 16person. And I've done that. It's very uncomfortable.

17 It's even more uncomfortable when the discovery is a 18 positive.19 And then there is the we had a scram, and 20 four people were involved, and we're not quite sure, 21and someone was injured. Then you might do a post-22 event, because there's been human error, to determine 23 whether or not drugs or alcohol were involved.

24 Then there's the follow-up if you had a 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 89positive. And they have a follow-up after the 1 positive to see if you're clean or not.

2 That is alive and well at every nuclear 3power plant in the country. It's alive and well in 4this building by the way. So those of us who have 5 been called, that's where we are. Next slide.

6So there were data for three years. There 7 were subversion attempts. Do you know what that is?

8 That's where someone tries to either alter their 9 specimen or give somebody else's specimen or do 10 something like that. It happens all the time.

11Amphetamines have been increasing. And 12 construction sites have the highest positive rate, 13particularly in pre-access. This is where you got 200 14 laborers coming on board. And you're required under 15Part 26 to do some screening. And lo and behold, you 16 screen out 10 or 15 of these people because they, for 17doing drugs from their last construction site. And 18they've had a big weekend. And they came in on a 19Monday morning. And there they are. They get caught.

20 Next slide.

21 The NRC submitted a proposed rule for 22 consideration that would address three multi-year 23 trends by lowering the cutoff levels for amphetamine 24 and methamphetamine and expanding testing measures 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 90 related to subversion as an attempt to not be 1 detected.2 There's no Commission response yet. But 3that SECI is up there for consideration. In other 4 words, it's a trimming of the rule. Next slide.

5So the data tells a story. So there is an 6 increase in subversion attempts. And, you know, you 7 stop and say what does that mean. Well, it means --

8 does it mean there are more people doing drugs?

9Probably. It means more people doing drugs or trying 10 to hide doing drugs.

11 And I would just sense that this data is 12 probably applicable almost anywhere you might travel 13 in a high tech industry.

14 And at the bottom there, 45 facilities 15with at least 1 subversion attempt. How many 16 facilities are there? What is it, 67, 68 facilities 17in the country? So three-quarters of the sites, 18 facilities have at least one subversion attempt.

19 Sixty-seven are pre-access. And 98 percent of those 20 are contractors or vendors. Next slide.

21 There's an overall positive rate around 22.75, .77, .76 percent.

It's low, but it continues 23upward. There was a slight reduction in calendar year 24'17 total tests. There's a continuing downward trend.

25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 91 About 64 percent of the calendar year '17's positives 1 and refusals occur pre-access.

2 For those of you who don't know, if a 3 contractor comes in and you're asked to go give us a 4 specimen, there will be some who will say no thanks.

5 And they'll take their lunch bucket and their backpack 6 and go back to the parking lot and leave the site.

7They'll just say, no, I'm not taking that test. That 8 happens.9 And the randoms identified about 22.7 10 percent, one-fifth of the abusers. And that next to 11 the last caret identifies more employees using 12substances than pre-access. That is what is alarming 13 to me.14 We have a culture, not only in nuclear but 15 throughout our society, where there are a lot of 16 people using substances, these substances that are 17 banned. Next slide.

18 For-cause testing continues to have the 19highest positive rate. That is where a supervisor or 20 someone points out an employee that needs to be 21escorted to the fitness-for-duty station. And the 22 substances that account for about 85 percent of the 23 positives are marijuana, alcohol, and cocaine.

24 I would just say back in 1966 when I 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 92 started that we were concerned about people coming on 1 the watch that had a couple beers. I don't know. I 2 don't even remember these words. Marijuana was 3something that happened in California. Those of us 4 from the east didn't, at least I didn't know about it 5in '67. I got my RO in '67, so never even thought 6 about it. And here --

7 (Off mic comments.)

8MEMBER SKILLMAN: Yeah, that's, I mean, 9 that's a culture I wasn't part of.

10 (Laughter.)

11MEMBER SKILLMAN: Yeah, I'm just saying 12 the reason I wanted to show this is because I think 13it's something that we as serious, engaged, active 14professionals just need to keep in the back of our 15minds. It's a shame that the culture is here. But 16 this is where it is. And this is where we live. We 17 need to be aware of it.

18MEMBER SUNSERI: Hey, Dick, does the data 19 suggest that since the legalization of marijuana in 20 several states is contributing to the trend in 21 marijuana use, or is there, is it indifferent to that?

22MEMBER SKILLMAN: Matt, that's a great 23question. I think it's too -- this is Dick Skillman's 24opinion. I think it's too soon to tell, because, you 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 93 know, it's just beginning to be legitimized as a 1medical treatment. I don't think we've seen the data 2 yet that would be the result of those changes.

3 I know I've been kind of watching Exelon.

4 And they're saying we're not changing our standards.

5 You know, if you're taking medical marijuana, 6recognize you can be called to fitness-for-duty on 7Monday morning. So medical or not, if you show up 8 positive, you're going to, you're either not going to 9 have a license or you're not going to be working here.

10 MEMBER SUNSERI: Yeah, I understand that 11 about the medical. But I was thinking, you know, so 12 you got Colorado, Oregon, you know, California.

13 PARTICIPANT: Massachusetts.

14 MEMBER SKILLMAN: So the reason I wanted 15 to present this information is so we can kind of have 16 this conversation and say what's going on here. You 17legalize marijuana. You go to a football game at 18 State College and you smoke up all Saturday afternoon.

19 And you come back and take a security watch on Monday 20 morning. I'm not sure. That's the issue.

21VICE CHAIRMAN RICCARDELLA: You know, the 22 other issue is, you know, evidence of marijuana can 23 stay in your bloodstream for 30 days, you know. And 24 it doesn't really, you know -- if you're at that 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 94 football game 30 days ago, it's not clear to me that 1it affects your performance. But it is what it is, I 2 mean.3MEMBER SKILLMAN: Next slide, please.

4 Fitness-for-duty data, understand that if you're at a 5site, if you're a director or manager of a site, 6 you've got some very clear guidelines for how you 7 handle this information.

8 Once the employee has been sent to the 9 fitness-for-duty station, if there's a positive, there 10are all kinds of reportings that do occur. And so 11 this will issue a fitness-for-duty as I think kind of 12 taken over at least a large portion of how the site 13 thinks about its health.

14 What is important is that for 2017 there 15 were 33 24-hour reportable events and half of those 16are supervisors and managers. That's alarming. At 17 least I find that alarming.

18 And so, you know, what do you do if you've 19 got a very key manager upon whom you depend for key 20decisions and you find that individual positive? Then 21 you enter your program, fitness-for-duty, that might 22take that person off site for 30 days. There are 23 major issues associated with this whole topic.

24 So I just wanted to present this so that 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 95 the members have a sense of perhaps what the site 1 leadership and the site executives have to deal with, 2but also what the NRC staff has to deal with. Our 3 culture has changed. Next slide.

4 Annual abnormal occurrence report to 5Congress, there was one, one item. This was an 6 Americium-241 ampule, probably as big as your little 7 finger, that leaked on a lab table up the street here 8 up at NIST in Gaithersburg. And that resulted in an 9overexposure. That is the only abnormal occurrence 10 that has been reported. Next slide.

11Colleagues, any questions? Okay. Don 12 Helton, we're ready for you to come forward, please.

13 This is the second piece that I wanted to 14talk about. And this is the topic of leading 15 indicators. Matt and I were kind of working our way 16 into this. I've got some strong feelings about what 17 is a leading indicator for future performance.

18 When I look at the identification of the 19 problems that ANO and at Pilgrim, you know, my own 20 involvement with 95002 and 95003 plants and 10350 21 plant, my view is that there are some items in the 22 culture at the site that will point to degrading 23 performance. And the question is at what magnitude.

24And that is what kind of triggered this 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 96 discussion. Could we actually make a rubric or some 1kind of a formula where you would say if you take a 2 look at these data over a reasonable time period, 3could you predict that that licensee is going to go 4into a ditch? I believe the answer is yes. But 5 that's what I wanted to talk about here.

6MR. BOWMAN: Yes, so neither of us are Don 7Helton in case you haven't realized that. I'm Greg 8 Bowman.9MEMBER SKILLMAN: I thought Don was going 10 to make the presentation. So --

11MR. BOWMAN: So I'm Greg Bowman. I'm the 12 Branch Chief for the ROP Assessment Branch --

13 MEMBER SKILLMAN: Okay, Greg.

14 MR. BOWMAN: Tom Hipschman is the Branch 15Chief for our Inspection Branch. And so Tom is going 16 to go through the first part of our presentation 17 talking about, you know, how our inspection program 18 picks up on degrading performance, whether it's due to 19 financial challenges or whatever.

20 And then I wanted to give a brief 21 discussion at the end about some recent developments 22 on ROP enhancements. We've gotten a lot of -- we're 23 in the process of dealing with a lot of proposals from 24 our stakeholders, both internal and external, for ways 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 97 we can make the ROP better.

1 And so we really just wanted to give you 2 a, you know, heads-up on something that's coming down 3 the road that you'll likely be involved in, you know, 4 helping review our, whatever we come up with. So --

5 MR. HIPSCHMAN: Thank you.

6 MEMBER SKILLMAN: Please proceed.

7MR. HIPSCHMAN: Hi, I'm Tom Hipschman.

8I'm the Chief of the Inspection Branch. This is my 9 first opportunity to present to the ACRS.

10 I took over the branch back in February of 11this year. Previously, I've been a senior resident 12 inspector at Indian Point, Diablo Canyon, the resident 13 inspector at Oyster Creek, a regional inspector and 14 also a regional branch chief and served on the 15 Commission as a TA.

16 I'll just kind of fill in a little bit of 17my perspective. It was a very good overview of the 18 ROP. So I'm not going to reiterate a lot of that.

19 But the ROP is designed as a process that 20 we use to identify declining licensee performance 21 prior to the loss of reasonable assurance or adequate 22 protection.

23 In part we do that by plants that will 24 progress through the action matrix, so columns 2, 3, 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 98 4, or possibly even 5. So we have increasing levels 1 of Agency response and oversight and management 2 oversight.

3 We also apply additional inspection 4 resources in response to greater-than-green findings.

5 And roughly our 9500X procedure, such as 95001, would 6 be used for a white finding, 02 for yellow, red for 3 7 roughly.8 And throughout those inspections, every 9 quarter following completion of the 95003 there will 10typically be a confirmatory action letter. The 11 regions will send out several inspectors on a 12quarterly basis to do CAL closeout. Additionally, 13 there will be probably annual problem identification 14 and resolution inspections.

15 So there's quite a bit of additional 16resources applied to plants that increase in the 17 action matrix.

18MEMBER SKILLMAN: Tom, how many plants 19 have been subjected to 95003? I'm thinking 5, 6.

20MR. HIPSCHMAN: I don't have a number off 21 the top of my head.

22 (Simultaneous speaking.)

23 MEMBER SKILLMAN: It's not 20. It's not 24 ten.25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 99 MR. HIPSCHMAN: No.

1MR. BOWMAN: I'm thinking probably five to 2 ten.3MR. HIPSCHMAN: Just off the top of my 4 head, Pilgrim, ANO --

5 MR. BOWMAN: Cooper.

6 MR. HIPSCHMAN: -- Indian Point.

7 MR. BOWMAN: Browns Ferry.

8 MR. HIPSCHMAN: -- Cooper, Browns Ferry, 9 Fort Calhoun. Those are the ones that come to mind.

10 MEMBER SKILLMAN: So maybe it's 6, 7 out 11 of 100.12 MR. BOWMAN: Yes.

13 MEMBER SKILLMAN: 6 or 7.

14MR. BOWMAN: Since the beginning of the 15 ROP in 2000.

16MEMBER SKILLMAN: Yeah, okay. Thank you.

17MR. HIPSCHMAN: So, notwithstanding, 18 there's been always interest in leading indicators.

19 The Commission in various forms has encouraged the 20staff thinking about this. I know my former boss was 21 also very interested in leading indicators.

22 We do routinely assess the effectiveness 23 of the ROP through annual self-assessments, biannual 24 procedure self-assessments where we look at 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 100 procedures, inspection procedures and findings.

1 Additionally, as you mentioned, there is 2an annual assessment meeting that all the regions 3 perform where they review each plant's performance.

4 And they have various discussions about certain areas 5 of concern. Next slide.

6 There have been -- various discussions 7 have highlighted ideas as, you know, what can we do.

8 Can we look at the trend and the number of green 9findings? While there are some plants that tend to 10 have greater number of green findings or challenges, 11 we haven't historically used that as any kind of 12 trigger for increased oversight.

13 However, again, it is an assessment piece 14 that the regions will look at during end of cycle 15 meetings.16 One aspect that sometimes has been 17 suggested is looking at cross-cutting aspects. With 18 those findings you will tend to accumulate in certain 19 areas, such as human performance or problem 20 identification, a number of cross-cutting aspects that 21 could give you insights into licensees' performance in 22 a particular area.

23 Other areas of findings that might be of 24 interest are QA, Appendix B related findings, 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 101maintenance rules. Other areas that the NRC is 1 interested is plant operations.

2 (Background noise.)

3CHAIRMAN CORRADINI: Just so you know 4 what's going on is that the wonderful system here cuts 5out if there's no communication. So we lost a member 6out in the far west. So we're going to try to 7 reconnect. So I apologize.

8MR. HIPSCHMAN: All right. So proceed or 9--10 CHAIRMAN CORRADINI: Keep on going.

11 MEMBER SKILLMAN: Please proceed, yes.

12MR. HIPSCHMAN: Plant operations, you 13 know, we'll look at --

14 PARTICIPANT: Do you have a question?

15MEMBER REMPE: I do, but I thought you 16were going on to the next slide. When you finish this 17 slide, I have a -- oh, I'll just say it now.

18 On some of these things where you're 19 talking about discussions of using the data, is there 20something that could be done? Like right now a hot 21 research area is big data, right, and using some sort 22 of system to help you analyze it.

23 Has that discussion ever gone that way 24 that maybe there's something that research could do to 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 102 help you use this data more effectively?

1MR. BOWMAN: So, when we get to the 2 transformation discussion or the enhancement 3 discussion at the end, some of those suggestions have 4come in to us. We haven't made a lot of progress on, 5 you know, shipping that work over to research to help 6us with. But we do need to assess that as one of our 7 ROP enhancement proposals.

8 MEMBER REMPE: So --

9 MR. BOWMAN: Most of the things that Tom 10 is discussing on this slide are already looked at as 11 part of our inspection program, so, but not from a big 12 data standpoint, not, you know --

13MEMBER REMPE: So that's something that's 14coming up in the DOE arena with industry. And so 15 there might be, and we can talk about it later, but 16 with some collaborative efforts --

17 MR. BOWMAN: Yeah.

18 MEMBER REMPE: -- with the Department of 19 Energy.20MR. HIPSCHMAN: With regards to plant 21 operations, inspectors will keep tabs on various 22 things such as work management. They'll look at the 23 number of corrective actions or corrective maintenance 24 in the backlog. They'll -- some plants tend to have 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 103 higher backlogs than others. You know, anecdotally, 1 that could be sort of an indication.

2 Other things that occur are to help the 3 inspectors relate to operation of the plants and 4 material condition of the plant. Some plants are in 5 obviously better condition and better upkeep than 6 other plants.

7 And one way that inspectors can be aware 8 of that is that they will do annual objectivity visits 9 to other sites so that they can get comparisons on how 10both residents and also other plant managements are 11 performing.

12 Also, there are frequent regional 13 management visits that, you know, are also very 14instructive for the resident inspectors. Just on the 15 list, you know, there's other things that the 16 inspectors can trend.

17 Interestingly, you know, staffing 18 resources such as, you know, what kind of license 19 operator pipeline is there is kind of also an 20 indicator of how well or, you know, of what's going on 21 at the plant. If there's a lot of license operators 22 in the pipeline or if there's a healthy number of 23 senior reactor operators, you know, that typically 24 seems to be a good indicator.

25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 104 You know, trend and conditions, 5072, 5073 1 reports, again, those are sort of, those are things 2that inspectors follow up on. They also will do an 3inspection of those after the fact. And that is also 4 reviewed during end of cycle meeting.

5 One of the outcome of end of cycle 6 meetings are senior management key messages that the 7regions will develop. And they're sort of talking 8 points for when the regional senior managers go out to 9the site. They're areas of performance that they 10 would like to emphasizes during their site visits.

11MEMBER SKILLMAN: Tom, let me ask this, 12 please. When Matt and I were trying to develop some 13 energy around leading indicators, the question that 14 was foremost in my mind is, has the staff ever taken 15 all the 95003 plants, just remove the name, just take 16 the data that were the initiating events that got the 17 plants in the 95003 and compared them and asked the 18 question are the conditions that brought the plant to 19 95003 similar enough that those specific conditions 20 would themselves become a leading indicator.

21 For example, at least one man's opinion, 22 the robustness of the system health reports, the 23 degree of connection between the corrective action 24 program and the work management program and the 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 105 expressed beliefs by senior leadership and their 1 knowledge of the Appendix B to 10 CFR 50, I keep 2coming back to that, those plants where senior 3 leadership understands that Appendix B is 18 pieces.

4If you pull them together right, if you've got a 5strong corrective action program and a strong work 6 management program, you might get a lot of greens, but 7 they're in the grass because people are taking care of 8 stuff, little things that happen.

9 But you're also moving the bigger items 10into strong root cause evaluations. And you're 11 actually fixing the underlying problem as opposed to 12 repair and move on.

13 So my real, the question I would ask is, 14 have you ever or has the staff ever considered looking 15 at the six or eight or ten 95003 plants a nd the one 16 0350 plant -- I don't know how many 0350s, but at 17 least one -- and asked is there any common set of 18 findings, which if given even a thicker magnifying 19 glass, might produce a fairly short list of leading 20 indicators.

21My view is the answer would be yes. But, 22 and I would sure like to have the time and energy to 23do that study, but I don't. But it seems like that's 24 a gold mine.

25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 106 MR. BOWMAN: Yes, so -- I'm not aware of 1us having done a study like that. It's a great 2 question.3 When we get to the -- and I hate to kick 4 everything to the enhancement discussion we're going 5to have shortly. But we've got a number of 6 suggestions both from internal stakeholders and from 7 external stakeholders about our performance indicator 8 program and whether that's where it should be, whether 9 it's giving us meaningful information.

10 And one of the things that I -- this is 11 just -- we haven't actually kicked off the review of 12 that yet. But I think doing something like that is, 13 would be very valuable in determining whether we can 14 identify different performance indicators or better 15 performance indicators to help us in that area.

16 I think that's something we had planned.

17We haven't really done that yet. I'm not aware of us 18 having done that historically.

19 But as we look at the performance 20 indicator program in the context of our opening 21 enhancement, I think that will be a very fruitful 22 exercise, going through and seeing, you know, what 23 95003 plants might have been experiencing before they 24 got there. That's a good, very good suggestion.

25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 107 MR. HIPSCHMAN: Yes, I'm not --

1MEMBER BLEY: Dick, can I cut in sometime?

2MEMBER SKILLMAN: Yes, sir. Dennis, good 3 to hear from you. Please.

4MEMBER BLEY: Yeah, well, I've been locked 5out for a while. I finally found out why. I'm back.

6 This business of leading indicators has 7been a hot topic since at least the mid-70s. And the 8 NRC sponsored a lot of research back in the 80s on 9 this, so have others.

10 The unfortunate thing that's happened is 11 when each of us thought we had it figured out, because 12 it kind of made sense, when you tracked it, you found 13that you kept getting surprised. The things the 14 leading indicators pointed to didn't turn bad and 15 other things turned bad that you weren't expecting.

16 A couple of years ago we had INPO up here.

17 And they claimed, at least the gentleman who was 18representing them, that they had developed a new 19 leading indicator model that really was working and 20 gave short term and longer term predictions that were 21 coming out true.

22 I wonder if the staff is aware of what was 23 going on there and if that's actually come to fruition 24at all. And I also like the idea, as Dick put out, of 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 108 things you might look at, because I think some of 1 those nobody has really looked at yet. That's all.

2MR. BOWMAN: Yeah, and this is Greg 3Bowman. I'm not aware of the INPO, what INPO's, what 4 they've been working on or whether it's come to 5 fruition or not.

6 We do try to keep a healthy amount of 7 separation between our oversight program and what INPO 8 does just to avoid -- you know, they obviously have 9 different objectives than we do. And we try to keep 10 them separate. But, yeah, so I'm not aware of where 11 INPO is with their process.

12CHAIRMAN CORRADINI: Dennis, can I just 13 follow up? Dennis, was it an open discussion or was 14 it a closed discussion? I remember it. But I can't 15 remember much about it.

16MEMBER BLEY: I think our whole meeting 17 with INPO was closed.

18 CHAIRMAN CORRADINI: Okay.

19MEMBER BLEY: And it followed on the heels 20 of Fukushima.

21 CHAIRMAN CORRADINI: Okay. Thank you.

22MR. HIPSCHMAN: One comment regarding 23 INPO, and I'll finish my answer on the other one, is 24 with respect to INPO evaluations, we do have a 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 109 memorandum of understanding with them.

1 And one of the things that the NRC 2 inspectors have access to is their evaluation report.

3 And inspectors will typically read that just to make 4sure that they haven't missed anything, verify, you 5 know, their conclusions as far as licensee 6performance. And if there's any new insights that 7 inspectors need to follow up, they will leverage those 8 reports as needed.

9 With regard to the 95003s, I am not aware 10of anything as Greg mentioned. However, we do have 11 sort of a lagging indicator, to some respects, is that 12 we do look at the 95003s after they're completed.

13 We do a lessons learned report, which 14tends to be rather extensive. There are several 15 corrective actions that are generated from each 16 lessons learned report that the staff follows up on to 17 look to see how inspection procedures, manual chapters 18can be changed. But I don't think we've done anything 19 that kind of looks at all of them in the whole.

20MEMBER SKILLMAN: I would sure like to 21 encourage finding a way perhaps to take the 95003 22 conclusions and seeing if there are not commonalities 23 among all of those 95003 inspection, the completed 24 reports, because I have a pretty strong sense that 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 110 there are diamonds down in there.

1 MR. BOWMAN: That's the first suggestion 2 we'll --3 MR. HIPSCHMAN: Yeah.

4 MR. BOWMAN: -- think about.

5MR. HIPSCHMAN: Yeah, I think it's 6 interesting --

7MEMBER SKILLMAN: Thank you. Please 8 proceed.9MR. HIPSCHMAN: Next slide, please. A 10 related point is other areas where we can adjust our 11 inspection program for plants is for plants that are 12experiencing financial issues. We do have guidance 13 for that for various things to, inspector to look at 14such as challenges to material conditions. And you'll 15see four plants that have been in that. And the 16 inspectors will comment in the inspection report.

17 Also, we do have guidance for plants that 18are nearing cessation of permanent operations. We'll 19 adjust, the inspectors will adjust their inspection 20 areas such as, you know, they'll look for is there an 21 increased attrition of, say perhaps, licensed 22 operator, are there any changes in material condition.

23 And so they'll look at those kind of 24things. And as appropriate, they'll follow up on the 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 111 baseline inspection program for areas that might 1 warrant that.

2 And the other things that we have as tools 3are safety culture initiatives. As part of our 4 biannual problem identification and resolution 5 inspections, we have a part of that that looks at 6safety culture. And also in our 95002 and 95003 we 7 have much more extensive guidance regarding looking at 8 safety culture.

9Let's see. Also, we do have other 10 inspection procedures in manual chapters such that 11 during plant walkdowns or senior manager site visits 12we do look at material conditions. We do look at 13long-standing issues. We do highlight those, bring 14 those up with licensee management.

15 And again, you know, overall the ROP is a 16 very flexible inspection program. Inspectors have a 17 lot of flexibility within the program to look at 18various things, use modules to look at degraded 19 material condition, look at human performance, look at 20problem identification. And they can review and 21 report those as appropriate.

22Again, in the action matrix, one of the 23 aspects of that is increased management presence, as 24 well as a site we'll go through the action matrix or 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 112increase in the action matrix. Not only is there 1 increased management presence, but there's also 2 increased management review, such as with the 3inspection reports and things like that and public 4 meetings.5 With that, I'll -- if there's not any more 6 questions, I'll turn it over to Greg.

7MR. BOWMAN: Okay. So, on the next slide, 8 so, at the very beginning of his presentation, Tom 9 talked about the self-assessment process.

10So the ROP is a mature program. It's been 11in place for about 20 years. One of the key 12 components of the ROP is self-reflection, figuring out 13 things, you know, on a periodic basis that we could 14 have done differently, could have done better to 15 improve the program.

16 So the beginning of this slide, the first 17 main bullet and the sub-bullets on this slide talk 18 about some of the ongoing initiatives we have to make 19 the ROP better.

20 You know, we've had a lot of work on the 21inspection report development process. We've 22streamlined our inspection reports. And we're nearing 23 the end of an initiative to automate the production of 24 our inspection reports to make it easier for our 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 113 inspectors so they can be focusing on inspection and 1 not inspection report writing.

2 Dick, when you presented, you talked about 3 the four levels of violations. There's a fifth down 4 there that we, that gets, that results in a lot of 5work but doesn't, you know, see often. Those are 6 minor violations, right.

7 So, if you have a -- you don't fill out a 8-- or you're working through a procedure and you don't 9 follow every step, but there's no impact from failing 10 to follow that step, we'd often characterize those as 11 minor violations.

12And Tom and I being inspectors can tell 13 you that we spend a lot of time with our management 14 discussing, and I'm using discussing gently, that's a 15 gentle term for what we wind up doing, with our 16 managers about whether an issue is minor or more than 17 minor.18 So we have work going on to sort of help 19 give better guidance to our inspectors so they don't 20have to deal with the, all the back and forth. Things 21 are more clear to them on what's minor and what's not.

22MEMBER SKILLMAN: Greg, thank you for that 23 clarification. That was an oversight --

24 MEMBER BLEY: Can I --

25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 114MEMBER SKILLMAN: -- on my part. Thank 1 you.2MEMBER BLEY: May I sneak in a question 3 here?4MEMBER SKILLMAN: Please, go ahead, 5 Dennis.6MEMBER BLEY: You know, from an overall 7 plant safety point of view, that seems to make sense.

8 From a human performance point of view, if it's the 9 same action that could have created a serious problem, 10 you really ought to look at it.

11 MR. BOWMAN: Yeah, and that's a function 12of how our program is set up. We look at, we 13 essentially look at what the consequence was of an 14 issue.15 There's often times cases where, if a 16 circumstance was slightly different, then the 17violation or the finding would be of greater 18 significance. And that's sort of a fundamental part 19 of how our program works.

20 I think one way we get to that issue is 21 through our cross-cutting issue program where we, you 22know, when we go through, we have a green finding. We 23 figure out what the causal factors are. And we look 24 to aggregate, you know, see if there's a trend in an 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 115 area.1But that's a fair point. And it's sort of 2 just a fundamental part of how our program works.

3 One thing I would add is that, you know, 4 on a biannual basis we do a problem identification and 5resolution inspection. And that inspection procedure 6 allows us to dig in a little more deeply on issues 7 that are less, that are minor.

8 So we can do -- we do some trending. We 9 document observations that normally wouldn't be 10 documented because of the low significance of the 11 issue.12MR. HIPSCHMAN: You know, notwithstanding 13 that, in my experience at the sites that I have been 14 at is that when the inspectors provide comments that 15 aren't more than minor or their observations, the 16licensees treat those very seriously. They put those 17in the corrective action program. And they do take 18 action.19 And also, the inspectors have the 20 opportunity to follow up those that -- you know, 21they'll keep those in mind. And if they start 22aggregating and they look for a bigger programmatic 23 type deficiency or trend, the opportunity is also to 24 have an opportunity to make a finding or violation 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 116 from that.

1MEMBER SKILLMAN: Yeah, Dennis, to your 2 point, my experience, my firsthand experience is very 3 commonly the minors or the debate about more than 4 minor normally falls to the director of operations or 5 whoever is running engineering, because it's either a 6 procedure failure, a failure to perform procedure 7 properly or it's a Criterion III design issue.

8 And the amount -- debate is the right 9word, but perhaps not politically correct. But the 10 amount of energy in discussing whether it's minor or 11 more than minor, at least in my view, addresses, 12 Dennis, your comment. Could there be something that 13 is very subtle whose impact is not fully appreciated?

14 And in the multiple events I've been 15 involved in, the extent of discussion and the tone of 16 the discussion has been very effective in sorting out 17 whether or not the minor issue was truly minor or 18 whether it was more than minor and could have been 19leading to a more subtle or more serious outcome. And 20 those debates --

21 MEMBER BLEY: Well, what --

22MEMBER SKILLMAN: -- have been very, very 23--24 MEMBER BLEY: What --

25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 117 MEMBER SKILLMAN: -- very spirited.

1MEMBER BLEY: What you say and what the 2other fellows just said is encouraging. But the idea 3 that Joe just got lucky and Harry gets nailed to the 4 wall because he wasn't lucky is a significant thing to 5 worry about.

6 MR. BOWMAN: And one thing I just wanted 7-- this is Greg again.

One thing I wanted to just 8amplify something Tom said. So, regardless of whether 9 a violation is minor or more than minor, the licensee 10still needs to fix it. They need to put it in a 11 corrective action program.

12 And if you have a case where there was a 13 near miss, we would expect the licensee's corrective 14 action program to include a robust causal analysis, 15you know, in-depth corrective actions. And as Tom 16mentioned, we can go back and inspect those even if 17 they were minor, even if it was a minor issue.

18 MR. HIPSCHMAN: You know, and one of the 19 things --20 MEMBER BLEY: Okay.

21MR. HIPSCHMAN: -- in assessing whether 22 it's minor or not, there are several questions that 23the inspectors have at their use. And one of those is 24 if that issue was left uncorrected, could it be a 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 118precursor to a more significant event. And if it is, 1 then it would be kicked up into more than minor.

2MEMBER BLEY: Okay. And since you used 3 the word precursor, the precursor program looks at 4 events and then says from this point on could this 5 have become serious. And it seems like that kind of 6 thinking, from what you're saying, is getting picked 7 up. So I think that's very important.

8 MR. HIPSCHMAN: Thanks.

9 MR. BOWMAN: So then the next, you know, 10 sort of sub-bullet on this slide is the work we've 11 been doing over the past couple years to improve the 12engineering inspection program. I believe we briefed 13you on that. And we're getting close to the point of 14sending a paper up to the Commission with 15 recommendations to make the program better.

16 So I mentioned at the beginning, you know, 17we established this transformation team at the NRC 18probably I guess it's been about a year ago now. That 19 sort of kicked off a lot of input coming in our 20 direction.

21 The transformation team was largely 22focused on making the NRC more agile, more able to 23 deal with new technologies, that kind of thing. But 24 they went out and solicited stakeholder input on 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 119 pretty much anything the NRC could do to be better.

1 A number of those recommendations were 2 related to the ROP. Because it wasn't really within 3 the transformation team's charter, they just 4 essentially passed those recommendations on to my 5 group to review.

6 You know, sort of coincident with that or 7 in conjunction with that, we got feedback from the 8 industry, from NEI, and from NRU, the National -- I'm 9 going to forget the acronym.

10 MR. HIPSCHMAN: Regional Utility --

11 MR. BOWMAN: Group.

12 MR. HIPSCHMAN: -- or Users Group.

13MR. BOWMAN: With additional proposals 14 from them on how to enhance the ROP.

15 And then we just recently got a follow-up 16 letter from NEI with sort of a consolidated list of 17 recommendations from the industry on what we could, 18 what they believe we should change in the ROP.

19So, on the next couple slides, I'll go 20 through just at a high level what some of that 21 feedback was.

22 So the input from the transformation team, 23 I mentioned there were about 70 recommendations, fit 24into generally these categories. And it was, you 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 120 know, it was raw, sort of unfiltered feedback from the 1staff. So, you know, often there were recommendations 2 that conflicted with one another or, you know, some of 3 them were not very well developed.

4 But, you know, probably the most common 5 recommendation had to do with the structure of the 6 regions, you know, whether the resident inspector 7 program, whether that should be expanded or done away 8 with, whether our Division of Reactor Safety and 9 Division of Reactor Projects should be organized 10differently. We got a lot of suggestions in that 11 area.12 And then again, you know, pretty much 13 every component of the ROP inspections, PI, 14 performance indicators assessment, and SDP, we got 15 some suggestions.

16 And so we have all those recommendations 17in. We've done some binning, you know, to group them 18together. But that's the extent of what we've done to 19 date. Next slide, please.

20 So the -- we got a letter from the 21 industry, from NEI, on the 19th of September, so just 22a week ago, a little over a week ago. And then we had 23 a public meeting with NEI the following day for them 24 to brief us on their recommendations.

25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 121 The one thing that I think we've heard 1 from both industry and from other stakeholders is that 2the ROP is a sound oversight program. You know, it's 3 one that countries around the world model. And it's 4 not one that I think any of us argue should be 5substantially changed. So, however, I think we all 6 acknowledge, both the staff, industry, other external 7 stakeholders, that there are areas we can improve.

8 So this slide provides sort of a synopsis 9of what the industry's recommendations were. And they 10 really fit into four areas.

11 Impact of white findings, and so that 12 would be things like do we issue a press release for 13 a white finding or do we not issue a press release, 14 how do we communicate the relative risk of a finding.

15 You know, a white finding is low to, we 16characterize it as a low to moderate risk. Is that 17really what a white finding is? And when we're in the 18 minus 6 range for a finding like that, is that an 19appropriate characterization? Does the public 20 understand what we're saying when we say low to 21 moderate? So that type of thing.

22 And then we got feedback also on the type 23 of inspection we do to follow up on a white finding.

24 As, Dick, you mentioned, we do a 95001 inspection, 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 122 which, you know, it's the lowest of the 9500X 1inspections, but it's a fairly intensive effort. And 2 so does that make sense given the low safety 3 significance of a white finding?

4We also got feedback from NEI on the 5 baseline inspection program, whether it's where it 6should be. You know, one of their suggestions was 7 take a look at licensee performance over the last 8several years. If a plant has been in column 1 or has 9 been doing well based on whatever indicators we come 10 up with, could they get something less than the 11 baseline inspection program?

12MEMBER SKILLMAN: I would like to just 13 jump in here because --

14 MR. BOWMAN: Go ahead.

15MEMBER SKILLMAN: -- I think that is 16 where, if one were to take a 95003 lessons learned and 17 refine them into action statement and then look at the 18 baseline inspection program, one might say, one might 19 conclude or assert that licensees that are performing 20 excellently against those 95003 --

21 MR. BOWMAN: Criteria.

22MEMBER SKILLMAN: -- items could almost be 23 exonerated from some level of inspection because their 24behavior is so opposite to what the findings are in 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 123 the 95003 category.

1 In other words, the work management 2program is bulletproof. The connection from the work 3 management program to the root causes is remarkable.

4I mean, they really do root causes. They go right 5down to the last nut and bolt. And by and large, they 6are on target. Their recordkeeping, their MT&E is 7 right where it needs to be.

8 You could almost say a plant that has that 9 type of performance and also has a safety culture that 10 is thoroughly positive, one might say we can probably 11back off. So what I'm suggesting is there's a 12 connection between that question and lessons learned 13 out of those 95003 inspections.

14 MR. BOWMAN: Yeah, that's exactly what I 15 meant when I said we would, this would be something we 16 would look at as part of this.

17 I think, you know, the inspection program 18 and the performance indicator program are meant to be 19complementary, right. You have performance 20 indicators, and then you have the things that can't be 21 readily measured by a performance indicator we go out 22 and inspect.

23 If we were theoretically able to identify 24 a better set of performance indicators that could 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 124 result in need for less inspection, but, you know, we 1 just started with this so I don't want to --

2 MEMBER SKILLMAN: Yep.

3MR. BOWMAN: -- you know, predict where 4 we're going to end with it.

5MEMBER SKILLMAN: I'm agreeing, I guess in 6 a way agreeing or maybe even championing a real 7 serious look at this because it might be to everyone's 8benefit. What it really does is it might make 9 resources available to do other really important 10 stuff.11MR. BOWMAN: Right, right. Similarly, 12 this doesn't really fit into the inspection program 13 cleanly, but industry had some suggestions on the 14 mitigating systems performance indicator and 15 specifically whether that indicator is really needed 16 anymore, whether it's -- you know, we don't have very 17 many -- I think it's very rare for us to have a 18 greater-than-green MSPI.

19 And most licensees, you know, when the 20 program was put in place, many of them took actions to 21 make those systems less risk significant. You know, 22 they made modifications to the plant.

23 So the value, one could argue that the 24 value of the indicator has gone away somewhat. It's 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 125very complicated to calculate. It's not easy to 1understand necessarily for the public. So the 2 feedback was, you know, what's -- I think whether we 3 could do things better with that indicator.

4 They had suggestions on improving the SDP, 5 some specific SDPs that they suggested we go back and 6 look at, and then some sort of infrastructure work to 7-- you know, we have a lot of interaction with 8 licensees as we're completing a significance 9 determination process for potentially greater-than-10 green finding.

11A lot of that is around the assumptions 12 that go into the SDP, things like recovery credit, 13common cause failure. Those things result in a lot of 14discussion. And so the recommendation was really can 15 we establish better ground rules up front so that NRC 16 and industry both know what sort of assumptions are 17 going to go into an analysis.

18 And then they had, the last, you know, 19 sub-bullet there, some feedback on resolution of 20inspection issues. So, and some of this is addressed 21 by our backfit initiative.

22 But is there a way we can more easily 23 resolve low risk compliance issues, things that -- you 24know, we have an inspection finding. We know based on 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 126a PRA that the finding is very low risk. But 1 sometimes we can spend a lot of effort trying to get 2that resolved. Sometimes there's action statements 3that require the plant to shut down. And is that 4 really where we need to be based on the risk of the 5finding? So that's sort of the fourth item. So next 6 slide.7 So we are in the very early stages of 8working through all these recommendations. As I 9 mentioned, the letter from NEI just came in a week and 10 a half ago. We've done some work to sort of bin all 11 the recommendations between what we got from internal 12 stakeholders and from the industry.

13 And what we're planning on doing is 14 setting up some working groups internal to the NRC to 15go through and evaluate the proposals. We had a 16kickoff meeting with our team. October 3rd was 17 yesterday. So it was just yesterday.

18 And then we already -- we meet with the 19 industry on a monthly basis to go over items of 20interest. We'll be using those meetings to sort of 21 engage with both the industry and stakeholders on 22 their proposals, talk through them with our next 23 meeting scheduled for the 18th of October.

24 We do -- some of the recommendations we 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 127 got, many of them actually, are fairly significant, 1 you know, either fairly significant from an evaluation 2 standpoint or fairly significant when it comes to 3implementation. And many of them would likely require 4 Commission approval if we were to choose to adopt 5them. And that process is laid out in that management 6 directive, the criteria for when we go to the 7 Commission versus when we can make changes on our own.

8 And then I guess I didn't put this on the 9slide. But obviously, the big changes, we would need 10to engage with ACRS to get your feedback before we 11 went to the Commission with any sort of proposals.

12MR. HIPSCHMAN: You know, Greg, also with 13 a Commission meeting, recently operating reactor 14 business line --

15 MR. BOWMAN: Right.

16 MR. HIPSCHMAN: -- it was a topic during 17the Commission meeting. And the Commission pretty 18much encouraged us, you know, take a look at these, 19 consider your path forward.

20MR. BOWMAN: So that was all we had.

21 Let's open it up for questions now if you --

22MEMBER SKILLMAN: Greg and Tom, thank you 23very much. Colleagues, what Greg and Tom are showing 24 here is, first of all, it's a heck of a lot better 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 128 than SOLP.

1 PARTICIPANT: I can agree with that.

2MEMBER SKILLMAN: For those who weren't 3 around, that was the Systematic Assessment of Licensee 4 Performance, and the ROP replaced that. And this is 5 much more objective and much more thorough. I would 6 say much more flexible, and it's not personal. So 7 this has been a huge enhancement at least from my 8 perspective.

9 And I would just like to keep thinking 10 about are there some tools readily available that will 11 help our industry colleagues and help the staff maybe 12 shape forward-looking activities that might result in 13 optimization of resources for everybody.

14 So thank you. Colleagues, any questions 15 for Greg or for Tom?

16 (Off mic comments.)

17MEMBER SKILLMAN: On the phone line, is 18 there anybody out there that would like to simply say 19 hello so we know you're there? Anybody in the room?

20 MR. BOWMAN: I think there's a comment.

21MEMBER SKILLMAN: Great. Sir, good 22 morning.23MR. THORPE: Good morning. John Thorpe 24with Office of the Inspector General. I'm their 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 129 Senior Technical Adviser. I'm a former Chief of the 1 Operating Experience Branch.

2 I've heard some discussion earlier 3 regarding the initiatives that INPO might be taking 4 relative to leading indicators and evaluating what 5 they can do to get a better pulse on what's happening 6 with their plants.

7 And I would recommend that you guys talk 8to the Operating Experience Branch. They have a 9routine. It's once or twice a year. I don't think 10they've stopped it since I was the Branch Chief. They 11meet with INPO. They do a compare notes kind of 12 session.13And I know for sure that in one of the 14 meetings that I sat in on we had INPO staff talk to us 15 about the leading performance indicators, the leading 16 indicators that they were trying to develop from their 17 perspective as INPO.

18 Now, obviously, they're shooting for 19different goals and thresholds. But they're also 20 really trying hard to try to find out what it is that 21 they can sense from all the data that comes in, from 22 EPIX, from all these other things. How can we sense 23 whether somebody's on the decline or are they 24 improving or remaining the same?

25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 130 So I think there's some valuable 1 information that may be available to you from the 2 Operating Experience Branch and just wanted to offer 3 that to you.

4 MEMBER SKILLMAN: Mr. Thorpe, thank you.

5 Thank you.

6 Before we come to close here, I want to 7 thank Derek for his effort to put this together.

8 About 24 sets of reports that I've been studying and 9 that Derek's been studying are the basis for the first 10 presentation. So, Derek, thank you for your effort.

11 Colleagues, any questions for me or for 12Greg or for Tom, please? Hearing none, Mr. Chairman, 13 back to you, sir.

14CHAIRMAN CORRADINI: Okay. Thank you very 15much to the staff. That clock does not work. Just so 16 we're all clear, it's still not -- we haven't -- we're 17 not like an airplane that arrived five years later or 18 whatever the hell that is.

19PARTICIPANT: It's an hour behind right 20 now.21PARTICIPANT: It's actually adjusting 22 itself.23 PARTICIPANT: It stopped.

24CHAIRMAN CORRADINI: Yeah, I think it's 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 131 dead. It's dead.

1 (Off mic comments.)

2CHAIRMAN CORRADINI: So thanks to the 3 staff. Appreciate it. And we're going to go into a 4 lunch break and be back at 1:00.

5 At 1:00 we're going to do the quality 6 review for the research topics of interest. And Dr.

7Rempe will, Member Rempe will lead us through. Okay?

8 Thank you.

9 (Whereupon, the above-entitled matter 10 went off the record at 11:47 a.m. and 11 resumed at 1:00 p.m.)

12CHAIRMAN CORRADINI: So we'll start again 13with the quality review. And, Dennis, I think you're 14 the first one up as chairman of the team and Joy will 15 take over.

16MEMBER REMPE: Okay. Just before you 17 start though I want to remind everyone we're going to 18 have two of these.

19 And as we go through this, think about to 20 the two chairs, Margaret and Dennis, how soon you can 21 turn around the input for the actual document because 22 that's a question we need to think about in when we're 23 going to issue the letter. So go for it, Dennis.

24MEMBER BLEY: Okay, thank you. On the 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 132 title slide I remind everybody our team was myself and 1Walt and Dick. Are we still on the record or is that 2 done?3CHAIRMAN CORRADINI: We're still on the 4 record.5 MEMBER BLEY: Okay. So we'll have notes 6from this. That's good because I don't think I can 7take them. Okay, second slide is the PIRT project 8 description high energy arc fault.

9 And it's been on the NRC's table for a 10while. But some things happened in the last few years 11that have changed their thinking a bit. These arc 12 faults seem to be, not seem to be, have been more 13 prevalent than people thought in the past.

14 Very severe arc events often involve 15 unanticipated break or coordination failure. Things 16 that you wouldn't normally know weren't properly 17 aligned if it was very high past current then they 18 don't work the way people expect them and that causes 19 multiple problems.

20 The existing models they have developed 21 did not comport with experiments that the staff 22sponsored. And you all saw the aluminum bus fire and 23 enclosure fire, hello.

24CHAIRMAN CORRADINI: Yes, we're still 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 133 here, Dennis. We can hear you.

1MEMBER REMPE: If anyone is out on the 2 webcast please put your phones on mute, okay.

3MEMBER BLEY: I am hearing a reprise of 4 Mike's introduction that's very loud.

5CHAIRMAN CORRADINI: I think, Dennis, 6 there seems to be a delay with the webcast so if you 7--8 MEMBER BLEY: Well there always is. But 9 before I did not hear it over my phone and now it's 10 really, I'm hearing 50 seconds behind and it's really 11 confusing.

12 CHAIRMAN CORRADINI: I understand.

13MEMBER BLEY: So if they can cut down that 14 crosstalk it would be great. But I'll go ahead.

15 CHAIRMAN CORRADINI: Okay.

16 MEMBER BLEY: And try to put that out of 17 my mind.18CHAIRMAN CORRADINI: I can even hear 19 myself.20MEMBER BLEY: Well once they ran into that 21 problem with the aluminum and other issues that kind 22 of surprised them they thought maybe doing a PIRT that 23 helped them set priorities for further research.

24 MEMBER REMPE: Dennis.

25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 134 MEMBER BLEY: The report --

1MEMBER REMPE: The webcast is coming 2through perhaps your system. Can you put your system 3 of the webcast on mute because we are hearing you 4 through the phone line, okay.

5 MEMBER BLEY: I didn't hear it because I 6 had the headset on.

7MR. BROWN: Dennis, you can't have the 8 webcast going at the same time you're speaking because 9 it's a ten minute, ten second delay.

10MEMBER BLEY: It's actually a 50 second 11 delay.12MR. BROWN: It's a delay so you can't have 13--14MEMBER BLEY: I didn't know my speaker --

15 MR. BROWN: You can't have both on.

16MEMBER BLEY: I didn't know my speaker was 17on because I had these headphones on. This should be 18 better now.

19 MEMBER REMPE: Thank you.

20MEMBER BLEY: We found that the PIRT 21 exercise was conducted in a reasonably satisfactory 22 way to frame the financial risk contribution in 23nuclear power plants from these events. And they also 24 tried to evaluate their own state of knowledge when 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 135 they did the PIRT which is a smart thing to do.

1 Next slide, documentation, clarity of 2presentation. Our consensus score was a five. I'm 3 going to take a little diversion because of the 4 conversation that went on in the last couple weeks.

5 We did this kind of the way I've been on 6 three or four of these, kind of the way we've, I've 7always done it. But others have said, gee, the way we 8 did this if one person is an outlier we just make 9 them, talk them into changing their score so they're 10 all about the same.

11Well we didn't do that. We followed what 12you do on elicitation. And if we were diverse and on 13 a few things we had scores as wide as, whoever is 14 moving the paper is driving me nuts.

15 We had things as diverse as an eight and 16a four or a three. So instead of saying let's come 17 together we talked about why each of us had the scores 18 we had.19 And when you do that you find out we were 20looking at slightly different things. And some 21 things, I don't know if you've played with this scale 22 much, some you could put in multiple places.

23 And some of us would put one issue under 24 clarity of presentation where the other person might 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 136put it under uncertainty. So we reached agreement on 1 where everything belonged and exactly what questions 2 we were trying to answer.

3 And then independently still, we 4reevaluated, boy, that sound keeps coming in. We 5 reevaluated our individual scores and you'll see those 6 at the end of the presentation.

7 After that point we had a final discussion 8 following the ideas that are in the Shack report 9 saying let's come to a consensus on the score that we 10 think represents kind of the best knowledge of 11technical people in this area. So we treated our 12 scoring like a properly done elicitation that was 13facilitated and I did the facilitation. Let's go 14 ahead then.

15MEMBER REMPE: Dennis, do you want to 16 point out to everybody on Slide 9 that you do have the 17 individual scores too just as they go through this?

18MEMBER BLEY: I was going to do that when 19 I got to Slide 9, but you pointed it out.

20 MEMBER REMPE: Okay, thanks.

21MEMBER BLEY: We had slightly different 22scores on this to begin with. When you start reading 23the report it's very good. But then it turns out that 24 really three of the chapters, 1, 2 and 4 are really 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 137 well written, clear and precise.

1 Chapter 2 provides a really good 2description of the PIRT process and how it ought to 3 work and including the things that would affect 4 uncertainty and helping identify important factors.

5 Chapter 4 was pretty tight and ordered on 6 the presentation of results, but at a high level.

7 They didn't dig into the details of the ordering out 8 at that point.

9 If you read Chapter 3 which is kind of the 10 results section and you don't know what you're 11reading. After you read the appendices you know 12 what's in Chapter 3 but you can't really tell it just 13 from Chapter 3.

14 It's a collection of results and tables 15 with not much to tell you what you're looking at and 16why it is the way it is. So overall we thought it was 17 pretty good and you'll see later that we think there's 18 enough information you can figure out what's in 19 Chapter 3.

20 Our individual scores on this one after we 21 had that reconciliation of what we were looking for 22was a five, six and a seven. But when we talked about 23 it, especially because of the way Chapter 3 is put 24 together, we all agreed we would give it a five.

25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 138 Next slide which is identification of 1major assumptions. This is a tough one and Joy 2 pointed out, you know, there are some things we 3 complain about that maybe it shouldn't be a five.

4 But one of the key things they had to do 5was figure out what to look at. And they found three 6 generic fault scenarios that were very well documented 7 and provided a reasonable baseline for the PIRT.

8 The report I thought, well we all thought, 9 clearly and objectively described the assignment of a 10 hierarchy of phenomena and employment of the 11methodology. They did very good, better than 12 satisfactory on description of the bases of the 13 phenomenon and defending their importance.

14 There were, however, some unstated 15 assumptions that took this away from being a lot 16 better than good, normal work, good, professional 17work. They give you the rankings they use but they 18 don't really have, what you ought to do is have a 19 plain English scale that describes what they mean so 20 you're all using them in the same way.

21 They had an unknown ranking and although 22 it's never explicitly stated the way they put the 23 scores together they essentially treated unknown as if 24 this particular phenomena has no value and that's not 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 139 right. If it's unknown it could be really important 1 or it might not be.

2 To treat it as absolutely unimportant is 3 putting a pretty strong bias in the report. This is 4 linked to something you'll see later on facilitating 5 the process.

6 They have a rank equation and it kind of 7 makes sense. It goes the right way when you put the 8scores together. But there's really no justification 9 of why it's especially, why it's the right way to put 10 them together.

11Excuse me a second. Wow, pardon me. And 12 finally they assume that their three scenarios span 13 the space of high energy arc faults.

14 Some place later they note that the people 15 on the Panel could have looked to see if there were 16 other things they hadn't looked at. But that should 17 have been, we think, explicitly a part.

18 So there were some unstated assumptions.

19 But they documented what they did very well and we 20came out with a five on that one. Our own scores were 21 five, five and seven.

22 I had a couple notes here. I've already 23covered those, very good. Next slide is justification 24of major assumptions. I also had a consensus score of 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 140 five on this one.

1 The overall assessment was reasonably 2good. The authors provided appropriate and useful 3 justification of the assumptions they identified as 4 assumptions.

5 The identification of the phenomena that 6 they were investigating created a little confusion 7 between cause and effect for us as readers especially 8 one of us. And that's a reasonable thing.

9 Later they give enough story that you see 10 that they're evaluating both cause and effect issues 11and it's a reasonable thing to do. But they didn't 12 introduce that as well as they should have.

13 I already noted the treating of the 14 unknown ranking as having no value is never justified.

15And it apparently is not recognized. If they're going 16 to get facilitation that should have popped out at us.

17 The rank equation, so we didn't really hit 18 them too hard on the previous one on these things that 19were assumptions. But over here under the assumptions 20 expanded it a little bit.

21 And the other one is one I talked about 22 before, little explanation of why the three scenarios 23 were picked and why that, you know, we ought to think 24that stands in place. But on the main issues they did 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 141 and on the assumptions they announced they did a 1 really good job of explaining them.

2 So there are some things that push well 3above a five, some things that push below a five. Our 4 scores here were four, five and six with one of us 5 thinking they really did a pretty good job and one 6 thinking not so good and one thinking down the middle.

7 After we discussed it given where we 8 scored other things related to this we agreed that a 9 five was a reasonable score for this issue. We'll 10 come back to those things that were hidden assumptions 11 later.12 Next slide is soundness of technical 13approach. Here they come out a little better. Our 14 final scores were, this was after we readjusted our 15 individual scores.

16We were a four, a six and an eight. So 17two of us above five, one of us below it. After we 18 had our discussion we kind of centered back on it, 19 they really did a pretty good job but you've got to 20 dig around to find it.

21 So the first bullet is talking about that.

22 If you take the whole report and you mentally 23 integrate the description of the methodology which is 24 very well done in Chapter 2, the pretty cryptic 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 142 summary in Chapter 3, the detailed results in the 1 appendices and the conclusion of Chapter 4 you can 2 evaluate the overall soundness of the report.

3 And we think it's a little better than a 4good, professional job. The approach used in 5 performing the elicitation from the six experts was 6 sound, well documented and produced a useful product 7 for informing a road map moving forward with HEAF.

8We have some caveats on that, that will 9show up on a later score. There are a number of 10 issues associated with the proper role of the 11facilitator. We have chosen to evaluate all of these 12 under the following section.

13 They could fit in other places like here.

14But we didn't evaluate them here. We grouped them all 15 in that one place and think that was a reasonable 16 thing to do.

17I'll take an aside. If you read other 18 PIRTs where you can find some guidance on doing PIRTs 19 you'll find they don't talk much about the facilitator 20 and why that's important.

21 There's vast literature on how you 22 facilitate things like this, things like expert 23 elicitation. If anybody is interested I can provide 24 you a paper or two or just a nice short summary that 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 143 links you to the literature.

1 Much of this goes back to the mid-70s when 2 a lot of these ideas were developed. Even the Shack 3 system when you get into that were very sparse on how 4to do this facilitation well. And if you don't do it 5 well you open yourself up to problems which we'll 6 discuss in a minute.

7 The last one results the objectives on 8uncertainties and sensitivities addressed. Our scores 9 on this after our readjustment that show the 10 individual score was a three, a five and a five.

11 After we had a discussion and thought where this fits 12 within the scope of everything we came up with a 13 consensus score of a three, not as good as it ought to 14 be.15 The first one is that they really had a 16 diverse background of people on the Panel who were 17volunteers from international organizations. Some 18 sent project managers. Some sent technical experts, 19 but it was a mix.

20 But it's got a nice diverse background for 21 gaining different perspectives and addressing and 22 ranking important aspects of the three scenarios they 23evaluated. This could have enabled uncertainties and 24 sensitivity to be well identified as Chapter 2 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 144 indicated they ought to do and addressed quite 1 objectively.

2 On the other hand, the diversity of the 3 experts working essentially independently with the 4 same data, the good thing about that is you don't get 5 a group think and this form of elicitation is 6 effective because it enables objective assessment that 7 accounts for uncertainties and sensitivity.

8 When you get to how things are facilitated 9 however, you really want to bring the group back 10 together and understand why independently they came up 11 with what they did and if there are places where 12 sharing information might be very helpful. In fact, 13 there are a lot of clues in the results that indicate 14 places they really should have delved into.

15 We, our little group, questioned the 16 completeness of the three scenarios which were derived 17 from actual events and say are there other 18possibilities that are not covered here. And there 19 could be.20 You can invent some if you think through 21 this thing. There's a little bit on the other side.

22 There were, I'll save that until we get to the next 23 one.24This is a continuation. This one 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 145especially bothered me. The evidence of facilitation 1that, is that could be is pretty strong. Chapter 2 I 2 think fairly thoroughly addresses this.

3We all agreed on this. But it appears 4 that no one forced the experts to identify the 5uncertainty in their own evaluations. It was also not 6 clear how the experts were advised and they were 7 advised to base their importance ranking on risk.

8 But if you're not a risk an analyst and 9 haven't done a lot of study they should have given you 10 some guidance on how to do that well. They may have 11 done it informally. But not even a hint of it shows 12 up in the report.

13 Chapter 2 gives a hint about how to seek 14consensus. But there's no discussion of consensus 15building. In cases where rankings span the full range 16 and this happened quite often, I forget what their 17 scale is.18 But say it was one to five. On the same 19 elicitation one person has a five, another had a one.

20They just leave it that way and use it. But that's a 21 real hint, as I said, we found out in our own that 22 they're evaluating different things.

23 The facilitator should have taken those 24 and said, okay, each of you explain why you give it 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 146 the score you did and when you go through that process 1you see you're doing things differently. You're 2 looking at different aspects of this issue.

3 Sometimes the right thing to do is say 4there are two different things here. Let's break this 5into two separate elicitations. And you usually find 6 that then you come much closer together.

7 Sometimes you would find that one person 8 had access to information that not everyone had access 9to and when you share that you do better. The 10 facilitator ought to dig into this stuff when it looks 11 funny and help seek resolution.

12 There was another area where they, on a 13 few issues said we don't know enough and didn't make 14the recommendation. The facilitator could really help 15 with that pushing on what you do know, what you don't 16 know, do we need some outside expertise to come in and 17 help you with it.

18 And that sort of thing wasn't done.

19Here's our final summary. Now remember, the scores 20 you see under the three of us were after we had first 21 decided how we needed to evaluate each of the points 22 that we judged and these scores are much closer 23 together than our original scores were because we were 24 evaluating the same thing.

25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 147 We didn't just do arithmetic on these 1 because there is a serious problem with just doing 2arithmetic on these kind of scores. And these are 3 fairly close together so it's not as big a problem.

4 But if you have a very high outlier and 5 you do an average, a mean value the high outlier 6drives the answer. You can play with some numbers and 7 see that.8 And then you say well instead of that 9let's do a geometric average. Well, if you did the 10 geometric average that kind of takes care of the high 11 outlier but now the low outlier drives the show.

12 And if you play with that you'll see 13that's true too. We come together and discussed these 14 and say we think a reasonable place the community 15 would come together on this is the clarity of 16 presentation of five.

17 After we get all our scores we go through 18 and do the weighting as indicated in the methodology 19and we come up with a final score. So that's our 20 whole story.

21We have to write text around it. I am 22 still a little slow and bleary so I am not sure I will 23get that done in October. There's a good chance I 24 will.25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 148 If I don't we'll certainly be able to 1resolve this in November and pull it together. My 2 expectation is I'll be with you folks in person in 3 November. But that still remains to be seen.

4 I've got a lot of things to go through 5 before they say, yes, hop on an airplane, go back 6 there. That's all I have to say.

7MEMBER REMPE: Thank you, Dennis. Do any 8 of the Members have any comments or, about the ranking 9 and the information that Dennis has presented here?

10 So actually since, did you want to say 11 anything to add to what Dennis has presented?

12 MEMBER SKILLMAN: No. I support Dennis.

13 I looked at the slides, you know, he sent them out for 14 comments and Walt and I got back to Dennis and I'm 15 comfortable with Dennis' explanation of what we've 16 done.17MEMBER BLEY: Walt agreed with that as 18well. I mean the slides come from our discussion, 19 pretty much straight from the discussion onto the 20 paper.21MEMBER REMPE: So I have one comment. And 22 again, it's just, when I looked at the report and what 23I was thinking about. And it's not, some of your 24 comments are actually just general guidance that I 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 149 think is useful for PIRTs in general.

1 And so I guess that's a comment, why I'm 2 giving this comment is that with respect to unstated 3 assumptions about the usefulness for the PIRT for 4 future research because the report actually, although 5 maybe it wasn't a stated assumption, they did go 6 through and talk about some of the limitations of the 7 information that they extracted from this PIRT, like 8 country specific influences.

9 And I think that again they took care of 10it in this document. But it might be something worth 11 highlighting in your text or something along that line 12 is why I'm bringing up that discussion because right 13 now we're hearing that the accident tolerant fuel 14program will be soliciting and performing PIRTs to 15 guide that research.

16 And so I think any thoughts we have that 17 are generally good thoughts or guidance on PIRTs could 18be documented here too. Any thoughts about that, 19 Dennis or Dick?

20MEMBER BLEY: Well it's, I kind of like 21the idea of the country specific stuff. And I will 22 try to read some of that in. I'm a little unsure of 23 us giving guidance in a review of the research report.

24 If we want to maybe we ought to do a 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 150 separate letter that says, you know, there are some 1 things that are missing in general in this PIRT 2process as implemented by some people. Now I think if 3 you read their Chapter 2 all of these things that I'm 4 talking about should have been done.

5 But it wasn't explicitly covered on some 6 things. So it, we can put a little outline of that.

7 We can put a little outline of what the facilitation 8 would be.9It seems like the wrong place. But I'm 10 not sure of that.

11MEMBER REMPE: Well actually another point 12 they raised is sometimes when they do a PIRT the cost 13 of getting the data influences how the experts weigh 14the importance of certain data. If they know it costs 15 a lot they basically didn't rank it very highly even 16 though it might be important to have.

17And so some of those insights I thought 18were good insights. And so I'd like to see it 19somewhere in your write-up. But it's up to you on 20 what you guys want to do. Any other comments?

21MEMBER BLEY: I don't quite remember that 22 one. I have to go back and try to find that.

23MEMBER REMPE: Yes, I thought it was kind 24 of an interesting nugget.

25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 151 MEMBER BLEY: I don't remember they said 1 it cost too much so we shouldn't think about it.

2MEMBER REMPE: They said it was like an 3 underlying influence on how the experts, I'm pretty 4 sure I saw that somewhere in the report because I 5 think that might be a true statement just in general 6 about person, it's a limitation that they noticed.

7MEMBER BLEY: I'm going to have to go find 8 that because that's a, that's really, you know, this 9shouldn't be a cost thing. This ought to be saying 10 for what's important here's what you ought to do then.

11 Then a manager says well, that costs too 12 much, I can't do that even though it would be very 13 important to us. But it shouldn't affect the scores 14they give them. And I missed that if that's in there.

15MEMBER REMPE: Well I don't think they 16encouraged people to do that. They just observed that 17 it was an underlying factor that influences experts.

18 And so I'll try to find that, the actual quote and 19 send it to you. But I thought it was --

20MEMBER BLEY: That would be helpful. And 21 it might show up in the place I was talking about 22 where they said we didn't have the expertise to 23 address this issue.

24And I think that falls, can be included 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 152 under discussion of what good facilitation for this 1 kind of process would be which is missing from many of 2 the lists of advice for doing this, for doing 3 elicitation, et cetera.

4 MEMBER REMPE: So my notes indicate --

5MEMBER BLEY: If you can find that I'd 6 like to highlight it and I can, I know I can link it 7 to the, we didn't have the right experts here issue.

8MEMBER REMPE: My notes indicate it was in 9Chapter 4. But I'll find the exact location for you.

10 MEMBER BLEY: Thanks.

11MEMBER REMPE: If there's no other 12 comments from anybody.

13MEMBER SUNSERI: I had one question really 14for the group. Can someone remind me what the scoring 15scale is? I mean a five represents good, sound, 16 technical work. Is that right?

17 MEMBER BLEY: That is correct.

18MEMBER SUNSERI: So the overall assessment 19 of this Working Group is that this product is a sound 20technical product but has some room for improvement 21 based on the recommendations you're making.

22 PARTICIPANT: That is correct.

23MEMBER BLEY: It would be, it's useful for 24going ahead and prioritizing the research. But you 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 153 could do better, yes.

1 And, Matt, if you look on the second page 2 of Chapter 2 of our report where our methodology is 3 laid out there's a table that tells you how the scores 4 go from outstanding to unacceptable.

5MEMBER REMPE: And Margaret actually 6 included in her --

7 MEMBER CHU: In my presentation I have a 8 slide on that.

9 MEMBER REMPE: Page 7.

10 MEMBER CHU: Yes.

11VICE CHAIRMAN RICCARDELLA: This is Pete.

12 I hate to --

13MEMBER BLEY: In our report it shows up in 14 Chapter 2. We didn't replicate it.

15VICE CHAIRMAN RICCARDELLA: This is Pete.

16I hate to show my ignorance. But what does PIRT stand 17 for?18MEMBER REMPE: Phenomenon importance 19 ranking.20MEMBER SKILLMAN: Phenomenon 21identification and ranking table. I was going to just 22 make one comment, Joy, and perhaps this will support 23 what Dennis is doing, has done.

24 This was not our review of the research.

25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 154 It was our review of the PIRT process. And so we 1really had two different bundles of information. One 2 is the data that research has provided including that 3 excellent video of the aluminum bus exploding.

4 But then we had the documentation of that.

5 And this was a review of the documentation of the 6 PIRT.7 MEMBER REMPE: That's what it should be, 8 yes.9VICE CHAIRMAN RICCARDELLA: So we were, if 10 you will, swapping between glasses and binoculars 11 making sure we were keeping clear in our mind which 12one we were evaluating. And it is the PIRT that we 13 were evaluating.

14 Now there was abundant, excellent, 15 remarkable data. But as stunning as the data was we 16were trying to evaluate the PIRT of that data. And at 17 least I found that to be kind of a wrestling contest 18 in my mind.

19 The other thing that, about what would be 20 in our letter relative to the PIRT, I think Dennis has 21 identified the three or four items regarding 22 uncertainty which when captured will help the next 23 effort for those who would evaluate a PIRT.

24 But that will be in the record so we can 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 155capture it for our letter. So I think the main things 1that Dennis was trying to capture will be in the 2 transcript.

3VICE CHAIRMAN RICCARDELLA: Are the 4 uncertainties relative to the data or to the PIRT?

5MEMBER SKILLMAN: To the PIRT, how they 6 handled it.

7 VICE CHAIRMAN RICCARDELLA: All right.

8MEMBER REMPE: So if there's no other 9 comments or questions should we move on to Margaret's 10 group?11 MEMBER CHU: Thank you.

12 MEMBER REMPE: Thank you, Dennis, and to 13 his team and to Walt and Dick also.

14 MEMBER CHU: We'll move on to the second 15review project. We have a review panel of three; 16Jose, Vesna and myself. And then the research project 17 title is correlation of seismic performance in similar 18 structures, systems and components.

19It's NUREG CR-7237, next please. I'll 20give you a little background. When an earthquake 21 occurs near a nuclear power plant it subjects all the 22 SSCs to ground motions and it depends on the level of 23 the ground motion one or more failure of SSCs could 24 occur.25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 156 So the Seismic Probabilistic Risk Analysis 1 requires an estimation of possible correlations among 2seismic failures of similar components. This 3 correlation is a large area of uncertainty.

4 And frequently, actually all the time 5 almost, very simple assumptions are made that 6 component failures are either fully coupled or 7 completely uncoupled depending on the design and the 8 relative locations of the SSC in the plant.

9 So basically as you have two equipments or 10 components that are co-located to each other you 11 assume they are 100 percent dependent on each other.

12 They fail, either they don't fail or fail 13 simultaneously otherwise they are totally independent.

14 Now this project was actually part of the 15 research office 2010 to 2014 Seismic and Structural 16Research Plan. And then the project was initiated to 17 determine the impact or correlation assumptions of 18 risk estimate and to recommend a better approach that 19 may reduce uncertainty.

20Next please. Now this project consists of 21four required tasks from the Research Office. The 22 first is the review of SPRA, seismic PRA in literature 23to understand how people treat correlation and the 24 impact of correlation assumptions on risk estimates.

25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 157 The second task is review of existing 1 literature on seismic correlation analysis 2methodology. The third is the review of existing data 3 from earthquake experience and shake table test for 4 their usefulness in the correlation issues.

5 The fourth one is with the help of experts 6 in a series of workshops, actually there were two 7 workshops, recommend methodology that better addresses 8 correlation issues. Next please.

9 Before I go to the scoring I just want to 10 briefly summarize what are the results of this 11project. First one from review of existing seismic 12 PRAs seven categories of SSCs are judged to be more 13 sensitive to correlation assumptions.

14 I'm going to have another view graph just 15after this. Now the earthquake data shake table test 16 are found not to be useful for this project.

17 And from literature reviews of 18 methodologies to treat seismic correlation, the 19 research team identified the Reed-McCann methodology.

20 That's from, I think 1985, also called separation of 21 independent and common variables methodology.

22 And that is recommended for further 23research. Next one please. When I say they identify 24 seven categories of SSC that are judged to be 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 158 important to correlation assumptions what the team did 1was upon reviewing the seismic PRAs these seven 2 categories were found to first dominate seismic risk 3 contributors.

4 And secondly, they were judged to have a 5 high degree of a potential correlation because of 6 their numbers within the plant and their typical 7 locations within the plant.

8 And this is the list of seven things:

9 masonry walls; electrical tanks; mechanical batteries 10 or racks and so on. Next please.

11VICE CHAIRMAN RICCARDELLA: Question, 12 Margaret.13 MEMBER CHU: Sure.

14VICE CHAIRMAN RICCARDELLA: Did they 15 actually look in those PRAs where they had considered 16 full correlation and then did the same analysis with 17 no correlation and, I mean we saw numbers yesterday 18 like 30 percent to 70 percent potential difference?

19 MEMBER CHU: I think they did.

20MEMBER DIMITRIJEVIC: I think my 21 impression is this is not easy to do because I think 22PRAs are already complex. And just to do sensitivity 23 study like that it would require a lot of 24 manipulation.

25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 159 So I think whatever they did has to be 1 some simplified way of looking, you know, between the, 2how was correlation treated. I didn't get the 3 impression how exactly they did.

4 But they say it's complex and a not easy 5project. That would be one research project of itself 6 to do sensitivity studies like that especially --

7VICE CHAIRMAN RICCARDELLA: Do the people 8 who do PRAs normally do sensitivity studies?

9MEMBER DIMITRIJEVIC: The couple I am 10 familiar with tried to do that by doing the scale, you 11 know, the scale correlation factor which is between 12zero and one. So they can then easily do the zero and 13 one.14 It wasn't this basic assumption. That's 15a little different method. If then they discover 16 strange results they actually, and then spend two 17 weeks trying to understand and didn't really come to, 18 the result was actually you will assume the full 19 correlation with a reduced, the highest CDF it 20 actually produced a lower CDF.

21 VICE CHAIRMAN RICCARDELLA: Really.

22MEMBER DIMITRIJEVIC: And it came out 23 because sometimes in the model when you separate this 24occur you go into different branches. And when you 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 160 put this together in correlation it was a very 1 complicated case.

2But this can also, it is not necessary 3that one is more conservative than the other. It 4often depends on the model of the PRA. But that's my 5 personal experience.

6 This was not discussed in the report in 7the task. That's one of our comments you will see 8 that we were not sure there was proof importance of 9 that.10VICE CHAIRMAN RICCARDELLA: So it was 11 basically judgment the factors that they said could 12 affect CDF by this much was basically from judgment 13 not from action?

14MEMBER DIMITRIJEVIC: Well looking in that 15in sequences and concepts, yes. I don't think that 16was detached sensitivity study. But I'm sure there 17 was technically some. That's my --

18 MEMBER MARCH-LEUBA: I thought they told 19us yesterday that the state of the practice was to 20 send it to see the one and see the difference and 21that's how they got to that 30 to 60 percent. But 22 you're the expert. That's what I thought I heard.

23MEMBER DIMITRIJEVIC: Yes. When we had 24 discussion with them.

25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 161MEMBER MARCH-LEUBA: That's what I thought 1 I heard, but I'm biased.

2MEMBER CHU: I have a feeling there's a 3 lot of judgment in there because they went to, with 4the experts in the workshop on these. So that made me 5 think it's sort of a, judgmental in many ways, yes.

6VICE CHAIRMAN RICCARDELLA: They didn't 7 present any hard data results in the report.

8 MEMBER CHU: No.

9MEMBER DIMITRIJEVIC: And it's not an easy 10 study to do. The model is complex.

11MEMBER CHU: Okay. And then the Reed-12 McCann methodology is the recommended separation of 13independent and common variables methodology. In this 14 methodology they develop a procedure to estimate 15 dependency between component failures by searching for 16common sources of variability in the response and 17 strength calculations.

18 And then the analyst needs to carefully 19 examine the component design and qualification 20 documents, material properties, installation methods, 21 et cetera to make judgment on what are the common 22 variables and then decide what's not.

23 And so we as a panel feel the methodology 24 could be difficult to implement and it could vary from 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 162 analyst to analyst because it's such a judgmental 1 cause.2Next one please. Now just to refresh 3 everybody's memory on what those scores mean as Matt 4requested, five is really the baseline. It's a 5 satisfactory score. It means you have satisfied the 6 research objectives.

7 And then it goes up and down, you know, 8 from zero to ten and a zero being unacceptable, ten is 9outstanding. But five is basically a satisfactory 10 score. Next please.

11 And then these are the individual scores 12from the three of us. I am number one. Jose is 13number two. Vesna is number three. You can see that 14 mostly that we're pretty even, okay.

15 So what we did was basically take an 16 average of the three scores and then everybody agrees 17is this about right. Sometimes we would round up a 18 little bit, sometimes we round down a little bit, 19 okay.20 Next please. And then the five areas of 21 measure, performance measures is, Dennis already went 22through this. Clarity of presentation, identification 23 of major assumptions, justification of major 24 assumptions, soundness of technical approach, 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 163 treatment of uncertainty sensitivities.

1 And then there are predetermined weighting 2factors for each. You notice that the soundness of 3 technical approach and the results basically is 50 4 percent of the overall score.

5 And then with this our consensus final 6score is five, satisfactory. Next please. I'm going 7 to go through a little bit on each of the performance 8 measures, yes.

9 The first one, clarity of presentation it 10gets a six. And we feel the report is very well 11written and easy to understand. It clearly 12 communicates the purpose, scope and technical approach 13 of the project.

14 Existing methodologies that deal with 15 correlations clearly presented the rationale for 16 selected methodology is clearly articulated. We did 17 find a few deficiencies in the report writing.

18 For example, the organization didn't 19follow the task sequence that was dictated. And then 20 there are also minor problems with the definition of 21 captions in tables or figures.

22 But overall, we felt it was a good report.

23So we gave it a six. Next please. Now identification 24 of major assumptions. We kind of struggled a little 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 164 bit with this one because basically the project, the 1 whole project is a literature review and literature 2 assessment.

3 So we, you know, so we felt the major 4assumptions are not clearly identifiable. So we 5 looked through examples of the assumptions discussed 6 in the report, okay.

7 These are the three big ones. The first 8 one is the correlation assumption is important to risk 9results and risk insights from seismic PRA. And then 10 the second one is that thumb rule of independence, 100 11 percent or zero percent that is being used right now 12 is unsatisfactory and needs to be improved.

13 And the third one is the recommended Reed-14 McCann methodology would require assumptions on design 15 qualification, installation and so on of SSCs. Next 16 one please.

17VICE CHAIRMAN RICCARDELLA: Those were 18 stated as assumptions in the report?

19MEMBER CHU: No, they were kind of buried 20 all over. And we kind of identified them.

21 MEMBER REMPE: It's simpler --

22 MEMBER MARCH-LEUBA: There is no chapter 23 that says assumptions.

24 MEMBER REMPE: And I note that's similar 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 165to what Dennis' group had to do too. Both of these 1 projects were a bit difficult and this is the way the 2 group decided to do this rather than eliminate the 3 metrics.4MEMBER CHU: Yes, because it is not the 5usual research project. You make assumptions and you 6do this and you do that. So that's the best we could 7 do.8 Now as to justification of major 9assumptions remember the three listed before. We felt 10 there was one that is not rigorously justified in the 11 report which is the correlation assumption is 12 important for the seismic PRA results.

13This is a very major assumption. You say 14 is it important or is it not. The report says it is 15 important but we felt it's not rigorously justified.

16 The following bullets are examples of the statement.

17 These are the statement from the report.

18 It says, for example, correlation assumptions may not 19significantly impact the seismic PRA results but it 20 could impact the risk insights. But it didn't quite 21 go into what that meant.

22 And second one for some seismic PRAs the 23 difference in seismic CDF could be as much as almost 24a factor of two. Typically it was a difference of 30 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 166 percent to 60 percent.

1 And then it says in some SPRAs the 2 dependent failure issues is not as important when the 3 overall seismic CDF is dominated by an accident 4 sequence that itself dominated by a PRA singleton, 5 that means a single failure.

6 And then it says for some key accident 7 sequences the difference could be as much as a factor 8 of two to four in the frequency of, notice it's the 9sequence, the frequency of the sequence. So with sort 10of these vague and all over kind of statements our 11 panel wonders are these factors or percentage changes 12 in the PRA number numerical results significant enough 13 to justify to the time consuming and costly new 14 methodology.

15VICE CHAIRMAN RICCARDELLA: Is a factor of 16two on CDF considered significant? I mean it seems to 17 me that's probably in the uncertainty band.

18MEMBER REMPE: But, Pete, they didn't say 19 it was, typically it was more 30 to 60 --

20VICE CHAIRMAN RICCARDELLA: Thirty to 60.

21 But even if it were two.

22MEMBER DIMITRIJEVIC: It's upper bound for 23 significant.

24 VICE CHAIRMAN RICCARDELLA: You know, if 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 167 it's two it's not.

1MEMBER BALLINGER: I read the document 2 like four times and could for the life of me not 3figure out what the actual uncertainty would be. But 4 I knew it was large.

5 MEMBER REMPE: Yes.

6MEMBER BALLINGER: And so a factor of two 7 compared to PRA results which are sort of like fatigue 8 results is a magnitude one way or the other.

9MEMBER CHU: So therefore you notice this 10one got the lowest score, four, okay. And the next 11 one is the soundness of technical approach and 12results. And then we feel that the tasks accomplished 13 were defined and then it was done competently.

14 And then these other tasks that are done 15 as required by the Office of Research and then they 16did every single one of them, okay. And so overall 17 it's done well with the exception of the previous 18 slide, okay.

19 Next please, treatment of uncertainties 20and sensitivities. It gets a five score. We believe 21 the project team has appropriately considered 22 uncertainty and sensitivity.

23 Some sensitivity analysis were performed.

24The report addresses how to obtain the uncertainty 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 168distribution for the results. In the Reed-McCann 1 methodology the analysts have to deal directly with 2 common variables and their epistemic uncertainty and 3 random variability.

4 I don't know how to pronounce that word, 5yes. And since expert judgment is needed in the 6 partition in between the independent and dependent 7 parts the proposed methodology are introduced new 8 model uncertainty, okay.

9Next please. Now we have some concluding 10remarks. I'm going to turn this most important slide 11 to Jose and Vesna.

12MEMBER MARCH-LEUBA: Vesna gave me the 13permission of starting. So there are three 14 conclusions that we arrived at only on the scope.

15 On the first bullet the name that sticks 16out is competence. When you read this NUREG you feel 17the competence of the people that were doing it. And 18 when we interviewed the staff and we tried to obtain 19 more information you really felt that they know what 20 they're doing.

21So that has to be stated. This was a 22competent effort. I'm sure it was the only good thing 23 we can say about the report because it's like point 24 number two says that even though it is a good 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 169mathematical method and it will work it's almost 1 impossible to obtain data to apply it because we have 2 difficulty getting the data for the known correlated 3 seismic PRA.

4You have such uncertainty. Now you bring 5 uncertainty too into two, one that is correlated and 6 one that's uncorrelated and it's almost impossible to 7 get because all the shaker table experience that, 8 doesn't tell you anything at all because you never 9 shake it to failure.

10So they never fail. So you never know how 11they correlate together. And experience from real 12 life earthquakes has so much variability that you 13 really cannot tell.

14 So we find that this would be very 15difficult to implement with accuracy. And actually 16 you would increase the uncertainty of the results.

17 On point number three is what we've 18 already been discussing over the whole presentation.

19 We questioned whether it is worth, given all the 20 uncertainty that we're going to increase and at best 21 we're going to get a factor of two, more likely 30 to 22 60 percent.

23 Is it worth implementing it? And Vesna, 24 you probably can tell us what a factor of two buys you 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 170 in this one.

1MEMBER DIMITRIJEVIC: These factors, 2 numerical values it's, they, I mean obviously the 3 factors of uncertainties and the importance measures 4in the PRA are not so significant request. But our 5 main impression was when the goals for this project 6 were assessed this report was a pleasure to read.

7 And I'm very thankful to doubters because 8 they have the good skills. They do a lot. They put 9 a lot in the report.

10 However, when they defined the goals in 11 the beginning that's exactly what I would want to 12 know, you know, if I was concerned about correlation.

13 Is it important?

14 And you have a feeling that it should be 15 important, right just like assumptions the components 16 failing the same time or, you know, that it's in the 17patterns between them. And then to say they're going 18 to look in the data to see can they justify something 19 and then they propose the method.

20 And if they said in the beginning we're 21 going to look in the different method, give you an 22 overview of that and tell you what we see in the 23 industry and don't set this goal this report would get 24 much higher value because it was our feeling that they 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 171 did not introduce goal in the way, as Margaret is 1 showing that slide, they didn't really make a 2 statement of how important it is.

3 When you read what they did you realize 4 yourself but it's not an easy task and it will require 5much more effort. So they could not really tell us 6 what's the importance of using one co-relationship, 7 whether it's other or not using it at all or, and what 8would be benefit for this which is very complex and 9 costly method which has to be done with a person who 10 knows fragility very well.

11 And there is not too many people we bring 12in to do these tasks. And to be, what is really true 13 that doubters are honest. They didn't overstate it.

14 They did not tell us, hey, this is 15important, you have to do it. They did not tell us 16these assumptions are bad or simple assumptions. They 17 were basically honest.

18 But they didn't reach their goal because 19 they set this goal very high. If they say listen we 20look in the literature. We cannot conclude because of 21 blah, blah, blah.

22We look in data. We couldn't find the 23 data to support. They said that in the project from 24 the data or from because you always ask, Joy, if there 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 172 is enough data in industry so that we can conclude 1 something about the co-relationship.

2 And they say but we present to you a lot 3 of, you know, not a lot but some selection of method 4 and we think that this method has promise. It would 5 be different type of evaluation if that was stated as 6 the goal.7 So we couldn't really basically conclude 8 that the data, that anybody would really, you know, 9 want to invest that this report, basically they have 10 a problem that some correlated components dominate the 11 risk. They may want to look into this methodology.

12PARTICIPANT: You know, I think the key 13words are on one of your slides. Slide 12 is the 14 impact risk insights.

15 And I think that's maybe my opinion about 16 PRA in general is it's not so much, I mean you don't 17 design things based on you've got to get 10

-7. But 18 you get some insights, some general insights.

19MEMBER DIMITRIJEVIC: Yes, and that's and 20we were hoping they would provide some examples. I 21 can think myself of some examples of this risk insight 22 would be that you conclude then that, let's say you 23 were like you could measure intake structure.

24 The intake structure is extremely 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 173important for, you know, the Westinghouse plants. But 1 however, if you have a service for the pumps which are 2 all on the same rotation and the same design that co-3 relationship between those pumps may be more important 4 than intake structure.

5 That could be one sort of insight because 6 insights from PRA in identifying important sequences 7 has important components, human actions and things 8like that. But they never, the only way they mention 9 that is in that one sentence.

10 It was never explored further in the 11 report.12MEMBER MARCH-LEUBA: I think, Peter, your 13 recommendation, your comment is very wise. And even 14 though when we write the report which Margaret has 15 volunteered to do we maybe, our job is to grade the 16 report.17 But maybe we should make a recommendation 18 that it, this methodology would be more valuable 19 applied for insights than for quantitative sharpening 20 of the pencil. We don't think the sharpening of the 21 pencil by itself is worth it.

22 But the analysis tells you what components 23 you can improve especially for new reactors with 24 multiple modules.

25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 174MEMBER SKILLMAN: I want to ask a question 1 here. Jose, you used the, you expressed the comment 2 because the shaker table doesn't take the device to 3 failure we end up, I think, not knowing or a comment 4 like that.

5 And that is a comment that was also made 6 yesterday afternoon.

7 MEMBER MARCH-LEUBA: That is correct.

8MEMBER SKILLMAN: I'd like to try to flip 9that at least in challenge. When general design 10criteria were set out, the revised general design 11 criteria was 1969 and 1970.

12 In '70, '71 and '72, I bet you would 13remember this, we were doing seismics. And we were 14doing square root some of the squares three 15 dimensionally, two horizontal, one vertical.

16 And Reg. Guide 1.48 was produced and that 17was active seismic testing. And then came Reg. Guides 18 1.26 and 1.29 that identified quality classification 19 levels one, two and three and what was seismic one, 20 what was seismic two and seismic three.

21 And I know for a fact because I was part 22 of it the team at VNW, the team at Westinghouse, the 23 team at combustion and the team at GE we were all the 24 plumbers and we were buying, we were designing and 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 175 buying basically ECCS and all the auxiliaries.

1 And scramble became wood shaker table 2tests are available. And there was one set available 3at the University of Alabama in Birmingham. There was 4 one set of all of them.

5 There was a lot of competition to get on 6those shaker tables. And we shook HPI pumps, 13 stage 7 and 12 stage pumps and we shook heat pumps. And I 8 know Bar W was doing the same thing, combustion was 9 doing the same thing.

10 And when those shaker table tests 11 concluded for the ground floor response that we put 12 into the shaker tables that if those devices did not 13fail that was a success. So now and we said fine, so 14 we installed this equipment all over the product line.

15 Now to say that because it didn't fail on 16 the shaker table sounds to me like fails in 17 correlation, sounds to me it's almost the opposite 18 because it didn't fail on the shaker table what should 19 conclude that remains operable if the earthquake 20 envelope has not been exceeded at that location for 21 that amount.

22MEMBER MARCH-LEUBA: The failure of one 23 and all components would be single.

24MEMBER SKILLMAN: Let me go further. I 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 176 think that same argument is the argument that wins on 1 passive components. And that is your piping and the 2 other robust components that also do not fail on that 3 seismic envelope.

4 Now so I'm struggling with because it 5didn't fail on shaker table it's somehow not 6 compliant.

7VICE CHAIRMAN RICCARDELLA: No, no, not 8that it's not compliant. It just didn't give you any 9 information about seismic correlation.

10 MEMBER SKILLMAN: It gave me a whole lot 11 of information about that component.

12VICE CHAIRMAN RICCARDELLA: About that 13component, yes. But it doesn't tell you whether if 14 you have three other components.

15MEMBER RAY: Dick, listen, the issue isn't 16 will it meet the design basis level of shaking. The 17 question is at what point above that does it fail.

18 And then I would tell you as somebody who has run more 19 shaker table tests than everybody here put together 20 that failure isn't a single defined term.

21 You're talking about the dam foundation 22 failing, you're talking about a breaker within the box 23 failing. What are you talking about failing?

24So trying to gather the data and define 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 177 what failure you're talking about when there are 20 1 different ways that something can fail, it's a complex 2 device and we had to do all of our own because we had 3 a two-thirds GE plant, we had to do all of our switch 4 gear, everything is, I just agree with Jose and I'll 5 shut up.6 VICE CHAIRMAN RICCARDELLA: You agree?

7MEMBER RAY: I agree that trying to get 8 the data that you need through testing, I mean it's 9 different to talk about how the flow and response 10 spectrum is different at different elevations in the 11 building and things like that.

12 But to talk about using a PRA in which you 13 have failure data to project the likelihood of they're 14 not being a failure at some level of shaking above the 15 design earthquake, which is what you need for a PRA, 16 is to me it's a hopeless enterprise.

17 MEMBER SKILLMAN: You know, I agree with 18that. If you're trying to project failure there's 19 never enough data. I agree with that.

20 VICE CHAIRMAN RICCARDELLA: If, okay, go 21ahead. You know, you and I have both done a lot of 22 fatigue analysis, right. We computer the cumulative 23 fatigue usage. And so we have a fatigue usage curve 24 or a fatigue design curve data.

25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 178MEMBER SKILLMAN: We've got a cuff and we 1 know how to use it.

2VICE CHAIRMAN RICCARDELLA: Imagine if 3 when you were setting up this fatigue analysis that 4 you said well my component needs to take, my pipe has 5 to take 400 cycles of this and 50,000 cycles of that.

6 And so I take my test specimen and I sampled it and I 7 tested it for 400 cycles of this and 50,000 cycles of 8 that and said it's fine, you know.

9 You're not getting any information versus 10when you take these specimens and you fail them and 11 you see that, wow, you get a huge degree of scatter in 12 the actual failure data from, you know, half a million 13 cycles to five million cycles.

14 That is information in testing to failure 15 that you don't get by that proof test.

16MEMBER SKILLMAN: I agree with that. I 17 certainly agree with that.

18VICE CHAIRMAN RICCARDELLA: That's all 19 we're talking about.

20MEMBER SKILLMAN: Okay. So we're talking 21 about testing to failure is about, as opposed to 22 testing for competency.

23MEMBER DIMITRIJEVIC: Yes, this is a very 24 good moment actually because we can, Harold introduced 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 179 something which shows seismic PRA complexity when it 1comes to this. The seismic PRA there is a failure 2 probability for let's say ten.

3 Usually the seismic intensity is picked 4 in ten intervals. And it is tightened for different 5 ground acceleration rate, the ground acceleration.

6 In these ten intervals, the last intervals 7 that the components are actually failing are least 8 important from correlation because all the failure 9probabilities are very high already, .9 or something.

10 And therefore, the correlation which is 11 not as significant and the frequencies of occurrence 12of that initial event are very low. And they say this 13 report, the co-relationship is most important in the 14 low or medium range of the seismic where the 15 components are not failing but they may fail.

16 Probability of that failure can be 10

-2 or 17something. And it's not clear what it is. How do you 18 measure for that failure probability because you can 19 only say component didn't fail or it doesn't fail on 20 the shaking table?

21 The same thing when you have a Fukushima 22 you can go around and check how many components 23 actually failed. But that doesn't tell you anything 24 about failure probability.

25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 180 Then you have a different failure mode.

1Will it fail to start? Will it fail to run? It's 2incredible. You have a ground acceleration, but it's 3a million combinations. It's very difficult to get 4 from data.

5VICE CHAIRMAN RICCARDELLA: If anybody is 6 interested in delving into this in more detail Dennis 7 sent me an old 1988 report and it's kind of like a 8 fundamental, you know, it's a paper on seismic, 9 probabilistic seismic risk analysis.

10It's very, very interesting. It's Kennedy 11 and Cornell and some of these guys that were, you 12 know, I could distribute that if people are 13interested. And it was a lot easier to read than that 14 NUREG. It didn't get into Boolean algebra.

15MEMBER SUNSERI: I would like to add some 16perspective here or a perspective. So those of you 17 who know me I'm not much of a curmudgeon outside of 18 the meeting.

19 So I just want you to recognize what chair 20 I'm sitting in and think back to past meetings and so 21my curmudgeon remarks might make more sense. So 22 research, why do we do research?

23 And the purpose of this review is to 24 provide feedback to the research organization to help 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 181 them do better research, right? So when I listen to 1 this conversation and I read the report and I think 2 about well research is going to be done to explore an 3 area that we have some unclarity or uncertainty of 4 we're going to isolate.

5 We're going to research. We're going to 6 review and we're going to elevate the level of 7 understanding of a certain topic so that decision 8 makers can apply that in a way and make better 9 decisions going forward.

10 That's fundamentally why we do research.

11 So when I see the conclusion that, you know, maybe we 12 have correlated or uncorrelated, it takes a lot of 13 effort, what do we do, is it going to make any 14 difference it causes me to have pause about the value 15 of this research or whether it was really actually 16 research or is this a report on a various topic.

17 So taking this a little further --

18CHAIRMAN CORRADINI: You mean a report in 19 progress versus some strong conclusion?

20 MEMBER SUNSERI: So I'm going to go back 21 to the statement justification of assumptions which 22 got the lowest score here which it seems to me the 23 whole purpose of this research would have been to 24 identify your assumptions, isolate them and then run 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 182 a process so that we get a sound technical result that 1 decision makers can use to make better decisions.

2 So I think the fact that this got high 3 marks for sound technical approach but low marks for 4 justification measures assumptions is a big 5 disconnect. And when we look at the final results I 6 don't think that this "research" is much use to 7 anybody because whether you correlate it or not 8 correlate it, it doesn't make that much difference as 9 the report says.

10 So I would have given it an overall much 11 lower score, maybe marginal because what, I mean what 12did it really do from a research perspective? I think 13that techniques that have been outlined here about 14 starting from a fundamental saying we've got to, you 15 know, there's too much variability in just looking at 16 literature or random earthquakes.

17 Let's define a specific systematic 18 assessment way to look at correlation, put stuff on 19 shaker tables, use different configurations, use 20 configurations that we know would be fully correlated 21 and shake it.

22 Use situations that are not correlated, 23shake it. See what the results are and draw some 24conclusions like that. So that wasn't done. So 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 183 therefore I'm not sure, one opinion, what research 1 this accomplished.

2MEMBER CHU: But, Matt, you know this has 3 given to them by the Office of Research.

4 MEMBER SUNSERI: So I'm just telling you 5--6 MEMBER CHU: We kind of --

7MEMBER SUNSERI: I don't disagree with 8 that. I'm not going to, and that is a fact. But my 9 point is though that's a bad start right, you know.

10 So if we're going to give feedback to the 11 research department on how to do better research maybe 12 there needs to be some comment around clarity of that.

13 MEMBER DIMITRIJEVIC: But, I can see how 14 you come to your conclusion and that could be maybe 15 our failure of how we presented this in the general 16 because we didn't have a chance to talk about it but 17 there are things that we just presented.

18We were looking. But I can completely see 19how you came to this conclusion. But I would disagree 20 with this because of the one point. First, they did 21 not have many seismic PRAs.

22 Seismic PRAs are just coming with 2.1.

23 And then you suddenly look now in importance of 24 correlation has money and therefore to analyze 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 184correlation I am sure they will be able to conclude 1 that co-relationship is important.

2 Also they used such a primitive actually 3 assumptions which may be good enough or not but we 4don't know. We just want to say that this report 5 didn't have enough material to conclude for this that, 6 you know, in the general, everybody who is doing that 7 has a feeling there should be something better out of 8 there to do this more scientifically.

9 As a matter of fact and theoretically 10pleasing, and it seems to me we'll have a sense. It's 11 simply complex and I am not a fragility expert so I 12 cannot say.

13 So they are proposing, they do show 14 different methods and they say, industry, that's what 15is variable. If you have a problem with co-16 relationship because your dominant card set is 17 correlated you can consider analyzing these methods.

18 They didn't do any shaker table experiment 19as that wasn't their job.

Their job was to look in 20 results for shaker table results.

21CHAIRMAN CORRADINI: But if I might, since 22 I didn't read the report so I'm totally able to do 23 this, I think where he's going with it, it would be I 24 expect this report to say okay, we've now developed a 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 185 methodology.

1 This methodology has certain pieces to it.

2Now of those pieces this is a key piece. Now go do an 3 experiment or go do something because, in other words, 4what are they proposing to advance it further. I 5 thought that's where --

6MEMBER REMPE: Well, okay, so hold on for 7a minute. A long time ago when we were discussing 8 this because I just review what they presented, the 9 very beginning of the report says they were supposed 10 to find a strong technical basis for a new method is 11 one of their objectives.

12 And I think as Margaret and the Panel have 13said today they didn't do that. They didn't give a 14 strong technical basis for what they are suggesting is 15 an approved methodology because they acknowledge there 16 aren't enough data and as Vesna has emphasized it's 17 not clear it's going to make much of an impact.

18 Now the authors of the report were 19contractors. They were not the Office of Research, 20right. And they basically, I think were a little 21 optimistic because they get paid for doing research.

22 But yesterday, which is not part of our 23review yesterday and I asked Jose about it. He said, 24 yes, it's something we'll look at if something else 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 186comes in. But we've done due diligence in evaluating 1 this possible methodology.

2 And so I think again, I think maybe 3 research was doing due diligence that this methodology 4 might be better and, yes, okay, the contractors were 5maybe a little more optimistic. They didn't fully 6maybe go as far as they could have gone on how, you 7 know, whether this approach should even be considered.

8 But they're contractors and I don't know 9 if the Office of Research can tell them to change the 10 words in your report and say there's a lot more 11uncertainty and all that. And that's another issue 12 with when you're doing research as a contractor for 13 the Office of Research. But --

14MEMBER SUNSERI: That's fine too and I 15understand that. But so maybe the, and let me back up 16 a little bit and say not all research has to be, for 17 it to be successful research doesn't have to result in 18 a positive outcome.

19 You can find it says that the research we 20 did in this area and there are no better ways to do 21this, right. That could have been the fundamental 22 conclusion.

23 MEMBER REMPE: It might have been nice.

24MEMBER SUNSERI: And that would have been 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 187 a great outcome, right.

1 MEMBER REMPE: If the project at the end 2had said this might be useful. But it does have some 3caveats in there if you look. It's not in the 4 conclusions as much as I would like to have seen or up 5 front in the abstract.

6But they did say, hey, the data are 7difficult. It's difficult with experts. They did say 8that. Again, that's a problem about you're dealing 9 with a laboratory where I used to work were doing 10 something that way we would have had trouble if we 11 knew our future bucks hung on that document.

12 So I mean that's just an issue with the 13system. But I understand where you're coming from.

14 But I think that, they did do a competent job, a good 15 professional job.

16 How they presented it, you know, it and I 17 think Margaret, Vesna and Jose are well aware of some 18 of the limitations of the report in how they write it 19 up.20MEMBER SUNSERI: Right. But if you asked 21 your research, you know, if you were, if Ron asked one 22 of his research students to do or they made a proposal 23to do some research in his lab and they wrote up a 24 great report but it had nothing to do with the topic 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 188 would he --

1MEMBER REMPE: But that's not what they 2 did. They --

3MEMBER MARCH-LEUBA: That's not what they 4did. Let me give you a completely different approach 5for the same problem. You know, my brain works 6 differently than anyone else's.

7The research was given a problem. Let me 8start by saying first that I agree with you 100 9percent. I wanted to give them a two on every single 10 item because I absolutely hate the results of the 11 report.12 And that's why I don't know if you 13 remember I spent with you a whole afternoon and Pete 14 was here. We spent six hours just going through the 15 whole methodology through how it is because I even 16 wrote my own Monte Carlo simulation of the problem to 17 get the feeling of how things work.

18 Then I was convinced that I was looking at 19 the report that they were given a problem and that 20 problem was go do a search of the literature. Don't 21 reinvent the wheel. See what has been done and pick 22 the best that's available.

23 And that's the way I understood this 24report to be. I will give the guys that wrote the 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 189scope an F. I would give a two to the guys that wrote 1 the scope of the problem.

2 But once you hear the scope research did 3 the best they could and came up with some bad product 4 because they were given a bad question.

5MEMBER SUNSERI: So let me understand.

6 And there was a user need and that was the scope.

7MEMBER MARCH-LEUBA: It wasn't user need.

8 It was a research program.

9 MEMBER REMPE: Part of their seismic.

10MEMBER CHU: My perspective is I think 11 Office of Research, they probably had a vague idea, 12 okay, and then they say the correlations should be 13important. And they say if you do literature search, 14 you do this and that I think they are very optimistic 15 something would come out.

16But they knew it was hard. Therefore, 17 they say get experts, field experts in this area and 18 then have a few workshops and ask them to help you.

19 So if you look at the scope it's an optimistic scope, 20 okay.21 And then these contractors got the scope 22and then went, diligently went through this. This is 23the best they could do. So we, I think we all 24understood that. I know we discussed many, many times 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 190 about what does it mean, yes.

1 And then this is what we came up with 2because they did everything they were told to do. And 3 then they tried to find all the information. They 4tried to see data, they could get data, okay. So this 5 is what we got.

6MEMBER SUNSERI: And I understand. So let 7 me just respond. And I appreciate the fact that the 8 research organization is going to be asked to do some 9 things that is not purely research, okay.

10 So the PIRT was another example. I mean 11a PIRT isn't research, right, it's a process. It's 12 just like okay, you know, go take a given process and 13 apply it to this phenomena and tell me what the 14 outcome is, right?

15 That's in my judgment not research. But 16 nonetheless.

17MEMBER MARCH-LEUBA: And that should be 18reflected in the report. I think that's what we need 19 to do.20MEMBER SUNSERI: And so maybe in 21 reflection of my feedback here it's back on us that we 22 should be more selective of what projects we select to 23 do our research quality review on to make them more 24 research oriented projects.

25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 191CHAIRMAN CORRADINI: I don't think we have 1 that luxury. The Research Office here is an applied 2research office. And a lot of the things they do are 3 relatively narrow in scope and incremental.

4 And what I heard Margaret, what I thought 5 I heard Margaret say is this is an incremental work 6 scope that they thought they knew the answer before 7 they were going to do it. They said go forth and do 8 it.9 And they come back and it's kind of like, 10 a result. So if you're going to write --

11 MEMBER BLEY: Can I get a word in?

12VICE CHAIRMAN RICCARDELLA: Sure. We 13 didn't even know you were there, Dennis.

14 MEMBER BLEY: Well I've been yelling for 15the last 45 minutes. So Ron must have had me cut off.

16 But he's now cut me back in.

17 I'll try to be very quick and brief. On 18 this last go around I really agree with the idea that 19 these, research that is applied and basic research and 20 the things we looked at today are to me in the applied 21 research area.

22And I think they're reasonable. Now to go 23 back to the earlier discussion about how much this 24 correlation can affect the risk assessment, I'm going 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 192 to take you back to the late 1970s.

1 One of the references in the report is a 2 Seismic Safety Margins Research Program in Livermore.

3 And those guys did a tremendous amount of work and 4 modeled all sorts of correlation.

5And it was quite interesting. But then 6over the next couple of years PRAs were being done.

7 And some of the things that drove this kind of 8 disappeared.

9 You know, we were very worried back then 10 that you have all of these segments of piping and if 11 they are 100 percent correlated, you know, we could 12 have some real problems. Well it turns out that all 13 the piping is, I think that's me causing trouble 14 again.15All the piping is very stout. Also the 16valves we were worried about. So that the issue kind 17 of disappears because you never get up to the levels 18 that take those out.

19And on the other hand, some things are 20very weak. And you go over their threshold whether 21 they're correlated or not.

22 In the early 80s Kennedy and Cornell wrote 23a piece that was kind of a sensitivity study on 24 varying the, looking at the correlation in some detail 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 193and also using these rules of thumb. And it showed 1 the rules of thumb work pretty well.

2 Vesna gave a real good discussion of the 3 idea that the real design of the plant is what affects 4whether correlation is important or not, the real 5 design of the plant and the absolute level of 6 fragility.

7 For the very low fragility it doesn't 8matter very much. For the very high fragility you 9 never challenge them whether they're correlated or 10not. For the stuff in the middle it matters but these 11 rules of thumb have worked pretty well.

12 So I kind of agree with all of your 13discussions. And I just wanted to get that historical 14note in there. We were worried about things that 15 would dramatically swing the results that impact the 16 correlation matter.

17 And they turned out not to matter because 18 of the details of the fragilities and the design.

19 That's all.

20MEMBER REMPE: So we are supposed to be 21 done at 2:30 and I need to let the audience, which is 22 the staff who has come and we've not let them have an 23 opportunity to talk about either project.

24 And then I would like Margaret to also 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 194 weigh in about timing to get her draft done because I 1 assume based on what Dennis has said we're not going 2 to see the draft report until December full committee 3 and I know you're looking forward to another item to 4 put on the agenda in December, right.

5CHAIRMAN CORRADINI: It doesn't have to be 6 done in December.

7MEMBER REMPE: Okay. I just wanted to 8make sure because then we might be February. Is 9 February okay?

10CHAIRMAN CORRADINI: When is it due, 11 March? We're not in a rush.

12MEMBER REMPE: So I wanted to make sure 13 that was okay because I didn't want to --

14MR. NOURBAKAHSH: We are not in a rush.

15 We could send it any time.

16MEMBER REMPE: Okay. So then if that's 17 the case then let's plan for February and everybody 18 won't worry about it. Staff, you have listened so 19 kindly without jumping up and down.

20 Do you have any comments about what you've 21 heard today for accuracy that we need to be 22considering? And tell us which project too just to 23 make sure we understand.

24 MR. HAMBURGER: Good afternoon. I'm Ken 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 195 Hamburger. I'm the author of the PIRT report, NUREG 12218. Briefly just want to say thank you for your 2 comments and your feedback.

3 This was my first NUREG so I will 4 certainly use this as an opportunity improve the 5quality of our research products. Two clarifications 6 that I think are worth making.

7 The first is where a value of unknown or 8uncertain was given and I said that has no value 9 that's not to say that the phenomenon was not of 10 value, simply that it wasn't given a numerical 11 ranking.12 It wasn't including in the arithmetic mean 13that we calculated. I was not saying that the 14phenomenon is not valuable. And the second thing that 15 I --16 MEMBER BLEY: I understand. That's what 17 I thought you meant.

18MR. HAMBURGER: Okay. The second thing 19 that I wanted to mention was although this was not 20 documented in the report we did have more than three 21 scenarios made up at the time of the PIRT.

22 And after we got through the first 23 scenario the group chose the remaining two scenarios 24 based on how much time we had left and how much time 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 196 we thought we would have to get through those 1 scenarios.

2 So there was some group judgment in 3 deciding which scenario we were going to use as part 4 of the PIRT though it wasn't documented and your 5 comments about the scope of the issue are certainly 6 well taken. Thank you.

7 MEMBER REMPE: Thank you. Do we need to 8 also open the line and we'll let the public, if 9 there's anyone out there can someone, Theron, can we 10 assume the lines are open for the public now?

11 CHAIRMAN CORRADINI: I hear noise.

12MEMBER REMPE: Usually they'll come on and 13 say line open.

14CHAIRMAN CORRADINI: Can somebody 15 acknowledge that they're out there on the public line 16 please if anybody is out there?

17MEMBER BLEY: This is Dennis. I'm on a 18 separate line.

19CHAIRMAN CORRADINI: Yes, we figured that.

20 We've heard you enough.

21MEMBER REMPE: Okay, well since we're not 22 hearing anyone else I'm going to assume that there's 23 no one out there that wants to speak. And we will 24 plan that the draft will be provided some time in 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 197 December right now if that's okay with Margaret and 1 Dennis.2MEMBER CHU: I think we can probably give 3 it to you in November.

4 MEMBER REMPE: Okay, well the sooner the 5 better and then we'll get a document together.

6 MEMBER CHU: We would also get it.

7 MEMBER REMPE: Yes, well before February 8 and plan to try to have this on the agenda in February 9 then. Great.

10CHAIRMAN CORRADINI: I just would 11 emphasize that it's important that you kind of get it 12 in a sense that you're happy with the result in terms 13of the writing and then you go with Joy. But we're 14 not in a, from a rush standpoint, we're not in a rush.

15 We'll fit it in where we can. If we can 16 fit it in November because you're done so quickly, 17fine. If we have to fit it in, in March, fine. It's 18 not as if we're on some sort of enormous schedule.

19MEMBER MARCH-LEUBA: Do we need to do 20 another full committee on it?

21 MEMBER REMPE: No. Typically what we do 22 is we give it to the whole committee about a month 23 before and if they have any burning comments they will 24 send them to Jose and the two chairs and myself and 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 198 we'll get them resolved.

1And then there's just like a single one 2 page letter. So --

3CHAIRMAN CORRADINI: There's the 4 transmittal letter and that's it.

5MEMBER MARCH-LEUBA: It will probably have 6 to be involved on P&P.

7MEMBER REMPE: Yes. Well it will be, 8 there will be time on the agenda and we'll go through 9the letter. And if someone didn't like the way their 10 comment was reserved that's the time to bring it up.

11 But usually it's more of a review offline.

12Thank you. And thank you again, to the two chairs and 13all the Members of the panels. Good discussion today.

14CHAIRMAN CORRADINI: Okay. Dennis, are 15 you going to stay with us for the letter or are we 16 going to say goodbye to you?

17MEMBER BLEY: As I said earlier, to you 18 privately, if you want me around for the first reading 19okay. But I don't intend to stay for the letter. You 20 ought to run that because I'm here I can't really 21 participate except being an annoyance.

22 CHAIRMAN CORRADINI: Okay, that's fine.

23MEMBER BLEY: I'm happy to let you guys go 24 ahead with it.

25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 199CHAIRMAN CORRADINI: That's fine. All 1right. So we're going to take a break, Dennis, and 2 then if you want to come back and listen to the 3reading that's fine otherwise that's fine too. But 4 we'll come back at quarter to three.

5 (Whereupon, the above-entitled matter 6 went off the record at 2:30 p.m.)

7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 1ACRSOctober 4, 2018Emergency Preparedness for Small Modular Reactors and Other New Technologies Proposed Rulemaking10 CFR Parts 50 and 52 NRC-2015-0225RIN 3150-AJ68*Project Manager:Andy Carrera (NMSS)

Technical Leads:Kenneth Thomas (NSIR)Arlon Costa (NRO) 2Emergency Preparedness for Small Modular Reactors and Other New Technologies Proposed Rulemaking

Purpose of Rulemaking

-Amend regulations for new alternative, performance

-based EP requirements for SMRs and ONTs.

-Address one of the policy, licensing, and technical issues identified in SECY 0034*Source term, security, and siting criteria are not affected by this rule.

3*Major provisions of this proposed rule:

-technology

-inclusive for future SMRs and ONTs, including medical radioisotope facilities and non

-light-water reactors

-performance

-based EP framework, including demonstration of effective response in drills and exercises-hazard analysis for contiguous facilities

-scalable approach for plume exposure pathway EPZ

-ingestion response planning for SMRs and ONTsEmergency Preparedness for Small Modular Reactors and Other New Technologies Proposed Rulemaking 4*Comments from ACRS Subcommittee Meeting August 22:

-Source terms

Governed by separate rules and guidance

Those rules and guidance are not within the scope of this rulemaking

-1000 MWt*No technical analysis

Existing large light

-water reactors

-Hazard analysis

Updated by the provisions in

§50.54(q)(2)Emergency Preparedness for Small Modular Reactors and Other New Technologies Proposed Rulemaking 5*Comments from ACRS Subcommittee Meeting August 22:

-96 hours*Used by the EPA in the PAG manual, section 2.2

Dose projections measure integrated 4

-day dose-Multiple modular considerations

Details are in DG, Staff Regulatory Position 8

-Documents are publicly available

Made public in September

-Use of "Other New Technologies"

Used in FRN, SECY, and DGEmergency Preparedness for Small Modular Reactors and Other New Technologies Proposed Rulemaking 6*EPZ size technical analysis

-The plume exposure pathway EPZ should encompass an area where prompt protective measures, such as evacuation and sheltering, may be needed to minimize the exposure to individuals.

-The analysis should consider radiological releases from credible accidents for the facility.

-Scalable EPZ

No NRC requirement for offsite radiological emergency preparedness programs

If EPZ > site boundary NRC requirements for offsite radiological emergency preparedness programsEmergency Preparedness for Small Modular Reactors and Other New Technologies Proposed Rulemaking 7Emergency Preparedness for Small Modular Reactors and Other New Technologies Proposed RulemakingExisting EP forNuclear Power ReactorsEP for SMRs and ONTsOnsite onlyOnsite and Offsite 8*Scalable approach for plume exposure pathway EPZ

-Consistent with the analyses documented in NUREG

-0396-Consistent with the existing graded

-approach afforded to:

Research and test reactors

Fuel cycle facilities

Independent spent fuel storage installations

Same level of protection afforded to public health and safety*Development of guidance supported by Office of Nuclear Regulatory ResearchEmergency Preparedness for Small Modular Reactors and Other New Technologies Proposed Rulemaking 9*Ingestion response planning

-Early phase of the response

Precautionary protective actions

-Washing garden products and food

-Placing livestock on stored feeds

-Longer term actions

Leading indicator drives response

-Biological contamination similaritiesEmergency Preparedness for Small Modular Reactors and Other New Technologies Proposed Rulemaking 10Status and Path Forward

Draft proposed rule due to Commission on October 12, 2018.

-Draft regulatory guidance is planned for issuance with proposed rule in early 2019 (pending Commission's approval).

-Public meeting during public comment period of proposed rule.

Draft final rule due to Commission in early 2020.

11Abbreviations ACRS -Advisory Committee on Reactor SafeguardsCFR -Code of Federal RegulationsOEDO -Office of the Executive Director of Operations EP -emergency preparednessEPZ -emergency planning zoneFEMA -Federal Emergency Management AgencyFRN -Federal Register NoticeMWt -Megawatts thermal (units for reactor power) 1Accident Source Terms for EPZ Size Considerations

Evaluate a range of accidents credible for the facility

-Use DBA and severe accident source terms developed to support other required safety and environmental assessments

-Multi-module is considered

Guidance exists for LWRs

-DBA releases (new reactors)

SRP 15.0.3, RG 1.183

-AST (based on NUREG

-1465)*SRP 11.1, 12.2

-coolant activity, radiation sources

SRP 3.11, RG 1.89

-EQ -Severe accident releases

SRP 19.0, 19.2

PRA standards ACRS 657 thFullCommittee MeetingSummary of Operating Experiencefor CY2017 and MY2018Gordon Skillman, ACRSOctober 4, 2018 1

2ROP Framework 3ROP Process 4CY2017 and MY18ROPTotal Number of Plants in ROP Evaluation:Region I = 25Region II = 34Region III = 23Region IV = 19 TOTAL = 101 5CY2017 and MY18ROPInspection Findings

-Quantitative ThresholdsRed CDF greater than 10

-LERFgreater than 10

-Yellow CDFgreater than 10

-5 and less than or equal to 10

-LERFgreater than 10

-6 and less than or equal to 10-LERF White CDFgreater than 10

-6 and less than or equal to 10

-LERFgreater than 10

-7 and less than or equal to 1 0-6LERF Green CDF less than or equal to 10

-LERFless than or equal to 10

-

6CY2017 and MY18ROPInspection FindingsGreen:a finding of very low safety or security significanceGreater-Than-Green:a finding of more than very low significance

-security cornerstone onlyWhite:a finding of low to moderate safety or security significanceYellow: a finding of substantial safety or security significanceRed:a finding of high safety or security significance 7CY2017 and MY18ROP 8CY2017 ROP* Snapshot at end of 4 thQTR 17 that shows all findings from previous 4 QTRs? = Red, Yellow, or White designation is SUNSI, reported as GTG. Number of PlantsInsp.Find.SafetyInitiatingEventsMitigatingSystemsBarrierIntegrityEmerg.Prep.Occ. RadSafetyPub. Rad.SafetySecurityRed 0 0 0 0 0 0?2 2 0 0 0 0?White 2 5 0 1 0 0?GTG------6Green 27 81 38 7 9 5 26 9MY2018 ROP* Snapshot at end of 2 ndQTR 18 that shows all findings from previous 4 QTRsOnly new (1stor 2 ndQTR 2018) presented in following slidesNumber of PlantsInsp.Find.SafetyInitiatingEventsMitigatingSystemsBarrierIntegrityEmerg.Prep.Occ. RadSafetyPub. Rad.SafetySecurityRed 0 0 0 0 0 0?0 0 0 0 0 0?White 1 5 0 0 0 0?GTG------3Green 43 86 43 9 20 10 42 10CY2017 ROPInspection FindingsFour Yellows (2017)

-Two Each at Arkansas 1 and 2 (Both CY2015 and 2016):

Initiating Events Cornerstone

Mitigating Systems Cornerstone

-Resolved and No Longer show (see MY2018 ROP) 11CY2017 ROPFour Yellows

-Arkansas Nuclear 1 and 2

-Historic:*Initiating Events Cornerstone:

-Failure to Follow the Materials Handling Program during the Unit 1 Generator Stator Drop -Initially in 1 stQTR 2014-LOOP of 6 days Unit 1; Partial LOOP Unit 2

Mitigating Systems Cornerstone:

-Inadequate Flood Protection for Auxiliary and Emergency Diesel Fuel Storage Buildings Concurrent with Stator Drop Event

-Over 100 Flood Barriers Inadequate

Placed in Col 4 of Matrix Assessment

Supplemental Inspection Procedure (IP) 95003 planned 12CY2017 ROPInspection Findings (Four Yellows

-ANO 1 and 2

-Historic (continued)

2 nd& 3 rdQTRs 2014

-Unplanned Scrams/7000 HrsWhite PISubsequent decision to also conduct Supplemental Inspections 95001 and 95002 3 Supplemental Inspections Conducted Jan & Feb 2016Entergy submitted "ANO Comprehensive Recovery Plan Area Action Plans," May 2016 to respond to recover from Col 4NRC Issued Confirmatory Action Letter (CAL) June 2016Finally -Inspection May 2018

-All actions completed, CAL closed, and assessment of performance of ANO 1 & 2 updated (reflected in 2018) 13CY2017 ROPInspection Findings Whites -8 Plants*2Plants in Initiating Events Cornerstone:

-Grand Gulf 1

-Parallel Performance Indicator White Finding -St. Lucie 1

-Loss of Configuration Control

5 Plants in Mitigating Systems Cornerstone

-Catawba; Clinton; Oyster Creek; Perry 1; Pilgrim

-Various Systems / Causes

1 Plant in Emergency Preparedness Cornerstone:

-Fermi -Failure to Maintain Effectiveness of Site EP 14CY2017 ROPInspection Findings (S&RP*White at Grand Gulf 1 (IE)

Parallel Performance Indicator White Inspection Finding.

PI = Unplanned scrams per 7000 hours0.081 days 1.944 hours 0.0116 weeks 0.00266 months

Inadequate and Weaknesses in response to White Performance Indicator reported by licensee in 3 rdQTR 16.*Poor Root Cause Analysis

White at Saint Lucie 1 (IE)

Failure to maintain configuration control of the Unit 1 main generator inadvertent energization lockout relay circuitry.*Resulted in reactor 2016. *2E-15CY2017 ROPInspection Findings (S&RP*White at Catawba (MS)

Failure to adequately develop and adjust preventive maintenance activities procedure.

Failed to develop preventive maintenance considered operating experience for the emergency diesel generator *White at Clinton (MS)

Failure to evaluate the change in the dropout voltages for 1 *in dropout voltages prevented the fan from operating during an under voltage *became inoperable.

16CY2017 ROPInspection Findings (S&RP*White at Oyster Creek (MS)

Tech Spec violation

-electromaticrelief valve incorrectly reassembled

.*.*Supplemental 95001 Inspection completed Sept 17 *White at Perry 1 (MS)

Failure to evaluate the effects of voltage suppression diode failure on circuit.*Introduction circuit.*inoperable and unable to start.

17CY2017 ROPInspection Findings (S&RP*White at Pilgrim (MS)

Failure to a significant condition *Failed to correct during a plant cool

-LOOP event.**Supplemental 95001

-collective rootcause evaluation represented a significant *cover this finding.

18CY2017 ROPInspection Findings (S&RP*White at Fermi (EP)

Failure to maintain the effectiveness of the EP and use adequate actual or potential offsite consequences of a radiological emergency.

Failure to accurately analyze the effect of increasing background radiation on the accident range radiation monitor (AXM*radioactive releases that are used determining EAL classification and development of PARs

.

19MY2018 ROPInspection Findings(S&RP*No More Yellow Findings

Whites -Reduced to 4 Plants

Only 1 New Finding since 4 thQTR 2017 *Mitigating Systems Cornerstone

-Davis-Besse 20MY2018 ROPInspection Findings (S&RP*White at Davis

-Besse(MS) *Failure to provide appropriate instructions to maintain adequate oil in auxiliary turbine bearing oil sumps.

Led to failure of auxiliary pump.*Pump found to be inoperable for > tech spec limit

.

21CY2017 and MY18 ROP 22CY2017 and MY18 ROPInspection Findings

-Plants Greater

-Than-Green; 4 in CY16; 6 in CY17; 3 in MY18 (not including Force

-on-Force)GTG is not a "new" category, it means the finding was either white, yellow, or redDetails are SUNSI and not publically availableBut NUREG-1885 -Annual non

-SUNSI report provided to Congress 23CY2017 ROPBaseline Inspection Findings 24CY2017 ROPSecurity -Inspection Findings Force -on -Force Exercises 25CY2017 ROPForce-On-Force Exercises

No Greater

-than-Green Findings

1 Ineffective Exercise:

-Licensees' inability to demonstrate an effective implementation of its protective strategy to defend designated target set components

3Marginal Exercises:

-In all three cases, licensees neutralized the adversary at a location, or in preparation to enter a location, that contained a single element target set.

Licensees all took appropriate corrective actions.

-Policy and procedure changes

-Physical security upgrades and technology improvements

-Personnel or security force enhancements 26-*NRC evaluates whether cross

-cutting issues exist at NPPs 2x/year: at mid

-cycle and end

-of-cycle assessments:

No New Open Issues in 2017*No Data reported yet for mid

-cycle 2018 assessmentsCY2017 and MY18 ROP 27CY2017 and MY18 ROP 28CY2017 and MY18 ROPPerformance IndicatorsGreen:performance within an expected level where all cornerstone objectives are metWhite: performance outside an expected range of nominal utility performance but related cornerstone objectives are metYellow: related cornerstone objectives are met, but with a minimal reduction in safety margin Red: significant reduction in safety margin in area measured by the PI 29CY2017ROPPerformance Indicators

NO Plants with Redor Yellow*Two Plants with White 30CY2017ROPPerformance IndicatorsIE04 -Unplanned Scrams with Complications

Columbia Generating Station

Unplanned scrams with complications occurred in 4 thQTR 16 and 3 rdQTR 17White Threshold > 1.0 1Q 2Q 3Q 4Q1.01.02.01.0 31CY2017ROPPerformance IndicatorsIE01 -Unplanned Scrams per 7000 Critical Hours

Watts Bar 2NR = Watts Bar 2 went online October 2016. Significant critical hours to require reporting not reached until 3Q17. White Threshold > 3.0 1Q 2Q 3Q 4Q NR NR1.43.1 32MY2018ROPPerformance IndicatorsNo Red , Yellowor White Thru first two quartersOf 2018 33CY2017 and MY18 ROPROP Action MatrixInspection Findings

+Performance Indicators

=Plant Assessment 34ROP Action Matrix AssessmentColumn 5: Unacceptable PerformanceColumn 4: Multiple/Repetitive Degraded CornerstoneColumn 3 Degraded CornerstoneColumn 2: Regulatory Response Column 1: Licensee ResponseCY2017 and MY18 ROP 35CY2017 and MY18 ROP 36CY2017 ROPROP Action Matrix AssessmentNumber of Plants in Each Column (per QTR)2017QTRLicenseeResponseRegulatoryResponseUnacceptable 1 82 16 00 2 84 14 00 3 86 12 00 4 89 9 00 37MY2018 ROPROP Action Matrix Assessmentnd-Number of Plants in Each Column (per QTR)2018QTRLicenseeResponseRegulatoryResponseUnacceptable 1 95 5 00x x x x x 3 x x x x x 4 x x x x x 38CY2017 and MY18 ROPAction Matrix Summary Highlights

Arkansas Nuclear 1 & 2

-Back in Column 1 in 2018 after being in Column 4 since 1st QTR 2015 39CY2017 and MY18 ROP*Pilgrim 1 -6 by 13consecutive 4 since then.

-*Grand Gulf 1

-(started 3rd QTR16

-*-4(starting 3 rdQTR17 -*-(all 4 QTRs of 2017

-

40CY2016 & 17 FFD*Electronic Reporting since 2009

Summary Reports available from some licensees

No Industry

-Wide Summaries being prepared as in the past*NRC staff provides annual briefing to Drug Testing Advisory Panel

Tests conducted on licensee and C/V employees 41CY2016 & 17 FFD*5 Types of Tests Conducted:

Pre-Access(applicants for employment)

Random(unscheduled/unannounced for employees)

For Cause (behavior of, or information received about employee)*Post-Event(after an event involving human error)

Follow-up(after a positive test) 42CY2016 & 17 FFD-*Three Multi

-year Trends were being tracked:*Subversion attempts prevalent since CY2011 (18 to 21% of violations: 143 to 187 events per year) with 54 to 66% of sites reporting at least one.

Amphetamine positive results increasing since CY2008 (from 3.8% (in 2008) to 10.6% (in 2014) of drug & alcohol positives.

Reactor construction sites have higher positive rates, primarily in pre-access and random tests, and have higher incidence of subversion attempts than operating reactor sites.

43CY2016 & 17 FFD-cont*-17-consideration.

those three multi

-year methamphetamineexpanding testing measures related to subversion attempt detection

.*

44CY2016 & 17 FFD

MEANWHILE

-Subversion attempts prevalent since CY2011 continue to rise*Subversion Attempt Trends:*2012 -*2013 -*2014 -*2015 -*2016 -*2017 -*Subversion Attempts in 2017

45 *67-*98 45CY2016 & 17 FFD*Overall industry positive rate CY17 = 0.77% (0.76% in CY16)Low, but continues upward trend since CY12 and 13 (both were 0.62%)

Total tests in CY2017 decreased by 3.6%Tests in CY2016 decreased by 5.9% from CY2015Continuing downward trend

Approx64% of CY17 positives and refusals occur at pre-accessPrevents access, directly protecting public health and safetyAlmost constant percentage (65%) for several yearsRandomtesting CY17 identified 22.7% of substance usersIdentifies more employees usingsubstances than pre

-access22.3% in CY2016; Continues upward trend since 2014 46CY2016 & 17 FFD*For cause testing continues to have highest positive rate (9.9%)(But is trending down 13.40% in CY13, 12%in CY14; 6% in CY16)

Three substances continue to account for 85% of positivesMarijuana (50%), Alcohol (23%), Cocaine (12%)Amphetamine positives continuing upward trend, now nearly same percentage as cocaine. Cocaine positives way down from high in 2006, but trending up again.Opiate positives way down (only 1% in CY 2017))

47CY2016 & 17 FFD*Events concerning individual employee violations must be reported to NRC within 24 hours2.777778e-4 days 0.00667 hours 3.968254e-5 weeks 9.132e-6 months

33 24-hourreportable events in 201715 involved Supervisors/Managers 482017 Abnormal Occurrence ReportAnnual Abnormal Occurrence Report to *No AOs at NPPs

11 AOs at Medical Facilities

1 Significant Event that does not meet definition of AO:*Exposure at National Institute of Standards and Technology (NIST), Gaithersburg MD 49Questions ?

Reactor Oversight Process UpdatesAnticipating Degrading Licensee PerformanceReactor Oversight Process Enhancement ProposalsGreg Bowman and Tom Hipschman, NRR/DIRSOctober 4, 2018ACRS Full Committee 1

Background

The Reactor Oversight Process (ROP) is designed to identify declining licensee performance prior to loss of reasonable assurance of adequate protection

-Columns 2-4 of the Action Matrix provide this "buffer"

-9500x procedures guide the associated agency response

That notwithstanding:

-ACRS and others have wondered about "leading indicators"

-The Commission has, in various forums, encouraged the staff to continue thinking about this topic

-We routinely evaluate the effectiveness of the ROP (e.g., the annual self assessment) and seek to continually improve 2 Previous Discussions

Various discussions have highlighted ideas such as:

-Trend in the # of findings

Green findings

Quality assurance / Appendix B

-related findings

Maintenance Rule

-related enforcement actions-Plant operations:

Work management

Material condition of the plant

Trend in unplanned outage rate relative to industry average*# of off-normal procedure entries

1. of unplanned Limiting Condition for Operation (LCO) entries

Staffing resources

-Trend in # of conditions requiring NRC notification (50.72/50.73)

Nature of these conditions (i.e., atypical?)

-Industry (e.g., INPO) evaluations

-Leadership performance 3

Examples of Tailoring the Existing ROP

A related point is how NRC adjusts the inspection program for plants that are:-Experiencing financial issues

-Nearing cessation of permanent operation

Inspection Manual Chapter (IMC) 2515, Appendix G-Provides inspection guidance for inspecting plants that are approaching shutdown and decommissioning

Staff also utilizes a handful of means for addressing these situations:

-Safety culture initiatives

-Supplemental guidance memos to address specific cases

-IMC governing resident activities calls out material condition and long

-standing issues

-Sample selection for inspection procedures that cover areas where financial difficulties could translate to degradation of performance (e.g., P&IR, Maintenance Effectiveness, Equipment Alignment)

-Increased NRC management presence 4

Enhancing the Reactor Oversight Process

NRC has a number of initiatives ongoing to make the ROP more effective and efficient. Examples include:-Improvements to the inspection report development process -Initiative to improve the minor/more

-than-minor component of the issue screening process-Work to improve the NRC's engineering inspection program*DIRS received input from stakeholders as part of the ongoing agency initiative associated with transformation:

-~70 recommendations from the internal stakeholders related to potential enhancements to the ROP-NEI publication and NRUG letter with initial industry proposals to improve the ROP-Follow-up letter from NEI with consolidated industry proposals 5

Internal Stakeholder Feedback

The Transformation Team provided DIRS with consolidated stakeholder feedback related to ROP transformation.

The feedback generally fit into several categories

-Changes to organization and staffing supporting oversight, primarily regional changes-Changes to frequency of inspections and focus of inspection procedures

-Changes to make performance indicators more effective-Changes to the various components of the assessment process-Changes to make the enforcement program more risk-informed-Changes to streamline or eliminate inspection reports*No assessment was done of the proposals; the input to consisted of direct feedback provide by NRC staff.

6 7Recent Industry Feedback

NEI letter submitted on September 19 (ML18262A322) and discussed at September 20 ROP public meeting

Acknowledgment that the ROP remains a sound and effective oversight program.

Provided some recommendations on how the ROP could be enhanced:-Impact of White findings (e.g., press releases, labeling and communication, type of inspection follow

-up)-Reassess baseline inspection program (e.g., focused evaluation of inspection areas, reduce burden of MSPI)

-Improvements to the SDP (e.g., improve specific SDPs, address differences in NRC and licensee models)

-Resolution of inspection issues (e.g., communication of potential issues, resolution of low risk compliance issues)

Next Steps

Establish working group(s) to obtain stakeholder input, evaluate proposals, develop recommendations

.-Initial kick

-off meeting took place on October 3

Establish recurring discussion at monthly ROP public meetings, with separate focused public meetings on specific ROP enhancements.

-Planning public meetings on October 18 and November 15 (tentative)

Commission approval will be needed for any significant changes to the ROP, per Management Directive 8.13.8 ACRS 657 thFullCommittee MeetingSummary of Operating Experiencefor CY2017 and MY2018Gordon Skillman, ACRSOctober 4, 2018 1

2ROP Framework 3ROP Process 4CY2017 and MY18ROPTotal Number of Plants in ROP Evaluation:Region I = 25Region II = 34Region III = 23Region IV = 19 TOTAL = 101 5CY2017 and MY18ROPInspection Findings

-Quantitative ThresholdsRed CDF greater than 10

-LERFgreater than 10

-Yellow CDFgreater than 10

-5 and less than or equal to 10

-LERFgreater than 10

-6 and less than or equal to 10-LERF White CDFgreater than 10

-6 and less than or equal to 10

-LERFgreater than 10

-7 and less than or equal to 1 0-6LERF Green CDF less than or equal to 10

-LERFless than or equal to 10

-

6CY2017 and MY18ROPInspection FindingsGreen:a finding of very low safety or security significanceGreater-Than-Green:a finding of more than very low significance

-security cornerstone onlyWhite:a finding of low to moderate safety or security significanceYellow: a finding of substantial safety or security significanceRed:a finding of high safety or security significance 7CY2017 and MY18ROP 8CY2017 ROP* Snapshot at end of 4 thQTR 17 that shows all findings from previous 4 QTRs? = Red, Yellow, or White designation is SUNSI, reported as GTG. Number of PlantsInsp.Find.SafetyInitiatingEventsMitigatingSystemsBarrierIntegrityEmerg.Prep.Occ. RadSafetyPub. Rad.SafetySecurityRed 0 0 0 0 0 0?2 2 0 0 0 0?White 2 5 0 1 0 0?GTG------6Green 27 81 38 7 9 5 26 9MY2018 ROP* Snapshot at end of 2 ndQTR 18 that shows all findings from previous 4 QTRsOnly new (1stor 2 ndQTR 2018) presented in following slidesNumber of PlantsInsp.Find.SafetyInitiatingEventsMitigatingSystemsBarrierIntegrityEmerg.Prep.Occ. RadSafetyPub. Rad.SafetySecurityRed 0 0 0 0 0 0?0 0 0 0 0 0?White 1 5 0 0 0 0?GTG------3Green 43 86 43 9 20 10 42 10CY2017 ROPInspection FindingsFour Yellows (2017)

-Two Each at Arkansas 1 and 2 (Both CY2015 and 2016):

Initiating Events Cornerstone

Mitigating Systems Cornerstone

-Resolved and No Longer show (see MY2018 ROP) 11CY2017 ROPFour Yellows

-Arkansas Nuclear 1 and 2

-Historic:*Initiating Events Cornerstone:

-Failure to Follow the Materials Handling Program during the Unit 1 Generator Stator Drop -Initially in 1 stQTR 2014-LOOP of 6 days Unit 1; Partial LOOP Unit 2

Mitigating Systems Cornerstone:

-Inadequate Flood Protection for Auxiliary and Emergency Diesel Fuel Storage Buildings Concurrent with Stator Drop Event

-Over 100 Flood Barriers Inadequate

Placed in Col 4 of Matrix Assessment

Supplemental Inspection Procedure (IP) 95003 planned 12CY2017 ROPInspection Findings (Four Yellows

-ANO 1 and 2

-Historic (continued)

2 nd& 3 rdQTRs 2014

-Unplanned Scrams/7000 HrsWhite PISubsequent decision to also conduct Supplemental Inspections 95001 and 95002 3 Supplemental Inspections Conducted Jan & Feb 2016Entergy submitted "ANO Comprehensive Recovery Plan Area Action Plans," May 2016 to respond to recover from Col 4NRC Issued Confirmatory Action Letter (CAL) June 2016Finally -Inspection May 2018

-All actions completed, CAL closed, and assessment of performance of ANO 1 & 2 updated (reflected in 2018) 13CY2017 ROPInspection Findings Whites -8 Plants*2Plants in Initiating Events Cornerstone:

-Grand Gulf 1

-Parallel Performance Indicator White Finding -St. Lucie 1

-Loss of Configuration Control

5 Plants in Mitigating Systems Cornerstone

-Catawba; Clinton; Oyster Creek; Perry 1; Pilgrim

-Various Systems / Causes

1 Plant in Emergency Preparedness Cornerstone:

-Fermi -Failure to Maintain Effectiveness of Site EP 14CY2017 ROPInspection Findings (S&RP*White at Grand Gulf 1 (IE)

Parallel Performance Indicator White Inspection Finding.

PI = Unplanned scrams per 7000 hours0.081 days 1.944 hours 0.0116 weeks 0.00266 months

Inadequate and Weaknesses in response to White Performance Indicator reported by licensee in 3 rdQTR 16.*Poor Root Cause Analysis

White at Saint Lucie 1 (IE)

Failure to maintain configuration control of the Unit 1 main generator inadvertent energization lockout relay circuitry.*Resulted in reactor 2016. *2E-15CY2017 ROPInspection Findings (S&RP*White at Catawba (MS)

Failure to adequately develop and adjust preventive maintenance activities procedure.

Failed to develop preventive maintenance considered operating experience for the emergency diesel generator *White at Clinton (MS)

Failure to evaluate the change in the dropout voltages for 1 *in dropout voltages prevented the fan from operating during an under voltage *became inoperable.

16CY2017 ROPInspection Findings (S&RP*White at Oyster Creek (MS)

Tech Spec violation

-electromaticrelief valve incorrectly reassembled

.*.*Supplemental 95001 Inspection completed Sept 17 *White at Perry 1 (MS)

Failure to evaluate the effects of voltage suppression diode failure on circuit.*Introduction circuit.*inoperable and unable to start.

17CY2017 ROPInspection Findings (S&RP*White at Pilgrim (MS)

Failure to a significant condition *Failed to correct during a plant cool

-LOOP event.**Supplemental 95001

-collective rootcause evaluation represented a significant *cover this finding.

18CY2017 ROPInspection Findings (S&RP*White at Fermi (EP)

Failure to maintain the effectiveness of the EP and use adequate actual or potential offsite consequences of a radiological emergency.

Failure to accurately analyze the effect of increasing background radiation on the accident range radiation monitor (AXM*radioactive releases that are used determining EAL classification and development of PARs

.

19MY2018 ROPInspection Findings(S&RP*No More Yellow Findings

Whites -Reduced to 4 Plants

Only 1 New Finding since 4 thQTR 2017 *Mitigating Systems Cornerstone

-Davis-Besse 20MY2018 ROPInspection Findings (S&RP*White at Davis

-Besse(MS) *Failure to provide appropriate instructions to maintain adequate oil in auxiliary turbine bearing oil sumps.

Led to failure of auxiliary pump.*Pump found to be inoperable for > tech spec limit

.

21CY2017 and MY18 ROP 22CY2017 and MY18 ROPInspection Findings

-Plants Greater

-Than-Green; 4 in CY16; 6 in CY17; 3 in MY18 (not including Force

-on-Force)GTG is not a "new" category, it means the finding was either white, yellow, or redDetails are SUNSI and not publically availableBut NUREG-1885 -Annual non

-SUNSI report provided to Congress 23CY2017 ROPBaseline Inspection Findings 24CY2017 ROPSecurity -Inspection Findings Force -on -Force Exercises 25CY2017 ROPForce-On-Force Exercises

No Greater

-than-Green Findings

1 Ineffective Exercise:

-Licensees' inability to demonstrate an effective implementation of its protective strategy to defend designated target set components

3Marginal Exercises:

-In all three cases, licensees neutralized the adversary at a location, or in preparation to enter a location, that contained a single element target set.

Licensees all took appropriate corrective actions.

-Policy and procedure changes

-Physical security upgrades and technology improvements

-Personnel or security force enhancements 26-*NRC evaluates whether cross

-cutting issues exist at NPPs 2x/year: at mid

-cycle and end

-of-cycle assessments:

No New Open Issues in 2017*No Data reported yet for mid

-cycle 2018 assessmentsCY2017 and MY18 ROP 27CY2017 and MY18 ROP 28CY2017 and MY18 ROPPerformance IndicatorsGreen:performance within an expected level where all cornerstone objectives are metWhite: performance outside an expected range of nominal utility performance but related cornerstone objectives are metYellow: related cornerstone objectives are met, but with a minimal reduction in safety margin Red: significant reduction in safety margin in area measured by the PI 29CY2017ROPPerformance Indicators

NO Plants with Redor Yellow*Two Plants with White 30CY2017ROPPerformance IndicatorsIE04 -Unplanned Scrams with Complications

Columbia Generating Station

Unplanned scrams with complications occurred in 4 thQTR 16 and 3 rdQTR 17White Threshold > 1.0 1Q 2Q 3Q 4Q1.01.02.01.0 31CY2017ROPPerformance IndicatorsIE01 -Unplanned Scrams per 7000 Critical Hours

Watts Bar 2NR = Watts Bar 2 went online October 2016. Significant critical hours to require reporting not reached until 3Q17. White Threshold > 3.0 1Q 2Q 3Q 4Q NR NR1.43.1 32MY2018ROPPerformance IndicatorsNo Red , Yellowor White Thru first two quartersOf 2018 33CY2017 and MY18 ROPROP Action MatrixInspection Findings

+Performance Indicators

=Plant Assessment 34ROP Action Matrix AssessmentColumn 5: Unacceptable PerformanceColumn 4: Multiple/Repetitive Degraded CornerstoneColumn 3 Degraded CornerstoneColumn 2: Regulatory Response Column 1: Licensee ResponseCY2017 and MY18 ROP 35CY2017 and MY18 ROP 36CY2017 ROPROP Action Matrix AssessmentNumber of Plants in Each Column (per QTR)2017QTRLicenseeResponseRegulatoryResponseUnacceptable 1 82 16 00 2 84 14 00 3 86 12 00 4 89 9 00 37MY2018 ROPROP Action Matrix Assessmentnd-Number of Plants in Each Column (per QTR)2018QTRLicenseeResponseRegulatoryResponseUnacceptable 1 95 5 00x x x x x 3 x x x x x 4 x x x x x 38CY2017 and MY18 ROPAction Matrix Summary Highlights

Arkansas Nuclear 1 & 2

-Back in Column 1 in 2018 after being in Column 4 since 1st QTR 2015 39CY2017 and MY18 ROP*Pilgrim 1 -6 by 13consecutive 4 since then.

-*Grand Gulf 1

-(started 3rd QTR16

-*-4(starting 3 rdQTR17 -*-(all 4 QTRs of 2017

-

40CY2016 & 17 FFD*Electronic Reporting since 2009

Summary Reports available from some licensees

No Industry

-Wide Summaries being prepared as in the past*NRC staff provides annual briefing to Drug Testing Advisory Panel

Tests conducted on licensee and C/V employees 41CY2016 & 17 FFD*5 Types of Tests Conducted:

Pre-Access(applicants for employment)

Random(unscheduled/unannounced for employees)

For Cause (behavior of, or information received about employee)*Post-Event(after an event involving human error)

Follow-up(after a positive test) 42CY2016 & 17 FFD-*Three Multi

-year Trends were being tracked:*Subversion attempts prevalent since CY2011 (18 to 21% of violations: 143 to 187 events per year) with 54 to 66% of sites reporting at least one.

Amphetamine positive results increasing since CY2008 (from 3.8% (in 2008) to 10.6% (in 2014) of drug & alcohol positives.

Reactor construction sites have higher positive rates, primarily in pre-access and random tests, and have higher incidence of subversion attempts than operating reactor sites.

43CY2016 & 17 FFD-cont*-17-consideration.

those three multi

-year methamphetamineexpanding testing measures related to subversion attempt detection

.*

44CY2016 & 17 FFD

MEANWHILE

-Subversion attempts prevalent since CY2011 continue to rise*Subversion Attempt Trends:*2012 -*2013 -*2014 -*2015 -*2016 -*2017 -*Subversion Attempts in 2017

45 *67-*98 45CY2016 & 17 FFD*Overall industry positive rate CY17 = 0.77% (0.76% in CY16)Low, but continues upward trend since CY12 and 13 (both were 0.62%)

Total tests in CY2017 decreased by 3.6%Tests in CY2016 decreased by 5.9% from CY2015Continuing downward trend

Approx64% of CY17 positives and refusals occur at pre-accessPrevents access, directly protecting public health and safetyAlmost constant percentage (65%) for several yearsRandomtesting CY17 identified 22.7% of substance usersIdentifies more employees usingsubstances than pre

-access22.3% in CY2016; Continues upward trend since 2014 46CY2016 & 17 FFD*For cause testing continues to have highest positive rate (9.9%)(But is trending down 13.40% in CY13, 12%in CY14; 6% in CY16)

Three substances continue to account for 85% of positivesMarijuana (50%), Alcohol (23%), Cocaine (12%)Amphetamine positives continuing upward trend, now nearly same percentage as cocaine. Cocaine positives way down from high in 2006, but trending up again.Opiate positives way down (only 1% in CY 2017))

47CY2016 & 17 FFD*Events concerning individual employee violations must be reported to NRC within 24 hours2.777778e-4 days 0.00667 hours 3.968254e-5 weeks 9.132e-6 months

33 24-hourreportable events in 201715 involved Supervisors/Managers 482017 Abnormal Occurrence ReportAnnual Abnormal Occurrence Report to *No AOs at NPPs

11 AOs at Medical Facilities

1 Significant Event that does not meet definition of AO:*Exposure at National Institute of Standards and Technology (NIST), Gaithersburg MD 49Questions ?

Reactor Oversight Process UpdatesAnticipating Degrading Licensee PerformanceReactor Oversight Process Enhancement ProposalsGreg Bowman and Tom Hipschman, NRR/DIRSOctober 4, 2018ACRS Full Committee 1

Background

The Reactor Oversight Process (ROP) is designed to identify declining licensee performance prior to loss of reasonable assurance of adequate protection

-Columns 2-4 of the Action Matrix provide this "buffer"

-9500x procedures guide the associated agency response

That notwithstanding:

-ACRS and others have wondered about "leading indicators"

-The Commission has, in various forums, encouraged the staff to continue thinking about this topic

-We routinely evaluate the effectiveness of the ROP (e.g., the annual self assessment) and seek to continually improve 2 Previous Discussions

Various discussions have highlighted ideas such as:

-Trend in the # of findings

Green findings

Quality assurance / Appendix B

-related findings

Maintenance Rule

-related enforcement actions-Plant operations:

Work management

Material condition of the plant

Trend in unplanned outage rate relative to industry average*# of off-normal procedure entries

1. of unplanned Limiting Condition for Operation (LCO) entries

Staffing resources

-Trend in # of conditions requiring NRC notification (50.72/50.73)

Nature of these conditions (i.e., atypical?)

-Industry (e.g., INPO) evaluations

-Leadership performance 3

Examples of Tailoring the Existing ROP

A related point is how NRC adjusts the inspection program for plants that are:-Experiencing financial issues

-Nearing cessation of permanent operation

Inspection Manual Chapter (IMC) 2515, Appendix G-Provides inspection guidance for inspecting plants that are approaching shutdown and decommissioning

Staff also utilizes a handful of means for addressing these situations:

-Safety culture initiatives

-Supplemental guidance memos to address specific cases

-IMC governing resident activities calls out material condition and long

-standing issues

-Sample selection for inspection procedures that cover areas where financial difficulties could translate to degradation of performance (e.g., P&IR, Maintenance Effectiveness, Equipment Alignment)

-Increased NRC management presence 4

Enhancing the Reactor Oversight Process

NRC has a number of initiatives ongoing to make the ROP more effective and efficient. Examples include:-Improvements to the inspection report development process -Initiative to improve the minor/more

-than-minor component of the issue screening process-Work to improve the NRC's engineering inspection program*DIRS received input from stakeholders as part of the ongoing agency initiative associated with transformation:

-~70 recommendations from the internal stakeholders related to potential enhancements to the ROP-NEI publication and NRUG letter with initial industry proposals to improve the ROP-Follow-up letter from NEI with consolidated industry proposals 5

Internal Stakeholder Feedback

The Transformation Team provided DIRS with consolidated stakeholder feedback related to ROP transformation.

The feedback generally fit into several categories

-Changes to organization and staffing supporting oversight, primarily regional changes-Changes to frequency of inspections and focus of inspection procedures

-Changes to make performance indicators more effective-Changes to the various components of the assessment process-Changes to make the enforcement program more risk-informed-Changes to streamline or eliminate inspection reports*No assessment was done of the proposals; the input to consisted of direct feedback provide by NRC staff.

6 7Recent Industry Feedback

NEI letter submitted on September 19 (ML18262A322) and discussed at September 20 ROP public meeting

Acknowledgment that the ROP remains a sound and effective oversight program.

Provided some recommendations on how the ROP could be enhanced:-Impact of White findings (e.g., press releases, labeling and communication, type of inspection follow

-up)-Reassess baseline inspection program (e.g., focused evaluation of inspection areas, reduce burden of MSPI)

-Improvements to the SDP (e.g., improve specific SDPs, address differences in NRC and licensee models)

-Resolution of inspection issues (e.g., communication of potential issues, resolution of low risk compliance issues)

Next Steps

Establish working group(s) to obtain stakeholder input, evaluate proposals, develop recommendations

.-Initial kick

-off meeting took place on October 3

Establish recurring discussion at monthly ROP public meetings, with separate focused public meetings on specific ROP enhancements.

-Planning public meetings on October 18 and November 15 (tentative)

Commission approval will be needed for any significant changes to the ROP, per Management Directive 8.13.8

ML18303A086

Text

Background

Background

Navigation menu

Search