ML20056D669
| ML20056D669 | |
| Person / Time | |
|---|---|
| Issue date: | 02/05/2020 |
| From: | Christiana Lui Advisory Committee on Reactor Safeguards |
| To: | |
| Lui, C, ACRS | |
| References | |
| NRC-0788 | |
| Download: ML20056D669 (262) | |
Text
Official Transcript of Proceedings NUCLEAR REGULATORY COMMISSION
Title:
Advisory Committee on Reactor Safeguards Reliability and Probabilistic Risk Assessment Docket Number: (n/a)
Location: Rockville, Maryland Date: Wednesday, February 5, 2020 Work Order No.: NRC-0788 Pages 1-176 NEAL R. GROSS AND CO., INC.
Court Reporters and Transcribers 1323 Rhode Island Avenue, N.W.
Washington, D.C. 20005 (202) 234-4433
1 1
2 3
4 DISCLAIMER 5
6 7 UNITED STATES NUCLEAR REGULATORY COMMISSIONS 8 ADVISORY COMMITTEE ON REACTOR SAFEGUARDS 9
10 11 The contents of this transcript of the 12 proceeding of the United States Nuclear Regulatory 13 Commission Advisory Committee on Reactor Safeguards, 14 as reported herein, is a record of the discussions 15 recorded at the meeting.
16 17 This transcript has not been reviewed, 18 corrected, and edited, and it may contain 19 inaccuracies.
20 21 22 23 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 www.nealrgross.com
1 1 UNITED STATES OF AMERICA 2 NUCLEAR REGULATORY COMMISSION 3 + + + + +
4 ADVISORY COMMITTEE ON REACTOR SAFEGUARDS 5 (ACRS) 6 + + + + +
7 RELIABILITY & PROBABILISTIC RISK ASSESSMENT 8 SUBCOMMITTEE 9 + + + + +
10 WEDNESDAY 11 FEBRUARY 5, 2020 12 + + + + +
13 ROCKVILLE, MARYLAND 14 + + + + +
15 The Subcommittee met at the Nuclear 16 Regulatory Commission, Two White Flint North, Room 17 T2D30, 11545 Rockville Pike, at 8:30 a.m., Vesna 18 Dimitrijevic, Chair, presiding.
19 20 COMMITTEE MEMBERS:
21 VESNA B. DIMITRIJEVIC, Chair 22 DENNIS BLEY, Member 23 JOY L. REMPE, Member 24 JOSE MARCH-LEUBA, Member 25 DAVID A. PETTI, Member NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
2 1
2 DESIGNATED FEDERAL OFFICIAL:
3 CHRISTIANA LUI 4
5 6
7 8
9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
3 1 CONTENTS 2 Opening Remarks . . . . . . . . . . . . . . . . . 4 3 Introductory Remarks . . . . . . . . . . . . . . 7 4 Overview of Risk-Informed Initiatives . . . . . . 12 5 Overview of PWROG-19027 and NEI 17-07 . . . . . . 41 6 Details of Planned RG 1.200 Update . . . . . . 136 7 Public Comments . . . . . . . . . . . . . . . . 169 8 Subcommittee Discussion . . . . . . . . . . . . 170 9 Adjourn . . . . . . . . . . . . . . . . . . . . 176 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
4 1 P R O C E E D I N G S 2 8:31 a.m.
3 MEMBER DIMITRIJEVIC: This meeting will 4 now come to order.
5 This is a meeting of the Reliability and 6 Probabilistic Risk Assessment Subcommittee of the 7 Advisory Committee on Reactor Safeguards.
8 I am Vesna Dimitrijevic, Chairman of this 9 Subcommittee meeting.
10 ACRS members in attendance are Dave Petti, 11 Joy Rempe, Dennis Bley, and Jose March-Leuba.
12 Christiana Lui of the ACRS staff is the 13 Designated Federal Official for this meeting.
14 The Subcommittee will hear presentations 15 and hold discussions with the NRC staff and industry 16 representatives of the Proposed Update to Reg Guide 17 1.200 Rev 2, An Approach for Determining the 18 Accessibility of Probabilistic Risk Assessment Results 19 for Risk-Informed Activities.
20 The Subcommittee will look at the 21 information, analyze relevant issues and facts, and 22 formalize a report position and action as appropriate 23 for deliberation by the Full Committee.
24 The ACRS was established by statute and is 25 governed by the Federal Advisory Committee Act.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
5 1 The NRC implements FACA in accordance with 2 the regulation found in Title 10 of the Code of 3 Federal Regulations Part 7.
4 The Committee can only speak through its 5 published letter reports. We hold meetings to gather 6 information and perform preparatory work that supports 7 our deliberations.
8 The rules for participation in all ACRS 9 meetings were recently updated and announced in the 10 Federal Register on June 13, 2019.
11 The ACRS section of the NRC public website 12 provides our charter, bylaws, agendas, selected 13 reports, and full transcripts of all open Full and 14 Subcommittee meetings, including slides presented in 15 those meetings.
16 The Meeting Notice and Agendas for those 17 meetings are posted there.
18 As stated in the Federal Register Notice 19 and in the Public Meeting Notice of the NRC website, 20 interested parties who desire to provide written and 21 oral comments may do so and should contact the 22 designated Federal Official five days prior to the 23 meeting as practicable.
24 We have received no such requests prior 25 for today -- prior to today's meeting.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
6 1 We do have a time set aside for spur of 2 the moment comments from anyone attending or listening 3 to our meeting during today's meeting.
4 Also, we have a bridge line established 5 for interested members of the public to listen in. To 6 preclude interruption of today's meeting, the phone 7 bridge will be placed in listen in only mode during 8 the presentations and Subcommittee discussions.
9 We will unmute this bridge line when we 10 proceed to the public comment agenda item.
11 A transcript of this meeting is being kept 12 and will be made available on the NRC public website 13 as mentioned. Therefore, we request the participants 14 of the meeting use the microphones located through the 15 meeting room when addressing the Subcommittee.
16 The speakers should first identify 17 themselves and speak with sufficient clarity and 18 volume so as they may be readily heard. Make sure 19 that the green light is -- on the microphone is on 20 before speaking and off when it is not in use.
21 At this time, I request the meeting 22 attendants and participants silence their cell phones 23 and any other audible electronic devices.
24 We will now proceed with the meeting. I 25 call upon Mike Franovich --
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
7 1 MR. FRANOVICH: Good morning, ACRS 2 members.
3 MEMBER DIMITRIJEVIC: -- of the NRC staff 4 to begin.
5 MR. FRANOVICH: Thank you.
6 First of all, I want to thank the 7 Committee and also the ACRS staff for fitting us in to 8 your busy schedules. I know projects like NuScale are 9 really dominating a lot of attention. There's a lot 10 of material to read. So, thank you, again, for 11 fitting us in.
12 As you know, the Reg Guide 1.200 really is 13 the backbone of a lot of the PRA work as well as the 14 ASME PRA Standard.
15 We've got a lot of experience with 16 applying Reg Guide 1.200, both industry and regulator, 17 and we have progressed to a point where we think this 18 update in Rev 3 is necessary to deal with a particular 19 issue. And, that issue is the treatment of newly 20 developed methods.
21 Why that is important is, while we can 22 make progress currently with newly developed methods 23 going through a regulatory review or what we would 24 call the licensing review, an alternative approach 25 that would help our stakeholders expedite review of NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
8 1 the newly developed methods has been proposed by 2 industry and the solution for many years has been 3 elusive until today.
4 We've made a great deal of progress. We 5 believe that taking the approach that's going to be 6 defined by -- outlined by the staff today helps 7 fulfill what the Commission actually originally 8 intended which to leverage the PRA Standard to help 9 obviate the need for detailed staff review of PRA 10 models.
11 Why this is also important and has a high 12 importance for senior leadership and NRC as well as 13 industry, the matter is a top priority for both risk-14 informed steering committees on both sides and that 15 is, we are approving a number of license amendments 16 for things like risk-informed completion times.
17 So, for folks in the field who actually 18 these risk tools, the objective here is to give them 19 the most realistic assessments of risk and not have it 20 skewed or distorted by methods that may be less than 21 complete, but are sufficient to move forward with 22 approving the model.
23 Realism is important. These operational 24 decisions that plants are making and taking equipment 25 out of service for extended periods of time is really NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
9 1 sort of a top issue.
2 There's a great deal of interest by our 3 stakeholders as we see more and more plants migrating 4 to risk-informed completion time submittals.
5 We are also approving other programs like 6 50.69, an alternate treatment of SSCs.
7 So, there's a lot of momentum. It's an 8 exciting time if you're in the risk business to see a 9 lot of progress forward.
10 But the newly developed methods area, we 11 believe this alternative approach will help fulfill 12 what was originally intended in terms of using the 13 peer review process in the ASME Standards.
14 So, you're going to hear a great deal of 15 details today about how we've been working with our 16 stakeholders to build the Standard out and Reg Guide 17 1.200 update which, in essence, we would endorse the 18 provisions that'll be built into several industry 19 documents, including in the PWR Owners Group document 20 which you'll hear about later today.
21 Ultimately, we would try to finalize Reg 22 Guide 1.200, but where we are today is we're looking 23 to move forward and incorporate feedback from the 24 Committee as well as some of our internal stakeholders 25 and get this draft document out for stakeholder NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
10 1 comment in accordance with our normal processes using 2 the Federal Register.
3 It is on an aggressive schedule, again, I 4 appreciate your time and making the time to hear us 5 today. And so, we look forward to your comments.
6 CHAIR BLEY: Michael, I don't want to 7 sound like a complete jerk, but I think I will. But 8 this is really addressed at --
9 (OFF MICROPHONE COMMENTS) 10 CHAIR BLEY: What can I do about it?
11 (OFF MICROPHONE COMMENTS) 12 CHAIR BLEY: It's really addressed at all 13 the speakers. When I read this newly developed 14 methods requirements, it seems the tautology, whereas 15 I don't find substance there. It spends a lot of time 16 figuring out, is this a new method?
17 And, it's the sort of thing that feels 18 like any good analyst would do without all of this 19 baggage to get them there. And, if they can't, maybe 20 they should be doing this kind of work.
21 So, I'm really interested in what this is 22 going to do to help. Because it seems to not get to 23 the real meat, but maybe there is some. I'm looking 24 forward to finding it today.
25 MR. FRANOVICH: I think one -- just to add NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
11 1 one perspective, at the end of today, we need to have 2 some level of assurance that these methods are 3 appropriate.
4 In particular, if you're taking equipment 5 out of service. So, while it's heavily process 6 focused, that really is the end objective.
7 And, I think that's what I'm getting from 8 you. Just if you're looking for technical like in 9 this particular method, a lot of the construct is 10 about process, bringing in the right individuals, and 11 using the high level requirements to give us, at the 12 end of the day, confidence that those methods are 13 appropriate.
14 CHAIR BLEY: Okay. And, when I go through 15 the flowchart which is all process, I don't see many 16 things there that -- and it's mostly being able to 17 say, oh, this is not a new method and doesn't need 18 review. Oh, this is a new method.
19 MR. FRANOVICH: I think you'll hear from 20 our stakeholders a little bit more why that is in 21 terms of the concept of -- or issue of PRA upgrade 22 versus maintenance, is a big factor, whether or not 23 you need to bring in a group to do a focused scope PRA 24 review.
25 So, it has tentacles and other trigger NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
12 1 points and other aspects that affect the stakeholders.
2 CHAIR BLEY: Thank you. I'm looking for 3 the meat. I'll be here.
4 MR. WEERAKKODY: Should I go ahead then?
5 I think that Mike then now -- yes.
6 So, I want to repeat what Mike said with 7 respect to the Committee and also the staff pulling us 8 in because we really wanted to get in front of you, 9 get some initial reactions, not necessarily 10 understanding it's not going to be a formal position 11 from the Subcommittee but your questions might be 12 insightful and we'll take notes and try to factor that 13 as much as possible in to the Revised Reg Guide before 14 we put it out for public comment.
15 So, thank you, again. My name is Sunil 16 Weerakkody. I'm the -- I'm one of the two senior 17 level advisors in PRA in the Office of Nuclear Reactor 18 Regulation. My special focus is operating reactors.
19 I have Dr. Reisi Fard here. He's got 20 significant experience of the actual application of 21 Reg Guide 1.200 in reviewing a large number of 22 licensing actions over the last, should I say decade 23 or five, six years in NRR.
24 And then, we have the Office of Research, 25 our lead, to update the Reg Guide 1.200 has been NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
13 1 extremely closely working with us to get this done on 2 the expeditious time scale that Mike has imposed on 3 us.
4 So, when I say the senior management, so, 5 thank you, again.
6 So, we have several specifically three 7 presentations for today from the three of us.
8 You know, my presentation for the next 9 half an hour has one primary objective. I want to 10 give you more details about what Mike spoke about 11 which is the most significant change to the Reg Guide 12 1.200 that's primarily driven by significant increase 13 by licensees in adapting risk-informed initiatives.
14 It's been significant because not just 15 because of the RITS-4b AOT, but then there are other 16 rules they are adapting.
17 Let's go to the next slide.
18 What I wanted to do is to -- I wanted to 19 communicate, I know there's a couple of members at 20 least who are very familiar in the PRA area, but I 21 really want to give the -- all Subcommittee members 22 the context in the next 25, I will use 12 slides to go 23 over the seven bullets I have highlighted below to 24 give you the context of what we are trying to do.
25 And then I'll give some details on the NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
14 1 most significant change we are making to the document 2 which is our plan to endorse the revised version of 3 NEI 17-07 and also the document that the PWR Owners 4 Group has created which lists out the review 5 requirements for new review of methods.
6 And, I believe in our -- Dennis, you 7 mentioned -- should I call you Dr. Bley or --
8 (OFF MICROPHONE COMMENTS) 9 MR. WEERAKKODY: Okay.
10 I think when the PWR Owners Group, they 11 are on here, they will go over some of the efforts we 12 have explained there with respect to coming up with a 13 clean, clear definition of a new review method.
14 So, review requirements is not the only 15 thing, but any questions that they do not 16 satisfactorily answer, we have a follow up 17 presentation with this.
18 So, first off, I will go through and give 19 you a historical summary of the evolution of the peer 20 reviews process, the objective that is to convey to 21 you that this is a process that the Commission started 22 in endorsing about 20 years ago. So, I will give you 23 a perspective of that.
24 And then, within that, I will summarize 25 the role of Reg Guide 1.200 and the role of Reg Guide NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
15 1 1.200, it's relationship to a number of other Reg 2 Guides that are related to risk-informed initiative.
3 Then, I use the work gap here. We did 4 find out as in -- during our implementation, 5 especially in the 805 that there si a gap in Rev 2 of 6 Reg Guide 1.200 with respect to newly developed 7 methods.
8 And then, we concluded that it is really 9 significant to close that gap. And, I have to thank 10 the industry representatives. They volunteered to do 11 a number of things to help us close that gap.
12 And then, I will go over the current 13 status and next steps and using a very brief summary.
14 CHAIR BLEY: Sunil, what led to this? Was 15 it that the staff and applicants couldn't agree on 16 what things meant?
17 MR. WEERAKKODY: I believe the short 18 answer is this, and that is a presentation -- there is 19 a bullet here that specifically speaks to that.
20 But in summary to your question, we 21 realized not having a good clear requirement with 22 respect to a definition of newly developed method or 23 how to review and accept a newly developed method was 24 creating a lot of inefficiencies.
25 This came up during the fire PRA reviews.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
16 1 The staff felt that there were a number of fire PRA 2 methods that have been used that were not properly 3 substantiated. And then, therefore, the staff 4 starting reviewing those methods because they've got 5 to fulfill their safety mission and make sure that 6 methods that are used is acceptable.
7 On the other hand, some of the licensees 8 felt that, hey, you know, we have the PRA review 9 process the Commission endorsed 20-some years ago is 10 the staff doing things that they don't have to in 11 terms of reviewing things that they have to.
12 So, that created a lot of negative energy 13 and a lost of trust in the peer review process.
14 CHAIR BLEY: Okay.
15 MR. WEERAKKODY: So, that's --
16 CHAIR BLEY: So, the hope is this will 17 help both the applicant and the staff figure out where 18 the staff has to dig in and do a more detailed review?
19 MR. WEERAKKODY: Yes, that is the hope.
20 And, I still call it a hope for a couple of reasons.
21 One is, I think we have done, in my 22 personal view, done a tremendous job in the technical 23 area with the support of the Owners Group in terms of 24 setting up a very detailed criteria so that the chance 25 of a deficient method getting through is very low.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
17 1 But there is another big element to it 2 which is, well, once we established that are there, 3 you know, and we are working with our Office of 4 General Counsel on this, in terms of moving that 5 responsibility back to the independent license 6 agreement was is, are there any legal concerns with 7 that?
8 So, we are addressing that. We have a 9 number of interactions ongoing with our Office of 10 General Counseling regarding that.
11 Today, we are going to focus on primarily 12 the credibility of what the industry has proposed, but 13 I will mention the other element in brief.
14 So, let's go to the next slide.
15 Now, what I have done here is listed some 16 key Commission communications. Because of the time 17 constraints, I am not going to go into details, but 18 what I will do is make an overarching remark on each 19 of these things. So, all members here, even if you 20 know this, will be refreshed with respect to the issue 21 of PRA quality, or acceptability how it originated 22 about 20 years ago, and then how we have addressed 23 that issue and implemented over the years.
24 First off, I am referring to SECY-99-256.
25 So, in October of 1999, the staff made a proposed NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
18 1 rulemaking on 50.69, you know, it was published in 2 2004. And, what it did was, during that proposal, the 3 staff made it clear to the Commission that the issue 4 of having PRAs of acceptability is a key concern.
5 And, staff also pointed out that these 6 would be burdensome to the industry to demonstrate 7 that they have developed models of acceptable quality 8 to the staff and highlighted two potential parts.
9 One is the staff reviews and approves 10 every PRA model, all the details.
11 A second is to rely on the peer review 12 process where are the independent peer review was --
13 will review it and the staff will choose to go -- take 14 a deep dive into any area that they choose is 15 important to safety. And, that's in SECY-99-256.
16 Then, if you look at COMNJD-03-0002, some 17 of you may remember that we had Chairman Diaz here, a 18 big proponent of PRA, he wrote to his two fellow 19 Commissioners basically saying that, this is a 20 significant policy issue.
21 He basically said to the other 22 Commissioners, it's not just 56.90 for all of these 23 risk-informed initially, it is very important for the 24 staff and the Commission to have a policy with respect 25 to how we issue acceptability. And, that's in COMNJD-NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
19 1 09-0002.
2 The two other Commissioners, Commission 3 Merrifield, and the other Commissioner, I can't 4 remember the name. I can only remember so many 5 details, but they agreed with Chairman Diaz. And, 6 based on that, directed the staff to prepare an 7 implementation plan which is SECY-04-0118.
8 Now, in SECY-04-118, which is titled, Plan 9 for Implementation of the Commission's Fast Approach 10 to PRA Quality, which was issued in 2004, the staff 11 communicated to the Commission, you know, they had 12 like a 23-page attachment.
13 In that attachment, the Commission told 14 the status of various tools that are being produced by 15 the industry and the SME and the staff to get to 16 basically to go with the peer review process.
17 At that time, I believe NEI had published 18 what we call NEI 002 that outlines the peer review 19 process. And the ASME/ANS has started developing the 20 Standard, excuse me if I say anything inaccurate.
21 And then, on the staff side, we had put 22 out Reg Guide 1.200 that we talk about today, the 23 trial version.
24 So, clearly in SECY-04-0118, the staff 25 informed the Commission with respect to the efficiency NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
20 1 and all of the other reasons our preferred path is the 2 peer review process.
3 The Commission issued the SRM pretty much 4 approving that plan. And, that's why I would 5 basically say that the peer review process was brought 6 to their attention by the Commission in as early as 7 2004 and was then approved by them through the SRM.
8 And then, of course, I'll go into a little 9 bit more details how the whole process is set up in my 10 next slide, we established the peer review process.
11 And, sometimes, there are questions with respect to 12 whether there is our -- there are regulations in NRC 13 that has qualified. I say, do we have a regulation 14 that has necessarily pointed to the peer review 15 process as an acceptable way of reviewing the actual 16 quality?
17 The answer is a definite yes, when you 18 look at 50.69, there is rule language, not Reg Guide, 19 rule language that specifically points to the peer 20 review process as our vehicle, at least as one way of 21 accepting the PRA quality.
22 Any questions on that slide? I just 23 wanted to give you sort of a summary of the -- in a 24 story kind of way with respect to where we are. If 25 not, let's go to the next slide.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
21 1 (NO AUDIBLE RESPONSE) 2 MR. WEERAKKODY: So, this is a slide that, 3 and I still, you know, feel sad. This is something 4 that Dr. Mary Drouin created which we use pretty much 5 in most of our presentations. She's no longer with 6 us, but her legacy continues.
7 This is a great picture to convey to 8 anyone in a very brief way the different documents 9 that we have used and how they interact with each 10 other to implement the peer review process.
11 So, if you look at the -- one of the 12 blocks that's title PRA Standard to Demonstrate 13 Conformance with Staff Positions. Some of the members 14 may have already looked at the Standard, but if you 15 haven't, if you open the Standard, what you would see 16 is they have a listing of technical elements. And, 17 under each technical element, they will list, okay, 18 here are the high level requirements, here are the 19 supporting requirements.
20 It's a very thorough way of making sure 21 that our independent peer reviewers go and do a peer 22 review. And, as Chairman Dimitrijevic knows, I have 23 been a licensee. And, I'll tell you, I have been 24 subjected to that peer review process.
25 I would personally be subjected to a staff NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
22 1 review then that peer review because they come in and 2 they spend the night in this looking at every corner 3 of your PRA and they do a good job. But that is my 4 dated memory. I don't know how things are today. I 5 think he knows more. So, that's one component.
6 The second component is where usually, not 7 usually, always we have the Nuclear Energy Institute 8 creating a document that delineates the process. It 9 points to how the peer reviewers should use the 10 Standard to do a thorough peer review.
11 And, a third document which is -- which 12 has the regulatory statute is Reg Guide 1.200. It not 13 a rule requirement, but in my personal view, because 14 we do not have a PRA rule, it basically fills that 15 vacuum in a big way with respect to PRA acceptability.
16 And, some of the licensees that I have 17 talked to almost looked at Reg Guide 1.200 pretty much 18 like a rule, even though it is not a rule.
19 So, what Reg Guide 1.200 does is it gives 20 us the clear, unambiguous position as a regulator to 21 make the final call.
22 What we do is, we point to the peer review 23 document and the Standard. And, basically endorse 24 them. And, if there's anything there that we do no 25 endorse, we highlight that.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
23 1 So, that's why when you open Reg Guide 2 1.200, there is a whole appendix that tells you, 3 clarifying, you know, where we agree, where we don't.
4 So, the reason I say that is, we call the 5 final call, even though we are using the industry 6 documents to establish the Standard.
7 Let's go to the next slide.
8 So, I wanted to convey a couple things.
9 Again, because of the time constraints, I'm kind of 10 rushing through this, but there are two points I 11 wanted to make wtih this slide.
12 One is, when it comes to PRA 13 acceptability, it's a function of the application.
14 Okay? How deep we look for the applications like 15 risk-informed ISI versus RITS-4b AOT very different.
16 And, when I say very different, with 17 respect to acceptability, there are four key 18 dimensions. One is the scope. Do we have the fire 19 PRA? The seismic PRA? Have you modeled internal 20 events? External events? That's one element.
21 The level of detail, you want to make sure 22 that when you use it for an application, it has 23 sufficient level of detail to match the needs for the 24 application.
25 The technical elements, HRA, you know, NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
24 1 does it have all the technical elements? Because for 2 each of those technical elements, when you go to the 3 Standard, you find those high level and supporting 4 requirements.
5 And, the Standard presentation, we want 6 the licensee to use a model that reflects the as-built 7 as operating plant. Ensuring that they do that is 8 very important for some applications, less important 9 for some of the other applications.
10 For example, if it's a risk-informed ISI, 11 if there's some deviation, it's not going to 12 influence. But if you're talking about changing the 13 allowable outage time, they have to always have a 14 situation where the plant is operating after the PRA 15 model.
16 So, with that, let me go to the next 17 slide. Okay?
18 Now, what I want to convey here is the 19 special emphasis to RITS-4b. Mike mentioned this, but 20 this is the application where licensees will use the 21 quantified numbers, and I want to emphasize the word 22 quantified numbers from the PRA model, typically 23 includes the internal events and the fire to compute 24 the allowable outage time for simulated stains that 25 have been in the tech specs.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
25 1 So, the staff have sensitivity and the 2 most detail review is on this application. Because in 3 comparison to this, the other applications have lots 4 of qualitative elements to compliment the -- what's 5 coming out of the PRA model.
6 I make that point strongly because the 7 subject matter of the issue today is most important to 8 that particular application. And, that's why the 9 staff is very much committed to getting this right for 10 RITS-4b applications.
11 MEMBER MARCH-LEUBA: You're having too 12 much talking to yourself.
13 MR. WEERAKKODY: I know, so please.
14 MEMBER MARCH-LEUBA: I'm going to correct 15 you for the rest of it.
16 I'm a big, big non-fan of risk-informed.
17 MR. WEERAKKODY: What is that?
18 MEMBER MARCH-LEUBA: I am always 19 complaining about risk-informed things. And, my 20 primary concern is that you don't really know the 21 risk. If you knew the risk, everything else, the MAAP 22 and application would work perfectly.
23 My complaint is, you don't know the risk 24 because of the completeness of your analysis. Okay?
25 So, it's always what physical mechanism, what failure NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
26 1 mechanism, what activity to the operator do you not 2 include on your model?
3 So, we already know this improves the 4 review of the completeness. And, how do you really, 5 really go into completeness?
6 MR. WEERAKKODY: So, let me give you a 7 high level answer to that covers all of those 8 applications. The completeness of the model that you 9 point to is a very well known, I would call it a 10 limitation of the PRA model.
11 And, if you go to Reg Guide 1.174 which 12 tells licensees to how they need to submit the 13 application, that particular uncertainty has to be 14 addressed in the manner that is acceptable to the 15 staff.
16 So, and then, the other thing we do for 17 each of these applications when we recognize that the 18 completeness, the uncertainties, sensitivities, they 19 are there as inherent of the PRA model.
20 MEMBER MARCH-LEUBA: And, what would you 21 say if I tell you that we are reviewing a new reactor, 22 imaginary new reactor and, therefore, got a very 23 important design basis event, they didn't even know it 24 existed.
25 And, it was not on a license Chapter 15, NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
27 1 it was not included in Chapter 19, the operators were 2 not aware of it, but it's a very important event.
3 Call it a tsunami, call it a small break LOCA --
4 MEMBER REMPE: Oh, let's just call it 5 transportation to the site and installation of a 6 reactor with a loaded core and then removal of an 7 operating reactor --
8 MEMBER MARCH-LEUBA: If you need an 9 example, let's go to the bottom solution and then 10 outcomes.
11 MEMBER REMPE: Yes.
12 MEMBER MARCH-LEUBA: But if there are not 13 a member of the plant, forget -- let's go with the 14 tsunami. Forget to include a tsunami on their PRA, 15 how can you say that I made a risk-informed -- if I 16 didn't include a tsunami on my PRA?
17 MR. WEERAKKODY: Okay. So, there are 18 certain things that you know that you're not including 19 in the PRA and you can, and we have, and we should if 20 we don't, consider that in the application specified 21 here. Okay?
22 Now, but you can still take me to a world 23 where how do you deal with unknown unknowns? And, my 24 standard answer would be, that's why we never rely on 25 the number only, we always supplement them by NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
28 1 difference in that, safety margins, and performance 2 monitoring.
3 So, we have put those measures in place.
4 MEMBER MARCH-LEUBA: But what I'm going to 5 is this unknown unknowns, I don't know how to find 6 them other than look very hard for it. And, that is 7 where the peer review and independent reviews and 8 blind reviews come into effect.
9 And, do you consider the tsunami or not?
10 And, if we start removing all those things, you're 11 making your -- your basic data more incomplete than 12 before or certainly less probability that it's 13 complete.
14 And, I don't have any problem with the 15 MAAP. I don't have any problem with the application.
16 I have a problem with, is your basic data correct?
17 CHAIR BLEY: But you have the same problem 18 with the traditional approach to licensing.
19 MEMBER MARCH-LEUBA: Absolutely, 20 absolutely.
21 CHAIR BLEY: Which can leave out the same 22 sorts of things.
23 MEMBER MARCH-LEUBA: Absolutely.
24 CHAIR BLEY: And, we try to find them 25 every way we can. And, once in a while, Mother Nature NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
29 1 teaches us a new one.
2 MEMBER DIMITRIJEVIC: But you know the 3 perfect solution doesn't exist in anywhere in the 4 universe. So, the thing is, if you're trying to 5 emulate can he take or separate out facility feed 6 water, pump out for two weeks instead of one week like 7 what you disallow the outage time, then does he have 8 a tsunami or not? What's your feeling? Is that 9 important or not? Probably not.
10 So, the things which we forget and usually 11 on peer review, doesn't influence the basic essential 12 things. So, you know, you are not making decision 13 about meaning of the life, you are making decisions 14 about some that you have to know lots of things --
15 MEMBER MARCH-LEUBA: If you have --
16 MEMBER DIMITRIJEVIC: -- to arrive to the 17 --
18 MEMBER MARCH-LEUBA: If you are able to 19 compartmentalize the problem you're trying to solve, 20 optionally, you should also be comparing them both and 21 see which one is best. That's a very good application 22 of this.
23 MEMBER DIMITRIJEVIC: Well, then --
24 MEMBER MARCH-LEUBA: But when you're --
25 MEMBER DIMITRIJEVIC: -- you know, but if NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
30 1 you're like was talking about tsunami protection wall, 2 obviously, I mean, they will go and say, wait a 3 second, this risk is not analyzed. It's not about --
4 your concern is about, you know, bore revolutions, 5 things like that, you will look to include that, the 6 systems which can prevent such events.
7 And, you know, what is their function?
8 And, if they said the systems in the questions and 9 that you can circle --
10 MEMBER MARCH-LEUBA: But the problem is 11 the basic phenomenon was when I didn't identity --
12 MEMBER DIMITRIJEVIC: And, you are 13 completely right. But, you know, look and work on 14 Chapter 15 so they think their exact science.
15 So, the thing is that, I mean, you --
16 there is no, you know, every time you update RELAP, 17 you get the new answers. I mean, obviously, there is 18 no perfect solutions to that.
19 MEMBER MARCH-LEUBA: Well, we have to make 20 it.
21 MEMBER REMPE: Isn't the problem really 22 that you're concerned because there's more confident 23 in the results from the PRA. So, you've taken out 24 some of the conservatisms that you like with your --
25 Because in both cases, the completeness is NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
31 1 an issue. You have design basis accidents with 2 conservative assumptions.
3 MEMBER MARCH-LEUBA: My concern is when 4 you remove peer review and staff review and you go 5 with the 50.59 where the interested party that it's 6 his money you're spending. Is the one has to do the 7 evaluation.
8 The completeness has got to go down.
9 They're going to be -- jump to making a conclusion, 10 that's not the problem, don't look at it.
11 MR. WEERAKKODY: So, can I respectfully 12 disagree with you on that point?
13 MEMBER MARCH-LEUBA: Might have in some 14 case.
15 MR. WEERAKKODY: And, I will make the 16 following statement. And, since you mentioned Chapter 17 15, I would submit that, number one, and I'm 18 specifically I can say this because I'm consulting in 19 the development of a safety guide for IAEA, and one 20 thing we have brought in, and I think the Agency is 21 doing this, as opposed to in the past where we created 22 a list of design basis events based on the best 23 guesses of --
24 MEMBER MARCH-LEUBA: Gut feelings.
25 MR. WEERAKKODY: -- we are now using NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
32 1 operating experiences to add to that list so that you 2 come up with a plan that is well founded on the actual 3 risk issues.
4 MEMBER MARCH-LEUBA: If you can do it on 5 operating experience, I'm all for it.
6 MR. WEERAKKODY: That is what we do 7 because if you look at the design of the new reactors, 8 and I'm not an authority on that subject, but since 9 you are reviewing the NuScale, I'm sure you are aware 10 in over there.
11 MEMBER MARCH-LEUBA: We won't use names.
12 MR. WEERAKKODY: That's fair. Okay, so, 13 my point is, if you look at what the international 14 committee is doing, and I'm sure we have done that 15 with respect whenever we go to new designs, we have 16 used the insights of PRA to really make sure we don't 17 miss out on important things.
18 And, as Professor Dimitrijevic has know 19 very well knows that, we used to try to design plants 20 without high pressure injection systems. We used to.
21 We wouldn't dream of it today.
22 So, I would say that, of course --
23 CHAIR BLEY: In any of the later talks 24 today, you know, it wasn't the purpose of this 25 meeting, but is anybody going to go through how the NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
33 1 peer reviews that are actually organized and worked?
2 I know I've been involved, not involved, 3 I've observed some of them and they bring in people 4 from outside, they're not people from the plant.
5 MR. WEERAKKODY: Yes.
6 (OFF MICROPHONE COMMENTS) 7 MR. WEERAKKODY: Nuclear energy --
8 CHAIR BLEY: Okay. If you can do that, 9 that would be great. And, I don't know if we've come 10 to the point yet of thinking hard, but the place we're 11 most likely to have a completeness problem, well, we 12 always had one, but the most likely to have a 13 significant one are on some radical designs where we 14 have no experience.
15 And, does the -- if we've reached a point 16 where we've developed a way for the peer reviews to 17 look at the creativity in the search for events, if we 18 can get to that, I think that would be very helpful to 19 all of us.
20 MEMBER MARCH-LEUBA: Yes, and I was 21 falling asleep. I wanted to start --
22 MR. WEERAKKODY: Well, thank you for 23 waking us all up.
24 (LAUGHTER) 25 MEMBER MARCH-LEUBA: And, it's not that.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
34 1 MR. WEERAKKODY: It's a very energetic 2 conversation, but can I -- I'm going to get back 3 because I don't want to take more time.
4 CHAIR BLEY: Any time you get a break, 5 keep going.
6 MR. FRANOVICH: Mike Franovich, again.
7 Just on that thought about innovation, new 8 vulnerabilities through designs, we are not -- that's 9 not coupled with this current version of 1.200.
10 In fact, the staff is working with 11 industry, in particular, with non-light water reactors 12 and coming up with a separate approach for dealing 13 with that.
14 So, we're not going to have probably today 15 a satisfactory or fulfilling answer on that part.
16 CHAIR BLEY: I didn't expect that, but I 17 was hoping.
18 MR. FRANOVICH: Yes, I know.
19 MEMBER REMPE: But we're kind of going off 20 topic. As part of what they're doing with industry, 21 are they starting to look about a way to think outside 22 the box and be innovative and think about new 23 challenges? That's good to hear because I have not 24 heard that so far in the discussion, so that's great.
25 MR. WEERAKKODY: So, I'm going to go with NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
35 1 the next three slides. The purpose of the next three 2 slides to just to -- if some of the members would 3 knock away some key initiatives to give you a high 4 level flavor.
5 We have something called Tech Spec 5b or 6 this is an effort where licensees are using inputs of 7 PRA in combination with other things like performance 8 monitoring to change the surveillance intervals about 9 75 percent or maybe if more of our plants have already 10 received approval to conduct this.
11 Let's go to the next slide.
12 MEMBER MARCH-LEUBA: Let me -- since I 13 wasn't ready. This application is perfect.
14 MR. WEERAKKODY: Thank you.
15 MEMBER MARCH-LEUBA: You have operating 16 experience for that particular component. You -- most 17 of them are PRA and I'm always the devil's advocate, 18 is you're a PRA expert and when you tell me one number 19 you use for the input data for your failure frequency, 20 you say, well, I got together with a bunch of my 21 friends and we all agree on ten to the minus two. I 22 mean, that's the answer I get, being honest.
23 But in this particular application, you 24 have got the pump, that excel pump, I mean, it's been 25 running for 40 years and I know it has failed only NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
36 1 once. And, therefore, I know what the frequency is 2 and it has not -- and we haven't mentioned this 3 problem that the seals are not degrading and, 4 therefore, I don't need to do it every 12 hours1.388889e-4 days <br />0.00333 hours <br />1.984127e-5 weeks <br />4.566e-6 months <br />.
5 CHAIR BLEY: Well, maybe not as perfect as 6 it smells because you've been doing maintenance on the 7 current frequency. And, if you slip out from 31 days 8 to 18 months, new failure modes can exhibit 9 themselves.
10 So, you really have to be careful after 11 you do that that nothing new is coming in and changing 12 that experience data we've collected.
13 MR. WEERAKKODY: Yes, that is why the 14 performance monitoring is extremely critical.
15 So, let's go --
16 MEMBER MARCH-LEUBA: Unknown unknowns 17 which is the issue of completeness.
18 CHAIR BLEY: But it's an area where from 19 other experience you know that can happen. So, it's 20 not too many unknowns.
21 MEMBER MARCH-LEUBA: Yes, but this doesn't 22 bother me as much as from the applications.
23 MR. WEERAKKODY: In another application, 24 this is one, even though the rule was published in 25 2004, industry's interest in adopting this is NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
37 1 extremely high.
2 A number of the licensees, I can't 3 remember exact count, have adopted these. And, what 4 we hear from the industry is most licensees will adopt 5 that and the faster this is, they will use PRA inputs 6 and the inputs of an expert panel to probabilistic of 7 plant operations and design to move -- to reclassify 8 a sub-zero safety related systems as safety related, 9 but lower significance and will enable licensees to 10 manage them using not necessarily pressed with 11 requirements, but other requirements.
12 This is --
13 MEMBER DIMITRIJEVIC: So, you have -- you 14 said there is a much more because up to the year or 15 two ago, there was not too -- I mean, you could count 16 them on one hand the applications in the --
17 MR. WEERAKKODY: Correct.
18 MEMBER DIMITRIJEVIC: And, that was a 19 pity. So, you said that you have a much more 20 application now?
21 MR. WEERAKKODY: Yes, we have a number 22 more applications, exactly. I think it's more than 23 ten. We have issued the approvals but industry has 24 informed us that large numbers of others will be 25 coming into this.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
38 1 MEMBER DIMITRIJEVIC: I see, all right.
2 MR. WEERAKKODY: But I want to make --
3 okay, let's go to the next one.
4 And, this is the one that should be the 5 most focus because this is the one that really gives 6 us the expeditious need to update Reg Guide 1.200.
7 The -- this is where the licensees will 8 use PRA inputs to, as you can see, the change in the 9 tech specs, it says three days or in accordance with 10 the risk-informed completion time program.
11 And, we have a number of licensees who are 12 using this. I recognize the members' concern with the 13 completeness. I don't want to kind of go into a 14 detailed discussion on that, but I would say that the 15 staff has thought about those things hard in terms of 16 before we approve the program.
17 So, if you don't mind, let's --
18 MEMBER DIMITRIJEVIC: Yes.
19 MR. WEERAKKODY: So, this slide, I wanted 20 to convey that Reg Guide 1.200 is foundational with 21 respect to the acceptable quality. But you have for 22 each application another Reg Guide that is more 23 directly lined up with the specific needs of that 24 particular application.
25 So, you have the 1.175 on in service NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
39 1 testing, 1.17 on technical specifications, and for 2 ISI, 1.205 is for 805.
3 The one I forgot to list here is as 4 foundational as critical, that's 1.1200, is 1.174, 5 that tells the licensee how they need to -- that's 6 where -- how -- where they will come and tell us how 7 they would manage the plant in safe way using PRA in 8 light of some of the limitations, uncertainties, and 9 so on and so forth.
10 Let's go to the next slide.
11 So, I already told you that for each 12 technical element, the ASME/ANS Standard provides high 13 level requirements and supporting requirements.
14 Now, here, I think I gave the, based on 15 your question, I gave an answer pretty much that 16 covers that. Things appears to be fine before 805 17 came along. When 805 came along, we had NUREG-6850 18 that gave screening method, but there was screen 19 method and some licensees used some in adverted 20 methods without good technical substantiation which 21 basically revealed that we had a chink in our armor 22 with respect to assuring PRA quality.
23 We call it a chink or a big hole in the 24 armor but that's what gave us pause and said, okay, we 25 want to rely on the PRA review process, but we'd NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
40 1 better close this gap as soon as possible.
2 And, we got industry very motivated to 3 help us out. And, I'll tell you how that happened 4 using the next slide.
5 The staff, and I, you know, rightfully so, 6 when we were approving RITS-4b AOT, staff was dead 7 against doing business as usual with respect to newer 8 methods because, you know, you use numbers from PRA to 9 adjust AOTs.
10 So, what I have put here is the current 11 condition that we have imposed on the licensee. And, 12 what we said, it's highlighted in red there is that if 13 you change the method, okay, and this only for this 14 application, you need to get prior approval from the 15 NRC before you incorporate it again.
16 MEMBER MARCH-LEUBA: Educate me. Method 17 is mathematical process to get there or is it a model 18 or is it an input? What is a method?
19 MR. WEERAKKODY: Okay. So, I am going to 20 defer that answer to someone who's more knowledgeable 21 from the PWR Owners Group. Because I don't know 22 whether you have in your presentation the definition 23 of the method.
24 MEMBER MARCH-LEUBA: I don't want to, 25 let's finish --
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
41 1 (OFF MICROPHONE COMMENTS) 2 MEMBER MARCH-LEUBA: If you talk, you have 3 to be on the record, you need to go to a microphone 4 and tell your name or else be quiet.
5 MR. LINTHICUM: How do I turn this on?
6 It's on? Okay.
7 So, this is Roy Linthicum, Chairman of the 8 Risk Management Committee for the PWR Owners Group.
9 So, in our presentation, we don't actually 10 have the definition covered, but we have the 11 definitions that we provided in our document, the PWR-12 OG-19027.
13 (OFF MICROPHONE COMMENTS) 14 MR. LINTHICUM: Oh yes, Victoria Anderson 15 has it in hers. So, we did recognize that defining 16 what a method is was critical. So, because we have 17 had differences of opinion. Is this a model change or 18 a method change? We need -- and then, we knew we 19 needed to have that nailed down.
20 MEMBER MARCH-LEUBA: Is this defined? I 21 mean --
22 MR. LINTHICUM: Yes, it's defined now.
23 MEMBER MARCH-LEUBA: I'm not looking for 24 a lawyer decision, just educate me, what is it? In 25 ten words or less, what's a method?
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
42 1 MR. LINTHICUM: So, in ten words or less 2 and I didn't have time to pull it up, but it is an 3 overall compilation of the model data and evaluation 4 techniques that are used to --
5 MEMBER MARCH-LEUBA: So, how do you use 6 it?
7 MR. LINTHICUM: Well, that they use to put 8 into a PRA model. It's not a PRA model itself, but 9 it's a piece of the PRA model.
10 MEMBER MARCH-LEUBA: Is the input data --
11 MR. LINTHICUM: Input data --
12 MEMBER MARCH-LEUBA: -- how you connect 13 the cutsets and how you process that?
14 MR. LINTHICUM: Well, not so much the 15 cutsets, the method is more what goes in before you 16 get into the cutsets.
17 MEMBER MARCH-LEUBA: Sure.
18 MR. LINTHICUM: So, it's --
19 MEMBER MARCH-LEUBA: How you connect it --
20 MR. LINTHICUM: But how you connect the 21 data to the results, what assumptions and what 22 certainties are associated with it?
23 MEMBER MARCH-LEUBA: It's under the whole 24 universe of evidence?
25 MR. LINTHICUM: It's a compilation of that NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
43 1 so it might have changed.
2 MEMBER DIMITRIJEVIC: There are a lot of 3 different methods in the PRA, depending on the type of 4 the, you know, for example, to calculate human 5 reliability, to calculate fire damages, I mean, to 6 address the flooding. I mean, there is million 7 different methodologies --
8 MEMBER MARCH-LEUBA: Yes, I'm going to be 9 able to focus my mind if you are talking MAAP or 10 you're talking input data or you're talking your tree 11 and you're talking every factor?
12 MEMBER DIMITRIJEVIC: No, and you know, I 13 don't really, I mean, like, you know, in Section 15 14 you use the method to address, you know, the pressures 15 and subjects there.
16 MEMBER MARCH-LEUBA: It was Chapter 15 I 17 had.
18 MEMBER DIMITRIJEVIC: I can do it.
19 MEMBER REMPE: It doesn't go as far as the 20 MAAP, for example, or -- right? That's not considered 21 in that?
22 MR. LINTHICUM: Well, I mean, it would 23 include the calculations needed to do -- you don't do 24 the different calculational technical but it doesn't 25 include like, you know, it would be like addition, NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
44 1 subtraction type, that's not a method.
2 MEMBER REMPE: No, and if they put a new 3 model into MAAP and suddenly they decided that a new 4 phenomena occurs and that affects severe accidents, 5 then that would need to go through this process for 6 evaluation.
7 MR. LINTHICUM: It could, yes.
8 MR. WEERAKKODY: So, let me try to wrap up 9 in the next ten to five minutes.
10 I think what I wanted to highlight here 11 was, you know, we basically put this license 12 condition. It assures safety but at the expense of us 13 having to review large numbers of license amendments.
14 Because theoretically, what would happen 15 is any time any licensee uses a new method, they have 16 to send us an amendment and we have to review them.
17 It's not something that the industry wants to do 18 because it is -- they are very inefficient.
19 And, also, for the staff also, it takes a 20 lot of resources.
21 CHAIR BLEY: They have to send you an 22 amendment to their license?
23 MR. WEERAKKODY: Yes.
24 CHAIR BLEY: Is the PRA part of the 25 license?
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
45 1 MR. WEERAKKODY: PRA is --
2 CHAIR BLEY: I don't understand that.
3 MR. WEERAKKODY: So, PRA, this is a very 4 important subtle point. PRA model is not part of the 5 licensing basis.
6 CHAIR BLEY: I know.
7 MR. WEERAKKODY: But the configuration 8 process, configuration control process which you have 9 imposed using a license condition like this, becomes 10 part of the licensing basis.
11 So, if you look at the -- these words are 12 part of the -- we are -- we have changed their license 13 to basically say, any time you use a PRA method, you 14 need to --
15 CHAIR BLEY: I think, reflecting it back 16 on the fire PRA, I won't say debacle, but something 17 approaching that, and this last few words explains to 18 me why what I thought was a tautology is viewed as 19 essential.
20 MR. WEERAKKODY: I missed your last part 21 of the sentence.
22 CHAIR BLEY: What I had thought was a 23 tautology, I see why the industry may look at it as 24 essential.
25 MR. WEERAKKODY: Right.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
46 1 MEMBER DIMITRIJEVIC: So, wait a second.
2 This PRA configuration process is only part of 3 licensing that the licensee has a risk-informed 4 applications.
5 MR. WEERAKKODY: Right.
6 MEMBER DIMITRIJEVIC: Otherwise, that's 7 not --
8 MR. WEERAKKODY: Otherwise, it does not, 9 yes, but what we do, yes.
10 MEMBER DIMITRIJEVIC: So, if licensee has 11 a risk-informed application which he already 12 submitted, then is approved like risk-informed which 13 almost everybody now has implemented. Do they still 14 have to submit -- even they're not changing anything, 15 do they still have to submit to you changes in the 16 methodologies?
17 MR. WEERAKKODY: I don't know the exact 18 license condition in risk-informed ISI. All I can say 19 is, Steve, okay, Steve Dinsmore.
20 MR. DINSMORE: Yes, hi, is this on?
21 (OFF MICROPHONE COMMENTS) 22 MR. DINSMORE: Yes, this is Steve Dinsmore 23 from the staff.
24 I guess what happens is when we do a 25 review, they give us a set of stuff to review about NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
47 1 the PRA. And, we'll go through that and we will 2 decide that the PRA is acceptable to use for this 3 application.
4 So, every time they come in with a new 5 type of application, we go through the PRA. And, what 6 this thing did was, when we got finished looking at 7 the PRA that they had at that time, we said that that 8 PRA is acceptable to use for 4b.
9 However, as you change it in the future, 10 that's why that other red stuff.
11 MEMBER DIMITRIJEVIC: But this is 12 completely different type of application because it's 13 one time. But the people who submit the application 14 will they using it right now up to the life of the 15 plant.
16 Do we ever -- like, for example, is the --
17 out in South Texas is a good example because they have 18 always performed applications, you know, doing the 19 50.69.
20 When their PRA updates, do you guys go 21 back and check implication on all existing risk-22 informed applications?
23 MR. WEERAKKODY: Not unless we do it as 24 part of our formal OSI process. I mean, we have like 25 several inspection procedures that you must --
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
48 1 MR. DINSMORE: Only if they submit it, do 2 we look at it.
3 MEMBER DIMITRIJEVIC: Only something new 4 then?
5 MR. DINSMORE: We at NRR. I don't know 6 about the inspections, but so, most of the 7 applications they can change it as they want, like for 8 50.69 because it's not in the condition.
9 But as Sunil was saying, this one is very 10 -- it's kind of special so we wanted to put controls 11 on the future changes.
12 MEMBER MARCH-LEUBA: So, again, for my 13 education, I come from the Chapter 15 universe. And 14 there, we license codes. For example, we approve, we 15 the staff, approve NRELAP5 for use in non-LOCA 16 transients.
17 And then, that approval puts an A at the 18 end of the of the number in the topical report and it 19 can be referenced in tech specs because it's in the 20 licensing basis.
21 So, if they want to use that code to 22 change a set point, they have to use the code that's 23 approved. They cannot use a different version, it has 24 to be that one.
25 Is this the same you are doing here?
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
49 1 You're -- by referencing a model, you put that model 2 into licensing basis. And then, if you want to change 3 anything in there, you have to do a license amendment 4 request?
5 MR. DINSMORE: No, well --
6 MR. WEERAKKODY: Not anything, the 7 mechanism.
8 MR. DINSMORE: -- I can't see it really 9 because I don't have my glasses. What is --
10 MR. WEERAKKODY: Method only, yes.
11 MR. DINSMORE: What it's supposed to say 12 is, if there is another method that's been approved by 13 the staff, you can put that in your PRA without you 14 coming in for --
15 MEMBER MARCH-LEUBA: If it has an A --
16 MR. DINSMORE: Well, if it has an A --
17 MEMBER MARCH-LEUBA: -- number.
18 (SIMULTANEOUS SPEAKING) 19 MR. DINSMORE: -- on the end, you can use 20 it for this. You don't have to come in once it's been 21 approved.
22 MEMBER MARCH-LEUBA: Is that the basis of 23 -- is that what you said?
24 MR. WEERAKKODY: I kind of missed the 25 context of the conversation but I can tell you what NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
50 1 I'm trying to say. I can repeat it.
2 If a plant has adopted RITS-4b, for those 3 plants only, if they change -- if they use a newly 4 developed method, before they use it, they need to 5 come and get our approval according to this slide.
6 MEMBER MARCH-LEUBA: Even though if it was 7 approved for a Plant B?
8 MR. DINSMORE: No.
9 DR. REISI FARD: No, under that scenario.
10 This is Mehdi Reisi Fard.
11 So, the current admin textbook language 12 says that if a method has been accepted as a part of 13 the review or method that has been accepted or 14 approved as a part of other licensing activities.
15 So, if for another plant, you have 16 accepted or approved that method, that wouldn't fit 17 into the newly developed method kind of framework.
18 MEMBER MARCH-LEUBA: Now, let me ask you 19 a question. Say that, in my method, I have decided my 20 failure -- the frequency probability of this type of 21 valves is ten to the minus twelve or one or two, 22 whatever.
23 And now, there's an obscure university in 24 Italy that has collected data from all Russian 25 reactors and comes up that it should be twice as much.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
51 1 And, they decide to go and change that number. Is 2 that a change of the method?
3 MR. WEERAKKODY: That's not how -- in my 4 mind, that's not a change in the method.
5 DR. REISI FARD: That's not changing the 6 method, no. That's part of the PRA configuration 7 control, PRA is updated according to the, you know, 8 the --
9 MEMBER MARCH-LEUBA: That's why I was 10 asking what the method is. The method --
11 MR. WEERAKKODY: No, that's not a method 12 because we have standard ways of doing those updates 13 that we have had exposure to that are acceptable. So, 14 as long as they stick to that.
15 MEMBER MARCH-LEUBA: They already have the 16 flexibility to --
17 MR. WEERAKKODY: Correct, yes.
18 MEMBER MARCH-LEUBA: Just makes a lot of 19 sense. And, you have to be making license amendment 20 requests for --
21 MR. WEERAKKODY: No, no, no, no, no, it's 22 not that bad.
23 MEMBER MARCH-LEUBA: Okay.
24 MR. WEERAKKODY: So, my point is, industry 25 recognized that in order for to -- they have an NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
52 1 request to change this so that, in my layman's words 2 here, methods can be peer reviewed and used without 3 prior NRC approval.
4 But they understood that in order to --
5 for us to even consider that proposal, there should be 6 clear criteria that says what is a new method and how 7 are you going to review and approve for the peer 8 process whether that method is acceptable.
9 Owners Group has those criteria. We 10 typically like to have them in NDMs standard, but 11 because of the expeditious nature, right now 12 tentatively, it will be in the Owners Group document.
13 And, NEI updated the peer review process 14 to accommodate basically describe how it needs to be 15 done.
16 Let's go to the slides, I'm taking too 17 much time from everybody else.
18 So, in terms of closing the gap, as I 19 said, PRW Owners Group, you'll hear details from the 20 industry, provide definitions related to NMDs, PRA 21 maintenance, and PRA upgrade, provide six high level 22 requirements and 21 supporting requirements for peer 23 review NDMs.
24 Seventeen-oh-seven delineates the process 25 that peer reviewers must use to peer review NDMs in NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
53 1 addition to other technical elements of PRA.
2 So, that's our strategy. Next one, 3 please?
4 Current standard, we have had -- I'm not 5 going to dwell on this because he's going to go 6 details. We have a number of meetings, observations 7 on these -- the work that the industry has done.
8 And, next steps, we want to complete the 9 update of the Reg Guide 1.200. Our goal is to give 10 you a, you know, after this meeting, create a version 11 of Reg Guide 1.200 that is -- and get the OGC 12 approval, management approval, put it out for public 13 comment.
14 And then, I have -- and then, of course, 15 incorporate those comments.
16 One of the key things I do want to 17 mention, we are briefing you with respect to the Reg 18 Guide, but there's another element of the Agency's 19 functions which we have enhanced to recognize the 20 importance of these initiatives.
21 We have updated about four inspection 22 procedures that would enable our inspectors to, on a 23 performance based, risk-informed basis, to go and do 24 some sample checks on whether they are following 25 through on their commitments to keep these models with NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
54 1 acceptable quality. So, most of that work has been 2 already done.
3 And then, the final item is the NEI 4 proposal, the industry proposal would change the tech 5 spec is very related to the Reg guide, updating the 6 Reg Guide is an essential component, but it does not 7 itself is the conclusion. That is something that is 8 going through a legal review.
9 And then, at some point in time, once we 10 get the Agency to agree or decide we, our senior 11 management will make a decision and communicate that 12 decision to the industry.
13 CHAIR BLEY: I have not made myself 14 familiar with the inspection procedures related to 15 PRA. Are those inspections like the physical 16 inspections run out of Region 2 or do the -- each 17 region use their PRA people to do these inspections?
18 MR. WEERAKKODY: They don't usually use 19 PRA people. There are -- they use that as their 20 inspectors. And, what happens is, if they get into a 21 situation where they need some PRA information --
22 CHAIR BLEY: They'll come back to --
23 MR. WEERAKKODY: -- they'll come back to 24 the Regional SRA OS. And, I know that Branch Chief is 25 not here, but we are doing a lot of training to get NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
55 1 the agents to go out and learning this area. But we 2 are not planning to make the inspectors PRA experts.
3 We will stay accessible to support them as needed.
4 CHAIR BLEY: Okay. Is this new or has 5 this been going on for a long time?
6 MR. WEERAKKODY: It's been going on for a 7 long time. I think what is new is that we took a 8 holistic look at all of our procedures. It came to 9 our attention primarily because of RITS-4b. But in 10 the process, we realized that it should be just 11 focused on that.
12 Every application should have a peer 13 review check and for some applications, we need to 14 verify maybe a one-time check on whether the licensee 15 has implemented the program.
16 If you are interested, I could send you 17 the list of inspection procedures for your awareness 18 that we have updated those.
19 CHAIR BLEY: Yes, that's a good idea.
20 MR. WEERAKKODY: Okay. I will take an 21 action item. I'll share the request and send the 22 inspection list.
23 MR. WEERAKKODY: So, that concludes my 24 presentation and thank you for waking everyone up 25 because I was having it too easy.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
56 1 MEMBER MARCH-LEUBA: It's our job to make 2 your life miserable.
3 MR. WEERAKKODY: That's impossible.
4 MEMBER MARCH-LEUBA: But I think you 5 enjoyed it.
6 MR. WEERAKKODY: Right, this is my life.
7 I enjoy this work.
8 MEMBER DIMITRIJEVIC: Yes, I'm sure you 9 do.
10 MR. WEERAKKODY: Yes, was not going to 11 attest to that, I have at least 30 years experience in 12 this area.
13 MS. ANDERSON: All right, so, my name is 14 Victoria Anderson. I'm with the Nuclear Energy 15 Institute where I work in Risk-Informed Regulation.
16 And, I am going to talk about NEI 17-07 17 which is the industry document that is going to go 18 through the peer review process in general as well as 19 specifically on newly developed methods and also give 20 a little bit of background and cover some of the 21 questions that were asked in the first presentation.
22 All right, so, I'm going to give just a 23 little background on how we got to NEI 17-07. I'll 24 talk about the guidance itself and how it relates to 25 supporting documents. And, I'm also going to go over NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
57 1 the extensive stakeholder interactions that we've had.
2 NEI numbers its documents with the first 3 number being the year it was developed. So, this was 4 developed in 2017.
5 So, as you can imagine, there has been a 6 lot of stakeholder interaction, many rounds of 7 comments between the staff, many public meetings.
8 MEMBER MARCH-LEUBA: Only two years, 9 that's not bad.
10 CHAIR BLEY: That's not bad at all.
11 MS. ANDERSON: I'm not saying it's bad, 12 but we did -- I mean, this was -- we were meeting very 13 regularly and passing comments back and forth and 14 really trying to make sure that we were all aligned.
15 MEMBER MARCH-LEUBA: Let me rephrase that.
16 It is bad, but it's not unexcepted or unusual. It 17 should be shorter.
18 CHAIR BLEY: Victoria, this new document, 19 it's all the guidance now on --
20 MS. ANDERSON: It's all. Yes, I will 21 actually get into that --
22 CHAIR BLEY: Okay.
23 MS. ANDERSON: -- in a couple of slides.
24 So, I think as we discussed during the 25 first presentation, the peer review process has really NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
58 1 been a vital component of the implementation of the 2 ASME/ANS PRA Standard since its inception.
3 And, just to talk a little bit about how 4 the peer review process works, it -- when we did sort 5 of like the early peer checks, I mean, back in like 6 the late '80s, early '90s, we didn't really have solid 7 criteria. We more had people going around talking 8 about, well, this state of the practice, is this what 9 we expect?
10 And, we've since then moved along to a 11 much more standardized set of expectations. And, 12 that's really what the ASME/ANS PRA Standard does. It 13 really lays out, here's exactly what we expect to have 14 done.
15 And, that's led to much more consistency 16 in the peer reviews. It's helped the licensees 17 develop their PRAs with those expectations in mind.
18 And, it's ultimately led to a higher quality of PRA 19 that has a better technical applications throughout 20 risk-informed regulation.
21 It provides a very rigorous process. As 22 Sunil mentioned, he thinks that licensees would rather 23 go through an NRC staff review than a peer review.
24 I think, in some cases, perhaps. I think 25 it's definitely very rigorous. But it's stable. I NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
59 1 think the expectations at this point are really well 2 known thanks to the ASME/ANS PRA Standard and the peer 3 review process documentation that we've put together.
4 This also reduces the NRC resources that 5 need to be expended on PRA technical adequacy. So, in 6 the course of the peer review process, the peer 7 reviewers write up findings that are against a 8 specific supporting requirement from the Standard.
9 And so, what the staff can do is they can 10 take those findings from that report and they can look 11 at where the potential weak points of the PRA are.
12 And, areas that they may need to review more closely 13 in their licensing reviews.
14 CHAIR BLEY: Is that something they would 15 audit or those submitted to --
16 MS. ANDERSON: They're audited. Well, 17 what gets submitted usually is the open findings.
18 CHAIR BLEY: Okay.
19 MS. ANDERSON: It's submitted, not the 20 entire peer review report because those are provided 21 there.
22 CHAIR BLEY: But the things they are still 23 working on?
24 MS. ANDERSON: Yes, the open findings will 25 usually get submitted with the licensing application, NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
60 1 depending on the specific licensing application and 2 what is and is not relevant.
3 And then, the report will usually be 4 available for audit and for staff review, but not 5 submitted on the docket.
6 So, this does definitely cut back on the 7 amount of review the staff has to do. And, really, it 8 helps them in focusing on what they need to review to 9 ensure that the PRA can support the decision being 10 sought.
11 MEMBER MARCH-LEUBA: Well, going back to 12 my original question of completeness, I'm a reviewer 13 for -- I do reviews for my living and we review often, 14 let's say, review a thousand pages a week we have to 15 go through.
16 So, it's very easy to nitpick on the three 17 conclusions and the three items that the staff or you 18 guys send me. And, it's very hard to try to figure 19 out what they forgot. When, because simply because of 20 the volume of it in our case.
21 So, do you give any thought -- my problem 22 is, do you remember to account for the tsunami? And, 23 when you go through this process when that checking 24 your MAAP and, oh, look at that conclusion two, I 25 don't agree with it. I would have done it this way NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
61 1 and I still would have reached this conclusion two.
2 You are not thinking about this or that.
3 So, I would like for the peer review -- I 4 don't want the staff to do it, because the staff would 5 always take 18 months to do it.
6 And, I'm going to their famous red 7 herrings which they always do the same one. But the 8 peer reviews should concentrate on what did you miss 9 on your model?
10 MS. ANDERSON: I think the peer reviews do 11 an outstanding job of that. We have at least one 12 licensee at the table and one in the back.
13 CHAIR BLEY: Before you go ahead, I want 14 to -- I've only looked at two or three for particular 15 utilities I had worked with in the past.
16 And, the ones I saw, the peer review team 17 really asked sophisticated questions and not some of 18 them were things they had to do. But, you know, you 19 might want to do a much better job in this area. And, 20 really searching for the missing things.
21 I've heard other people say they've seen 22 some years ago, after the Standard was in place, but 23 some years ago, they had seen some that really didn't 24 delve deeply at all and were not extraordinarily good.
25 What kind of, as you talk through this, NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
62 1 let us know what kind of controls you have and checks 2 you have on the folks who go out and do these peer 3 reviews --
4 MS. ANDERSON: We have --
5 CHAIR BLEY: -- to see consistency and 6 thoroughness in their examination.
7 MS. ANDERSON: We have -- so, I mean, NEI 8 has a peer review task force that includes all the 9 peer review team leads. Roy's Owners Group has a 10 project to work on that.
11 MR. LINTHICUM: Yes, this is Roy Linthicum 12 from the --
13 (OFF MICROPHONE COMMENTS) 14 MR. LINTHICUM: Once again, this is Roy 15 Linthicum from the PWR Owners Group.
16 So, within our process and we, as an 17 Owners Group, do more peer reviews than any other 18 organization just because of our size.
19 We actually -- we have specific 20 requirements above and beyond even what's in the NEI 21 guidance. We do ensure anyone we have in our peer 22 review is qualified in the area that they're going to 23 be reviewing to their company standard.
24 You know, we don't -- everyone has a 25 different set of qualifications.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
63 1 We also ensure that they're aware of the 2 Standard and the peer review process. So, we don't 3 take someone for a peer reviewer that has never been 4 part of a peer review before. And, that could be --
5 (OFF MICROPHONE COMMENTS) 6 MR. LINTHICUM: Well, we have what we call 7 working observers. So, that's -- it's either through 8 being a working observer or, if you're a utility 9 person, defending your PRA as part of a peer review 10 was also a way we get through that process.
11 MEMBER MARCH-LEUBA: Do you have subject 12 matter experts?
13 MR. LINTHICUM: And, we have subject 14 matter experts.
15 MEMBER MARCH-LEUBA: A PRA expert? The 16 physics guys?
17 MR. LINTHICUM: The physics guys, well, we 18 did have a challenge on a recent peer review on 19 external flooding trying to find external flooding PRA 20 people. They really don't --
21 MEMBER MARCH-LEUBA: Yes, PRA people, it 22 has to be a --
23 MR. LINTHICUM: Right, but once again, 24 finding -- so how do you -- but how do you find 25 someone that's not a PRA person that also has some NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
64 1 knowledge of those things?
2 MEMBER MARCH-LEUBA: But you're --
3 MR. LINTHICUM: So, we do address all 4 that.
5 MEMBER MARCH-LEUBA: How about, sorry to 6 interrupt, how about the role of the moderator?
7 Because whenever you have a group of reviewers, the 8 guy with the biggest mouth with, in this case, it's 9 me, that dominate the decision?
10 MR. LINTHICUM: So, that -- yes, so we do 11 have what we call peer review lead and Andrea Maioli 12 sitting next to Victoria here is our --
13 MEMBER MARCH-LEUBA: Not lead --
14 MR. LINTHICUM: -- most experienced. But 15 while we call it -- the lead is the moderator.
16 MEMBER MARCH-LEUBA: Yes.
17 MR. LINTHICUM: So, the primary role of 18 the peer review lead is not to actually perform the 19 review, even though they can.
20 MEMBER MARCH-LEUBA: But that --
21 MR. LINTHICUM: But to herd the cats, so 22 to speak and to ensure that the loudest voices are 23 being heard. So, the primary role is to ensure that 24 all the reviewers actually get to a consensus before 25 they have the finding.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
65 1 MEMBER MARCH-LEUBA: Okay.
2 MR. LINTHICUM: And, that's a tough 3 challenge, especially when you're dealing with PRA 4 people that always tend to be a very opinionated. But 5 we do deal with that.
6 CHAIR BLEY: These teams, when they go out 7 on a peer review, are they from many different 8 utilities or do they all come from the same one?
9 MR. LINTHICUM: No, well, we try and avoid 10 having multiple people from the same utility for a 11 number of reasons.
12 One, we want a broader range of expertise, 13 plus most utilities don't want to support multiple 14 reviewers because there is a kind of expense involved 15 in time and resources.
16 But we try actually for a 50/50 split when 17 we can of utility and consulting -- consultants. We 18 don't always achieve it, but that's kind of our --
19 that's kind of where we kind of target.
20 MEMBER DIMITRIJEVIC: What would be 21 interesting and in my knowledge, that would happen is 22 actually to do the peer review of the same plant with 23 two different teams who don't know each other. And, 24 which plant will pay, because usually utilities pay 25 for that and then maybe NEI can sponsor that just to NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
66 1 see what's the consistency of that peer review.
2 MS. ANDERSON: You might run into some --
3 the Heisenberg uncertainty principle there that PRA 4 model gets affected simply by the peer review team 5 being there. So, it would be hard to do.
6 You know, it's pretty common for a review 7 team to come in and they'll have suggestions for the 8 PRA that don't necessarily rise to the level of a 9 finding that are pretty easy fixes. So, then, they 10 wind up changing the model as a result of the review.
11 MEMBER DIMITRIJEVIC: Well, I have been 12 part of five different teams from five different 13 nations doing the PRA of the same reactor and you can 14 see it's not the same PRA.
15 MS. ANDERSON: Yes, it's --
16 MR. LINTHICUM: Well, yes.
17 MEMBER DIMITRIJEVIC: There's a lot of 18 similarities but it's interesting the different teams.
19 That was fascinating from my point of view. And, I 20 was thinking it would be the same fascinating to have 21 a review done with that.
22 MR. LINTHICUM: Well, yes.
23 MEMBER DIMITRIJEVIC: It's all dependent 24 on the human assumptions and priorities and, yes.
25 MR. LINTHICUM: Right. There is that NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
67 1 which is why we strive for a diverse team and everyone 2 on the team brings their own set of kind of beliefs 3 and interpretations and what's important for them.
4 But in general, I mean, we do find it is 5 a -- we do really, you know, shake the tree, so to 6 speak and get to all the important elements. That's 7 been a consistent process.
8 Utilities aren't always happy with the 9 results. They don't like findings necessarily. But, 10 you know, it is what it is and we -- I think we have 11 been very successful at improving the quality of PRAs 12 over the last 20 to 30 years.
13 MEMBER REMPE: So, maybe this isn't the 14 best slide to bring this up on, I was reading your 15 document. I was curious on why you wouldn't allow the 16 author of the method to be typically should be a peer 17 reviewer because of, I think about MAAP, again.
18 That's kind of where my angle comes from is in the 19 phenomena assessments.
20 If they weren't involved in actually doing 21 the application of the method for that plant, it seems 22 like a developer of a method might be a good person to 23 have because they'd know whether the analyst had 24 correctly implemented them.
25 MS. ANDERSON: Yes, I --
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
68 1 MR. LINTHICUM: So, that's different.
2 MS. ANDERSON: That's separate, yes.
3 MR. LINTHICUM: So, what we're looking at 4 is the peer review requirements of the method. So, we 5 don't want the method developer reviewing their own 6 method.
7 MEMBER REMPE: Okay.
8 MR. LINTHICUM: No, now, the -- it's 9 perfectly okay -- so, any new method, once the utility 10 puts that method into their PRA, has to have an 11 implementation period.
12 MEMBER REMPE: That makes sense. Okay, I 13 --
14 MR. LINTHICUM: But now, that 15 implementation peer review, it would be perfectly okay 16 to have the method author be part of that 17 implementation peer review.
18 MEMBER REMPE: You're right, I 19 misunderstood those.
20 MS. ANDERSON: Yes, I guess the only case 21 where that would be potentially problematic is if you 22 were reviewing the implementation and the method at 23 the same time which is allowable by the guidance. But 24 I don't see that necessarily happening very much.
25 MR. LINTHICUM: Yes, well, our utilities NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
69 1 in general have said they'd rather make sure the 2 method is good before they spend the time and 3 resources putting it in their model.
4 MEMBER REMPE: So, I mean, MAAP's never 5 been reviewed by the staff has it? Is it an approved 6 method?
7 MS. ANDERSON: Well, it's a state of 8 practice --
9 MR. LINTHICUM: It's a state of practice.
10 MS. ANDERSON: -- on a consensus method.
11 Well, sort of get -- that's sort of part of the 12 definition --
13 MR. LINTHICUM: Part of one of the 14 definitions.
15 MS. ANDERSON: -- of newly developed 16 method. We sort of addressed the concept of, you 17 know, obviously, not everything is going to go through 18 either NRC staff review or this peer review process 19 because we've accepted these for decades and we've 20 been using them and we have experience with them.
21 MEMBER REMPE: Okay. So, then, I'll throw 22 you a curve ball. We're learning a lot from 23 Fukushima. Those vessels have failed for the BWRs.
24 I wouldn't be surprised if you're going to see some 25 updates in MAAP.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
70 1 And so, you're going to have to do 2 something about that and so will the MELCOR folks, 3 too. But it'll be an interesting situation.
4 MR. LINTHICUM: So, that is, you know, 5 part of the question that needs to be answered. And, 6 we are planning on developing a set of examples and 7 MAAP updates is one of those.
8 So, depending on the extent of a MAAP 9 update and what the revision is, that may or may not 10 -- that new revision may or may not be a new method.
11 But that's something that would have to be evaluated 12 on a case by case basis.
13 CHAIR BLEY: I don't know if you've read 14 it, but must be 20 years ago now, Alan Swain did a 15 review of HRA methods for the Germans. And, he lays 16 out all the methods in his book.
17 And then, in the appendix, he has each of 18 the developers of each of the methods kind of evaluate 19 all the others and their own.
20 And, some of it was a surprise to him, but 21 not so much to me. It turned out that almost all of 22 the developers didn't like any of the other methods 23 and said they could not be used for these applications 24 but their own method because they understood how to 25 adapt it, could be used. And it was -- it's a pretty NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
71 1 interesting report if you've never seen it. I 2 recommend it to you.
3 MS. ANDERSON: I have to look it up.
4 All right, any other questions on the peer 5 review process in general before I move on to NEI 17-6 07 in newly developed methods?
7 (NO AUDIBLE RESPONSE) 8 MS. ANDERSON: All right, so as was 9 alluded to earlier, we took all these peer review 10 guidance documents and supporting documents and we're 11 putting them all into NEI 17-07.
12 We previously had actually four documents 13 just for peer review, one on external hazards, one on 14 fire, one on internal events, and then, also we had an 15 appendix on closing peer review findings.
16 So, we had a lot of guidance out there and 17 we put it into one document that would make it easier 18 for utilities to implement and also for the NRC to 19 endorse.
20 CHAIR BLEY: The lower right one on newly 21 developed methods --
22 MS. ANDERSON: Yes?
23 CHAIR BLEY: -- it's -- that source is 24 this PWR Owners Group report?
25 MS. ANDERSON: It's related to that. This NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
72 1 -- NEI 17-07 has the process and qualifications and 2 all those kinds of things. The PWROG document has the 3 technical requirements.
4 And, we kept those separate for a very 5 specific reason --
6 CHAIR BLEY: Okay.
7 MS. ANDERSON: -- because if you sort of 8 look to the analog of NEI guidance for peer review 9 versus the ASME/ANS PRA Standard.
10 The PRA Standard has requirements. Our 11 peer review guidance document is a guidance document.
12 It does not have requirements.
13 So, the PWROG document includes 14 requirements and criteria.
15 MR. LINTHICUM: Right. Now that is 16 intended to be short-term. Now, well, relatively 17 short-term in nuclear space. Because the intent is to 18 have the requirements that we have in our document put 19 into the next edition of the ASME Standard.
20 And, in fact, they are currently --
21 CHAIR BLEY: Oh, okay, that's what I said.
22 MR. LINTHICUM: They are currently in the 23 draft version that's out for comment ballot right now.
24 CHAIR BLEY: Okay.
25 MR. LINTHICUM: But we -- because of NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
73 1 what's happening with risk-informed completion times 2 and the need to clarify that license condition, we 3 wanted to move sooner than the standard process.
4 So, once that's fully put into the ASME 5 Standard then a subsequent or revision of Reg Guide 6 1.200 would be expected to just reference the ASME 7 Standard and our guides.
8 MEMBER DIMITRIJEVIC: How do you put fire 9 when it was done after you explained to us the '17 10 year and they're obviously, fire was then after the 07 11 because it was 012. So, how did you manage to put 12 something in --
13 MS. ANDERSON: Oh, sorry, that's a typo, 14 that should be '0712.
15 MEMBER DIMITRIJEVIC: Oh, I see.
16 MS. ANDERSON: Sorry, operator error.
17 MEMBER DIMITRIJEVIC: So, we're already 18 advancing into the future this week.
19 MS. ANDERSON: Yes.
20 (LAUGHTER) 21 MS. ANDERSON: That would have made it 22 hard to get through NFP-805.
23 All right, so, in 17-07, we didn't really 24 make that many changes compared to the original peer 25 review documents. We did incorporate some lessons NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
74 1 learned, namely related to how we use observers as Roy 2 talked about earlier, how we confirm reviewer 3 qualifications, just some process clarifications.
4 Most of our changes were related to the 5 support of newly developed methods. And, this 6 provides an alternative to explicit NRC approval of 7 PRA methods.
8 So, as promised, we have our definition of 9 a newly developed PRA method. And, this may also kind 10 of answer the MAAP update question, too.
11 So, I'll just read this here, a newly 12 developed PRA method is one that has either been 13 developed separately from a state of the practice 14 method or is one that involves a fundamental change to 15 a state of practice method. So, therefore, it is 16 neither a state of practice method or a consensus 17 method.
18 When we say consensus method, that's also 19 defined in the document, the PRA review document and 20 it's something that's done by a large group versus 21 like one individual contractor, one individual 22 utility.
23 So, the most obvious example we have in 24 front of us is the work jointly that the Electric 25 Power Research Institute and NRC do on fire PRA.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
75 1 That's considered a consensus method because there is 2 so much input from so many stakeholders, it's 3 essentially accepted by the industry and the 4 regulator, the entire technical community as a whole.
5 So, there is not really any value to be 6 gained in a newly developed method peer review.
7 MEMBER MARCH-LEUBA: So, I'm still 8 confused maybe because I don't know what is it. Is 9 the fire PRA a method?
10 MS. ANDERSON: The fire PRA is an 11 approach. So, that's an approach to modeling fire 12 risk.
13 MEMBER MARCH-LEUBA: I'll quit.
14 MS. ANDERSON: But like a --
15 (OFF MICROPHONE COMMENTS) 16 MS. ANDERSON: Yes, there are many methods 17 within a fire PRA. So, like the method that you use 18 to model electrical cabinet fire heat release rates, 19 that's a method within your fire PRA which is the 20 approach to modeling fire risk.
21 MEMBER MARCH-LEUBA: But then you would 22 call it a subroutine? And that's why --
23 MS. ANDERSON: Sure. And, I think that's 24 probably pretty analogous.
25 MR. LINTHICUM: Right. But you're getting NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
76 1 to the point, it's the question was asked, you know, 2 in the beginning, why is this important to the 3 industry?
4 Well, these questions, you know, what does 5 this mean? You know, they seem important and you've 6 asked me what a method is, I may have a definition, 7 you may have something different.
8 MEMBER MARCH-LEUBA: I simply only have 9 one.
10 MR. LINTHICUM: Right, so we needed to 11 make sure that everyone had the same concept and 12 definitions. So --
13 MEMBER MARCH-LEUBA: If the methods go 14 down to the subroutine level, then you have so many 15 and during licensing basis, are there something wrong 16 with their strategy?
17 MR. LINTHICUM: Well, I wouldn't say we 18 would say a method goes down to the subroutine level.
19 It would -- but it would include any of the 20 subroutines.
21 So, the big -- probably the biggest area 22 in that case would be something like MAAP that has a 23 lot of different algorithms, a lot of different 24 assumptions imbedded in that.
25 MEMBER MARCH-LEUBA: So, if I have a plant NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
77 1 that is fully gung-ho on risk-informed, how many 2 methods do I have on my licensing basis? Two?
3 Twelve? A hundred and twenty?
4 MR. LINTHICUM: Well, in your thousands.
5 MS. ANDERSON: Yes.
6 MEMBER MARCH-LEUBA: So, it goes down to 7 the subroutine level?
8 MR. LINTHICUM: Well, no, I mean we're 9 talking about a PRA in whole, I mean, you're talking 10 about how do you --
11 MEMBER MARCH-LEUBA: And, I really think 12 the change --
13 MR. LINTHICUM: -- quantify the model?
14 So, are you, you know, I mean, physically, how do you 15 quantify the cutsets in some subset? That's a method.
16 MEMBER MARCH-LEUBA: One thousand methods?
17 MR. LINTHICUM: There are probably if 18 there's too many to call out and we have been asked by 19 the staff to previously, can we list all the methods 20 that they've accepted? And, we said, that's an 21 impossible task just because there's so many.
22 MEMBER MARCH-LEUBA: Okay, this is --
23 MEMBER DIMITRIJEVIC: In the thousands, 24 but --
25 MEMBER MARCH-LEUBA: There's something NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
78 1 wrong with that.
2 CHAIR BLEY: If I get a little -- if I get 3 the motivation that led us here, right, and had a lot 4 to do with the fire PRAs. And --
5 MR. LINTHICUM: It did.
6 CHAIR BLEY: -- there was a lot of 7 bickering back and forth between various licensees and 8 the staff. And, as that evolved, and I remember, I 9 forgot what they called them, but the staff would come 10 up with new criteria and they had a name for them.
11 And, that just grew and grew of things 12 that they decided people weren't doing right and they 13 needed to do better.
14 But a lot of those were pretty small 15 changes within a method. So, it doesn't feel like 16 this will help in that kind of situation.
17 MS. ANDERSON: Yes.
18 CHAIR BLEY: Do you think it would?
19 MS. ANDERSON: I think, so, I think you're 20 referring to the FAQ process that we had for --
21 CHAIR BLEY: Yes.
22 MS. ANDERSON: -- NFP 805 and fire PRA.
23 CHAIR BLEY: I forgot what they called 24 them, but --
25 MS. ANDERSON: So, this doesn't replace NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
79 1 that, but if you look at back, you talked earlier in 2 this meeting about that flowchart that's there.
3 What that would do is it would say this is 4 something that goes through normal licensee process 5 controls for maintaining the PRA.
6 So, that part of what we've done addresses 7 that. This part with the newly developed method PRA 8 peer review addresses those things that cannot be 9 addressed solely by the licensee maintenance and 10 upgrade process.
11 CHAIR BLEY: And, your hope is, this is 12 will really focus NRC's involvement in the reviews?
13 MS. ANDERSON: Yes.
14 MEMBER DIMITRIJEVIC: Let me ask you 15 something. Let's say that the Subcommittee can use 16 old fashion --
17 (OFF MICROPHONE COMMENTS) 18 MEMBER DIMITRIJEVIC: So, let's say the 19 utility have used some old fashioned model for 20 something. So, it's not using the state of practice.
21 But decide to update to state of practice. Would that 22 be considered newly developed method?
23 MS. ANDERSON: No, it wouldn't be a newly 24 developed method but it would be a new method at that 25 plant so they may need -- or it could be a new method NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
80 1 at that plant and so they may need a focus scope peer 2 review to evaluate how to implement it.
3 MR. LINTHICUM: Right. That goes to the 4 definition of upgrade versus maintenance.
5 MS. ANDERSON: Right.
6 MEMBER DIMITRIJEVIC: I see. No, but this 7 is the model upgrade. I mean, the model upgrade that 8 --
9 MS. ANDERSON: Right.
10 MEMBER DIMITRIJEVIC: -- right. But 11 they're not fitting in your definition?
12 MS. ANDERSON: Correct.
13 MR. LINTHICUM: Right.
14 MEMBER DIMITRIJEVIC: Okay.
15 MR. HYSLOP: This is J.S. Hyslop. I'm on 16 the staff in NRR and I was involved in the fire PRA in 17 805. And, there were a lot of changes that the staff 18 did feel need to be made for this fire PRA. Some were 19 small and sometimes they weren't challenged by the 20 peer reviews.
21 And, if there had been some process that 22 had some of these assumptions had to go through, then, 23 that certainly would have given us more confidence.
24 It doesn't mean we would have agreed 25 necessarily, but it would have given us more NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
81 1 confidence.
2 But everything wasn't small, you know, the 3 main control room abandonment evolved through the RAIs 4 process that went through 805. And, that became much 5 more robust. And, in some cases, the analysis didn't 6 seem to be well considered necessarily at the 7 beginning of the RAI process, but it was later.
8 So, if there had been a process that 9 licensees had to go through and someone had flagged 10 that sort of thing as a newly developed method, then, 11 yes, I think it would have been better before it got 12 to us.
13 MS. ANDERSON: And, I think one of the big 14 advantages of the process we've developed in concert 15 with the NRC staff is the main advantage is that we 16 have these concrete criteria for evaluating newly 17 developed methods.
18 So, while the ASME/ANS PRA Standard does 19 make provisions for peer review teams to review 20 methods themselves, it doesn't lay out the criteria 21 for what should this method do.
22 So, that helps both with the evaluation of 23 them and it helps the method developers know what 24 criteria they're trying to meet versus let's make 25 something that's acceptable or good and what does that NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
82 1 really mean?
2 And so, it really helps them focus on 3 exactly where they need to be getting their data from 4 and what kinds of considerations they need to be 5 taking in.
6 And, we found that when we piloted this 7 newly developed PRA method peer review process that 8 the peer reviewers were -- they were very hard on the 9 methods. They were extremely rigorous in their 10 review, but they were also very focused. And, it was 11 very clear at the end of the review what needed to be 12 done to the method to make it acceptable versus in 13 some past situation where we might have just been 14 passing methods back and forth and saying, is this 15 good enough? No, it's not.
16 And, that just took so much time, this was 17 a much faster process, but it still included a good 18 deal of rigor. So, I think we've -- this is a very 19 good process that can help both the industry and the 20 NRC both move quickly and improve the technical rigor 21 of the review.
22 So, I think I've moved beyond this slide.
23 MEMBER REMPE: Just out of curiosity, this 24 high energy arcing fault situation, have you been 25 thinking about how what you're doing might assist NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
83 1 what's going on with that issue?
2 MS. ANDERSON: Yes, it's possible. I 3 think any kind of work that would be done relative to 4 the high energy arcing fault issue where there's a 5 potential to change the zone of influence for high 6 energy arcing faults based on some testing data, I 7 think anything that would be done under there would be 8 considered a consensus method because it would be --
9 there's the project plan involves support from 10 Electric Power Research Institute, National Labs, NRC, 11 it's a broad technical community doing the work.
12 So, I don't --
13 MEMBER REMPE: So, it would be a new 14 method that's a consensus method is what you're 15 hoping?
16 MR. LINTHICUM: So, it'll be -- so, you 17 know, I look at the little bit in our presentation.
18 But the newly developed methods peer review process is 19 one way of getting acceptance but not the only way.
20 We still have the topical report or NUREG type 21 approach.
22 So, that's another way that you can have 23 a method accepted by the NRC. And, if it's accepted 24 by the NRC, then it's a consensus method and you don't 25 have to go through the separate peer review.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
84 1 So, my expectation is, the high energy 2 arcing fault would go through that process.
3 MS. ANDERSON: I mean, I don't think it 4 needs acceptance.
5 MR. LINTHICUM: It's --
6 MS. ANDERSON: But it would be -- it's the 7 broad technical community is involved.
8 MEMBER DIMITRIJEVIC: But we have another 9 thing today on the agenda in the afternoon which is 10 very good example that was there is a, you know, a 11 couple tests in NUREGs done on the leakage time on the 12 instrumentation of cable which are far from being 13 implemented in the PRA because you had to 14 differentiate between losing signal and getting false 15 signal.
16 So, that will be totally new method in the 17 current situation, only that it's cooled down a lot.
18 MS. ANDERSON: Right. So, it would be a 19 new method. But, again, that might be considered a 20 consensus method, depending on how many --
21 MEMBER DIMITRIJEVIC: Especially if 22 there's nothing existing at this moment.
23 MS. ANDERSON: Right.
24 MEMBER DIMITRIJEVIC: Okay. So, new 25 method, it has to replace something existing and in NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
85 1 this case it's the PRA.
2 MEMBER PETTI: So, can I ask a question?
3 I know everybody is aware of the ASME/ANS Standard.
4 For a facility like that's in the design phase like 5 NuScale, they've done a PRA as best as they can.
6 Does this help them at all? And, because 7 it seems like it's very focused on the industry, 8 utilities, you know, that really have a tremendous 9 amount of operating experience.
10 Is there something useful for the designs 11 that are coming in that don't have a lot of that 12 operating experience, even if it's, you know, the 13 intent of what's going on instead of the letter of the 14 law?
15 MS. ANDERSON: Yes. Potentially, it could 16 help with some of the new designs because if they are 17 going to be taking a new approach that's outside what 18 the technical community uses right now, they could 19 include that peer review as part of the peer reviews 20 they do to support their design certification.
21 MEMBER PETTI: I mean, I would imagine 22 even some of these advanced reactors would be using 23 different methods and approaches.
24 (OFF MICROPHONE COMMENTS) 25 MEMBER PETTI: Yes, yes, yes, right, yes.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
86 1 MR. LINTHICUM: Yes, but we did, yes, we 2 did have input from the advanced reactor community as 3 well.
4 MEMBER PETTI: Okay, you have?
5 MEMBER DIMITRIJEVIC: I am very interested 6 in a lot of related to that subject. So, we're going 7 to wait for a decision to come back.
8 MS. ANDERSON: Yes, I mean, I think it's 9 --
10 MEMBER DIMITRIJEVIC: Any of the new 11 reactors.
12 MS. ANDERSON: It's sufficiently generic 13 in technology neutral that this process could be used 14 for, I mean, design.
15 All right, so, I think we already got 16 through all of these key points that, yes, you can 17 review a newly developed method either in parallel 18 with or separately from implementation in a plant PRA.
19 As a matter of practice, we think for the 20 most part, you will see it done separately because 21 licensees want to make sure that it's a viable method 22 before they invest in putting into their PRA.
23 MEMBER DIMITRIJEVIC: Wait, wait, wait.
24 So, who does it? Licensee does it separately?
25 MS. ANDERSON: Well, the licensee has the NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
87 1 review for the implementation of the method done.
2 MEMBER DIMITRIJEVIC: And, who does it 3 separately?
4 MS. ANDERSON: And then, whoever owns the 5 method or whoever developed it, we call it the method 6 developer --
7 MEMBER DIMITRIJEVIC: I see.
8 MS. ANDERSON: -- they will be responsible 9 for getting the peer review done. So --
10 MEMBER DIMITRIJEVIC: So, does EPRI 11 factory develops method?
12 MS. ANDERSON: Well, for the most part, 13 EPRI methods would be consensus methods because they 14 work, again, with like National Labs and the broader 15 technical community.
16 But, for example, some of the -- we had 17 one method that was developed by the NEI fire PRA task 18 force.
19 MEMBER DIMITRIJEVIC: Okay, all right.
20 MS. ANDERSON: And, we piloted that and we 21 had consultants that did the peer review. And, I 22 think the Owners Group is going to talk about their 23 method developing experience. So, in their case, the 24 method developer was Westinghouse, the Owners Group.
25 MR. MARKLEY: Yes, it was developed for NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
88 1 the Owners Group from one specific vendor and we had 2 other vendors or members with expertise involved.
3 MS. ANDERSON: Right, yes.
4 I think one thing that's also important to 5 note is that a newly developed method with open 6 findings cannot be used in a PRA licensing 7 application. So, once those newly developed methods 8 review gets done, if there are open findings, a 9 licensee can't use that method in a PRA that supports 10 a licensing application.
11 MEMBER DIMITRIJEVIC: After those findings 12 are closed, right?
13 MS. ANDERSON: Yes, once those findings 14 are closed, it's all good to go, it can go to support 15 a licensing application. And, this is really --
16 MEMBER DIMITRIJEVIC: All right. So, now, 17 on the closure of those findings --
18 MS. ANDERSON: Yes --
19 MEMBER DIMITRIJEVIC: -- which is also 20 interesting question --
21 MS. ANDERSON: Yes, and we actually 22 conducted that as part of our pilot. So, we had one 23 method that had several findings I think on the order 24 of like 12 to 14. And, they conducted a closure 25 review of that.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
89 1 Actually, using the original team that 2 conducted the first review, so they were already 3 familiar with the method. They were familiar with 4 some of the gaps that needed to filled and --
5 MEMBER DIMITRIJEVIC: And, those are ones 6 the team identified those findings?
7 MS. ANDERSON: Right, yes. So, they 8 identified the findings and then, the method developer 9 was able to come back and explain how they addressed 10 all of them.
11 And, there was a lot of time saving there 12 because we didn't have to go over familiarity with the 13 method and all of that. And, it was a pretty 14 efficient process and it was also, again, very 15 rigorous, the method developer had to put a lot of 16 detail into how he addressed each of those findings.
17 So, it was a pretty successful process and I think we 18 got a good product out of it.
19 Okay, so I think we've covered everything 20 else there. Just a couple of other changes in 17-07.
21 As I mentioned earlier, we enhanced the discussion on 22 the concept of unreviewed versus not reviewed which is 23 sort of a fine point related to which supporting 24 requirements actually got reviewed or did not get 25 reviewed because there wasn't sufficient information.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
90 1 MR. MARKLEY: Can I offer a comment to the 2 Committee?
3 This is Mike Markley. I'm the Chief of 4 Licensing for Region 2 plants.
5 And, this is my opinion, I don't share the 6 views of the staff and NEI or the industry with regard 7 to the tech spec change on the last bullet on the 8 previous slide or Slide 13 on the previous 9 presentation where Mr. Weerakkody.
10 The tech spec is the requirement for them 11 to use NRC approved methods. And, just like your 12 Chapter 15 and issues with PRA and how we do tech 13 specs.
14 The tech specs are founded on using NRC 15 approved methods. I worry that we'll lose control of 16 the design on the licensing basis of the plant through 17 PRA if we don't have oversight of this piece in a very 18 strong manner. That's all.
19 MEMBER MARCH-LEUBA: At a minimum, now 20 that you bring that up, once you have a list of all 21 the approved methods, and I just heard that that's an 22 impossible task. I mean, it's worrisome.
23 I mean, it's not just that you don't do a 24 review of them or probably you have. But that you 25 cannot even list them?
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
91 1 (SIMULTANEOUS SPEAKING) 2 MS. ANDERSON: I want to address the 3 specific question with respect to what methods need to 4 be approved with respect to tech specs.
5 So, this goes back to 505 Risk-Informed 6 Completion Times, and there is the original safety 7 evaluation on NEI 06-09 which goes through the process 8 for risk-informed completion times --
9 MR. WEERAKKODY: If I may, you know, I'm 10 the lead staff member in NRC on this task. I just 11 want to emphasize that as the process goes on with 12 respect to the acceptability of the NEI proposal, we 13 are going to be considering all these including the 14 risk that Mike has expressed.
15 So, it will be part of our process, we are 16 going through that right now. There are some -- a 17 number of us who feel that the PRA report says once 18 you incorporate this original tech until it is 19 sufficient, but there are some of us who feel 20 differently.
21 We have Agency processes to appropriately 22 consider all of those things in making an informed 23 decision. Just wanted to share that.
24 MEMBER MARCH-LEUBA: When I am not reading 25 the submittals trying to prepare for this meeting, so NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
92 1 reviewing stuff and I'm doing computer programming.
2 And, I see it is so easy to when it's your own program 3 and you have to send it to somebody else to change the 4 subroutine, but I meant from an energy float, there 5 should be an intergen.
6 And then, two months later, you realize 7 that that just killed you. And, it's a pain in the 8 neck. I mean, you are used to us, a vehicle from NRR 9 and doing peer reviews -- not peer reviews, approvals 10 of topical reports and it takes 18, 24, 36 months to 11 get anything done, which is a little abusive and it's 12 wrong.
13 But if you change that process forces you 14 to be thorough and methodical and to documenting.
15 And, maybe we are going too much, too far. But if you 16 remove it completely, then you're losing that inertia 17 based on change.
18 MR. WEERAKKODY: Well, yes, I think that 19 I'll say this and I don't want to divert the 20 discussion, but one of the things we are considering 21 based on feedback we got from legal is that clearly 22 some type of threshold if all the criteria which if it 23 exceeded would require a particular method to come to 24 us for prior approval.
25 So, that is under consideration. We don't NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
93 1 know how we -- we haven't discussed how exactly to go 2 that line, but your point, like we take.
3 MEMBER MARCH-LEUBA: It forces, I think, 4 real -- I mean, let me use my example. The main value 5 that ACRS provides is the fact that you know that 6 you're coming here. We don't know anything, okay?
7 But just because you have to come and 8 confess to us, you're doing a good job.
9 MR. WEERAKKODY: Thanks.
10 MEMBER MARCH-LEUBA: And, the same thing 11 happens when you have to get a dash A on your report.
12 The review -- the staff review doesn't add anything to 13 value but they did a good job because they knew they 14 were coming here.
15 And, if you remove that, then people have 16 to be more thorough and more dedicated and there is 17 money pressures all the time.
18 MEMBER DIMITRIJEVIC: I have a question 19 because I want to understand. This sort of methods 20 change on tech specs, there was a matter related to 21 tech specs simulation or made to the changes PRA is 22 that you are going to tech specs?
23 MS. ANDERSON: Yes, it's -- so, what the 24 tech specs -- what the supporting documentation for 25 risk-informed tech specs referenced, when it said --
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
94 1 when that documentation said method, if you really dig 2 into what was written there, and that safety 3 evaluation, it's clear that what they meant was fire 4 PRA for modeling fire risk.
5 Seismic PRA for modeling seismic risk.
6 Seismic margins analysis for modeling seismic risk.
7 And so, we sort of went back and forth and 8 said, well, how do we make sure it's clear what's 9 meant? And, I think with the staff, we came to the 10 conclusion we needed to use the word approaches.
11 So, fire PRA is an approach to modeling 12 your fire risk. Seismic margins analysis is an 13 approach to modeling your seismic risk.
14 And, that's what needs to be reviewed and 15 approved, well, not approved by the staff, but 16 reviewed explicitly by the staff and approved for use 17 in risk-informed completion times. And, there was no 18 change to that.
19 We still --
20 MEMBER DIMITRIJEVIC: That's already 21 approved for using the PRA, right?
22 MS. ANDERSON: It's approved for use --
23 MEMBER DIMITRIJEVIC: I mean, let's say 24 that you have a --
25 MR. LINTHICUM: For that application.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
95 1 MS. ANDERSON: Right, for that 2 application.
3 MEMBER DIMITRIJEVIC: Yes, well, let's say 4 that you have a peer reviewed PRA with all elements, 5 fire, seismic, blah, blah, blah.
6 MS. ANDERSON: Right.
7 MEMBER DIMITRIJEVIC: And, somebody's 8 using it that's for tech specs, why would that be the 9 question?
10 MS. ANDERSON: It would have to be 11 something that the safety evaluation that that 12 licensee got from the NRC. It would have to 13 explicitly say, this program is conducted using fire 14 PRA, internal events PRA, and seismic PRA.
15 So, if I originally got my application 16 approved doing internal events and fire PRA, and then 17 seismic margins analysis and a seismic penalty factor 18 which several licensees have done, if I then developed 19 a seismic PRA and wanted to explicitly use that in my 20 risk-informed completion time program, I have to go 21 back to the NRC staff to get approved to use that 22 seismic PRA in my risk-informed completion time 23 program.
24 So, there's still --
25 CHAIR BLEY: And, that's for focused NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
96 1 review kind of thing you were talking about?
2 MS. ANDERSON: Yes, it would be a focused 3 review because they wouldn't be looking at your entire 4 program again. They would be looking to make sure 5 that that new approach, that new seismic PRA you 6 wanted to use was technically acceptable to support 7 your program.
8 MEMBER DIMITRIJEVIC: And, I just want to 9 add something for the people who doubt, let's look 10 what the tech specs debate. What's the tech spec, 11 let's say, I don't know, high pressure injection pump, 12 can we get out for two weeks based on what? Based on 13 the tech spec with existing, you know, what 14 Westinghouse first time --
15 MEMBER MARCH-LEUBA: If you want --
16 MEMBER DIMITRIJEVIC: Yes, it would -- it 17 doesn't have any basis and two weeks, why two weeks?
18 Why not three? Why not four? Nobody knows, but 19 suddenly, it's a Bible.
20 MEMBER MARCH-LEUBA: If you --
21 MEMBER DIMITRIJEVIC: And now, when we 22 want to introduce some risk inputs to that, people get 23 nervous. They should, just think of the origin of the 24 deterministic regulation and what that debate is 25 first.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
97 1 MEMBER MARCH-LEUBA: When, not if, when 2 you're reading the transcript and go back 12 pages, 3 you'll find out I said, this was a perfectly 4 acceptable application of PRA in my mind.
5 MEMBER DIMITRIJEVIC: Yes, I know. That's 6 what you said.
7 MEMBER MARCH-LEUBA: I agree with you.
8 MEMBER DIMITRIJEVIC: Wait until actually 9 just stated to something better or if we don't even 10 know what that was.
11 MEMBER MARCH-LEUBA: When you have a tech 12 spec that says three days, somebody pick out of the 13 air. And, probably was based on the fact that they 14 estimated it would take three days to fix the problem, 15 so let's give them three days. And, that's what it 16 came from.
17 MS. ANDERSON: The other rationale is that 18 it's one percent of a year, so it can't be that much 19 impact.
20 MEMBER MARCH-LEUBA: Maybe we'll use that, 21 we'll use that. You think probabilistic.
22 MEMBER DIMITRIJEVIC: And, they're doing 23 life extension, I was told, right? Whatever it is 24 that is -- which was based on the internal -- the 25 manual originally advisement in 1960, I mean, 40 years NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
98 1 of life and now we have a very scientific way to 2 extend it.
3 MS. ANDERSON: So, I think I've 4 essentially covered my conclusion slide. But I do 5 just want to underscore, we had a lot of stakeholder 6 interactions, several public meetings.
7 We completed three pilots of the newly 8 developed method process and NRC observed all three 9 with a rather large team at all three instances.
10 (OFF MICROPHONE COMMENTS) 11 MS. ANDERSON: Three different newly 12 developed methods, yes.
13 And, we revised NEI 17-07 to incorporate 14 lessons learned after each pilot.
15 MEMBER DIMITRIJEVIC: And, all those 16 lessons are fine?
17 MS. ANDERSON: They all had findings.
18 MR. LINTHICUM: They all had findings.
19 MS. ANDERSON: But they went through the 20 process okay.
21 MEMBER DIMITRIJEVIC: Okay. They're all 22 in the fire PRA?
23 MS. ANDERSON: No.
24 MR. LINTHICUM: Only one, only one.
25 MS. ANDERSON: Yes.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
99 1 So --
2 (OFF MICROPHONE COMMENTS) 3 MR. LINTHICUM: Yes, it would be located 4 --
5 MS. ANDERSON: Yes, it's in the next one.
6 So, at this point, we've done a lot of 7 revision of 17-07 to address NRC comments and right 8 now, we don't have any outstanding NRC comments to 9 address. If more come our way, we'll be happy to 10 address those.
11 And, with that, I think I am done and we 12 are ready to talk more about the --
13 MEMBER MARCH-LEUBA: Let me subject you to 14 a review.
15 MS. ANDERSON: Sure.
16 MEMBER MARCH-LEUBA: Can you go back to 17 Slide 7? This is just nitpicking. On the next to the 18 last sentence says, the NRC will endorse all of the 19 above.
20 Are you making up NRC's mind? Or it's a 21 fact?
22 MS. ANDERSON: Well, it could -- well, we 23 actually have seen a draft of 1.200 and it does 24 endorse all the above. And, when I say --
25 MEMBER MARCH-LEUBA: It has been, though?
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
100 1 MS. ANDERSON: No, it hasn't been endorsed 2 because 1.200 isn't final. When I say will endorse, 3 that could also include endorsement with exception.
4 MEMBER MARCH-LEUBA: Okay. An applicant 5 yesterday, I chastised them on their SER -- SAR for 6 using aspirational goals in their statements. So, you 7 really mean you have an agreement that they will do 8 it, right?
9 MS. ANDERSON: They will endorse it and if 10 there are exceptions, that's obviously the staff's 11 prerogative, but --
12 MEMBER MARCH-LEUBA: And, you have read 13 the draft and you are in violent agreement and --
14 MS. ANDERSON: Yes.
15 MEMBER MARCH-LEUBA: -- there is no 16 disagreement between the two of you?
17 MS. ANDERSON: I mean, because the 18 alternative would be that the NRC staff would have to 19 find some alternative document to endorse and there 20 isn't.
21 (SIMULTANEOUS SPEAKING) 22 MR. LINTHICUM: Yes, the industry has --
23 MS. ANDERSON: I guess they could write 24 their own, but I don't see that happening.
25 MEMBER MARCH-LEUBA: Well, we could keep NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
101 1 on behalf and we've kept the reactor safe for the last 2 50 years. It's very expensive, very cumbersome, but 3 it works.
4 MR. LINTHICUM: I mean, the industry has 5 provided comments on the draft. Not -- I wouldn't say 6 they're --
7 MS. ANDERSON: Groundbreaking.
8 MR. LINTHICUM: -- I mean, they are more 9 clarifications.
10 MEMBER MARCH-LEUBA: But the real way, are 11 you happy with the way the draft reads?
12 MR. LINTHICUM: Yes, I would say we expect 13 the staff to endorse.
14 MEMBER MARCH-LEUBA: Okay. I hate 15 aspirational goals.
16 MR. LINTHICUM: I understand.
17 MR. WEERAKKODY: If I may just for the 18 record, the original version of 17-07, we had about 70 19 comments.
20 MS. ANDERSON: It was more like a 107.
21 MR. WEERAKKODY: We had 90 public 22 comments.
23 MS. ANDERSON: To the person who addressed 24 them, it was more like 107.
25 MR. WEERAKKODY: But I want to very NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
102 1 clearly say what the staff will do. So, we have 2 gotten to a point where we have a version that we 3 think is reasonable and we said we have no comments.
4 But we recognize that as we go through the 5 public review process, we may get comments from the 6 public, you know, we may get comments from you which 7 will come back to think that, hey, you know, we need 8 some additional changes and we will comment them to 9 NEI.
10 And, in the end, our part is to endorse 11 17-07. It will be great if we can endorse it without 12 any exceptions, but if that becomes necessary, we will 13 do so.
14 MR. MARKLEY: This is Mike Markley, NRR 15 again.
16 I'd just like to remind you that these are 17 guidance documents that are full of shoulds and very 18 few shalls that you'll find anywhere. And the 19 requirement is in the tech spec.
20 MEMBER MARCH-LEUBA: Yes, we all say that, 21 but if they come in with something that's to satisfy 22 URG, you approve it. And so, very rarely disapprove 23 it. So, it is not that requirement, but this has 24 sufficient -- it's not the necessary but it has 25 sufficient condition.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
103 1 MS. ANDERSON: Okay, we're ready to turn 2 it over to --
3 MEMBER DIMITRIJEVIC: So, I hope you --
4 you have more slides or --
5 MS. ANDERSON: I don't have any more 6 slides, but the Owners Group does.
7 MEMBER DIMITRIJEVIC: Okay, all right.
8 MS. ANDERSON: Do you want me to pull up 9 the Owners Group presentation?
10 (OFF MICROPHONE COMMENTS) 11 MEMBER DIMITRIJEVIC: I think -- so, we 12 are running a little behind, surprise, surprise. So, 13 let's make a break now and come back in 25 minutes 14 before 11 and then we will go through the Owners Group 15 and then back to Dennis, so, 15 minutes break.
16 (Whereupon, the above-entitled matter went 17 off the record at 10:20 a.m. and resumed at 10:36 18 a.m.)
19 MEMBER DIMITRIJEVIC: Okay, we are on the 20 record.
21 MR. LINTHICUM: Okay, now I can start.
22 So, good morning. This is Roy Linthicum again from 23 the PWR Owners Group.
24 My part of this is going to be very brief 25 and then, I'm going to turn it over to Andrea Maioli NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
104 1 who, Andrea and Reed LaBarge were the primary authors 2 of our document, so they can answer any detailed 3 questions you may have.
4 MEMBER DIMITRIJEVIC: Okay.
5 MR. LINTHICUM: Next slide?
6 And, when I say very brief, I was going to 7 go over the background and purpose, but we've pretty 8 much discussed a lot of this.
9 I will say the important part to note from 10 my perspective is, we didn't actually go down this 11 path because the fire PRAs, we went down this path 12 because -- we started down this path because there is 13 a lot of disagreement between what constituted an 14 upgrade and a maintenance change, or the PRA model.
15 And, when I say differences, there are 16 differences in interpretation between the NRC staff 17 that were doing audits of submittals and the 18 licensees. There were differences between peer 19 reviewers through our licensee people and other 20 licensees.
21 And, when you look at the current version 22 of the ASME Standard, the -- what constitutes an 23 upgrade and what constitutes a maintenance update are 24 not mutually exclusive. So, that was a lot of the 25 problem. So, we felt they needed to be well defined.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
105 1 And then, as we saw, and it evolved into 2 the risk-informed completion time where we're having, 3 you know, the definition of method and acceptability 4 of methods and newly developed methods as part of a 5 licensing condition, we wanted to avoid the same types 6 of disagreements.
7 Especially as a licensing condition, the 8 industry just can't live with something that was not 9 well defined. That didn't serve our needs.
10 And, it actually worked well, it didn't 11 really serve the NRC needs either. So, we were able 12 to reach a mutually, I would say, agreeable position 13 that this is something that needed to be addressed.
14 MEMBER MARCH-LEUBA: If you look at the 15 example, and I would encourage you to talk to your 16 colleagues, of the computer codes that I use for, 17 let's call it Chapter 15, since she likes that 18 nomenclature.
19 We approve an issue with a dash A for a 20 topical report for a computer code. But then, the --
21 if, during the application of that particular revision 22 4.22a, the applicant finds a mistake on an output card 23 or a comma missing or even parameters of just 24 correlation having to be found incorrect, they are 25 allowed to change it without notifying anybody.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
106 1 And, you're supposed to keep your code 2 maintained. Now, you decide to change your CSF 3 correlation from this to that, I mean, a whole new 4 server thing. And, that is an update.
5 And, once you have the update, you are 6 required to do all of your QA, your full QA, you have 7 to run your 10,000 cases and verify they correlate and 8 work and all that.
9 So, there's a difference between 10 maintenance and update and I think you can use that --
11 MR. LINTHICUM: Right, and that concept is 12 factored into the decision. So, like I say, when we 13 say something's a newly developed method, it's a 14 compilation of all the inputs.
15 So, a correction of an error, something 16 along those lines where you're not fundamentally 17 changing the method would not be a newly developed 18 method so it would not have to go through this 19 process.
20 So, it would just be a revision, you know, 21 that would be issuing this.
22 MEMBER MARCH-LEUBA: I mean, is that well 23 understood? Because I tell you, for codes it's not, 24 it's done but it's not really what -- I've been on QA 25 audits.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
107 1 MS. ANDERSON: Thanks to this document it 2 is.
3 MEMBER MARCH-LEUBA: On QA -- I mean, are 4 we being clear on this on the -- on a vendor's place 5 where we claim them because they have corrected some 6 output cards on their card but that's not the 7 approval.
8 So, there has to be a well understanding, 9 be well defined. You can get in trouble on it.
10 MR. LINTHICUM: Right, right. And, that's 11 why we actually went through and actually made some 12 definitions and created definitions to support the 13 method.
14 And, the last thing I'll say really before 15 I turn it over to Andrea is this -- our document is a 16 PWR Owners Group document but we had inputs from a lot 17 of stakeholders, including the BWR Owners Group, NEI, 18 the Joint Committee on Nuclear Risk Management that 19 owns the PRA Standard, the NRC, and even had some 20 advanced reactor input as well.
21 So, we did address a large number of 22 stakeholder comments and inputs into the final 23 process. We do recognize that, you know, as this gets 24 published in the Federal Register through Reg Guide 25 1.200, we may have to address some additional NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
108 1 comments. But we'll deal with those, you know, as any 2 comments come forward.
3 CHAIR BLEY: Ray, did any of those non-PWR 4 groups of potential vendors get involved with this 5 with you?
6 MR. LINTHICUM: I don't -- there was one 7 specific vendor that did, but I don't feel comfortable 8 mentioning them by name.
9 CHAIR BLEY: That's fine, but only one?
10 MR. LINTHICUM: But only one, yes.
11 CHAIR BLEY: Don't know if they knew what 12 was going on or --
13 MR. LINTHICUM: No, they did.
14 CHAIR BLEY: Oh, they did?
15 MR. LINTHICUM: They did.
16 CHAIR BLEY: They did.
17 MR. LINTHICUM: Yes.
18 CHAIR BLEY: They didn't really --
19 MR. LINTHICUM: Yes, but only one actually 20 wanted to actually engage.
21 CHAIR BLEY: Interesting. Okay.
22 MR. MAIOLI: I would say through JCNRM, 23 though, the advanced reactor is represented. The 24 advanced reactor community is represented with 25 multiple vendors there. And, they have all been kept NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
109 1 in -- up to speed to what this process was about and 2 some of those participated as well through the 3 workshops.
4 MS. ANDERSON: And also, through NEI as 5 those vendors are all NEI members. So, NEI did 6 consult with them on a lot of these updates.
7 CHAIR BLEY: Thanks.
8 MR. LINTHICUM: Okay, Andrea?
9 MR. MAIOLI: Okay, so, good morning. My 10 name is Andrea Maioli. I'm with Westinghouse. I am 11 supporting the Owners Group for this activities with 12 LaBarge. We are supporting this project.
13 As Roy mentioned, I'm also one of -- well, 14 both Reed and myself also one of the peer review leads 15 for the Owners Group and we have supported a number of 16 peer reviews and both involved also, I think, 17 actually, everybody at this table is actually a JCNRM 18 member supporting the evolution of the PRA Standard.
19 So, PWROG-19027 is really the document 20 where we are documenting the requirements that are 21 hopefully, likely, potentially, being endorsed by the 22 NRC and the Reg Guide 1.200 Revision 3.
23 Victoria mentioned before that, this PRA 24 Standard was used and an important role in putting 25 more structure in the review of the PRAs and, of NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
110 1 course, also in suggesting how the PRA should be 2 developed.
3 I think the addition of the newly 4 developed method, the technical element that is 5 documented in these documents and this suggested or 6 recommended to be included in the next edition of the 7 JCNRM Standard is an evolution of this.
8 The standard always adds short statements 9 saying, if there is a new method, that was -- didn't 10 go through a peer review, it is up to the peer review 11 team to assess the technical adequacy of that method.
12 And, that was the only statement, there 13 was no structure. So, when you go through the peer 14 review, you find a new method that is used for 15 anything, for flood calculation, for seismic 16 fragility. And, if you find something new, that would 17 be up to the peer review team to do the PRA review.
18 Well, this process and these supporting 19 requirements put structure in that review as well as 20 all the standards had done in the previous years for 21 all the other elements of the standard.
22 We have talked about definitions before 23 going to the actual supporting requirements. These 24 are the six key definitions that were either 25 introduced or changed from what they are currently, NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
111 1 documented in the current verison of the standard or 2 Reg Guide 1.200.
3 And, it's important to look at these 4 definitions because the newly developed method section 5 of the standard and this document is a part of the 6 configuration control process of the PRA.
7 A lot of the questions that came out today 8 were how things changes, if there is an error on a 9 method, for example, well, that's captured in the PRA 10 configuration control which has its own set of 11 requirements, its own set of items that would need to 12 be looked at when a peer review is done for the PRA.
13 Newly developed method is another element 14 in the PRA configuration control. When you have an 15 upgrade, due to the fact that you are introducing a 16 new method in your PRA, but this new method is also 17 newly developed for the industry.
18 So, of course, newly developed method, the 19 key definition then all the supporting definitions as 20 matter to trying to, as we discussed before, trying to 21 put a box around a method for a -- and there may be a 22 lot of them in a PRA.
23 And, what the stated practice is. We 24 talked before, it's not the intent of this document or 25 of the process to go back and re-peer review through NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
112 1 the newly developed method full tree analysis method.
2 That's sort of state of practice and it should be --
3 if you want to grandfather in the process.
4 And then, the definition of consensus 5 method and model which is based on NRC approval or 6 usage in an application.
7 And then, a clarification on the PRA 8 upgrade and maintenance review, removing that overlap 9 in the definition, that Roy was talking about. So, 10 that both the licensee and the peer reviewer are 11 looking at a change in the PRA may have an easier life 12 in identifying whether that's an upgrade, a better 13 process, re-peer review or its maintenance that 14 doesn't require a peer review.
15 And, the flowchart that was mentioned at 16 the beginning really goes through the configuration 17 control process and where the newly developed method 18 place in the -- play in the configuration control 19 process.
20 So, if you are familiar with --
21 (OFF MICROPHONE COMMENTS) 22 MR. MAIOLI: Oh, you have the printout.
23 (OFF MICROPHONE COMMENTS) 24 MR. MAIOLI: So, the newly developed 25 method is written in the same format of every other NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
113 1 technical element and every other part in the 2 standard. So, technical elements, another technical 3 element, for example, may be initiating events. So, 4 initiating events analysis is one technical element in 5 the standard.
6 So, the newly developed method is another 7 technical element in the standard. Sunil had a 8 summary of how the standard is structured with a high 9 level shower requirement and then supporting technical 10 element.
11 So, the newly developed method --
12 technical element has this six high level requirements 13 that puts every subject for a sort of structure in the 14 review of the method.
15 CHAIR BLEY: A quick question for 16 Victoria. Is the new NEI document on peer review, 17 does it call out the Owners Group report as for the 18 requirements?
19 MS. ANDERSON: Yes.
20 CHAIR BLEY: Okay. And, eventually, we 21 hope that will change and would --
22 MS. ANDERSON: Right.
23 CHAIR BLEY: -- be a part of it?
24 MS. ANDERSON: It'll just be a part of the 25 standard and then we can just strike that when we NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
114 1 revise 17-07.
2 MR. MAIOLI: So, the six key high level 3 requirements --
4 CHAIR BLEY: I'm sorry to interrupt you.
5 I'm just remembering back to all the elements of the 6 standard and how many years it took to beat agreement 7 out of everyone involved to publish them.
8 Has this gone through the group that will 9 be approving the standard eventually? They're on 10 board with it?
11 MR. MAIOLI: Is it going through? We have 12 provided this in draft form to JCNRM up to the ballot 13 for the next version of the standard just it's about 14 to close. And, it includes --
15 CHAIR BLEY: This?
16 MR. MAIOLI: -- this.
17 CHAIR BLEY: That's right, okay.
18 MR. MAIOLI: JCNRM provided it --
19 (SIMULTANEOUS SPEAKING) 20 MR. MAIOLI: JCNRM provided some initial 21 feedback which was the reason for the most recent 22 update of the document in December, included some 23 feedback from JCNRM.
24 So, JCNRM will go through their own 25 consensus process through the ballot with comments and NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
115 1 address those. But it's done in collaboration.
2 MEMBER DIMITRIJEVIC: But let me 3 understand how that will work. So, you have a peer 4 review team which is reviewing, let's say, I don't 5 know, for this we'll say the initiating event.
6 And, then, there is a new method in 7 initiating events, let's say. So, then what happens?
8 This is going to be on end of all, you know, high 9 level requirements.
10 And then, so, there is a Person A 11 reviewing the initiating event and then, what, there 12 is a Person A1 reviewing the method?
13 MR. MAIOLI: It is possible.
14 MEMBER DIMITRIJEVIC: Or how do they know 15 that there is new method?
16 MR. MAIOLI: So, 17-07 identified two 17 alternatives. You can do a dedicated peer review of 18 the method itself outside of its application.
19 Sometimes it's possible, sometimes it's maybe more 20 challenging, depending on the method.
21 But and a lot of utilities have told, 22 well, I'm not going to use in my PRA a method that has 23 not been gone through this process.
24 If a method is peer reviewed along with 25 this application, there will likely be dedicated NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
116 1 people in the team looking at the method which may or 2 may not be the same people also looking at how the 3 method is implemented.
4 It is an additional scope to a peer 5 review. So --
6 MEMBER DIMITRIJEVIC: So, they should 7 define that before asking for -- so it would not be 8 standard peer review team, you may have to have 9 experts?
10 MS. ANDERSON: Yes, I --
11 MR. MAIOLI: If we have identified that 12 the scope included a newly developed method, or the 13 alternative is that during the review and that you 14 identify the newly developed method, then the review 15 may be stopped and the scope and the other team.
16 That's part of how we normally do the process.
17 MR. LINTHICUM: And, this is -- let me --
18 I mean, this is Roy Linthicum.
19 So, we try and avoid those challenges mid 20 review. So, we do ask the utilities to identify any 21 change in methods or any new methods that they've used 22 in their models so we make sure we have the right 23 review team going in.
24 Now, sometimes you do get surprised at 25 what they consider a new method. It might not be what NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
117 1 we do, so you may get surprised by that and that's 2 where you would potentially run into a situation where 3 the review team would say, we don't have the right 4 people. So, this part we can't review. You'll have 5 to schedule a, you know, a follow up focused peer 6 review.
7 MS. ANDERSON: Yes.
8 MR. LINTHICUM: If we don't have the right 9 expertise.
10 MS. ANDERSON: And, that gets documented 11 in the peer review report. And, we'll, essentially, 12 this portion of the PRA and these high level 13 requirements were not reviewed.
14 CHAIR BLEY: It's a finding?
15 MS. ANDERSON: It's a type of fact 16 observation.
17 MR. LINTHICUM: It's a type of -- it's a 18 fact --
19 MS. ANDERSON: It gets documented, it's 20 something that the staff would see in the licensing 21 application.
22 MR. LINTHICUM: The application, right.
23 MS. ANDERSON: And, I think is the 24 important thing.
25 MEMBER DIMITRIJEVIC: So, all your peer NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
118 1 review teams will have to be very knowledgeable about 2 this.
3 MR. MAIOLI: All the peer review teams go 4 through --
5 MEMBER DIMITRIJEVIC: Definition of that.
6 MR. MAIOLI: -- go through --
7 MEMBER DIMITRIJEVIC: What's your 8 definitions are.
9 MR. MAIOLI: Right. All the peer review 10 teams go through refresh on the standard, on the 11 process and we are going to include this as part of 12 the training before ever peer review. That happens 13 before they kick off, before any material is made 14 available, all the peer review teams go through that 15 training.
16 MEMBER DIMITRIJEVIC: Okay, thank you.
17 MR. MAIOLI: So, at the high level, I 18 mean, I'm not going through the details here because 19 it's hard to read and it's in the report.
20 But the six elements here within this 21 document requirement, six high level requirements have 22 to do with purpose and scope. It's the first one.
23 The second one is essentially detecting 24 all bases.
25 The third one is on the data used, how NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
119 1 it's used, where it's -- well, the data sources, how 2 it's manipulated.
3 There is a dedicated high level 4 requirement on uncertainties.
5 And then, a high level requirement on 6 their other results or should expect to fit with the 7 scope and the end scope of the method.
8 CHAIR BLEY: I'm pleased that in both the 9 high level requirements and the lower level 10 requirements you don't make a distinction between the 11 two capability categories. The same thing applies at 12 --
13 MR. MAIOLI: Right, right.
14 MR. LINTHICUM: Right.
15 CHAIR BLEY: It's about time.
16 MR. MAIOLI: yes, we kept the structure 17 because we provide it as a plug-in for the standard 18 with all the capability being the same. But there is 19 no differentiation in capability category for the NDM 20 technical elements.
21 The last one is on documentation. The 22 last high level requirement is on documentation with 23 two focuses. One, the same focus that every other 24 technical element has which is to provide trustability 25 of the work.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
120 1 But the other one here, it's very specific 2 to what the differentiation between reviewing the 3 method and reviewing it's application. So, making 4 sure that documentation is clear on how a newly 5 developed needs to be implemented in the PRA.
6 And this provides the structure for the 7 review.
8 MEMBER DIMITRIJEVIC: And, the that was 9 reviewed by NRC? Is that method if those are reviewed 10 by the NRC?
11 MR. MAIOLI: If the method was reviewed by 12 the NRC, it's a different part where it's the, let's 13 say, the normal submittal as a topical of the method 14 to the NRC. These requirements are not applicable in 15 that case. These are what the industry looks at.
16 MEMBER DIMITRIJEVIC: So, wait a second.
17 So, a new method, where is the definition of the new 18 method, does that involve if there's no review but --
19 CHAIR BLEY: It's this flowchart.
20 MR. MAIOLI: The consensus method and 21 model? So, if a method is a consensus method, it 22 means it's approved by the NRC for use.
23 MEMBER DIMITRIJEVIC: Then -- okay, so, 24 it's not --
25 MR. MAIOLI: Then it doesn't go through NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
121 1 this process. It's one way in that flowchart to say, 2 well, this is good. There is no need for a review of 3 the technical adequacy of the method.
4 CHAIR BLEY: I have, yes, you're not going 5 through the details of the lower level requirements.
6 I'm pleased to see that you have one on uncertainties 7 and that's pretty well through the standard.
8 I'm uncomfortable in your later lower 9 level requirement to ensure uncertainties do not 10 preclude meaningful use of the newly developed method 11 results. I rather wish you had said, make sure you 12 present the results including uncertainties in a 13 meaningful way.
14 This looks like a way for a people to duck 15 out of doing the uncertainties because, oh my God, 16 nobody could understand it.
17 MR. MAIOLI: There is a lot of 18 wordsmithing in the developing supporting requirements 19 of the standard and, here, it was really not -- the 20 standard comment in here was really not different.
21 The goal was definitely not to let out 22 people from looking at uncertainties. It was to make 23 sure that the uncertainties were addressed and the 24 results were still applicable.
25 CHAIR BLEY: Okay, okay.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
122 1 MR. MAIOLI: We went through --
2 CHAIR BLEY: We got what you're saying, 3 yes.
4 MR. MAIOLI: Yes, I understand.
5 CHAIR BLEY: The one before that, since 6 all new methods are expected to be improvements, at 7 least that's what I heard earlier today, having the 8 secondary requirement that we should compare the 9 results with newly developed methods without some 10 explanation.
11 It worries me, again, a little because it 12 could lead to a spot like Jose was describing where 13 NRC says, my God, they're different. And, yes, 14 they're different on purpose because we're now 15 addressing something we weren't addressing before.
16 It's just worrisome.
17 MR. MAIOLI: Right.
18 MS. ANDERSON: Yes, I think what's -- it 19 meant there is that if it has -- it says identify the 20 causes and I think the idea is that you want to 21 understand why you get different results.
22 CHAIR BLEY: Good. I hope it doesn't 23 backfire.
24 MR. LINTHICUM: Well, yes. We did -- I 25 mean, we tried to choose the words to say what we NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
123 1 didn't want to happen was to say, you've got the 2 results that you expected to get when you developed 3 the method because you're unexpected results might be 4 real.
5 CHAIR BLEY: Yes.
6 MR. LINTHICUM: So, just because you 7 didn't get what you expected doesn't mean it's not a 8 valid method. So, but you need to understand why 9 there's a difference and being able to point to the 10 fact of the reason was the intent of that supporting 11 requirement.
12 CHAIR BLEY: Let's hope it works that way 13 in practice.
14 MR. LINTHICUM: Well, it has to be. So, 15 the three we've done.
16 MR. MAIOLI: I presented a few like the 17 underlying supporting requirements only for the first 18 one and just for awareness.
19 This follows, again, the same structure of 20 the supporting requirement for the rest of the 21 standard.
22 So, they are written in, of course, a 23 generic fashion because they are not specific to a 24 method. If you want the challenge of this specific 25 technical element that needs to be wide enough to NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
124 1 accommodate different methods.
2 It could be a fire method, a seismic 3 method, a method dedicated to data analysis, a method 4 dedicated to fragility, a method dedicated to 5 different elements. So, it needs to be generic 6 enough.
7 It's also not prescriptive of how 8 something needs to be addressed like all the other 9 supporting requirements in the standard, it's telling 10 what is the topic, what is the thing that needs to be 11 addressed, but not how to address it. And, that also 12 needs to provide that same level of flexibility like 13 everything else in the standard.
14 So, and, I think the example that was 15 raised before on uncertainties is a very good example.
16 It also needs to not be open ended. So, as you were 17 pointing out before, compare maybe open ended, but 18 then identify where you -- try to understand the 19 differences. It's trying to close the loop on that.
20 So, those are the supporting requirements 21 for the first technical element.
22 Next few slides on the actual pilots that 23 we went through and a few words on the field review 24 report.
25 The same process is essentially used for NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
125 1 the peer reviews with specific differences and 2 specific caveats that are included in 17-07.
3 The report is also very similar. The peer 4 review reports, an NDM peer report is very similar in 5 structure to a peer review report that is provided for 6 PRA for implementation.
7 The main difference, if you want, is the 8 addition of this non-proprietary appendix at the very 9 end that you need to understand the context of the 10 process.
11 So, a method developer developed the 12 methods. They want to go through peer review to, 13 let's say, bless the method, stamp it with the NDM 14 peer review. The method may be proprietary, may have 15 some proprietary information in that.
16 So, the way we work this out was there 17 would be a non-proprietary appendix or a self-18 sustained document that summarized the review. That 19 would be non-proprietary and that can be made public.
20 It can go in ADAMS, it can go in some other structure 21 that are public.
22 So, the then plant implementing that 23 method can call it and reference it and close the loop 24 in that way.
25 CHAIR BLEY: Have you -- you haven't done NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
126 1 much of this yet. Have you run into any, I guess what 2 I'm hanging on the whole PRA there on each PRA is on 3 a specific design, plant specific basis. Some of that 4 is proprietary.
5 MR. MAIOLI: Correct.
6 CHAIR BLEY: Here, we talk about 7 proprietary methods. I don't remember in the standard 8 if there's any mention of proprietary --
9 MR. MAIOLI: The standard does not care if 10 you want the method is or something is proprietary or 11 not.
12 CHAIR BLEY: So, have you run into any --
13 MR. MAIOLI: But the process --
14 CHAIR BLEY: -- problems with reviewer --
15 getting reviewers who are acceptable to the people who 16 own the proprietary information?
17 MR. MAIOLI: We face that situation 18 multiple times and I think every time it was -- what 19 we found the solution, sometimes -- so, realize that 20 the peer review process takes five weeks.
21 CHAIR BLEY: Yes.
22 MR. MAIOLI: The week on site, the full 23 week before. Normally, the material is made available 24 to the reviewers maybe on a SharePoint or something 25 like that.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
127 1 When there is some proprietary 2 information, maybe that proprietary information, if 3 they -- if the vendor or the utility is not 4 comfortable in posting it, it's only made available 5 during the week of the on site review.
6 CHAIR BLEY: On site?
7 MR. MAIOLI: It's only printed. But --
8 CHAIR BLEY: It hasn't been an issue?
9 You've been able to deal with it?
10 MR. MAIOLI: Yes, we were able to deal 11 with that successfully every time. And, there have 12 been cases like that that I've observed.
13 MEMBER DIMITRIJEVIC: So, how many slides 14 you still have planning to present?
15 MR. MAIOLI: Two or three more slides.
16 MEMBER DIMITRIJEVIC: Okay. Because now 17 we are getting a little concerned because we need to 18 leave the and now so we should speed it up.
19 MR. MAIOLI: Yes, so, I'll not go to much 20 more through in detail on the peer review report.
21 MEMBER DIMITRIJEVIC: No, no, that doesn't 22 mean maybe only like several comments.
23 MR. MAIOLI: But maybe it's worthwhile to 24 just spend a few words on the three pilots.
25 MEMBER DIMITRIJEVIC: Okay, no, no, that NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
128 1 is fine.
2 MR. MAIOLI: These are the three methods 3 that have been piloted, different kind of methods.
4 Two methods from the Owners Group, one method on the 5 review was managed by NEI.
6 So, topic here would be the emergency 7 diesel generator failure data or refined room cooling 8 effect screening and modeling or the fire in cabinets 9 method that was.
10 MEMBER DIMITRIJEVIC: Very nice, very 11 interesting. Okay.
12 MR. MAIOLI: There are -- there were three 13 dedicated teams. We decided to use methods that were 14 relatively simple. We didn't want to challenge the 15 overall process with a method that was contentious for 16 some reason, just to make sure the process worked.
17 MEMBER DIMITRIJEVIC: So, this is 18 different failure data, it wasn't just changing the 19 data, it was changing methods?
20 MR. MAIOLI: It was changing the way --
21 MR. LINTHICUM: Depending on how you 22 analyze it.
23 MR. MAIOLI: -- data is looked at to 24 generate failure rates. It was not just changing --
25 swapping data from two different references, but re-NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
129 1 looking at how the data is characterized or the events 2 are characterized to generate data that it's used in 3 the PRA.
4 So, it's not only like I'm using Reference 5 A, now I'm using Reference B and this newly developed 6 method, it's Reference B is massaging the same data in 7 different way and generating different failure rates.
8 So, yes, I'm changing the number but there is 9 something underneath.
10 CHAIR BLEY: I have a question about that 11 one because I'm a little fuzzy.
12 Some years ago, the NRC published its, it 13 started to call it the Data Handbook, it was 14 eventually called, I forget the exact name.
15 Were there real methods here that weren't 16 somehow included in that reference document?
17 MR. LINTHICUM: Yes. So, the answer is 18 yes. And, as the Owners Group, we've actually had 19 separate meetings with NRC to research on this 20 approach and those discussions area ongoing as well.
21 So, as a result of this, this may actually 22 end up being more of a consensus method if staff 23 accepts our approach and changes the way they look at 24 it and how they publish the underlying formula data.
25 CHAIR BLEY: Well, you've gotten my NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
130 1 interest, I may have to delve into that.
2 MR. LINTHICUM: So, I mean, and in the 3 interest of time, though, I think that I'll just 4 ahead. But so, the real result is we did three 5 pilots, the result of the pilots, we did make 6 revisions to the peer review criteria.
7 Those have been incorporated and then 8 revised as a result of JCNRM input and other inputs.
9 And, that's where we're at today.
10 So, we piloted successfully. We learned 11 lessons learned. And, that is all culminated in our 12 final report.
13 With that, I think we can just open up for 14 questions.
15 MEMBER DIMITRIJEVIC: You changed your 16 criteria based on this?
17 MR. LINTHICUM: Yes, we did some 18 clarifications, some reordering based upon the lessons 19 learned from the first couple peer reviews.
20 MR. MAIOLI: Yes, if you look at the, for 21 example, this slide as some feedback on the newly 22 developed method, number two, if you look at the total 23 number, it's 20 SRs.
24 If you look at the next one, it's 27.
25 And, if you --
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
131 1 MEMBER DIMITRIJEVIC: Twenty-seven?
2 MR. MAIOLI: And, if read where the 3 presentation is 21. The reason is, we started 20 4 looking at our -- the scope, the intent.
5 As a feedback from the first two reviews, 6 we split apart some of the SRs. Some were too big, 7 some needed to be refined and we ended up with 27.
8 And, actually the JCNRM then helped us 9 saying, well, these two are actually redundant, this 10 may need -- you may want to merge that in a different 11 way and we ended up with 21, so back closer to where 12 we were.
13 CHAIR BLEY: I read through them and they 14 seem pretty straightforward. Do you have any concern 15 that, in a year, you're going to really have to revise 16 these extensively?
17 MR. MAIOLI: I hope not extensively. I 18 don't think extensively. We realize the standard has 19 been around for so many years and SRs are continuously 20 tweaked. So, I wouldn't be surprised if a word or two 21 changes, but I think the concept is there and the 22 majority of the wholes are there.
23 And, we put a lot of thought in the action 24 word and made sure that it's consistent what we wanted 25 the what to be. What the reviewer looks at.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
132 1 CHAIR BLEY: When do you expect the 2 standards groups, and you guys are probably on it now 3 --
4 MR. MAIOLI: Right.
5 CHAIR BLEY: -- to actually take this up 6 and maybe incorporate it?
7 MR. MAIOLI: So, this19-027 has been 8 provided to JCNRM. It has been included in the 9 current ballot.
10 CHAIR BLEY: Oh, they're voting on it?
11 MR. MAIOLI: Yes, which were -- yes. So, 12 standard time is realistic, but --
13 CHAIR BLEY: That's good, I understand.
14 MR. MAIOLI: -- it should be there.
15 CHAIR BLEY: This year or five years from 16 now.
17 MR. LINTHICUM: Or five years, right.
18 (SIMULTANEOUS SPEAKING) 19 MR. MAIOLI: What you want to takeaway is 20 that we take relatively straightforward method, if you 21 want. We didn't want to change the process. But it 22 was very detailed. Every method came up with some not 23 mets and the number of F&Os, some of those were 24 documentation, some of those were identifying better 25 the scope, some of those was challenges, some of the NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
133 1 technical basis or the data used.
2 So, I think there was a lot of feedback 3 provided. One of the method closed all the F&Os, the 4 other two methods, the older one that used previous 5 version of the SRs will be re-peer reviewed with the 6 same the most updated versions to clean them up.
7 And, that's the current plan. So, there 8 are more details but in the interest of time, unless 9 you have questions, we can --
10 MEMBER DIMITRIJEVIC: Yes, too bad, 11 actually. I wish we had more time. Well, time is 12 clearly is about and I may ask that even they come 13 back on.
14 Let's save them I know the respect of like 15 was the room cooling identifies some issue which a 16 problem could exist in many utilities. Would that may 17 have a raise to the level of the, you know, some 18 generic issue and what will be done in that case? I 19 will ask that they come back, it's not.
20 But I mean, just see --
21 MR. MAIOLI: That would be part of the 22 configuration control portion of a PRA. There is the 23 expectation that the utility is looking out for 24 updates or use of this time, hey, there is a method 25 that was used before, now it's wrong, which was not NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
134 1 the case here. This was like an evolution of the 2 method, actually, two methods put together.
3 But that would be captured through the 4 configuration control process of the PRA. So, the 5 utility will see the information and would need to act 6 on that information if there is ay.
7 MEMBER DIMITRIJEVIC: So, I think I 8 expressed my concern about like, for instance, I 9 always was concerned about this room heat up, 10 especially in the case of fire which is always in back 11 that just heat up, you know, when the ventilation gets 12 closed and you have a fire, it's not just, you know, 13 what is above cabinet and things like that.
14 So, but then that's not the case here. I 15 understand. But there may be a case that somebody 16 goes and looks at that and sees that that's a problem.
17 That would be problem everywhere, but they're doing 18 the -- this is not a consensus issue, it's done as a 19 part of that peer review.
20 They finish that peer review, they peer 21 reviewed everything is fine. But the industry doesn't 22 know about what they discovered by the new method.
23 MS. ANDERSON: Yes, well, I think -- so 24 the question you're asking is, if in developing a new 25 method, it's discovered that what we currently do is NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
135 1 substantially insufficient enough that we have safety 2 vulnerabilities we're not aware of?
3 MEMBER DIMITRIJEVIC: Yes, yes.
4 MS. ANDERSON: And, I think what would 5 happen in that case, you know, the newly developed 6 method we talked about that non-proprietary appendix, 7 that gets provided to the NRC for information.
8 And, when we talked about that SR about 9 comparing your previous methods and why you have the 10 differences would state that.
11 So, I think the NRC staff would be made 12 aware. The industry is very good about sharing OE 13 with each other. So, I think informally people would 14 definitely raise that with each other. You know, 15 staff would become aware and then if it were 16 significant enough, yes, that be an issue.
17 MR. LINTHICUM: Yes, and both Owners 18 Groups have processes where if there is a significant 19 industry issue that we identified through what we have 20 processes to formally make that available to all of 21 our utilities and even to the NRC if needed if we know 22 that the NRC is relying on some information that we 23 now know may be insufficient.
24 So, we have other processes that would 25 cover that.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
136 1 MS. ANDERSON: Yes.
2 MEMBER DIMITRIJEVIC: So, they would 3 benefit from like different interpretation of this 4 generated data. You know, I mean, I would just wonder 5 what's the way to industry to share this if it's --
6 MR. WEERAKKODY: This is Sunil Weerakkody.
7 We will give you some information with 8 respect to how we have put some checks and balances in 9 place to catch and react to situations like that.
10 MEMBER DIMITRIJEVIC: All right, okay.
11 Thank you.
12 MR. LINTHICUM: Thank you.
13 MR. MAIOLI: Thank you.
14 MEMBER DIMITRIJEVIC: Thank you very much, 15 it was too bad that the issue of the time.
16 MR. GILBERTSON: So, good morning, 17 Subcommittee Members, my name is Andres Gilbertson.
18 I'm a reliability and risk analyst in the Office of 19 Nuclear Regulatory Research.
20 Mehdi Reisi Fard is a reliability and risk 21 analyst in the Office of Nuclear Reactor Regulation.
22 This morning, we will be presenting --
23 continuing our presentation to give you just a summary 24 of some of the changes that are being proposed for the 25 next revision of Reg Guide 1.200.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
137 1 I'm going to just go over some general 2 overview and then Mehdi is going to talk more about 3 the external stakeholder engagement and how we've been 4 considering feedback externally, and, again, also 5 internally as well.
6 So, first, you know, we're going to have 7 -- actually, if you can go to the next slide?
8 I just wanted to take a step back and just 9 give you sort of the higher view of the plans for Reg 10 Guide 1.200. We are, obviously, we're working on 11 Revision 3 currently. That is active in progress.
12 Revision 4, we are looking forward and 13 anticipating endorsement of the three standards listed 14 there. So, as has been mentioned previously, the next 15 edition of the ASME/ANS Level 1 PRA or Level 1 LWR PRA 16 Standard is under ballot right now. It's in the 17 process. So, you know, perhaps sometime before the 18 end of this calendar year, that may be published as an 19 ANSI Standard.
20 Potentially, similarly, with the Level 2 21 PRA Standard, and then also the LWR PRA Standard we 22 expect to include in Revision 4.
23 So, our schedule, you know, is dependent 24 on the Standards Development Organizations, in this 25 case, the Joint Committee on Nuclear Risk Management NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
138 1 which is ASME/ANS.
2 And so, we haven't set out a firm schedule 3 for Revision 4 yet, but it will be considering their 4 schedule as well.
5 And then, I also wanted to just point out 6 that the advanced non-LWR PRA Standard which is being 7 developed by the JCNRM, that is going to be endorsed 8 in a new regulatory guidance document and we have a 9 separate effort to address that, the review and 10 endorsement of that document.
11 Next slide, please?
12 Okay, I will hand it over to Mehdi.
13 DR. REISI FARD: Good morning, 14 Subcommittee Members. My name is Mehdi Reisi Fard.
15 I'm a reliability and risk analyst in the Office of 16 Nuclear Reactor Regulation Division of Risk 17 Assessment.
18 The purpose of this portion of the 19 presentation is to go over the NRC review of the 20 overall framework for peer reviewing newly developed 21 methods. That includes the reviewing the 22 requirements, the peer review process, and all the 23 associated definitions that you've seen so far.
24 As a part of that, I'll briefly discuss 25 some of our observations from pilot peer reviews.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
139 1 And, at the end, I'm going to switch gears to some --
2 a couple of other important clarifications that we 3 made in Reg Guide 1.200, not directly related to newly 4 developed methods on the definition of PRA upgrade and 5 also addressing key assumptions in risk-informed 6 applications.
7 Next slide, please?
8 Let's start with the discussion on newly 9 developed methods. NRC staff developed a set of 10 criteria for peer reviewing newly developed methods 11 about two years ago.
12 Around the same time, PWR Owners Group 13 started a series of workshops to refine and start to 14 develop those criteria.
15 And, NEI also consolidated all the peer 16 review guidance documents for fire, external events, 17 and internal events into one document, NEI 17-07.
18 And, that consolidated guidance also includes the peer 19 review for newly developed methods.
20 Once we determined that the PWR Owners 21 Group criteria and NEI 17-07 were ready, we conducted 22 -- the industry conducted three pilot peer reviews of 23 newly developed methods. We observed them, I'll 24 discuss them later. And, as a result of the comments 25 that we provided and the comments that -- a large NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
140 1 number of comments that also peer review team members 2 identified, both PWR Owners Group documents criteria 3 and documents as well as the NEI 17-07 were revised 4 which led us to issuing Draft Guide 1362. At this 5 time with no exceptions or clarifications.
6 As Sunil explained, as we go through the 7 process, we may come across new issues and we'll 8 address them appropriately at that time.
9 One point I want to emphasize here is 10 that, in the past 18 months or so, we've had 11 significant -- we've provided significant input and 12 contributions to the overall process.
13 Yesterday, I was trying to count the 14 number of public meetings and PWR Owners Group 15 workshops that we attended. I don't have the exact 16 count, but it's close to 15 just in the past 18 17 months.
18 So, we've had significant interactions 19 with the industry on --
20 MEMBER DIMITRIJEVIC: Just on these three 21 methods?
22 DR. REISI FARD: Some of it was with the 23 new methods, some of it was also about like the 24 definition of PRA upgrade and some other issues, but 25 mostly it was on newly developed methods.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
141 1 Next slide, please?
2 MEMBER DIMITRIJEVIC: So, is this 3 something you are very interested?
4 DR. REISI FARD: Oh very much so.
5 With respect to pilot peer reviews of 6 newly developed methods, as I mentioned, in May and 7 June of last year, we observed three pilot 8 applications of industry NDM peer review process.
9 Staff observed the on site portion of the 10 peer review which means the interactions between the 11 method developers and the peer review team.
12 We also had access to supporting 13 documentation through SharePoint sites, a wide range 14 of documentation including the discussion of --
15 description of the method, the technical bases, self-16 assessments by the method developer, the peer review 17 results, so on and so forth.
18 MEMBER DIMITRIJEVIC: It must be something 19 we are likely to see on here or do you think it's 20 about one time?
21 DR. REISI FARD: So, at least --
22 MEMBER DIMITRIJEVIC: And, those three 23 methods something you expect to see again or you think 24 it's just one time?
25 DR. REISI FARD: So, for the fire methods, NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
142 1 the industry closed all the findings. So, I guess the 2 next step for them is to -- and I'm going to go 3 through the rest of the presentation I'll talk about 4 -- the next step according to the process is for the 5 method developer to send NRC a report describing some 6 details about the method.
7 For the other two, industry is going to 8 have another peer review of the room cooling method 9 sometime later this month.
10 And, for the other one, it seems like 11 they're working through the Office of Research to 12 handle it.
13 So, we are going to see -- we are going to 14 observe the peer review of the room cooling one, the 15 other one on EDG failure rates, they're working with 16 the Office of Research.
17 And, I think we should see -- we should be 18 seeing some documentation with respect to the fire 19 method as well.
20 Next slide, please?
21 This slide at a high level explains our 22 objectives for observing the peer reviews.
23 First of all, we wanted to make sure that 24 -- or we wanted to determine whether the high level 25 requirements and supporting requirements are adequate NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
143 1 for determining the acceptability of methods.
2 We also wanted to see whether there are 3 differences between the process that is used for peer 4 reviewing the implementation of methods versus the 5 process that is used for determining the acceptability 6 of the method. And, I'm going to talk about that 7 later on.
8 And, finally, are there specific 9 considerations in relation to oversight activities of 10 NDMs? I'm not going to talk about this aspect much.
11 Sunil touched on this one when he talked about making 12 revisions to inspection procedures.
13 I'll talk about the reporting, some of the 14 reporting criteria later on.
15 But I'm mostly focus on the first two 16 bullets here.
17 Next slide, please?
18 In summary, we found that the process and 19 requirements provide a well structured approach for 20 reviewing NDMs.
21 Nevertheless, the NDM technical 22 acceptability peer review has significant differences 23 that the process has differences from compared to the 24 process that is used for reviewing the implementation 25 of the method.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
144 1 And, once, finally, once the process is 2 properly implemented, and all the supporting 3 applicable supporting requirements and high level 4 requirements are met, then the method will be 5 acceptable to be used in risk-informed decision 6 making.
7 Next slide, please?
8 The outcome of the NDM observations, as 9 you heard earlier, several high level requirements and 10 supporting requirements were revised based on peer 11 reviewers and NRC comment -- staff -- comments from 12 NRC staff.
13 There were no significant changes, but 14 there were some deletions and additions and kind of 15 consolidation of comments based -- requirements 16 basically.
17 NEI 17-07 was also revised to address some 18 unique aspects of peer reviewing the acceptability of 19 methods. And, the three bullets on this page kind of 20 provide at a high level what are those differences.
21 First of all, for peer reviewing 22 implementation of the method, it's a sampling process.
23 They don't look at all aspects of the implementation.
24 For determining the acceptability of a 25 method, it's beyond a sampling process. They need to NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
145 1 have a more in depth knowledge of all aspects of the 2 method.
3 Secondly, we provided comments to NEI with 4 respect to ensuring that the peer review team has the 5 right expertise to peer review the method.
6 A number of times, this expertise is non-7 PRA expertise. So, we included some language to make 8 sure that that expertise exists for peer reviewing the 9 method.
10 And, finally, the NDMs with finding level 11 F&Os cannot be used in PRAs supporting risk-informed 12 applications.
13 In the next slide, I discuss the basis for 14 that. We found that this is an important issue in the 15 context of the peer review of NDM peer reviews.
16 The peer reviewers in the peer review 17 framework, the peer reviewers determine whether 18 supporting requirements have been met or not. It 19 wasn't clear if their open findings, how the peer 20 reviewer, at a high level, will determine that a 21 method is acceptable for risk-informed application or 22 not.
23 So, for that reason, we said all the 24 findings need to be closed before they move on to 25 implementing it for risk-informed application.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
146 1 Also, it wasn't clear how licensees or 2 peer reviewers of implementation can justify the use 3 of NDM considering the expertise that is needed and 4 the detailed knowledge of the NDM.
5 For typical implementation issues, 6 licensees, at times, justify that certain findings 7 don't impact the application. They -- it doesn't --
8 it may not take a whole lot of non-PRA knowledge to 9 make that determination.
10 But for newly developed methods requires 11 specific expertise and requires a detailed knowledge 12 of NDM. So, it's not something that licensees can do 13 generally on their own.
14 And, finally, NDM documentation issues are 15 very important for implementation. Again, for peer 16 review implementations, a number of -- in many cases, 17 licensees argue that they provide justification that 18 the documentation issues don't impact the results 19 because they are simple documentation issues.
20 In the case of NDM, documentation issues 21 should actually impact the implementation. So, we 22 found that, you know, all the -- again, another reason 23 that they need -- all the F&Os documentation or 24 otherwise need to be closed before the method is used 25 in risk-informed decision making.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
147 1 Next slide, please?
2 I'll say a few words on NDM peer review 3 reports that will be provided to the NRC to support 4 our oversight activities.
5 NEI 17-07 lays out that peer review 6 reports should include, and these are some of the 7 items from 17-07, a clear discussion on conclusions 8 regarding the NDMs, a description of the method that 9 was peer reviewed, the technical justification, and a 10 summary of the review against each of the requirements 11 that the method was peer reviewed against.
12 This information will be provided to the 13 NRC and it will be publically available. Obviously, 14 if there's proprietary information, it will be 15 redacted appropriately consistent with our processes.
16 But this will provide a starting point for 17 the staff in case we need to have further interactions 18 with respect to oversight activities.
19 MR. WEERAKKODY: This is a good point to 20 address requests for -- sorry -- this is a good point 21 to address the question, I would say it's a largely 22 safety question that you raised.
23 The -- as part of the NEI industry reports 24 with the tech specs, they are also proposing that they 25 will send us a report on that he describe.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
148 1 And, we are in the process of developing 2 how we will use our oversight process, if necessary, 3 to expeditiously engage the licensees.
4 If we see anything that we don't like, 5 what I'll do as I committed to Dr. Bley earlier, I 6 will send you that inspection -- those inspection 7 reports and with, you know, a summary description of 8 how they would be used.
9 MEMBER DIMITRIJEVIC: Okay, all right.
10 Thanks.
11 DR. REISI FARD: In summary, staff 12 provided significant inputs to the development of NDM 13 review criteria and peer review guidance through 14 public meetings, workshops, observations.
15 We believe that NDM criteria provides a 16 well structured framework within the existing peer 17 review process for reviewing NDMs.
18 And, finally, we will periodically audit 19 implementation of the NDM peer review process to 20 ensure proper implementation and correct understanding 21 of the criteria and process in the future.
22 With that, I'm going to switch gears to 23 the other two subjects that I wanted to --
24 CHAIR BLEY: Before you go, the pilot 25 studies that the industry did, are those reports NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
149 1 available in ADAMS?
2 DR. REISI FARD: So, the reports are not 3 --
4 CHAIR BLEY: If you've got them, and can 5 you pass them on to Chris?
6 DR. REISI FARD: I'll check whether they 7 are publically available. I believe at least part of 8 them are publically available. No, no, so, we have 9 seen, again, it's not now, the appendix will be, the 10 summary appendix will be.
11 CHAIR BLEY: Right.
12 DR. REISI FARD: so, that part of the peer 13 review report that I described a couple of slides 14 earlier, that will be provided to the NRC at some 15 point when they close F&Os and it's ready for 16 implementation.
17 MR. WEERAKKODY: Next slide?
18 DR. REISI FARD: Next slide, please?
19 So, I have one slide on PRA operate, 20 determining what PRA changes constitute PRA upgrade is 21 an important element of Reg Guide 1.200 framework 22 because once it's determined that a change is PRA 23 upgrade, there needs to be a focused scope peer review 24 of the change.
25 The current definition considers changes NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
150 1 in scope and capability that impacts significant 2 accident sequences or significant accident progression 3 sequences as upgrade.
4 In the past several years or so, in the 5 licensing reviews that we've had, we've had a lot of, 6 you know, back and forth and RAIs on what constitutes 7 PRA upgrade and the licensees have provided further 8 justification on, you know, their determination on PRA 9 upgrade versus maintenance.
10 So, the goal here was to provide a more 11 clear and streamlined kind of definition of PRA 12 upgrade. And, basically, you know, you have the 13 definition there. I'm not going to read the entire 14 definition.
15 What is does is that it basically focuses 16 on changes in the scope and method would constitute 17 PRA upgrade without necessarily linking it to the 18 significant change in accident sequences and accident 19 progression sequences.
20 So, as simple as that. If it's a change 21 in the scope or method, then it's an upgrade.
22 Next slide, please?
23 On the issue of key assumptions, at a high 24 level, Reg Guide 1.200, obviously, needs for detail of 25 the PRA and allows the NRC staff to focus on peer NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
151 1 review findings and key assumptions.
2 And, that makes evaluation of key 3 assumptions as a pre-qual element of NRC review.
4 We -- in the Draft Guide, we've clarified 5 the guidance for identifying and dispositioning key 6 assumptions based on the recent experience that we've 7 had in 50.69 and specifically 50.69 and 65 of fire 8 reviews.
9 Next slide, pleas?
10 So, basically, it's a three step process.
11 The, say that the key assumptions are generally 12 identified for an application from the assumptions and 13 approximations in the base PRA.
14 ASME/ANS PRA Standard requirements has --
15 they have -- there are several requirements for 16 identifying assumptions when utilities develop PRAs.
17 And, identifying assumptions, that could be a starting 18 point. Those assumptions that have been identified 19 and have been peer reviewed, that could be a starting 20 point for identifying assumptions.
21 And the next step, those that are key to 22 the application are identified, meaning that they may 23 impact or they may influence the decisions.
24 And, those that are key will be 25 characterized and addressed using appropriate NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
152 1 sensitivity analyses or consistent with, you know, the 2 guidance in NUREG-1855 if there are other approaches 3 to the address them, 1855 also has a detailed -- more 4 detailed guidance on how to address, you know, key 5 assumptions.
6 With that, that ends my portion of the 7 presentation.
8 MR. GILBERTSON: Okay. If there are any 9 questions?
10 MEMBER DIMITRIJEVIC: But I mean, not to 11 all new PRAs based on the 1.206 have key assumptions 12 identified in the FSAR, but all PRAs didn't have key 13 assumptions, right?
14 MR. WEERAKKODY: Yes.
15 MEMBER DIMITRIJEVIC: I mean, so, does 16 this key assumptions, I don't remember the key 17 assumptions required in the standard.
18 CHAIR BLEY: I don't remember either.
19 DR. REISI FARD: They key --
20 CHAIR BLEY: But they should have been.
21 DR. REISI FARD: So, are you referring to 22 the PRA standard?
23 MEMBER DIMITRIJEVIC: Yes.
24 DR. REISI FARD: So, the PRA Standard in 25 several parts and under several technical elements has NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
153 1 as specific requirements for identifying assumptions.
2 MEMBER DIMITRIJEVIC: Right, assumptions, 3 but the reason --
4 (SIMULTANEOUS SPEAKING) 5 DR. REISI FARD: Yes.
6 MEMBER DIMITRIJEVIC: So, what does this 7 slide mean, that you're going to go on the key 8 assumptions that for every application you're going to 9 track what key assumptions are applicable?
10 DR. REISI FARD: As a part of, yes, as a 11 part of all applications, the licensees provide a list 12 of key assumptions that they have identified using the 13 guidance.
14 So, they need to identify those 15 assumptions that influence the decision. And --
16 MEMBER DIMITRIJEVIC: Okay, let's just 17 start with the CFR 50.69. Every assumption influences 18 decisions.
19 DR. REISI FARD: And so, the --
20 MEMBER DIMITRIJEVIC: And, plus, I don't 21 even know what the licensee have, you know, I mean, I 22 don't really know what is the status on this standard 23 issue.
24 MR. DINSMORE: Yes, hi, this is Steve 25 Dinsmore from NRR again.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
154 1 Key assumptions is a bit of a difficulty 2 we're working through. Obviously, they have lists of 3 assumptions that they get from the peer review teams 4 for each of the elements.
5 And then, they have their own assumptions, 6 well, there's a bunch of assumptions in two EPRI 7 documents, one on internal events and one on fires.
8 And, the NUREG-18 -- whatever it is --
9 DR. REISI FARD: 1855.
10 MR. DINSMORE: -- it tells them to go, for 11 each application, you're supposed to go through those, 12 all those assumptions and identify those that might be 13 key. And, if you identify some that might be key, you 14 can either do a sensitivity study to demonstrate that 15 they're not for that application or you can keep a 16 sensitivity study in your process which are kind of 17 the two options.
18 The only problem is it's a little 19 difficult that one step from going from assumptions to 20 those that are key is very dependent maybe on the 21 decision making at the time.
22 So, but we're working through the process 23 but that's how supposed to work.
24 DR. REISI FARD: So, the intent is not to 25 identify every assumption that impacts the results.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
155 1 Obviously, all the assumptions impact the results one 2 way or another.
3 Then the real criteria that it may 4 influence the decision, meaning that, based on certain 5 assumptions for 50.69, you go from HSS to LSS or the 6 other way.
7 So, if it impacts the decision, does it 8 impact the results so much that it would impact the 9 decision. That's kind of is the criteria that we used 10 in recent reviews.
11 MEMBER DIMITRIJEVIC: Okay.
12 MR. WEERAKKODY: Can I go to the next 13 slide?
14 MR. GILBERTSON: Yes, please.
15 Okay, I will try to go through this as 16 rapidly as I can.
17 I think in many regards, the redlines 18 strike out that we provided to you, it sort of self-19 demonstrates a lot of the changes that we made. So, 20 I'll just summarize a lot of them at a high level.
21 And, please, just stop me if you have any questions, 22 obviously.
23 So, in general, the changes that we made 24 to Reg Guide 1.200 were focused mostly on NDMs and the 25 guidance on the peer review process.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
156 1 We did look at it more holistically to 2 look at other parts of it, other pieces of language.
3 We had a few parking lot items that we also worked in.
4 So, but the big items are the new staff 5 endorsements, the NEI guidance document, the PWR 6 Owners Group document as well as the seismic ASME/ANS 7 PRA Seismic Code Case which provides a set of 8 alternative requirements to the Part 5 Seismic PRA 9 Requirements.
10 MR. WEERAKKODY: Should I do a page down?
11 MR. GILBERTSON: Yes, you can -- next 12 slide.
13 So, this is -- and, we've already kind of 14 touched on this. These are just a little more details 15 about the documents.
16 NEI 17-07, it's consolidates guidance from 17 the predecessor documents on the different hazards.
18 It incorporate Appendix X which was developed for 19 those documents and relates to the F&O independent 20 assessment.
21 And then, also, it points out to the newly 22 developed methods requirements.
23 The Case 1, that's the seismic PRA code 24 case. The NRC wrote an acceptance letter on that.
25 And so, we've just brought our comments and our NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
157 1 position in that letter into this endorsement.
2 And then, of course, the Owners Group 3 document, we have brought in definitions. We brought 4 in -- by bringing, I mean, we are endorsing 5 definitions, a process for determining whether a 6 change is an upgrade or a maintenance. And then, 7 also, the requirements for the newly developed methods 8 peer review.
9 Okay, next slide?
10 (SIMULTANEOUS SPEAKING) 11 MR. WEERAKKODY: I will also send you the 12 ML number that's highlighted.
13 MR. GILBERTSON: Oh, yes, you know what?
14 I can call that out just so that it's on the record, 15 it's ML-20030A437. So, apologies for not including 16 that.
17 CHAIR BLEY: NEI 17-07 Appendix X?
18 MR. GILBERTSON: So, previously --
19 CHAIR BLEY: I don't see one.
20 MR. GILBERTSON: Right, there's no -- it's 21 not Appendix X in NEI 17-07. It was previously called 22 Appendix X, I think the X was just sort of a 23 placeholder, you know, identifier.
24 And, this was intended to go along with 25 NEI 00-02, 05-04, and 07-12. And so, they both -- I NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
158 1 believe it's Appendix E.
2 MR. LINTHICUM: I believe it's Appendix E 3 in 17-07 but it was Appendix X because in a different 4 peer review guidance documents, they were different 5 appendices.
6 MR. GILBERTSON: Right, right, so, yes.
7 Okay, next slide?
8 Okay, so, just in general, the 9 enhancements and clarifications summary rely --
10 related to the key assumptions source of uncertainty 11 as Mehdi was talking about, it touches on risk-12 informed decision making.
13 We included a glossary of terms, a listing 14 of hazards in a new appendix. And then, there's a 15 discussion on peer acceptability. I'll talk about 16 that in a little more detail.
17 Organization, we did reorganize some of 18 the contents of Sections A, B, and parts of C and that 19 was just to create a more smoother narrative flow.
20 Next slide, please?
21 So, again, Sections A and B, the guidance 22 that we received from our internal process for some of 23 the sections in the guide, they're fairly distinct.
24 And so, the Revision 2 has almost like a 25 running narrative in terms of it blends in from NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
159 1 background, from purpose, it all sort of runs 2 together. So, those are separated out in this new 3 revision.
4 And, of course, we have a discussion on 5 PRA acceptability which is what Sunil had showed that 6 three triangle diagram. It really just describes that 7 that paradigm is. That had never really been 8 explicitly discussed in 1.200. So, and that is 9 consistent with our resolution of that DPO 2016-01.
10 Next slide, please?
11 Okay, and so, Section C.1, we used or we 12 used language that was a little more precise. We 13 wanted to, in may places, we refer to a PRA in 14 general. And so, but it's more appropriate to refer 15 to the base PRA. So, we used that kind of language.
16 We more specifically referred to the PRA 17 Standard or the Standard as the ASME/ANS Level 1/LERF 18 PRA Standard. So, we're just being more explicit.
19 And, we reorganized the technical elements 20 in Reg Guide 1.200 just to be consistent with the 21 organization in the PRA Standard for Level 1 LERF.
22 We also separated out all of the 23 requirements for the staff position for low power and 24 shutdown PRA. We did not change any of those staff 25 positions, they were simply moved to a new section, NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
160 1 their own section. And, again, that's just for 2 clarity.
3 Next slide, please?
4 Section C.2, again, this is just the lead 5 in to the discussion on the consensus PRA Standard and 6 industry PRA program -- peer review programs. So, 7 it's just a general introduction.
8 We talked about the code case and we made 9 some revisions for general clarity.
10 Next slide?
11 Again, this is just, we -- this is a short 12 paragraph or paragraph or a few paragraphs and it just 13 introduces the notion that we're going to talk about 14 in th additional guidance and the subsequent sections.
15 So, go ahead and --
16 Okay, and so, C.2.2, this is really where 17 most of the changes were made. We divided this 18 section up into five subsections based on the peer 19 review, the base PRA, upgrade or newly developed 20 method, and then, the discussion of facts, an 21 observation, independent assessment.
22 So, next slide, please?
23 So, we, in 2.2.1, we talk about the peer 24 review process. We included changes to the team 25 qualifications, the documentation, and this is NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
161 1 consistent with NEI 17-07 and also several of the 2 points that are brought in from the PWR Owners Group 3 report.
4 We do include, you know, as far as, say, 5 you know, team qualifications, just with relation to 6 previous questions, you know, we talk about 7 independence. And, you know, requirements that the 8 team members be independent from the work that's being 9 performed. They also need to be -- they should not 10 have supervised work. They can't peer review work that 11 they have supervised.
12 So, we're just trying to separate out 13 those potential conflicts of interest. And so, those 14 are built in. So, regardless of how the PRA is 15 actually -- the peer review is actually performed, if 16 it's with a, you know, a base PRA peer review or if 17 it's a focused scope, we still expect those 18 requirements.
19 CHAIR BLEY: Can they be from the same 20 utility company but a different plant or do they need 21 to be independent of the --
22 MR. GILBERTSON: Notionally, yes. I think 23 they could be. The requirements --
24 CHAIR BLEY: The organizational conflict 25 that you're worried about?
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
162 1 MR. GILBERTSON: Right, right. So, you 2 know, a staff member shouldn't be reviewing their 3 supervisor's work, for example. But if they're 4 reviewing another, you know, supervisors work in a 5 different component of their, that should be okay.
6 CHAIR BLEY: Okay.
7 MR. GILBERTSON: Okay, next slide?
8 So, this is the section that provides the 9 guidance on whether a change to the PRA is an upgrade 10 or it's PRA maintenance.
11 It's a relatively short paragraph or two 12 and really just calls out to Appendix C which is where 13 we are endorsing the process that's provided in the 14 Owners Group document. And, that is, you know, 15 getting to the flowchart that's provided in that 16 report.
17 And, we have some other -- we have 18 additional discussion that goes along with that in 19 Appendix C.
20 Next slide?
21 So, this section is for the PRA peer 22 review of an upgrade. And so, again, we're calling 23 out NEI 17-07 related to how that peer review is 24 performed. 17-07 has the guidance for performing the 25 focused scope peer review on an upgrade.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
163 1 And then, also, it calls out and endorses 2 the requirements in the Owners Group document for that 3 purpose.
4 Next slide?
5 And, this section is brand new and it's 6 just pulling in the requirements that are discussed in 7 the Owners Group document and NEI 17-07 provides the 8 definition of a newly developed method.
9 And, again, all of these definitions are 10 provided as well in the glossary for 1.200 which 11 doesn't exist. There's no glossary right now in 12 Revision 2 of 1.200.
13 And, it also pulls in the requirements for 14 documentation of the newly developed method peer 15 review.
16 Next slide?
17 And so, Section C.2.3 is focused on the 18 independent assessments. Again, we're, by and large, 19 we're just providing a description of what the F&O 20 independent assessment is and then we're referring to 21 and endorsing the NEI 17-07 guidance.
22 It's consistent with the letter that I 23 have mentioned before, the acceptance letter on 24 Appendix X. So, we did -- we sought not to change our 25 position on that.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
164 1 And, it brings in the new requirements as 2 well for -- from the PWR Owners Group report.
3 Okay, and so, Section C.3, there's really 4 just a handful of clarifications here and it's along 5 the lines of what Mehdi had already mentioned, some 6 similar language to what he provided in his slides.
7 Next slide?
8 And, Section C.4 is just related to 9 documentation. So, we pulled in all of the related 10 requirements for documentation for a newly developed 11 method peer review, of peer review and upgrade and the 12 F&O independent sections.
13 Okay, and this is just a listing of the 14 glossary of terms that we're going to include in the 15 Reg Guide.
16 The main thing I want to point out here on 17 these next two slides is that the endorsement of the 18 2009 ASME/ANS PRA Standard remains unchanged.
19 We have brought that over from Revision 2.
20 So, Appendix B is going to have the code case 21 endorsement.
22 And, on the next slide, we have Appendix 23 C as the guidance for classifying changes to the PRA.
24 And then, Appendix D, this provides a 25 listing of other hazards. So, it's really just to --
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
165 1 it's a listing of other hazards, well, I should say, 2 hazard. It's intended to be complete.
3 So, it talks about internal hazard or 4 internal events, et cetera, internal flood, the 5 typical ones as wells others that are included. But 6 it provides a description as well.
7 So, it's an aid to help try and scope out 8 what a PRA analyst should be looking at. And, it does 9 include things like tsunami and, you know, take you --
10 pick your random hazard, meteor strikes, whatever you 11 like, it's intended to be comprehensive.
12 Okay, so the next steps, as Sunil had 13 mentioned, we will be considering feedback from 14 external stakeholders and internal stakeholders that 15 includes public, ACRS Members that are in this 16 meeting, NRC legal, et cetera.
17 After we finish this briefing, we're going 18 to start preparing the document for our final 19 publication process and approval by our management and 20 legal review.
21 And then, after that, it will be issued 22 for public review, a formal public review and comment 23 through the Federal Register.
24 So, just to be clear, DG-1362 hasn't been 25 issued formally yet, it was a draft working copy was NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
166 1 provided for a public meeting on the 16th and also to 2 you for your consumption.
3 CHAIR BLEY: And, you're not anxious to 4 get a letter from us at this time, will you be after 5 it's a final draft?
6 MR. WEERAKKODY: That is correct, I think 7 what I was -- sorry -- what I was thinking is after 8 the Committee -- Subcommittee Members who are here 9 have a chance to caucus, you know, mean for Christiana 10 Lui and get some, you know, no?
11 MEMBER MARCH-LEUBA: It doesn't work that 12 way. We are not allowed to give you feedback as ACRS 13 Members.
14 MR. WEERAKKODY: No, I was referring to 15 questions on the letter.
16 MEMBER MARCH-LEUBA: We can write you a 17 letter as a private individual.
18 MR. WEERAKKODY: No, we are not asking for 19 a letter like that.
20 MEMBER MARCH-LEUBA: I mean, I can give 21 you opinions or something.
22 MR. WEERAKKODY: Oh yes, yes, right.
23 MEMBER MARCH-LEUBA: It wouldn't be, so 24 you understand, telling you ACRS things.
25 MR. WEERAKKODY: No, we understand.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
167 1 MEMBER REMPE: Even the members can't 2 decide about a letter. It goes to the Full Committee 3 to make the decision.
4 MEMBER MARCH-LEUBA: So, you will get a 5 letter or you don't get nothing.
6 MR. WEERAKKODY: Mike, did you want to say 7 something about that?
8 CHAIR BLEY: Except what you got today.
9 MR. WEERAKKODY: Okay.
10 MR. FRANOVICH: I don't think I have much 11 to add to that other than to say, you know, we always 12 value the, you know, endorsement from the Committee 13 with comments and exceptions, I understand.
14 But we might benefit better from seeing 15 all the stakeholder comments collected on the Draft 16 Guide then weigh in via letter.
17 MEMBER MARCH-LEUBA: In a sense, if we 18 were to write you a great letter saying everything 19 looks great, go ahead and publish it, it wouldn't do 20 you any good.
21 MR. FRANOVICH: I don't think so and it 22 would eat up a lot of your valuable time.
23 MEMBER MARCH-LEUBA: Unless we have 24 something to say, you don't want to hear from us.
25 CHAIR BLEY: But you will come back after NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
168 1 public comment?
2 MR. FRANOVICH: We would be absolutely 3 happy to come back.
4 CHAIR BLEY: I just think at some point we 5 need to write a letter on it eventually.
6 MEMBER MARCH-LEUBA: Oh, yes, eventually, 7 we'll need to published, but I think we need to talk 8 among ourselves if we have something to say. If --
9 MEMBER DIMITRIJEVIC: Well, that's true.
10 (SIMULTANEOUS SPEAKING) 11 MEMBER DIMITRIJEVIC: And, we have to 12 bring it to the Full Committee. I mean, we cannot 13 write a letter without Full Committee.
14 MEMBER MARCH-LEUBA: If you two think that 15 there was something wrong in one particular area, then 16 we need to have a Full Committee letter. If not --
17 MEMBER DIMITRIJEVIC: If you think we 18 would tell them today.
19 MR. WEERAKKODY: So, we have talked about 20 is definitely what we do is, we have provided you a 21 version about a month before this meeting.
22 After we go through the public comment 23 period, we will provide you a version that clearly 24 shows changes things that changed version.
25 MEMBER MARCH-LEUBA: And, for that final, NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
169 1 writing a positive letter of recommendation and you 2 did a great job is valuable. So, intermediate unless 3 we have anything or something bad to say.
4 MR. WEERAKKODY: No, I was forewarned by 5 Christiana that the Subcommittee Members reaction does 6 not constitute any formal ACRS positions. I wasn't 7 asking for one.
8 MEMBER MARCH-LEUBA: You get what we think 9 and it may affect some of your decisions.
10 MEMBER DIMITRIJEVIC: Should we ask for 11 public comments?
12 CHAIR BLEY: yes.
13 MEMBER DIMITRIJEVIC: If we have any, open 14 public line.
15 CHAIR BLEY: And in the room.
16 MEMBER DIMITRIJEVIC: Or for the people in 17 the room if anybody has a comment to make, please find 18 a microphone and do so.
19 Chris, can we open the public line?
20 MEMBER MARCH-LEUBA: They improved the 21 lines. We don't have no docket anymore.
22 MEMBER DIMITRIJEVIC: You have to go so I 23 cannot ask you that.
24 So, is there any -- do we have anybody on 25 the public line who is listening to the meeting today NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
170 1 and who would like to make a comment?
2 (NO AUDIBLE RESPONSE) 3 MEMBER MARCH-LEUBA: I think the five-4 second rule applies.
5 MEMBER DIMITRIJEVIC: Five seconds? Okay.
6 And, hearing nothing, we will assume that nobody has 7 a comment. All right.
8 So, we can down table, we'll ask Joy, she 9 had to go, but I will go down and finish on your side.
10 Dave?
11 Well, we were interested on the 12 applicability of a lot of things which were brought to 13 us for new plants. And, obviously, that's going to 14 come in your version Rev 4 which I just said the 15 standards for the advanced light water reactor will be 16 applicable for that or the new plants, will that be --
17 will that say anything about like design 18 certification, COLA applicability, or not? We can 19 discuss that, okay.
20 MR. GILBERTSON: Yes, yes, yes, we're 21 planning to include.
22 MEMBER DIMITRIJEVIC: Okay.
23 Dave? So, you don't have any comments?
24 (NO AUDIBLE RESPONSE) 25 CHAIR BLEY: Nothing more.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
171 1 MEMBER MARCH-LEUBA: Yes, I do have some 2 philosophy. I like the idea of the stakeholders 3 taking responsibility for their actions. Yes, and 4 they have -- and it was to have to write anything and 5 make the final decision.
6 I'm conflicted on the value of the staff 7 review. And, it has value just for the fact that it 8 exists, that's the main value. But it forces into not 9 cut corners and do it right.
10 But on the other side, I've seen so many 11 red tape, 18, 24, 36 month reviews that shouldn't take 12 more than two days that -- so, I'm conflicted on this.
13 And, one way I see that this can be fixed 14 at the Agency level is let them make the decisions, 15 let them do all the work and we just audit the results 16 here and there. I mean, do a quality control.
17 And, at the beginning when you have 18 something new, you do quality control 80 percent on 19 their submittals.
20 After we know everything is working you 21 quality control on it 20 percent on the submittal.
22 So, it's still a review and everything we pick and 23 choose which ones we want to do an audit which is not 24 the high quality as our review, but it doesn't take as 25 long.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
172 1 So, I'm very supportive of the 2 stakeholders taking responsibility for their plants.
3 And, I think they would like to do it, too. But we 4 cannot let them alone. If you leave them alone, and 5 don't audit it, you're asking for trouble.
6 MR. WEERAKKODY: Thank you.
7 MEMBER DIMITRIJEVIC: Well, and, I have to 8 say, you know, as much as I was listening, I heard 9 something I really worry about and I think in this 10 moment we are making risk-informed regulations so 11 complicated and more and more complicated every day.
12 And, they -- maybe there is a time for a 13 new evolution of this type. We're already using PRA, 14 but we have it came officially in '75, this is now 45 15 years since this policy statement on user PRA that is 16 '95, 25.
17 We have so many years we will start using 18 and applying PRA, maybe we should make regulations 19 such that what we learn is already implemented before 20 all of this check and balances.
21 That may give some idea, you know, what we 22 were doing on this new model of risk-informed sites or 23 something. We already learned something, you know, we 24 replaced two weeks with this, you know, risk-informed 25 the tech specs.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
173 1 And now, we have this living organism 2 which is the PRA which breathes and breathes and 3 change and moves a little here and a little there.
4 You know, so what is this now? You know, three weeks 5 or two weeks?
6 It's all depending on this little changes 7 which will be upgrade or maybe operate, but we already 8 learned from this PRA what is important and we can 9 say, okay, if it's important don't keep it longer than 10 three weeks, it is not important keep it as long as 11 two months.
12 We can make it as simple as those two 13 weeks where have been if we have enough data and 14 experience. Otherwise, it scares me when I see how 15 much requirements we are putting on this.
16 And, especially it scares me because I see 17 that we have new plants which will benefit from 50.69 18 more than anybody because they're doing procurement 19 and things like that.
20 And, new plants are, of course, afraid, 21 because their PRA is not any state of completion but 22 how many changes we will see and will that change see 23 some risk achievement was changing from, you know, the 24 1.9 to 2.3 and something, something becomes important.
25 Don't get me wrong, I love a PRA, I can NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
174 1 indulge in those numbers a million times but somehow 2 just listening to you today, this is just a new 3 method.
4 There are so many methods in the PRA, 5 nobody went through and bothered checking those. They 6 were just also taken in, hey, that's how we are doing 7 it. And, we were doing it for years and now this 8 suddenly states the licensees state and this is what 9 we have to form the new and much broader state of 10 consensus so we cannot not anymore have a butterfly in 11 Beijing flips his wings, oh let's check on it.
12 Somehow if we can find a way, which I 13 don't really have a solution, but I think that we 14 should really take benefit of experience we have in 15 all of this here.
16 MEMBER MARCH-LEUBA: While you were 17 talking, I was thinking, there's a false sense of 18 security on the complexity of the analysis. So, it 19 isn't difficult and so complex it has -- whenever we 20 have a PRA I see in there, my whole tree has a 21 thousand, million cutsets.
22 Yes, well, but you're missing the 23 important one. And, because there is so much 24 complexity that you feel that it is good, you tend --
25 it's so difficult to do that you tend to not NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
175 1 concentrate on what's missing because you don't have 2 time to do it.
3 MEMBER DIMITRIJEVIC: No, complexity, I 4 completely disagree. I think complexity reflects lack 5 of knowledge. Whenever you have enough knowledge you 6 can make things simple.
7 MEMBER MARCH-LEUBA: Absolutely, you're 8 right. I'm with you.
9 MEMBER DIMITRIJEVIC: And, my famous 10 standing charter once said that one of the courses 11 that he said, that's unfortunately there is limited 12 how simple things we can make, but there is no limit 13 to complicate.
14 (LAUGHTER) 15 MEMBER DIMITRIJEVIC: So, that's why we 16 have to be careful. It's like much, we like MAAP and 17 everything, let's don't make things too complicated, 18 try to keep them simple and identifying -- I will make 19 these comments today in the afternoon too because we 20 are talking about risk and review that maybe the new 21 direction is not to have risk-informed application, 22 but let's make regulation risk-informed.
23 We learned something from risk area, I 24 mean, you know? Let's put this into something and 25 let's don't really get afraid every time and something NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
176 1 change, we were using two weeks for, you know, and 2 look at how industry make it.
3 And, anytime that some things go in 4 industry wrong it wasn't something because it's in the 5 PRA.
6 You know, tsunami or the some of the 7 Chernobyl making letters of commission so, you know, 8 the wrong training in Three Mile Island.
9 All right, thank you, guys.
10 MR. GILBERTSON: Thank you.
11 MEMBER DIMITRIJEVIC: Off the record.
12 (Whereupon, the above-entitled matter went 13 off the record at 12:03 p.m.)
14 15 16 17 18 19 20 21 22 23 24 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
Westinghouse Non-Proprietary Class 3 © 2020 Westinghouse Electric Company LLC. All Rights Reserved.
Global Expertise
- One Voice Roy Linthicum, Andrea Maioli Newly Developed Method Requirements February 2020 ACRS meeting
Westinghouse Non-Proprietary Class 3 © 2020 Westinghouse Electric Company LLC. All Rights Reserved.
Background & Purpose
- Develop process/requirements that allows the technical adequacy of a newly developed method to be accepted through the PRA Peer Review Process.
- Definitions, requirements and peer review process developed during multiple dedicated workshops (PWROG, BWROG, NEI, JCNRM, NRC)
- Three peer review pilots informed the final draft wording (requirements, report content, etc.)
- Results of the workshops were transmitted to JCNRM for considerations for inclusion in the next edition of the standard (i.e., through the normal consensus process by JCNRM)
Westinghouse Non-Proprietary Class 3 © 2020 Westinghouse Electric Company LLC. All Rights Reserved.
Key Document
- PWROG-19027 - Revision 1
- Documents the requirements for the review of a Newly Developed Method (NDM) recommended to be added in the PRA Standard
- Revision 1 includes feedback from JCNRM (New edition of the Standard being balloted now)
Westinghouse Non-Proprietary Class 3 © 2020 Westinghouse Electric Company LLC. All Rights Reserved.
Key Definitions
- Newly Developed Method
- PRA Method
- State-of-Practice
- Consensus Method/Model
- PRA Upgrade
- PRA Maintenance
Westinghouse Non-Proprietary Class 3 © 2020 Westinghouse Electric Company LLC. All Rights Reserved.
NDM Technical Element Designator Requirement HLR-NM-A The purpose and scope of the newly developed method shall be clearly demonstrated.
HLR-NM-B The newly developed method shall be based on sound engineering and science relevant to its purpose and scope.
HLR-NM-C The data (note that data can be numeric or non-numeric in nature) shall be relevant to the newly developed method, technically sound, and properly analyzed and applied.
HLR-NM-D Uncertainties in the newly developed method shall be characterized. Sources of model uncertainties and related assumptions shall be identified HLR-NM-E The results of the newly developed shall be understandable and reasonable given the assumptions and data, and given the purpose and scope of the newly developed method.
HLR-NM-F The documentation of the newly developed method shall provide traceability of the work and facilitate incorporation of the newly developed method in a PRA model.
Westinghouse Non-Proprietary Class 3 © 2020 Westinghouse Electric Company LLC. All Rights Reserved.
New NDM SRs Index No.
Capability Category I Capability Category II NM-A ENSURE that the stated purpose of the newly developed method (i.e., what is being achieved by the newly developed method) is NM-A1 consistent with the scope (established boundary) of the newly developed method.
ENSURE the applicability and limitations of the newly developed NM-A2 method are consistent with the purpose and scope in NM-A1.
Based on the limitations and applicability of the newly developed method, IDENTIFY which areas of the PRA the newly developed method is intended to be used for (e.g., hazards, NM-A3 technical elements, plant features, SRs impacted by the newly developed method) and, as appropriate, which areas of the PRA the method is not intended to be used for.
Westinghouse Non-Proprietary Class 3 © 2020 Westinghouse Electric Company LLC. All Rights Reserved.
NDM Peer Review Report
- Similar in structure to a normal Peer Review Report (SR assessment, F&Os)
- Main differences
- Explicit global assessment of the method from the review team
- Non proprietary appendix with minimal key information for public availability (e.g., on a method developer web site, in ADAMS, etc)
Westinghouse Non-Proprietary Class 3 © 2020 Westinghouse Electric Company LLC. All Rights Reserved.
NDM Peer Review Report Non Proprietary Appendix (piloted in PWROG-19019 and 19020)
- Minimal set of information that can be shared to confirm that the method went through the NDM review process (and be referenced in future implementations of the method)
- Basic information
- Unique identification of the method
- Team composition
- SR met/not met
- F&O listing
- List of SRs to be peer reviewed in a plant PRA focused scope review following method implementation
- Explicit technical adequacy statement
Westinghouse Non-Proprietary Class 3 © 2020 Westinghouse Electric Company LLC. All Rights Reserved.
NDM Pilot Peer Reviews
- Three recently developed methods have been peer reviewed in 2019 to pilot the NDM peer review process (developed before the NDM process and SRs)
- Refined room cooling effect modeling (PWROG)
- Fire in cabinets (NEI)
- Three dedicated teams of 2/3 people each (qualifications addressed for method)
- Lessons learned resulted in refinement/finalization of the NDM requirements and definitions in PWROG-19027
- Development of public available appendix for the NDM review
Westinghouse Non-Proprietary Class 3 © 2020 Westinghouse Electric Company LLC. All Rights Reserved.
NDM Pilot Review #1
- The method used to estimate the EDG reliability parameters in this NDM, specifically the fail-to-load and fail-to-run failure modes, is distinct from the method used in NUREG/CR-6928 and in the USNRC Dataset (2015) o The USNRC data sources identify the FTLR parameter as a per hour failure rate o This NDM identified that this success data is reported by utilities to INPO as demand events. As a result, FTLR was calculated as a per demand failure rate in this NDM
- Findings were primarily related to documentation and uncertainty o This method was not originally intended to be reviewed as a stand-alone method o Scope and limitations needed to be identified o Assumptions and uncertainty needed to be documented and characterized
Westinghouse Non-Proprietary Class 3 © 2020 Westinghouse Electric Company LLC. All Rights Reserved.
NDM Pilot Review #2
- Two methods merged together for effects of room cooling failure (screening + probability of failure beyond EQ limit)
- Findings associated with interface between the two methods that were merged
- Needed Clarification of method boundary/scope
- Need better documentation of technical basis for one of the screening criteria
- Need uncertainty characterization (for the failure probability method)
Westinghouse Non-Proprietary Class 3 © 2020 Westinghouse Electric Company LLC. All Rights Reserved.
NDM Pilot Review #3
- Guidance on developing scenarios for in-cabinet fire damage of a Group 4 Electrical Cabinet multi-function control cabinet
- Reviewed against SRs updated following the first two pilot reviews
- Observations focused on:
- Improving the documentation of the limitations and assumptions of the method
- Improving the guidance for implementation of the method
- Clarifying the technical basis, which in this case specifically refers to clarifications on the selection and analysis of fire events data in EPRI Fire Events Database
Westinghouse Non-Proprietary Class 3 © 2020 Westinghouse Electric Company LLC. All Rights Reserved.
Observations
- NDM peer review along with expected documentation helps clarifying and standardize the SR(s) that need to be reviewed during the implementation review
- Clarifies the scope
- Spells out the technical SRs in the other Parts of the Standard
- Provide traceability of the work developing the method (similar to other documentation SRs)
- Ensure implementation guidance is clearly documented to minimize misuse of the method
Westinghouse Non-Proprietary Class 3 © 2020 Westinghouse Electric Company LLC. All Rights Reserved.
Feedback to process
- An F&O closure can be used to close NDM F&Os, but the definition of upgrade and maintenance is slightly different for an NDM
- Examples of NDM maintenance activities
- a correction of an error that does not change the intent or the conclusions for the method;
- the processing of more input data with the same process that does not change in the intent of the conclusion of the method;
- the expansion of documentation for data and assumptions already used (but not appropriately documented in origin);
- performance of more sensitivities to discuss uncertainties and or to confirm the applicability of the method within the original intended range of application;
- clarification of the documentation in support to implementation of the method.
Westinghouse Non-Proprietary Class 3 © 2020 Westinghouse Electric Company LLC. All Rights Reserved.
Feedback to process
- Examples of NDM upgrade activities
- extension/change of the scope/applicability of the method;
- a fundamentally different way to process input/output data (beyond usage of a different tool to perform the same process function)
Westinghouse Non-Proprietary Class 3 © 2020 Westinghouse Electric Company LLC. All Rights Reserved.
Global Expertise
- One Voice www.pwrog.com
Westinghouse Non-Proprietary Class 3 © 2020 Westinghouse Electric Company LLC. All Rights Reserved.
Backup Slides - NDM SRs
Westinghouse Non-Proprietary Class 3 © 2020 Westinghouse Electric Company LLC. All Rights Reserved.
NDM HLRs Designator Requirement HLR-NM-A The purpose and scope of the newly developed method shall be clearly demonstrated.
HLR-NM-B The newly developed method shall be based on sound engineering and science relevant to its purpose and scope.
HLR-NM-C The data (note that data can be numeric or non-numeric in nature) shall be relevant to the newly developed method, technically sound, and properly analyzed and applied.
HLR-NM-D Uncertainties in the newly developed method shall be characterized. Sources of model uncertainties and related assumptions shall be identified HLR-NM-E The results of the newly developed shall be understandable and reasonable given the assumptions and data, and given the purpose and scope of the newly developed method.
HLR-NM-F The documentation of the newly developed method shall provide traceability of the work and facilitate incorporation of the newly developed method in a PRA model.
Westinghouse Non-Proprietary Class 3 © 2020 Westinghouse Electric Company LLC. All Rights Reserved.
HLR-NM-A - Scope Index No.
Capability Category I Capability Category II NM-A ENSURE that the stated purpose of the newly developed method (i.e., what is being achieved by the newly developed method) is NM-A1 consistent with the scope (established boundary) of the newly developed method.
ENSURE the applicability and limitations of the newly developed NM-A2 method are consistent with the purpose and scope in NM-A1.
Based on the limitations and applicability of the newly developed method, IDENTIFY which areas of the PRA the newly developed method is intended to be used for (e.g., hazards, NM-A3 technical elements, plant features, SRs impacted by the newly developed method) and, as appropriate, which areas of the PRA the method is not intended to be used for.
Westinghouse Non-Proprietary Class 3 © 2020 Westinghouse Electric Company LLC. All Rights Reserved.
HLR-NM-B - Technical Basis Index No.
Capability Category I Capability Category II NM-B ESTABLISH the technical bases for the newly developed method using analysis or engineering/science founded on established mathematical NM-B1 and/or engineering and/or science principles (e.g., established through operating experience, tests, benchmarking, or acceptance by the scientific community).
ENSURE that if empirical models are used, they are supported by sufficient data which is relevant to the newly developed method. To NM-B2 the extent possible, ENSURE that the experimental data are shown to be repeatable.
IDENTIFY assumption used to develop the technical bases of the newly NM-B3 developed method.
JUSTIFY the rationale for the assumptions identified in NM-B3 (e.g.,
NM-B4 backed by appropriate operational experience).
Westinghouse Non-Proprietary Class 3 © 2020 Westinghouse Electric Company LLC. All Rights Reserved.
HLR-NM-C - Data Index No.
Capability Category I Capability Category II NM-C IDENTIFY the data needed to support the development of the newly developed method (e.g., relevant plant-specific data, NM-C1 industry-wide current operating experience and data, or experimental or test data).
COLLECT relevant data consistent with current technical state-NM-C2 of-practice.
DEMONSTRATE that the data used, including experimental data NM-C3 or test data, is relevant to and supports the technical basis of the newly developed method.
NM-C4 PROVIDE basis for exclusion of data identified in NM-C1.
ANALYZE data (e.g., modifications to the data, use of data in a NM-C5 different context or beyond the original ranges, statistical analysis) using technically sound basis or criteria.
ENSURE that data is applied consistent with the purpose and NM-C6 scope of the newly developed method.
Westinghouse Non-Proprietary Class 3 © 2020 Westinghouse Electric Company LLC. All Rights Reserved.
HLR-NM-D - Uncertainty Index No.
Capability Category I Capability Category II NM-D CHARACTERIZE the parameter uncertainties associated with the newly developed method; this characterization could NM-D1 include, for example, specifying the uncertainty range, qualitatively discussing the uncertainty range, or identifying the parameter estimate as conservative or bounding.
IDENTIFY the sources of model uncertainty associated with NM-D2 assumptions identified in NM-B3.
CHARACTERIZE the model uncertainties (identified in NM-D2)
NM-D3 associated with the newly developed method; this characterization could be in the form of sensitivity studies.
Westinghouse Non-Proprietary Class 3 © 2020 Westinghouse Electric Company LLC. All Rights Reserved.
HLR-NM-E - Results Index No.
Capability Category I Capability Category II NM-E REVIEW the results from the newly developed method to NM-E1 determine that they are reasonable and understandable.
COMPARE the results of the newly developed method with NM-E2 existing methods and, when possible, IDENTIFY causes for substantial differences.
ENSURE uncertainties do not preclude meaningful use of the NM-E3 newly developed method results.
Westinghouse Non-Proprietary Class 3 © 2020 Westinghouse Electric Company LLC. All Rights Reserved.
Index No.
Capability Category I Capability Category II NM-F HLR-NM-F - Docume DOCUMENT the newly developed method specifying what is used as input, the technical basis and the implementation expectations and limitations. ADDRESS the following, as well as other details needed to fully document how the set of the NM SRs are satisfied:
a) the purpose and scope of the newly developed method b) the intended use of the newly developed method NM-F1 c) the limitations of the newly developed method d) the detailed technical basis for the newly developed method e) the data source, collection process and data manipulation performed in support of the newly developed method f) the assumptions and uncertainties associated with the newly developed method g) the interpretation of the results of the newly developed method in the framework of the intended use and application DOCUMENT the process by which the newly developed method can be applied to a PRA model consistently with the intended NM-F2 used of the newly developed method and taking into account the purpose, scope and limitations.
NEI 17-07 and Newly Developed Method Peer Reviews Victoria Anderson, NEI February 5, 2020
©2019 Nuclear Energy Institute
Overview
Background
NEI 17-07: PRA Peer Review guidance Relationship between supporting documents Stakeholder interactions
©2019 Nuclear Energy Institute 2
=
Background===
Peer review process has been a vital component of implementation of ASME/ANS PRA standard since inception
- Provides rigorous process for review of licensee PRAs prior to use in licensing applications
- Reduces NRC resources expended on PRA tech adequacy NEI undertook effort to improve process and documentation after over a decade of experience
- NEI 17-07: Performance of Peer Reviews Using the ASME/ANS PRA Standard
- Latest version: Revision 2, August 2019
©2019 Nuclear Energy Institute 3
Major Product: NEI 17-07 NEI 17-012 (Fire PRA Peer Review)
NEI 05-04 NEI 12-13 (Internal (External Events PRA Hazard PRA Peer Review) Peer Review)
NEI 17-07 Appendix X (PRA Peer Newly Review Developed Finding PRA Method Closure) Guidance
©2019 Nuclear Energy Institute 4
NEI 17-07: Body of Document Few changes compared to original peer review documents
- Confirmation of reviewer qualifications
- Role of observers
- Use of walkdowns
- Post-on-site review week work Most changes
- Support of review of newly developed methods Provides alternative to explicit NRC approval of PRA methods
©2019 Nuclear Energy Institute 5
Newly Developed PRA Method Definition: A method that has either been developed separately from a state-of-practice method or is one that involves a fundamental change to a state-of-practice method.
Not a state-of practice or a consensus method.
Accompanied by detailed description and justification of its technical basis.
©2019 Nuclear Energy Institute 6
Review of Newly Developed Methods
- Provides guidance on review process NEI 17-07
- Describes reviewer qualifications, review documentation PWROG Criteria
- Gives technical criteria (supporting requirements) for newly developed methods Document
- Provides definition of key terms ASME/ANS
- Provides relevant technical
- Next edition will include supporting PRA Standard requirements for newly developed methods
- Will endorse all of the above RG 1.200 R3
- Provides regulatory footprint for process
©2019 Nuclear Energy Institute 7
Key Points on Peer Review of Newly Developed Methods Can be reviewed in parallel with, or separately from, a licensee PRA model peer review Cannot use a newly developed method with open findings in a PRA licensing application
- Finding closure is an option NRC review via topical report process remains an option Will be explicitly referenced in new tech spec admin section for licensees adopting TSTF-505 (Risk Informed Tech Spec Completion Times)
©2019 Nuclear Energy Institute 8
Additional Changes in NEI 17-07 Incorporated guidance on closure of findings Augmented discussion on concept of unreviewed/not reviewed Addressed lessons learned from over a decade of peer reviews Enhanced discussions on reviewer qualification and documentation
©2019 Nuclear Energy Institute 9
Stakeholder Interactions Completed three pilots of newly developed method process
- NRC observation of all three
- Multiple public meetings and teleconferences over 2 years
- No outstanding NRC comments remain
©2019 Nuclear Energy Institute 10
PRA Acceptability and Status of Regulatory Guide 1.200 Sunil Weerakkody, Ph. D.
Senior Level Advisor in PRA Division of Risk Assessment Office of Nuclear Reactor Regulation February 5, 2020 February 5, 2020, ACRS Reliability and PRA Subcommittee Meeting 1
OBJECTIVES
- Provide some details on the most significant change
February 5, 2020, ACRS Reliability and PRA 2
Subcommittee Meeting
OUTLINE
- Evolution of the peer-review process.
- Role of Regulatory Guide (RG) 1.200.
- Gap in Rev. 2 of RG 1.200 with respect to peer-review of newly-developed methods.
- Significance of closing this gap, specifically for (Risk-Informed Technical Specification (RITS)-4b).
- Strategy to close this gap using PWROG-19027 and NEI 17-07.
- Current Status and Next Steps.
February 5, 2020, ACRS Reliability and PRA 3
Subcommittee Meeting
EVOLUTION OF PEER-REVIEW PROCESS
- SECY-99-256: Rulemaking Plan for Risk-Informing Special Treatment Requirements, October 29, 1999.
- COMNJD-03-0002, Stabilizing the PRA Quality Expectations and Requirements, September 8, 2003.
- SECY-04-0118, Plan for the Implementation of the Commissions Phased Approach to Probabilistic Risk Assessment Quality, July 13, 2004.
- SRM-SRM-SECY-04-0118, Plan for the Implementation of the Commissions Phased Approach to Probabilistic Risk Assessment Quality, October 6, 2004.
- Establishment of the peer-review process using RG 1.200 and consensus standards.
- Peer-review process acknowledged in regulations (10 CFR 50.69, November 2004).
February 5, 2020, ACRS Reliability and PRA 4
Subcommittee Meeting
THREE ELEMENTS OF PRA ACCEPTABILITY Regulatory Guide (RG) 1.200 provides NRC All 3 elements staff position have to work together to PRA demonstrate PRA Standard ACCEPTABILITY Peer Review PRA acceptability to demonstrate to demonstrate conformance with conformance with staff position PRA Standard This process is to obviate the need for a detailed staff review of PRA February 5, 2020, ACRS Reliability and PRA Subcommittee Meeting 5
RG 1.200: AN APPROACH FOR DETERMINING TECHNICAL ACCEPTABILITY OF BASE PRA Evaluate base PRA model acceptability for the intended application
- Scope
- Address all hazard groups pertinent to the requested change
- Level of detail
- Sufficient detail to model the impact of the proposed change
- Technical elements
- RG 1.200 provides one acceptable approach to ensure PRA technical acceptability
- Plant representation
- PRA represents the As-Built, As-Operated plant to the extent needed to support the application February 5, 2020, ACRS Reliability and PRA 6
Subcommittee Meeting
PRA MUST BE SUITABLE FOR THE APPLICATION RITS-4b, Risk-Informed Completion Times NFPA-805, Risk-Informed
- Greater reliance Required Fire Protection on PRA scope, level of
- detail, technical
- More flexibility 50.69 SSC Categorization robustness, and for licensee plant representation
- More complex TSTF-425, Surveillance Frequency Control staff review Program Risk-Informed Inservice Inspection February 5, 2020, ACRS Reliability and PRA Subcommittee Meeting 7
Surveillance Frequency Control Program implemented at most US plants
- Adopted by greater than 75% of industry (Limerick pilot plant)
Move to 92 days 24 Hrs 3 Days 7 Days 31 Days 92 Days 6 Months 18 Months 8 February 5, 2020, ACRS Reliability and PRA Subcommittee Meeting
50.69 Allows Treatment of SSCs According to Safety Significance
- Most licensees plan to adopt
- Adjust scope of SSCs subject to special treatment controls
- Rule consists of three major elements
- Categorization Process
- Alternate ~ 25% ~ 1%
Treatment
- Feedback and ~ 75% ~ 99%
Process Adjustments 9 February 5, 2020, ACRS Reliability and PRA Subcommittee Meeting
RITS-4b: Risk Informed Completion Times R - Risk I - Informed C - Completion T - Times Risk-Informed Existing TS Completion Time Component Completion Time (RICT) Limit Inoperable Frontstop Backstop RICT Limit Risk Management Actions 10 February 5, 2020, ACRS Reliability and PRA Subcommittee Meeting
APPLICATION SPECIFIC REGULATORY GUIDES FOR RISK-INFORMED DECISIONMAKING
- RG 1.175, An Approach for Plant-Specific, Risk-Informed Decisionmaking: Inservice Testing (ADAMS Accession No. ML003740149)
- RG 1.177, Rev. 1, An Approach for Plant-Specific, Risk-Informed Decision Making: Technical Specifications (ADAMS Accession No. ML100910008)
- RG 1.178, Rev. 1, An Approach for Plant-Specific Risk-Informed Decisionmaking for Inservice Inspection of Piping (ADAMS Accession No. ML032510128)
- RG 1.205, Rev. 1, Risk-Informed, Performance-Based Fire Protection for Existing Light-Water Nuclear Power Plants (ADAMS Accession No. ML092730314)
February 5, 2020, ACRS Reliability and PRA 11 Subcommittee Meeting
A GAP IN REGULATORY GUIDE 1.200 &
ASME\ANS PSA STANDARD
- For each technical element, ASME\ANS PSA Standard provides high-level review requirements (HLRs) and supporting requirements (SRs).
- Current version of the ASME\ANS PSA standard does not provide HLRs or SRs for newly-developed methods (NDMs); Furthermore, there is no definition of what constitutes an NDM.
- This gap resulted in inefficiencies in the staffs review of NFPA 805 applications and loss of confidence of the peer-review method to adequately peer-review NDMs.
February 5, 2020, ACRS Reliability and PRA 12 Subcommittee Meeting
CURRENT SOLUTION TO GAP
- For RITS-4b applications, staff has imposed the following Administrative TS\License Condition:
The risk assessment approaches and methods shall be acceptable to the NRC. The plant PRA shall be based on the as-built, as-operated, and maintained plant; and reflect the operating experience at the plant, as specified in Regulatory Guide 1.200, Revision 2. Methods to assess the risk from extending the completion times must be PRA methods used to support this license amendment, or other methods approved by the NRC for generic use; and any change in the PRA methods to assess risk that are outside these approval boundaries require prior NRC approval.
- Industry voluntarily developed PWROG-19027-NP and updated NEI 17-07 to specifically address NDMs to support a less restrictive Admin TS.
February 5, 2020, ACRS Reliability and PRA 13 Subcommittee Meeting
CLOSING THE GAP
- PWROG-19027-NP:
- Provides definitions related to NDMs, PRA maintenance, and PRA upgrade.
- Provides 6 High-Level Requirements and 21 Supporting Requirements for peer-review of NDMs.
- Delineates the process that peer reviewers must use to peer review NDMs in addition to other technical elements of the PRA.
February 5, 2020, ACRS Reliability and PRA 14 Subcommittee Meeting
CURRENT STATUS & NEXT STEPS Status
- Held large number of meetings with industry to discuss and reach alignment on PWROG-19027-NP and NEI-17-07.
- Observed three NDM pilots using HLRs and SRs in draft PWROG-19027-NP.
- Shared draft RG with key internal and external stakeholders.
Next Steps
- Complete update to RG 1.200.
- Complete updates to inspections procedures.
- Decide whether industry request to modify administrative technical specification can be approved.
February 5, 2020, ACRS Reliability and PRA 15 Subcommittee Meeting
Status Briefing on DG-1362, Update to RG 1.200, Revision 3 Anders Gilbertson Mehdi Reisi Fard, Ph.D., P.E.
Reliability and Risk Analyst Reliability and Risk Analyst Office of Nuclear Regulatory Research Office of Nuclear Reactor Regulation Anders.Gilbertson@nrc.gov, Mehdi.Reisifard@nrc.gov 301-415-1541 301-415-3092 February 5, 2020 ACRS Subcommittee on Reliability and PRA
OVERVIEW
- Planned updates to RG 1.200
- Summary of external stakeholder engagement
- Proposed changes to RG 1.200 for Revision 3
- Summary
- Details
- Planned next steps Status Briefing on Draft RG 1.200, Revision 3 17 February 5, 2020 Public Meeting
Planned Updates to RG 1.200
- Revision 3 draft guide (DG-1362) in progress
- Next edition of the ASME/ANS Level 1/LERF PRA standard; and
- ASME/ANS Level 2 PRA standard
- Advanced non-LWR PRA standard to be endorsed in a new RG Status Briefing on Draft RG 1.200, Revision 3 18 February 5, 2020 Public Meeting
Objectives
- Discuss the NRC review of NDM review requirements, process, and associated definitions
- Discuss observations from pilot peer-reviews of NDMs
- Discuss enhancements in the draft guide related to PRA Upgrade and addressing Key Assumptions 1 9
PWROG held NRC workshops to PWROG/NEI developed develop/refine Three NDM requirements pilot peer- revised draft of NDM and definitions reviews documents NRC issued review conducted based on DG-1362 requirements NEI issued NRC and peer-review comments definitions (Jan. 2020) guidance (May-Jun. 2019) (Sep.-Dec. 2019)
(Aug. 2018) for NDMs Several PWROG workshops and public meetings were conduced since September 2018 2
0
Pilot Peer-Reviews of NDMs In May and June 2019, staff observed three pilot applications of industrys NDM peer review process and associated guidance.
Staff had access to Staff observed on-site peer documentations via a review discussions between SharePoint site (method reports, method developers peer reviewers and method self-assessments, resulting peer review reports, and developers. associated documentation).
2 1
Objectives of NDM Peer-Review Observations
- Are there differences in the process guidance and reporting due to differences between peer reviews confirming the proper application of methods versus peer reviews of acceptability of NDMs?
- Are there specific considerations in relation to oversight activities of NDMs?
2 2
Summary of Observations Process and requirements provide a well-structured approach for review of NDMs.
NDM technical acceptability peer-review has key differences compared to implementation peer reviews.
By meeting all applicable SRs under all HLRs, NDM will satisfy the intent of HLRs and therefore the method will be technical acceptable.
2 3
Outcome of NDM Observation Several HLRs and SRs were revised based on peer-reviewers and NRC staff comments NEI 17-07 was revised to address unique considerations for peer-reviewing NDMs
- A detailed examination of supporting information is needed for NDM methods beyond a sampling review
- Team should include expertise needed to review the newly developed method
- NDM with finding-level F&Os cannot be used it in PRAs supporting risk-informed licensing applications 2 4
Importance of Closing NDM Open Findings
- Peer-reviews determine whether requirements of the Standard are met; framework for NDM to be deemed acceptable is unclear unless all SRs are met
- Unclear how licensees/peer-review of implementation can justify use of NDM with findings (considering lack of expertise, detailed knowledge of NDM, etc.)
5
NDM Peer-Review Reports
- Peer- review reports include (in part):
- a clear discussion of conclusions regarding any NDMs reviewed by the peer review team
- a description of the method reviewed
- the technical justification provided
- a summary of the review against the NDM PRA requirements endorsed by the NRC as well as SRs relevant for the implementation of the newly developed method
- This portion of the peer review report will be provided to the NRC by the method developer.
2 6
Summary
- Staff provided significant inputs to development of NDM review criteria and peer-review guidance (e.g.,
public meetings, workshops, peer-review observations).
- NDM criteria provide a well-structed framework within the existing peer-review process to review NDMs.
- Staff will periodically audit implementations of the NDM peer review process, as well as review a sampling of the final peer review reports. 2 7
PRA Upgrade
- Current definition considers changes in scope and capability that impact significant accident sequences or significant accident progression sequences as PRA Upgrade.
- Challenges in implementing the current definition.
A change in the PRA that results in the applicability of one or more Supporting Requirements that were not previously included within the PRA [change in scope], an implementation of a PRA method in a different context, or the incorporation of a PRA method not previously used [change in methods]
2 8
Key Assumption Evaluation of key assumptions is a critical element of NRC review.
RG 1.200 allows reviewers to focus their review on key assumptions and areas identified by peer reviewers as being of concern [].
Reviewers ensure that key assumptions [] identified as having the potential to significantly impact the particular PRA results have been characterized in an acceptable manner given the current state of knowledge [].
Staff clarified the guidance related to definition, identification and disposition of key assumption based on recent reviews.
2 9
Key Assumption (Cont.)
Key assumptions for an application are identified from the assumptions and approximations identified in the base PRA.
An assumption is key to a RI decision when it could affect the PRA results that are used in a decision and, consequently, may influence the decision.
Identified key assumptions will be used to identify sensitivity studies as input to decision-making.
3 0
Summary of Proposed Changes to RG 1.200, Revision 3 (1 of 3)
- Proposed changes provide additional clarity, improve process efficiency, and enhance safety
- Proposed changes in RG 1.200, Revision 3, include:
- New staff endorsements
- Enhancements/clarifications to guidance
- Updates related to organization of RG content Status Briefing on Draft RG 1.200, Revision 3 31 February 5, 2020 Public Meeting
Summary of Proposed Changes to RG 1.200, Revision 3 (2 of 3)
- NRC staff endorsements:
- NEI 17-07, Revision 21
- Appendix X - Facts and Observations (F&Os) independent assessment process
- Newly developed method (NDM) peer review requirements
- ASME/ANS RA-S Case 1 for seismic PRA2
- PWROG-19027-NP, Revision 13
- requirements for peer review of newly developed methods (Sections 4 and 5) 1 See Agencywide Document Access and Management System (ADAMS) accession No. ML19241A615 2 Available via http://www.asme.org 3 See ADAMS accession No. MLXXXXXXXXX Status Briefing on Draft RG 1.200, Revision 3 32 February 5, 2020 Public Meeting
Summary of Proposed Changes to RG 1.200, Revision 3 (3 of 3)
- Enhancements/clarifications to guidance:
- Key assumptions and sources of uncertainty
- Risk-informed decisionmaking
- Glossary of terms
- Listing of hazards
- Discussion on PRA acceptability
- Updates related to organization of content:
- Reorganization of Sections A and B and parts of C Status Briefing on Draft RG 1.200, Revision 3 33 February 5, 2020 Public Meeting
Proposed Revisions to Sections A and B
- Reorganized for clarity
- Provides current status of PRA standards efforts
- Discussion on PRA acceptability added consistent with resolution of DPO-2016-0014 4 See ADAMS accession No. ML17013A015 Status Briefing on Draft RG 1.200, Revision 3 34 February 5, 2020 Public Meeting
Proposed Revisions to Section C.1
- Revised to provide clarity and consistent language
- Subsections on PRA technical elements arranged consistent with Parts in the ASME/ANS Level 1/LERF PRA standard
- Staff position on low-power and shutdown PRA unchanged, but moved into separate subsection in C.1 Status Briefing on Draft RG 1.200, Revision 3 35 February 5, 2020 Public Meeting
Proposed Revisions to Section C.2
- Revised for clarity
- Includes language related to ASME/ANS RA-S Case 1 for seismic PRA Status Briefing on Draft RG 1.200, Revision 3 36 February 5, 2020 Public Meeting
Proposed Revisions to Section C.2.1
- Includes language regarding PRA state of practice and peer review of a newly developed method Status Briefing on Draft RG 1.200, Revision 3 37 February 5, 2020 Public Meeting
Proposed Revisions to Section C.2.2
- Includes language related to the peer review of newly developed methods
- Divided into three main subsections:
2.2.1 Peer Review of a Base PRA Model 2.2.2 Peer Review of a PRA Upgrade or Newly Developed Method 2.2.2.1 Peer Review of a PRA Upgrade 2.2.2.2 Peer Review of a Newly Developed Method 2.2.3 Facts and Observation Independent Assessment Status Briefing on Draft RG 1.200, Revision 3 38 February 5, 2020 Public Meeting
Proposed Revisions to Section C.2.2.1
- Added language on the peer review process, team qualifications, and documentation, consistent with NEI 17-07, Revision 2 Status Briefing on Draft RG 1.200, Revision 3 39 February 5, 2020 Public Meeting
Proposed Revisions to Section C.2.2.2
- References Appendix C, which endorses process in PWROG-19027-NP, Revision 1 Status Briefing on Draft RG 1.200, Revision 3 40 February 5, 2020 Public Meeting
Proposed Revisions to Section C.2.2.2.1
- Defines PRA upgrade
- Endorses requirements in PWROG-19027-NP, Revision 1, related to focused-scope peer review Status Briefing on Draft RG 1.200, Revision 3 41 February 5, 2020 Public Meeting
Proposed Revisions to Section C.2.2.2.2
- Defines newly developed method and provides guidance on a peer review thereof
- Endorses requirements in PWROG-19027-NP, Revision 1, for the peer review of a newly developed method; includes documentation requirements and expectations on the outcome of such peer reviews Status Briefing on Draft RG 1.200, Revision 3 42 February 5, 2020 Public Meeting
Proposed Revisions to Section C.2.2.3
- Endorses NEI 17-07, Revision 2, guidance on F&O Independent Assessments.
- Guidance is consistent with the staff position documented in the NRC letter5 on the Nuclear Energy Institute (NEI) Appendix X to NEI 00-02, NEI 05-04, and NEI 07-12
- Provides overall endorsement of NEI 17-07, Revision 2, as a means of satisfying the peer review requirements in ASME/ANS RA-Sa-2009 5 See ADAMS accession No. ML17079A427 Status Briefing on Draft RG 1.200, Revision 3 43 February 5, 2020 Public Meeting
Proposed Revisions to Section C.3
- Provides clarifications regarding key assumptions and sources of uncertainty Status Briefing on Draft RG 1.200, Revision 3 44 February 5, 2020 Public Meeting
Proposed Revisions to Section C.4
- Revised to include documentation requirements related to a PRA upgrade, the use of newly developed method, and F&O independent assessments Status Briefing on Draft RG 1.200, Revision 3 45 February 5, 2020 Public Meeting
Proposed New Glossary of Terms
- as-built, as-operated
- newly developed method
- as-designed, as-to-be-built, as-
- PRA to-be-operated
- PRA acceptability
- assumption
- PRA application
- base PRA
- PRA maintenance
- consensus method/model
- PRA method
- conservative
- PRA upgrade
- current good practice (or state-
- realism of-practice)
- risk significance
- key assumption
- significant accident sequence
- key source of uncertainty
- significant basic
- level of detail event/contributor
- model Status Briefing on Draft RG 1.200, Revision 3 46 February 5, 2020 Public Meeting
Proposed Appendices (1 of 2)
- Appendix A: Endorsement of ASME/ANS RA-Sa-2009, unchanged from RG 1.200, Revision 2
- Appendix B: Endorsement of ASME/ANS RA-S Case 1, Case for ASME/ANS RA-Sb-2013 Standard for Level 1/Large Early Release Frequency Probabilistic Risk Assessment of Nuclear Power Plant Applications Status Briefing on Draft RG 1.200, Revision 3 47 February 5, 2020 Public Meeting
Proposed Appendices (2 of 2)
- Appendix D: Other Hazards Status Briefing on Draft RG 1.200, Revision 3 48 February 5, 2020 Public Meeting
Planned Next Steps
- Endorsements will include consideration of comments from internal and external stakeholders (e.g., public, ACRS, NRC Legal, etc.)
- February 2020 - ACRS Subcommittee Briefing; Receive ACRS Reliability and PRA Subcommittee members feedback
- Prepare DG-1362 for Management approval and Legal review
- Issue DG-1362 for public comment Status Briefing on Draft RG 1.200, Revision 3 49 February 5, 2020 Public Meeting
Acronyms ANS American Nuclear Society NRR Office of Nuclear Reactor ASME American Society of Mechanical Regulation Engineers PRA probabilistic risk assessment CFR Code of Federal Regulations PWROG Pressurized-Water Reactor CT completion time Owners Group F&O Fact and Observation RES Office of Nuclear Regulatory HLR high-level requirement Research LAR license amendment request RG Regulatory Guide LERF large early release frequency RICT risk-informed completion times LPSD low power and shutdown RISC Risk-Informed Safety Class NDM newly developed method RITS risk-informed technical specifications NEI Nuclear Energy Institute SR supporting requirement NFPA National Fire Protection TS technical specification Association NRC U.S. Nuclear Regulatory TSTF Technical Specification Task Force Commission Status Briefing on Draft RG 1.200, Revision 3 50 February 5, 2020 Public Meeting