ML21223A043
| ML21223A043 | |
| Person / Time | |
|---|---|
| Issue date: | 07/20/2021 |
| From: | Advisory Committee on Reactor Safeguards |
| To: | |
| Brown, C, ACRS | |
| References | |
| NRC-1596 | |
| Download: ML21223A043 (127) | |
Text
Official Transcript of Proceedings NUCLEAR REGULATORY COMMISSION
Title:
Advisory Committee on Reactor Safeguards Metallurgy & Reactor Fuels and Structural & Seismic Analysis Subcommittees Docket Number: (n/a)
Location: teleconference Date: Tuesday, July 20, 2021 Work Order No.: NRC-1596 Pages 1-84 NEAL R. GROSS AND CO., INC.
Court Reporters and Transcribers 1323 Rhode Island Avenue, N.W.
Washington, D.C. 20005 (202) 234-4433
1 1
2 3
4 DISCLAIMER 5
6 7 UNITED STATES NUCLEAR REGULATORY COMMISSIONS 8 ADVISORY COMMITTEE ON REACTOR SAFEGUARDS 9
10 11 The contents of this transcript of the 12 proceeding of the United States Nuclear Regulatory 13 Commission Advisory Committee on Reactor Safeguards, 14 as reported herein, is a record of the discussions 15 recorded at the meeting.
16 17 This transcript has not been reviewed, 18 corrected, and edited, and it may contain 19 inaccuracies.
20 21 22 23 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 www.nealrgross.com
1 1 UNITED STATES OF AMERICA 2 NUCLEAR REGULATORY COMMISSION 3 + + + + +
4 ADVISORY COMMITTEE ON REACTOR SAFEGUARDS 5 (ACRS) 6 + + + + +
7 JOINT SUBCOMMITTEES ON METALLURGY & REACTOR FUELS 8 AND STRUCTURAL & SEISMIC ANALYSIS 9 + + + + +
10 TUESDAY 11 JULY 20, 2021 12 + + + + +
13 The Subcommittees met via Teleconference, 14 at 2:00 p.m. EDT, Peter Riccardella, Chair, presiding.
15 16 COMMITTEE MEMBERS:
17 PETER RICCARDELLA, Chair 18 RONALD G. BALLINGER, Member 19 VICKI M. BIER, Member 20 DENNIS BLEY, Member 21 CHARLES H. BROWN, JR. Member 22 VESNA B. DIMITRIJEVIC, Member 23 GREGORY H. HALNON, Member 24 WALTER L. KIRCHNER, Member 25 JOSE MARCH-LEUBA, Member NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
2 1 DAVID A. PETTI, Member 2 JOY L. REMPE, Member 3 MATTHEW W. SUNSERI, Member 4
5 ACRS CONSULTANT:
6 STEPHEN SCHULTZ 7
8 DESIGNATED FEDERAL OFFICIAL:
9 CHRISTOPHER BROWN 10 11 ALSO PRESENT:
12 RAJ IYENGAR, RES/DE/REB 13 DAVID RUDLAND, NRR/DNRL 14 PATRICK RAYNAUD, RES/DE/REB 15 DAVID DIJAMCO, NRR/DNRL/NVIB 16 SCOTT MOORE, ACRS 17 THOMAS DASHIELL, ACRS/PMDA 18 19 20 21 22 23 24 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
3 1 CONTENTS 2 Opening Remarks and Objectives . . . . . . . . . 4 3 Staff Opening Remarks . . . . . . . . . . . . . . 6 4 Need for PFM Guidance . . . . . . . . . . . . . . 9 5 RES Perspectives on PFM Analyses and 6 on a Graded Approach for PFM . . . . . . . 17 7 NRR Presentation on the use of PFM Guidance . . . 69 8 Public Comments . . . . . . . . . . . . . . . . . 80 9 Adjourn . . . . . . . . . . . . . . . . . . . . . 84 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
4 1 P R O C E E D I N G S 2 (2:00 p.m.)
3 CHAIR RICCARDELLA: It is now 2 p.m.
4 Eastern Time and the meeting will come to order.
5 This is a meeting of the Metallurgy and 6 Reactor Fuels and Structural and Seismic Analysis 7 Subcommittees of the Advisory Committee on Reactor 8 Safeguards. I'm Pete Riccardella, chairman of today's 9 session.
10 ACRS members in attendance are Joy Rempe, 11 Dennis Bley, Jose March-Leuba, David Petti, Matthew 12 Sunseri, Ron Ballinger, Vesna Dimitrijevic, Walt 13 Kirchner, Vicki Bier, Gregory Halnon. Our consultant 14 Steve Schultz is also on the phone.
15 Christopher Brown of the ACRS is the 16 designated federal official for the meeting.
17 I did not call Charles Brown. Charles, 18 are you here?
19 (No response.)
20 CHAIR RICCARDELLA: Evidently not.
21 During today's meeting the subcommittee 22 will discuss NRC's perspectives on probabilistic 23 fracture mechanics analysis and on a graded approach 24 for PFM. The joint subcommittee will hear 25 presentations by and hold discussions with the NRC NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
5 1 staff from NRR and Research and other interested 2 persons regarding this matter.
3 The rules for participation in all ACRS 4 meetings, including today's, were announced in the 5 Federal Register on June 13th, 2019. The ACRS section 6 of the USNRC public website provides a charter, 7 bylaws, agenda, letter reports, and old transcripts of 8 all full and subcommittee meetings, including slides 9 presented there.
10 The meeting notice and agenda for this 11 meeting were posted there. We have received no 12 written statements or requests to make an oral 13 statement from the public.
14 This subcommittee will gather information, 15 analyze relevant issues and facts, and formulate 16 proposed positions and actions as appropriate for 17 deliberation by the full committee. Lists for 18 participation in today's meeting have been announced 19 as part of the notice of this meeting previously 20 published in the Federal Register.
21 A transcript of the meeting is being kept 22 and will be made available as stated in the Federal 23 Register notice.
24 Due to the COVID pandemic, today's meeting 25 is being held over Microsoft Teams for ACRS members NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
6 1 and NRC staff. There is also a telephone bridge line 2 allowing participation of the public over the phone.
3 When addressing the subcommittee, the participants 4 should first identify themselves and speak with 5 sufficient clarify and volume so that they may readily 6 be heard.
7 When not speaking, we request that 8 participants mute your computer or phone.
9 We will now proceed with the meeting. And 10 I'd like to stop by calling on Raj Iyengar for opening 11 remarks.
12 Raj, are you there?
13 MR. IYENGAR: Yes, I am, thank you. Can 14 you hear me?
15 CHAIR RICCARDELLA: Yes, we can.
16 MR. IYENGAR: Okay. I really appreciate 17 this opportunity to appear in front of you members and 18 to let our staff provide a briefing on this very 19 significant topic on probabilistic fracture mechanics 20 and the application results of those for reactor 21 component safety.
22 As you may all know, this topic people 23 have been working on probabilistic fraction mechanics 24 for several decades. There have been different kinds 25 of tools available. But until recently, there has not NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
7 1 been a concerted effort to develop a strong basis for 2 application of these tools in order to provide 3 reasonable assurance of adequate protection for 4 components.
5 I think the NRC staff from Research as 6 well as from NRR joined forces to develop a PFM, 7 probabilistic fraction mechanics guidance in early 8 2016. And that's been published by technical letter 9 report on this topic in 2018. Since then we've gotten 10 quite bit, a lot of feedback on that.
11 And as the subject of this endeavor, the 12 staff had proposed to develop a probabilistic fraction 13 mechanics regulatory guide for to better understand 14 and provide better guidance for licensees when they 15 use probabilistic fraction mechanics tools for license 16 amendments, or relief requests, licensing actions.
17 So that I think we've done a pretty great 18 collaboration between research and NRR, with the help 19 of Sandia National Labs, to develop the current new 20 technical basis document, as well as the reg guide 21 that you'll hear being talked about.
22 And I did want to mention, throughout the 23 process we've had several public interactions with our 24 stakeholders. We've been keeping with our good 25 principles of regulation, in particular with EPRI who NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
8 1 provided very constructive input and developed a white 2 paper on a graded approach to PFM. Which you will see 3 that some of the aspects were reflected in the current 4 document.
5 The draft documents associated with the 6 NUREG technical basis, NUREG and the draft reg guide 7 are still under internal review, so we will not be 8 able to make it public at this time. However, the 9 staff was very open to providing some kind of 10 considerations that we have in developing this 11 regulatory guide.
12 Today we'll hear from our regulator NRR 13 about why they believe NRC needs to be of some 14 guidance and why at this time. We are going to save 15 that thunder for Dave Rudland.
16 Then Research will, staff will present our 17 latest perspectives on PFM analysis and on a graded 18 approach for PFM. Research and NRR are presenting 19 their thoughts of PFM guidance in regulatory 20 application. As you know, that is the most important 21 aspect of what we do.
22 We look forward to having a productive 23 meeting and, hopefully, answering all your insightful 24 questions to the best of our ability.
25 I actually want to commend our staff from NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
9 1 NRR as well as Research who really worked very hard in 2 the last several years to make this happen.
3 With that, I'd like to turn this over to 4 Dave Rudland to provide his thoughts on PFM.
5 Thank you.
6 MR. RUDLAND: Thanks, Raj. So, as Raj 7 mentioned, my name is Dave Rudland and I am a senior 8 technical advisor for materials in the Division of New 9 and Renewed Licenses in NRR. And I'm going to provide 10 today a couple of slides on the need for probabilistic 11 fraction mechanics guidance.
12 And as Raj pointed out, this has been a 13 long effort, a couple of year effort in developing 14 this guidance and understanding what we think we need 15 in our discussions with industry.
16 As the NRC has been transitioning from 17 deterministic methodology to more risk-informed 18 methodology, the staff had developed an integrated 19 decision making process that includes the results from 20 probabilistic fraction mechanics in terms of the 21 change in risk to core damage frequency or large early 22 release frequency. And coupling that with 23 investigations in how it impacts safety margins, 24 defense in depth, whether it meets the current 25 regulations, and how performance monitoring is done NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
10 1 creates a integrated risk-informed solution. And it's 2 important to point out that this is risk-informed and 3 not risk-based.
4 In recent times, though, the agency has 5 been trying to expand the use of risk beyond PRA and 6 calculation of CDF by using more risk insights, and 7 especially for passive component integrity where when 8 you'd be talking about components that may not be 9 actually modeled within the PRA. So, that's kind of 10 what we're going to be focusing on today.
11 Through the course of this development in 12 risk-informed space, the staff has developed 13 applications, be it how to change license basis using 14 risk-informed methodology or how to categorize SPCs in 15 risk-informed space.
16 And with those, of course, they've also 17 developed specific guidance, guidance that is directly 18 related to the application in question. And also 19 they've developed generic guidance.
20 As shown here, Regulatory Guide 1-200, 21 which is one approved approach for determining if the 22 technical adequacy of probabilistic fraction mechanics 23 is sufficient to provide confidence in the result.
24 In addition to what I've shown here, 25 there's other licensing actions that are being NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
11 1 considered, and guidance processing tools are still 2 under development in those areas. And one of those 3 areas if probabilistic fraction mechanics.
4 I think Raj alluded to it a bit, but for 5 those that aren't familiar probabilistic fraction 6 mechanics analyses are basically probabilistic 7 analyses that leverage fraction mechanics and the 8 uncertainties in their inputs and models in developing 9 probabilities of failure. In PRA space, you know, you 10 have initiating events, scenario, development, and 11 consequence, probabilistic fraction mechanics can be 12 used in determining initiating event frequencies.
13 But, typically, PFM is used more for 14 requests for inspection relief, or determining LOCA 15 frequencies in the calculation of leaks, or breaks, or 16 other things. And it's those areas where we are, 17 where we are lacking guidance.
18 If I can talk about the licensing reviews 19 that I just mentioned and put them in more of a 20 graphical form, you can see that on this, on this 21 schematic. On the X factors it is kind of a reliance 22 on licensees' PRA information, the risk information.
23 And the Y axis is the acceptability of that PRA, those 24 PRA results.
25 The processes that I talked about earlier NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
12 1 are here on the right-hand side where we have high 2 reliance and high acceptability on those particular 3 processes.
4 Over on the other end of the scale where 5 we have relief requests for maybe relief from the SME 6 code, we may not rely much on PRA information. We may 7 use more qualitative risk arguments. And, therefore, 8 the acceptability of any particular PRA is not as 9 important to that review.
10 I'm going to be talking about stuff and 11 we're going to be talking about stuff today that's 12 kind of in the middle where we may be using risk 13 outside the defined processes, or outside of formal 14 risk-informed licensing basis processes, but may use 15 probabilistic fraction mechanics. Those processes are 16 still under development.
17 Recently, the staff has released LIC 206 18 which is integrated risk-informed decision making for 19 licensing reviews where some guidance is from the 20 staff on how they can try to use this outside of these 21 normal processes. But in terms of PFM, there's 22 currently no, no real guidance. And to make, to make 23 decisions on requests that use this, we need really 24 adequate and consistent information, and we need to 25 make sure that we have confidence in the result.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
13 1 Like Raj had mentioned also, PFM has been 2 around for quite some time. And there have been 3 submittals in the past, and successful submittals, 4 that have used PFM. However, typically there's a very 5 complex, lengthy review because these codes are very 6 complex. There's a lot of different models on how 7 uncertainties are being propagated that's very 8 computer code-specific. The reviews can be very 9 complex.
10 Many staff believe these codes are like 11 black boxes where input's in and output's out without 12 even knowing what's happening within the code. And 13 that may be due to insufficient documentation, or 14 insufficient vetting of the code and the input. And 15 this all leads to very low confidence in the output.
16 And this becomes particularly an issue 17 when we are, of course, using that as a basis for 18 modifying or changing things like long 19 (unintelligible) inspection programs.
20 Back when we first started this effort we 21 had a public meeting with the industry to talk about 22 PFM and kind of our thoughts and ideas and our 23 position on what should kind of go into these analyses 24 and what should be reported. And we presented this 25 table which shows some of the past EPRI reports and NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
14 1 categories and which we think are important to discuss 2 and to consider when using PFM in a regulatory 3 submittal. And what we did was we kind of labeled 4 what we thought -- whether we thought these things for 5 these reports met the staff's position for acceptable 6 PFM.
7 And as you can see, there's a lot of --
8 there's some yeses, some noes, a lot of partials. But 9 what's consistent in here is that there was not much 10 consistency across the reports, in a particular report 11 or across many reports in terms of how the information 12 was provided to the NRC.
13 There are a lot of hurdles still in PFM in 14 moving forward. Now, the submittals that we've had 15 with PFM are -- have been inconsistent, have been 16 complex. The level of acceptability has been mixed 17 also as we have considered PFM as a basis for a 18 regulatory position.
19 This risk-informed decision making is 20 still very much in process, in the process of being 21 implemented at the agency, especially for the passive 22 components that may not be modeled in traditional PRA.
23 The PFM is just one part of this risk-informed 24 decision making process, as I mentioned. And both the 25 industry and the staff need to keep that in mind as we NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
15 1 are working through these kind of processes.
2 And we're still in the development, not 3 just the PFM reg guide, but other processes in terms 4 of how to use risk-informed for these types of 5 components that are under development. But use of PFM 6 for licensing is increasing rapidly. And especially 7 recently we've been seeing more and more applications 8 of using probabilistic fraction mechanics codes and 9 unique codes in that fact to -- as a basis for trying 10 to make a regulatory change.
11 And so, because of all this we really 12 think that a PFM reg guide will help both the 13 licensees as well as the staff to determine what is 14 consistent and adequate information that needs to be 15 supplied to the NRC when using this type of code for 16 licensing applications.
17 So, that is all that I have. And with 18 that, I will transfer the control over to Patrick 19 Raynaud who will be giving some details about the 20 draft guide.
21 CHAIR RICCARDELLA: Yes, thank you, David.
22 Members, are there any questions on 23 David's presentation at this time? So let's then 24 proceed.
25 MEMBER BLEY: Yeah. Pete, this is Dennis.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
16 1 CHAIR RICCARDELLA: Hi, Dennis.
2 MEMBER BLEY: Early on your mentioned some 3 of the problems with the uncertainty treatment and 4 certain people not having a lot of confidence. Has 5 that condition changed?
6 I know we have used this in some 7 regulatory areas so far. Where do we stand right now 8 on that side of things?
9 MR. RUDLAND: Well, I think the, you know, 10 the applications in the past have been very 11 inconsistent in how they have handled uncertainties 12 and reported it to the staff. I think as the more and 13 more it's being used, the better it is.
14 I think Raj pointed out that EPRI had 15 published some thoughts on this in a white paper in 16 terms of how to present uncertainties, how to present 17 the results, what to present. And that's helped in 18 the most recent applications. But I think, you know, 19 to be consistent across the board, something like this 20 kind of guidance is needed.
21 MEMBER BLEY: If we go back to the PTS work 22 some years ago, --
23 MR. RUDLAND: Uh-huh.
24 MEMBER BLEY: -- this had been chased 25 pretty hard at that time. Was it done more rigorously NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
17 1 there or is it just we've looked deeper into it since 2 then?
3 MR. RUDLAND: Well, you know, I think 4 especially with PTS it was a long process that took a 5 lot of time to get to where we wanted to be. Right?
6 So, it was not a, it was not a simple, short-term 7 effort.
8 And through the multitude of iterations I 9 think we got to the right place.
10 MEMBER BLEY: Okay. You're right, it took 11 a long time.
12 MR. RUDLAND: Yeah. And there were a lot 13 of good lessons learned that we've learned from that 14 effort that set into the stuff we're going to see in 15 this, in this draft guide.
16 MEMBER BLEY: Okay, thanks.
17 CHAIR RICCARDELLA: Thank you, David.
18 Patrick, are you ready to go?
19 MR. RAYNAUD: I am. Thank you.
20 I hope you can see my slides, for those of 21 you who are on the Teams meeting. I think I got it 22 right. Let me know if that's not the case.
23 CHAIR RICCARDELLA: Some of us aren't 24 getting slides. So, I would appreciate it if you 25 would just tell which slide you're on.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
18 1 MR. RAYNAUD: Of course.
2 CHAIR RICCARDELLA: Thank you.
3 MR. RAYNAUD: So, right now I'm just on the 4 cover slides. And I want to introduce myself.
5 I'm Patrick Raynaud. I'm a senior 6 materials engineer in the Reactor Engineering Branch 7 in the Division of Engineering and Research. And I've 8 been aiding the project to develop the PFM guidance 9 for the NRC side since late 2016 or somewhere about 10 there.
11 And today is our first time speaking to 12 the ACRS. And I'm going to present some of the 13 research perspectives on PFM analyses and some 14 detailed thoughts on a graded approach for PFM.
15 I'm on Slide 2. I just want to highlight 16 the objectives today.
17 First, you know, we want to describe, as 18 I said, our perspectives on the analytical steps that 19 could be used to create high-performance PFM 20 demonstrations for regulatory purposes, and to 21 demonstrate that you have an adequate solution to a 22 technical problem that would be relevant for a 23 regulatory application.
24 And then we want to describe what a graded 25 approach for PFM analysis and documentation could look NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
19 1 like when you consider a number of factors such as the 2 safety significance of a particular problem that you 3 were trying to solve, and the complexity of that 4 problem, as well as of the tools that you're using to 5 solve that problem.
6 Now, a few things that -- a few 7 disclaimers for today is that, you know, the slides 8 I'm showing today are all pre-decisional content 9 that's not yet an official position of the NRC. And 10 nothing presented here is completely final since 11 they're still under internal review and could change.
12 And so this content is not intended to be guidance, 13 final guidance on what constitutes an acceptable 14 approach for PFM submittals to the NRC. Hopefully, 15 we'll get there soon.
16 So, my presentation has two parts today.
17 I'm on Slide 3. First I'm going to present the 18 analytical steps in a PFM demonstration. And then I'm 19 going to focus on this graded approach that we have 20 been discussing so far.
21 Slide No. 4.
22 I'll start with describing at a high level 23 what we think are the necessary analytical steps in a 24 probabilistic fraction mechanics demonstration. And 25 really if you look at that in the chart that we have NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
20 1 here, we have three big parts.
2 We have a planning stage, an analysis 3 stage, and a synthesis stage. And there are five 4 steps in total.
5 So, the first part, planning, only has one 6 step. And I'm going to go into each one of these five 7 steps in the subsequent slides in more detail.
8 But the major step there is to translate 9 your regulatory requirements into an analysis plan and 10 decide what you need to do and how before you launch 11 into it.
12 The second major part is to perform an 13 analysis. And here we have three main steps.
14 First, you need to characterize the model 15 input uncertainty. And that is essentially how you 16 are going to represent your inputs that might have 17 random values.
18 The second big step is to estimate your 19 quantities of interest -- you'll see this acronym 20 often, Q of I's, throughout my slides -- and the 21 uncertainty that is associated with those quantities 22 of interest.
23 And the last step, which you may or may 24 not need, depending on your situation, is to conduct 25 sensitivity studies to assess the credibility of your NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
21 1 modeling assumptions and to better understand how your 2 problem changes depending on those assumptions.
3 The last part of the analysis would be to 4 synthesize your results and to draw conclusions from 5 your analysis.
6 And one thing that I really want to 7 emphasize here is that we believe that this is a 8 recursive process, you know, or an iterative process 9 where, you know, you might have some thoughts, initial 10 thoughts on an analysis and perform those analyses.
11 And then based on your results you may often have to 12 revisit some of your initial assumptions or initial 13 modeling choices to refine your results and get a 14 result that is more defensible, perhaps, for whatever 15 you're trying to prove or demonstrate as part of that 16 regulatory analysis.
17 So now I'm moving on to Slide 5. And I'm 18 going to go into each one of these five steps in some 19 more detail.
20 So, the first big step is, as I mentioned, 21 to translate the regulatory requirements into an 22 analysis plan. And here we have come up with four 23 actions that you may have to perform.
24 First you need to look at your regulatory 25 context and, you know, how your PFM analysis is going NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
22 1 to be used to support a decision on a regulatory 2 action. And by that we mean how is the technical 3 basis using PFM going to be input to the regulatory 4 decision.
5 And, specifically, here what might be 6 important is what criteria are going to be used to 7 support a proposed regulatory action. There may be 8 cases where the criteria are well-known. If it's a 9 specific regulation that you're trying to demonstrate 10 that you meet, or maybe the criteria are not so well-11 defined and then, you know, the criteria would need to 12 be described and justified.
13 The second action here would be to define 14 the quantity of interest and how it relates to what 15 our model or PFM code can provide as output, and then 16 to compare that also to the acceptance criteria. So, 17 specifically we're looking at mapping what quantities 18 are used to show that a regulatory requirement might 19 be satisfied, how they're mapped onto specific outputs 20 of our PFM code or model.
21 And we want to make sure that our model is 22 able to predict the needed and relevant quantities.
23 The third action would be to -- and it's 24 related to the third one -- to the second one, is to 25 determine the suitability of the probabilistic NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
23 1 fraction mechanics code to the specific application.
2 What we mean here is that you want to be sure you have 3 a code that is able to model the relevant physics for 4 the problem that you're trying to solve, in the 5 appropriate ranges, and that it has all the necessary 6 physics and numerical and computational features that 7 are needed to solve your problem.
8 And the fourth and final action of this 9 first big step is to identify the key elements of the 10 problem that might impact your analysis choices. For 11 example, if your model is very computationally 12 extensive, what simplifications may you need to take 13 in order to be able to solve your problem in a 14 reasonable time or in a way that doesn't require, you 15 know, a supercomputer.
16 So, now I'm on Slide 6.
17 And the second big step, which is the 18 first step of the analysis portion of our grand scheme 19 here, is to characterize the model input uncertainty.
20 So, first of all two actions here. The 21 first action is to identify which model inputs are 22 going to be characterized as uncertain or random. And 23 then, also, if you are separating aleatory and 24 epistemic uncertainty, how are you going to categorize 25 the uncertainty for each one of these inputs?
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
24 1 And the second action here is to specify 2 probability distributions for those uncertain or 3 random inputs in order to be able to then represent 4 them in a numerical way and propagate those 5 uncertainties through your model.
6 And so, typically this is done by choosing 7 probability distributions of which there are a great 8 number, and setting their parameters, perhaps getting 9 into data and so, and then to propagate that 10 uncertainty to estimate your quantities of interest 11 for your problem.
12 The third step is to actually estimate the 13 quantity of interest and the associated uncertainty.
14 Four main actions here.
15 The first one is to select the sampling 16 scheme that you're going to use to propagate your 17 uncertainty through your model. There are many 18 choices there. The simplest, or the simplest in terms 19 of numerical implementation might be simple random 20 sampling using Monte Carlo methods. But there are 21 many different ways to sample an input space.
22 And the goal here is, obviously, to get a 23 good estimate of your quantity of interest and be able 24 to characterize the uncertainty associated with that 25 quantity of interest.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
25 1 The second action is to assess the 2 sampling uncertainty, which is also performing 3 statistical conversions analysis. You want to assess 4 that the quantity of interest that you've calculated 5 by propagating uncertainties through your model is 6 sufficiently converged to give a solution to your 7 problem given the sampling scheme that you've chosen.
8 And you may, as part of that iterative 9 process that I alluded to earlier, you may need to 10 adjust that sampling scheme if initially you don't 11 have a converged enough solution. That might be 12 adding samples, looking at different random seeds, or 13 completely changing your sampling strategies, maybe 14 using more advanced sampling strategies if you're 15 dealing with, for example, very low probability, if 16 that's where you need to sample more in the tails of 17 distributions to see certain things happen and have 18 more reliable outputs.
19 The third action, which may or may not be 20 needed depending on your case, is to conduct 21 sensitivity analyses to determine the input 22 uncertainty importance. Sensitivity analyses are 23 analyses that correlate the uncertainty of inputs with 24 the uncertainty of outputs and help you understand 25 which inputs drive most of the uncertainty in your NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
26 1 outputs of interest. And so this is essentially akin 2 to identifying the problem drivers.
3 This can also help you confirm that your 4 model is behaving as expected. You may find that some 5 input distributions may need to be refined to get 6 better convergence on your final estimation of the 7 quantities of interest that you need.
8 You may also be able to identify which 9 assumptions in your modeling assumptions may be 10 candidates for sensitivity studies, so you're making 11 a distinction between sensitivity analyses, which is 12 understanding how uncertainty propagations are 13 modeled, and sensitivity studies which are more like 14 "what if" analyses if you change your model or if you 15 change a distribution.
16 And also you can, by identifying the 17 problem drivers, you can identify which are the best 18 candidates for more targeted, more advanced sampling 19 methods like importance sampling, for example, for 20 those that drive the problem.
21 And the final action in this Step 3 is to 22 conduct output uncertainty analysis. And by the way, 23 I think there was a typo in one of my slides, so that 24 title might be different than what you have from the 25 slides that we sent earlier.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
27 1 So, the title here is conduct output 2 uncertainty analysis. And the goal is to provide a 3 final estimate and the associated uncertainty of the 4 quantities of interest. And, as always, it's very 5 helpful to visualize results when possible from a 6 reader perspective.
7 The fourth step, which is the last step of 8 the analysis portion in our scheme, is to perform 9 sensitivity studies, meaning "what if" analyses to 10 assess the credibility of some of the modeling 11 assumptions that you may have made.
12 And there are two actions here. First of 13 all, to choose which sensitivity studies are going to 14 be performed. And that means identifying important 15 assumptions that need further scrutiny and need to be 16 better understood, and to understand what happens if 17 we change these importance assumptions.
18 And the second action is to actually 19 conduct those sensitivity studies and summarize and 20 present the results.
21 There are many ways to perform sensitivity 22 studies. But in any case, there are common elements 23 to pretty much any method you use. You need a 24 reference case, maybe your baseline case. And then 25 you're going to have perturbations from there, maybe NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
28 1 change a model, change some input distributions. And 2 then you're going to compare the output of these new 3 realizations of your model, where you've changed 4 something, to the reference case and understand the 5 differences, see if there are differences or not, and 6 whether those changes are significant.
7 And that can help build confidence in 8 showing that perhaps the overall model is behaving as 9 expected, or to cover the various options that you may 10 need to cover to represent operational occurrences and 11 such.
12 The final step is to draw conclusions from 13 our analyses.
14 MEMBER KIRCHNER: Patrick, this is Walt 15 Kirchner.
16 MR. RAYNAUD: Yes.
17 MEMBER KIRCHNER: Before you go on could 18 you just, you know, you had sensitivity analyses 19 previously when we were looking at estimating, you 20 know, your quantity of interest and such. What are 21 the difference, what's the difference in the 22 sensitivity analyses done before this step? I guess 23 it was in Step 3.
24 MR. RAYNAUD: Uh-huh.
25 MEMBER KIRCHNER: Just is it at a micro-NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
29 1 level so to speak? Or in Step 3 you also were doing 2 sensitivity analyses. So, this looks like a more 3 macro look at functions.
4 CHAIR RICCARDELLA: You may mean the 5 difference between sensitivity analyses and 6 sensitivity studies. Maybe you could clarify.
7 MR. RAYNAUD: Correct. Correct. And I was 8 going to reinforce that. Perhaps I didn't make the 9 point enough.
10 There is a difference between sensitivity 11 analysis and sensitivity study. And I know for me it 12 took me a long time to realize that there was a 13 difference. But sensitivity analysis really is about 14 understanding what input uncertainty has a large 15 influence on the output uncertainty.
16 So, essentially it's studying the 17 relationship between input uncertainty and output 18 uncertainty.
19 Whereas sensitivity studies is, you know, 20 given that relationship, if I change something how 21 does the output change?
22 So, again, it's studying the relationship 23 between uncertainties versus doing "what if" analyses, 24 you know, what if I change a model, what if I change 25 a distribution, what if I change values for this NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
30 1 parameter or that one.
2 So, they're two different things in our 3 scheme here. Two different types of analyses.
4 MEMBER KIRCHNER: Okay, thank you. Yes, in 5 my simplistic world the first one is more a micro at 6 separate model levels, or material selections, or 7 whatever you're looking at. And this last step to me 8 is more at a macro level: you know, how does it impact 9 the overall results.
10 But I thank you for the clarification.
11 MR. RAYNAUD: You're welcome.
12 And they're definitely intimately related.
13 But they are, down in the detail of how you perform 14 these, they are different and they provide different 15 insights.
16 Okay, so unless you have more questions, 17 I'll continue.
18 So, we were on Slide No. 9 and our fifth 19 step of our scheme to perform PFM analyses.
20 And this last step is to draw conclusions 21 from analysis results. And, you know, two actions 22 here.
23 First, we're going to interpret the 24 results we have by synthesizing the information that 25 was gathered in the previous steps. And then we're NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
31 1 going to try to draw conclusion. We may be able to 2 draw final conclusions by comparing to our acceptance 3 criteria, or we may realize that we need additional 4 analyses or to refine our analyses to create a more 5 complete evidence package to be able to draw our 6 conclusions.
7 And the second actions, which is very much 8 related to what I just said, is to iterate on the 9 analysis process and to refine the model results. You 10 know, in many cases you will need to provide more than 11 one final analysis because PFM is very complex and 12 there are so many moving parts that, you know, this 13 iterative process and providing a more complete 14 picture of how the model behaves and how the results 15 change for different assumptions can be very important 16 in PFM space.
17 So, I'll move on now to the second part of 18 my presentation which has to do with thoughts on our 19 graded approach. That's Slide No. 10.
20 And, first, before I dive into any details 21 I want to highlight some overarching, higher level 22 concepts.
23 And the first thing I want to really 24 emphasize, and Dave before me alluded to that, is that 25 PFM is generally complex. You know, if you could NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
32 1 solve problems deterministically, that is, obviously, 2 much simpler and perhaps the first thing that everyone 3 tries, and that we have been doing for quite some 4 time.
5 So, when you're dealing with something as 6 complex or potentially as complex as probabilistic 7 fraction mechanics, the depth and the breadth of the 8 analyses that you're going to perform, and perhaps how 9 much you're going to have to document those things, 10 can depend on a number of factors, which is why it 11 does make sense to take a graded approach.
12 And, you know, I think this was discussed 13 in previous public meetings as far as maybe around 14 2018 or so, in particular with EPRI who brought some 15 of these concepts up. And so I want to credit our 16 stakeholder feedback for putting us onto this path.
17 So, the graded approach not only applies 18 to how you perform your PFM analyses, but of course 19 how you document them to present an evidence package 20 in a regulatory context that is something that is 21 going to be reviewed and scrutinized by the NRC.
22 The general principles that sort of guided 23 our thoughts here are that if you have higher safety 24 significant of the application at hand, higher 25 complexity, or higher degree of novelty associated NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
33 1 with, you know, the regulatory request for application 2 or the codes and tools that are used to perform the 3 analyses, then that probably is going to translate 4 into having to perform more analyses, have a more 5 complete documentation package. And, ultimately, it 6 will mean a higher burden to create a more defensible 7 and rigorous evidence package.
8 And so what we've done to try to develop 9 a graded approach or some thoughts on a graded 10 approach is come up with all the main topics that we 11 think need to be addressed in a probabilistic fraction 12 mechanics analysis and submittal context. And we came 13 up with eight of those, which I'll list in a second.
14 And then for each, we came up with some 15 categorization schemes and recommendations or thoughts 16 on what might be needed for analysis or documentation.
17 So, the eight broad topics that we came up 18 with are:
19 First, software QA and V&V; then models; 20 then inputs; then uncertainty propagation; then 21 convergence; then sensitivity analyses; then quantity 22 of interest uncertainty characterization, and; 23 finally, sensitivity studies.
24 So, I'm going to go into some detail for 25 each one of these. I have some pretty detailed tables NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
34 1 so that once you have all the slides, you have all the 2 information. But I may not cover everything in detail 3 because I think it would just be a lot of reading off 4 and talking. So, I'll try to highlight some of the 5 higher level distinctions we made.
6 CHAIR RICCARDELLA: Just a general comment 7 for my colleagues from someone who's been involved in 8 a lot of probabilistic fraction mechanics analyses.
9 And one of the things that's unique, I think, about 10 fraction mechanics that makes it so amenable to this 11 technique is we generally see much more scatter, much 12 more input scatter in the material properties in 13 fraction mechanics than we do in other strength of 14 materials type applications.
15 And I think that, you know, growth rates, 16 for example, you could see orders, an order of 17 magnitude difference, you know, in essentially the 18 same property from different samples that are tested.
19 And maybe not quite an order of magnitude, but on the 20 fracture toughness you see much more scatter in that 21 property than you would see in, for example, the 22 tensile test, yield strength or tensile strength.
23 So that's one of the things, I think, that 24 has led to probabilistic fraction mechanics being used 25 so much.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
35 1 MR. RAYNAUD: Thank you for that.
2 So, I'm going to go to my Slide 11. And 3 I'm going to go through each one of these eight topics 4 and present our current thoughts which are, again, 5 decisional not final, can still change, but our 6 current thoughts on our categorization and our graded 7 approach for these different topics.
8 Start with on Slide No. 11 our thoughts on 9 a graded approach for software quality assurance and 10 verification and validation.
11 First of all, let me just state that in 12 general safety demonstrations for the NRC usually 13 require that a QA program be in place. So, that's 14 sort of a prerequisite thought. And we also believe 15 that because of the complexity of probabilistic 16 fraction mechanics and all the intricacies of this 17 potential graded approach, if that's where we end up, 18 presubmittal meetings and discussions between 19 applicants and the NRC would probably be very useful 20 to make sure everybody's sort of on the same page as 21 to, you know, where we fall within this, you know, 22 rather complex graded approach.
23 So, when it comes to quality assurance and 24 V&V our categories are labeled QV-1, which stands for 25 QA and V&V-1, through QV-3. And, essentially, our NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
36 1 criteria here to determine which category you are in 2 are still with how familiar we are with the code. And 3 when I say "we," I mean the NRC in this case.
4 So, you know, the base case is if you have 5 an NRC-approved code that's exercised within its 6 validated range, previously approved, et cetera, then, 7 you know, the path in terms of documentation of the 8 code might be pretty simple and pretty basic. Just 9 show that your -- that the code works for the 10 application basically.
11 And then as you move away from that, for 12 example if you use the code outside of its previously 13 validated and accepted range, then you may have to 14 provide additional evidence that, you know, the code 15 is still applicable for what you are doing, and so on.
16 If you then modify a previously NRC-17 approved code, then you know there would be probably 18 some questions and some documentation that would be 19 helpful about what's changed and what kind of V&V and 20 QA did you perform on those changes, and what 21 documentation is available for that.
22 The second category for QA and V&V is QV-23 2, would be for commercial off-the-shelf software 24 designed specifically for the purpose of the 25 application. I don't think there are many such PFM NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
37 1 codes that are specifically, you know, off-the-shelf 2 available for purchase. I can't think of any, but 3 there may be some.
4 And the final category, QV-3, is if you 5 have a completely new and custom code. And then, 6 obviously, you know, much more documentation and 7 description of the QA and V&V would probably be needed 8 to instill confidence and show that the code behaves 9 as expected and so on.
10 So, the next big topic that we have is 11 models. And for this we have categories that we 12 labeled M-1 through M-5. And I really want to 13 emphasize that it's possible that in a large PFM 14 software tool different models could belong to 15 different categories. Because we sort of tied these 16 categories to the QA and V&V categories that I just 17 discussed previously.
18 So, for example, the easiest category, M-19 1, would be if you used a model from an approved NRC 20 code within its validated range, then, you know, 21 presumably we're familiar with it, we've reviewed it, 22 we've accepted. So, the path is pretty easy to show 23 that, to give us confidence that this works.
24 Conversely, if you start using that same 25 model outside of its validated range, then we'd want NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
38 1 to perhaps understand, you know, a demonstration of 2 why these models are still applicable in this new 3 range of application, and so on.
4 If you take a model from an NRC code but 5 you change it then, obviously, more description of the 6 changes would be needed.
7 If you take a well-established model that, 8 but that's not from an NRC code, maybe it's a 9 published model or something that others have used 10 extensively in other codes, or maybe something from 11 the ASME code, something that's widely accepted but it 12 hasn't been previously put into an approved software, 13 then, you know, obviously we'd probably want to 14 understand exactly how that was done and see some QA 15 and V&V trace of how that -- what data was used and 16 how well the model predicts and such. So, obviously, 17 the documentation starts to increase.
18 And, of course, if it's a first-of-a-kind 19 model, which is our final category, M-5, that, you 20 know, brand new, never published, just something 21 completely new for a new phenomenon that has occurred 22 then, obviously, really detailed documentation and 23 perhaps sensitivity studies and analyses to show how 24 the model behaves, and such, may be needed to instill 25 the right level of confidence.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
39 1 So, again, quite a wide range. And you 2 can see this progression from what we're most familiar 3 with to something completely novel with, you know, an 4 increased emphasis on more documentation and more 5 analyses and data analysis to show that the model is 6 behaving correctly and that we can have confidence in 7 that software.
8 Topic inputs in this one is it's a little 9 bit more complicated. So, I'm on Slide 13. And here 10 we're looking to determine our categories we thought 11 about what types of inputs we might have. And some 12 inputs in a code might be deterministic, meaning a 13 single value was provided. And others might be random 14 inputs or uncertainty inputs.
15 So, we made that distinction in our 16 categories when we numbered them I-1, for input one, 17 through I-4, for input four. But then we have a D for 18 deterministic and an R for random after those. And 19 depending on whether you have a deterministic or a 20 random input, obviously, you know, the type of 21 documentation you may need to provide may be different 22 because those models are inherently different -- or 23 those inputs are inherently different in how they are 24 characterized.
25 And then we focused on how important is NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
40 1 this input with regards to the output that we're 2 interested in? And then how well do we know the 3 characteristics of that input?
4 So, in our I-1 category, you know, it's 5 something that you know very well and, on top of that, 6 has low importance with regards to the output. And in 7 that case, you know, you may need very little 8 documentation. Just list your values and that's it, 9 or your distributions.
10 But as you increase the importance that 11 that input has on the output, and to know that by the 12 way, that's where sensitivity analysis may come into 13 play. And so, the more important the input is and the 14 less knowledge about that input you have, then the 15 more uncertainty you have.
16 So, if you have a lot of uncertainty on 17 something that's really important with regards to your 18 quantities of interest and your output, obviously, you 19 know, more scrutiny would be placed on such an input 20 value. And so more and more documentation and 21 analyses to show that the input representation covers, 22 you know, all the possibility -- well, "all" is a big 23 word -- but, you know, the relevant possibilities 24 would definitely be of interest in that case.
25 And, yes, one other important thing that NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
41 1 I want to say here is that every input you have may 2 fall in a different category, much like every model 3 you have in your overall code might fall in a 4 different category. So, there's a real detailed work 5 of categorization here that has to happen so that you 6 make sure that you're doing, you know, what you should 7 do but not too much or too little. And so it really 8 takes some care. And that's why I think, you know, a 9 lot of discussion, interactions with the NRC perhaps 10 would be helpful.
11 The next big topic is uncertainty 12 propagation. I'm on Slide 14.
13 And here, you know, we alluded to this 14 iterative process in your PFM analysis overall where 15 you might have to run several analyses within the 16 scope of your entire demonstration. Well, here, for 17 instance, your propagation may be different analyses 18 that you're going to run using different sampling 19 schemes, different features of how you model the 20 uncertainty and propagate it, might mean that 21 different analyses in your overall package and 22 demonstration might fall into different categories.
23 So, here we have categories UP, sensor 24 uncertainty propagation 1 through 2-Bravo. And, you 25 know, here we sort of left the QA and V&V aspects NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
42 1 aside and we focused more about on what the specifics 2 of that analysis might be, and what you might have to 3 document in those cases.
4 So, if you just run a very straightforward 5 Monte Carlo analysis, you know, obviously it would be 6 relevant to describe what methods specifically you 7 used for uncertainty propagation. You know, was it 8 simple random sampling? Did you use Latin hypercube 9 sampling? You know, you might want to describe your 10 sample sizes, your random seeds, what type of pseudo-11 random number generator you used, and so on. If you 12 separated aleatory and epistemic uncertainties, how 13 did you do that? If you used importance sampling, how 14 did you do that, and what regions of the input space 15 and why?
16 But then, you know, in some cases if your 17 model is very computationally extensive, you may only 18 run a handful of realizations and then fit a model 19 through that output, essentially create a surrogate 20 model. Some people believe this is machine learning.
21 And in that case, you know, you'd be bumped into the 22 next category because now you're using a more advanced 23 feature, something a little bit more complex that adds 24 maybe some uncertainty to the overall methodology.
25 And so, you know, we'd want to hear about NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
43 1 that surrogate model and how you implemented it, and 2 what are the assumptions, approximations, and how did 3 you validate it?
4 If you use a surrogate model for 5 sensitivity analyses specifically, you know, once 6 again we would want to hear about that because you 7 could use a surrogate model for the overall 8 uncertainty propagation, or maybe only, for instance, 9 to the analyses or sensitivity studies. So, it 10 depends on what you used the surrogate model for, and 11 we'd be looking for descriptions of what was done 12 specifically.
13 A broad topic -- I'm on Slide 15 -- is 14 convergence. And, first, let me say that here I'm 15 really talking about statistical and probabilistic 16 convergence.
17 And so, you know, we assume that temporal 18 and spatial conversions, if you have finite element 19 model, for example, as part of your fraction mechanics 20 calculation, that that part of the convergence, the 21 discretization convergence, pretty much has to be 22 achieved for you to have valid results.
23 So that's, that's sort of a prerequisite, 24 if you want to think of it that way. Otherwise, your 25 results really can't be valid. But once you have NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
44 1 valid discretization convergence, and good enough, 2 then you want to looks at statistical convergence.
3 And here is really what we focused on when we looked 4 at our graded approach.
5 And once again, much like for uncertainty 6 propagation, you know, you could fall into different 7 categories for each one of the analyses you do as part 8 of your overall evidence package.
9 And here the criteria we used to come up 10 with our categories focused on how close you might be 11 to an acceptance criterion so, you know, how much 12 margin you have essentially, and how simple your 13 analysis is over all.
14 So that category SC, which stands for 15 statistical convergence, SC-1, which is the lowest 16 category would mean, you know, you meet your 17 acceptance criteria with plenty of margin, and you 18 haven't done anything fancy: no importance sampling, 19 no surrogate models. In that case, you know, what 20 you've done is relatively simple and there are some 21 pretty simple methods, perhaps, to show that you've 22 achieved statistical convergence.
23 Or, you know, if you really have a ton of 24 margin, maybe you don't even really need to focus too 25 much on statistical convergence at all.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
45 1 But then as you get closer to your 2 acceptance criteria, or if you use some more advanced 3 methods, like importance sampling or surrogate models, 4 or if you separate aleatory and epistemic uncertainty, 5 then, you know, we definitely want to hear more about 6 how you assessed your statistical convergence. There 7 are many ways to do this.
8 You know, if you're still relatively far 9 from your acceptance criteria that your analyses are 10 using more advanced methods, you could demonstrate 11 statistical convergence just using one method of your 12 choice. There are many to choose from.
13 But as you get closer to your acceptance 14 criteria, it would be a good idea to use more than one 15 method to assess your statistical convergence.
16 So, that's kind of the gist of how we came 17 up with our categories here in our graded approach.
18 The next big topic is sensitivity 19 analyses. So, again, this is understanding the 20 relationship between the uncertainty and the inputs, 21 and the uncertainty and the outputs. And here for 22 sensitivity analysis we have six categories labeled 23 SA-1 through SA-6.
24 And these categories are very much tied to 25 which QA and V&V category your software falls in from NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
46 1 the very first category I discussed. So, you know, if 2 you're in the category of an approved code with, you 3 know, within the previously validated ranges, 4 presumably we know that code well and you may not even 5 need sensitivity analysis in some cases.
6 However, as you start getting further away 7 from that, you know, if you have a code that's a 8 modified previously approved code, or a first-of-a-9 kind code, and also as you improve the number of 10 independence input variables that go into your 11 software, that means you're essentially increasing the 12 complexity, as well as getting further away from what 13 we previously approved and what we know well.
14 So then, you know, we'd want to hear a lot 15 more about how you did your sensitivity analysis. Was 16 it local, global? What were the quantities of 17 interest that you looked at? You know, what sampling 18 scheme were your sensitivity analyses done for? And 19 what were the results, you know, what were the most 20 important inputs that you identified and how much of 21 the variance could be explained by those, and so on.
22 If you separated aleatory and epistemic, 23 obviously you would, you know, maybe want to study the 24 sensitivity analyses for both things. And the more 25 variables you have, independent variables as inputs to NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
47 1 your model, the more relevant and important the 2 analyses might be to narrow down, you know, which 3 inputs you really need to study and reply to the 4 solutions for, and so on.
5 And so, those are the kinds of things that 6 I think are important features of an analysis and that 7 really need to be explained so that you can show how 8 you built more and more evidence and confidence into 9 your final result.
10 The next big topic is quantity of interest 11 uncertainty characterization. And we often refer to 12 this as output uncertainty characterization. So our 13 categories are labeled O-1 through O-3. "O" stands 14 for output.
15 And here we focus on, you know, how far 16 are you from the acceptance criteria? And then if 17 you're close, you know, how strong is your basis and 18 your confidence on your input distributions and 19 uncertainty classification?
20 And, you know, I would say the less margin 21 you have and the less certainty you have about your 22 inputs, the less knowledge you have about your inputs, 23 the more you're going to want to study the output 24 uncertainty.
25 I'm hearing a lot of echo in the line.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
48 1 So, anyway, you know, in any case for at 2 the lowest level you're going to want to give a best 3 estimate of your quantity of interest and the 4 uncertainty that you calculated, perhaps represent 5 that graphically. How you calculated your best 6 estimate is important.
7 You know, did you separate, for example, 8 aleatory and epistemic uncertainty?
9 And what is the fraction in each of those 10 uncertainties?
11 And, you know, try to summarize what the 12 major assumptions were that went into the output that 13 you got and into the uncertainty, and what effect did 14 each of those assumptions have on the uncertainty?
15 And, you know, again, once you get closer 16 and closer to your acceptance criteria we're going to 17 want to have a stronger explanation, if you will, of 18 why your uncertainty that you calculated, you know, 19 you know it well enough to be confident that you're 20 not going to exceed your criteria.
21 The last topic now that we have is 22 sensitivity studies. And, again, this is studies 23 where you're going to change something in reference to 24 a base analysis that you performed, perhaps change 25 some models, change property of uncertainty, change NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
49 1 how you represent some distributions, and so on, and 2 look at the outcome, the result on -- look at how that 3 changes your results.
4 And here we have six categories from SS-1 5 to SS-6. SS stands for sensitivity studies. And 6 these categories are very much tied to which category 7 of QA and V&V you were in earlier on.
8 And if you're in, you know, category QV-1 9 where we know your code really well, you know, you may 10 not even need to do sensitivity studies. You may be 11 able to summarize what's been done previously.
12 However, as you move away from that, if 13 you have an approved code but the quantities of 14 interest that you're looking at were not previously 15 documented or studied or approved in any way, then, 16 you know, you want to provide more information on what 17 sensitivity studies you might have done.
18 If you have a very limited number of 19 independent variables in your code, you may not need 20 as many sensitivity studies or you may not need them 21 at all. It depends on exactly the specifics, you 22 know, how different is your code from a previously 23 approved code.
24 However, if you have a code that's newer 25 and you have a great number of independent variables, NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
50 1 then you're going to definitely want to perform and 2 document some more sensitivity studies. And the goal 3 here is to, you know, explain what your assumptions 4 were, what the impacts they had, or what impacts they 5 had on the final results of your study. And try to 6 describe, you know, what questions might arise, what 7 "what ifs" might arise with regards to your specific 8 problem, and how you address those via sensitivity 9 studies.
10 So, that covers the eight major categories 11 of PFM analysis that we envision. And, honestly, 12 there's a lot of intricate detail in these graded 13 approaches that we tried to come up with. So, you 14 know, it's a lot to deliver in the short amount of 15 time that I share, but we still have some time for 16 questions.
17 I'll just go to my last slide really quick 18 to highlight some of our next steps before maybe we 19 can have some discussion.
20 Our next steps with regards to this 21 project overall would be to publish some draft 22 regulatory guidance, as well as a draft technical 23 basis NUREG for public comment. And, of course, 24 gather and address the comments that we receive 25 If we feel like public meetings might be NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
51 1 needed to, you know, to gather additional feedback or 2 have discussions with our stakeholders, of course we 3 will organize some. And, you know, we expect that 4 perhaps once these documents actually are out and 5 finalized at the draft level, that maybe ACRS might 6 want to be able to discuss the various specifics of 7 those documents. So, of course, we're always 8 available for that.
9 And that ends my presentation for today.
10 Hello?
11 CHAIR RICCARDELLA: Yes, this is Pete. I'm 12 sorry. I had my microphone on mute.
13 I have sort of a general comment/question.
14 There seems to me that in a graded approach something 15 should depend, the amount of detail you get into and 16 the amount of validation, it depends on the results 17 and how the results are going to be used.
18 And I think there's a difference in my 19 experience between if you want to make a quantitative 20 statement of comparison to some acceptance criteria, 21 like 10 to the minus 5th, or a delta risk of 10 to the 22 minus 6th, you know, sort of the Reg Guide 1.174 type 23 of criteria, versus more of a qualitative use of PFM 24 results such as comparing two actions, action A versus 25 action B. It might be hydro test versus some form of NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
52 1 inspection. Or it might be different types of 2 inspections.
3 And that qualitative use of PFM results 4 is, to me, a useful, risk-informed approach that 5 should be considered, you know. If A, if option A 6 turns out to be an order of magnitude better than 7 option B, say, and then you do sensitivity studies and 8 you change some of the input assumptions, and maybe 9 the absolute probabilistic results change by an order 10 of magnitude, but option A always comes out better 11 than option B, you've demonstrated something that's 12 pretty useful that I think that kind of a use of PFM 13 should be included somehow in a graded approach.
14 MR. RAYNAUD: I do have some thoughts on 15 that.
16 I think, first of all, I agree. But I 17 think, ultimately, you know, my understanding of what 18 we're trying to do or show here is that we need to 19 have confidence in the results and the codes used.
20 And to do that, to be able to believe, you know, the 21 types of arguments that you've just described, even if 22 they're qualitative in nature, we need to have high 23 confidence that, indeed, the code that was used 24 produced results that we believe.
25 And I think that's a big focus of what NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
53 1 we're trying to do here. It is very rigorous and 2 perhaps demanding, but it really has the ultimate goal 3 of removing this feeling of the black box that has 4 been described, and instilling confidence in the 5 reviewers and the public that, yes, these results that 6 show these qualitative differences can be believed.
7 CHAIR RICCARDELLA: Uh-huh.
8 MEMBER BALLINGER: Are you all set, Pete?
9 CHAIR RICCARDELLA: Yeah.
10 MEMBER BALLINGER: Yeah, this is Ron 11 Ballinger. In reading through this, it was sort of 12 like reading the old sections of the ASME code before 13 it was computerized and you had a hard copy, and you 14 had to have 27 fingers to locate yourself into various 15 places in the document to feed things through.
16 This thing kind of cries out for a set of 17 examples that could be included which would give 18 somebody that wants to use this a little bit of 19 initial direction.
20 Maybe I'm not using the right terminology, 21 but it just seems like there should be examples in an 22 appendix or something like that that says, you know, 23 this is a simple one and this is a complex one, that 24 kind of illustrates where the nub of the problem is 25 likely to be.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
54 1 DR. RAYNAUD: Yeah, I understand that.
2 We, in fact, have drafted two reports that illustrate 3 some of the concepts that we have here. You know, 4 we're limited to the tools that we have, which are NRC 5 tools, at least the ones we're familiar with that we 6 can really use to the full extent that we want to, and 7 so it's a little bit hard to go through the graded 8 approach, but we do have some draft reports in the 9 works that are pretty much ready to go.
10 When the guidance is issued for public 11 comment, we'll post those as well, that at least 12 illustrate concrete examples with a fictional problem 13 of how you solve a PFM analysis and, you know, what 14 the documentation might look like and so on, and I 15 know you haven't seen that yet.
16 MEMBER BALLINGER: Yeah, so that would be 17 very instructive, and these are referenced 18 specifically in 1382?
19 DR. RAYNAUD: 13 --
20 MEMBER BALLINGER: Or whatever the NUREG 21 turns out to be?
22 DR. RAYNAUD: Oh, I don't know if there 23 are specific references yet because nothing being 24 public, you know, it's hard to reference nonpublic 25 things, but, yes, we can certainly add that in. We do NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
55 1 have the reports in a draft version.
2 MR. RUDLAND: And Ron, this is Dave 3 Rudland. If we do, you know, have a public meeting 4 once these are released, it might be a good idea for 5 us to go through those at the public meeting so that 6 you can see the examples and how they work.
7 MEMBER BALLINGER: Yeah, that's probably 8 an excellent idea. I mean, I remember a workshop that 9 Pete and I went to on xLPR, enough said.
10 DR. RAYNAUD: I was actually going to 11 mention xLPR, which of course is one of the NRC's main 12 probabilistic fracture mechanics codes for piping.
13 There have been a great number of 14 publications related to that code, including a lot on 15 sensitivity studies, sensitivity analyses, and so on 16 that could also be a basis for drawing examples.
17 MEMBER BALLINGER: Yeah, those, I'm 18 assuming that in this NUREG, there will be a set of, 19 what do you want to call it, references, not just 20 references, but a bibliography of sources that a user 21 could go to which would include some of the things 22 that you just mentioned.
23 DR. RAYNAUD: We can make sure that that's 24 there.
25 MR. RUDLAND: But realize, Ron, that xLPR, NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
56 1 and FAVOR for that matter, you know, the staff are 2 very, very familiar with those codes, and very, very 3 familiar with, you know, the bases, and so it would 4 not --
5 It might not be as useful to use those 6 codes as examples of how to step through this because, 7 you know, the basis is really for new or unique codes, 8 you know, where it might be more complex to use the 9 regulatory guide.
10 MEMBER BALLINGER: Yeah, I would think of 11 xLPR or FAVOR as what I would call a dead ringer 12 example, and then another example would be, you know, 13 I don't know, use another acronym, the killer example, 14 the one that's the most complicated.
15 CHAIR RICCARDELLA: Well, I would guess 16 xLPR and FAVOR are all in the one category --
17 (Simultaneous speaking.)
18 MR. RUDLAND: That's correct, yeah.
19 DR. RAYNAUD: So, you know, a lot of the 20 documentation, the QA and V&V documentation that 21 exists for both of those codes, but especially for 22 xLPR because it's a more modern code, and as a result, 23 the standards that were used in developing all of the 24 documentation are more up to today's standards, you 25 can see that that documentation is extensive.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
57 1 You know, we're talking I don't even know 2 how many reports and thousands of pages. Obviously, 3 that's not really something that is realistically 4 reviewable, you know, within a licensing submittal.
5 It's just too much, but I think that type 6 of rigor that was performed there and that exists, it 7 would be nice if that sort of thing existed for very 8 complex new codes that we've never heard of such that 9 if we have specific questions or needed to see 10 specific parts of the V&V and QA, that that paper 11 trace exists, and I think that's been one of the 12 things that's been very hard to get to in past PFM 13 reviews.
14 MR. MOORE: Chairman Riccardella, Members 15 Bier and Dimitrijevic have their hands raised.
16 CHAIR RICCARDELLA: Oh, okay, I'm sorry.
17 I didn't see that. Vicki, would you want to --
18 MEMBER BIER: Sure, I can go first. I 19 have a couple of questions that are mainly kind of to 20 understand the context for this.
21 You mentioned that some of these 22 components that would be analyzed this way might be 23 passive components that wouldn't even appear in the 24 PRA, but just kind of a what if this failed, you know, 25 how likely is that and do we need to worry about it.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
58 1 Could this also be used for components 2 that are in there, like for, you know, getting 3 fragilities for seismic PRA or would you imagine that 4 would be a different process?
5 DR. RAYNAUD: Dave, do you want to try and 6 take that one?
7 MR. RUDLAND: We're talking specifically 8 here about fracture mechanics types of codes, but 9 there's no reason why the process that we have here 10 couldn't be applied to other probabilistic style 11 codes.
12 MEMBER BIER: Okay.
13 MR. RUDLAND: Yeah, I don't see why not, 14 but this particular regulatory guide is really focused 15 just on the probabilistic fracture mechanics codes, 16 yeah.
17 MEMBER BIER: So, I guess the question I 18 have which, you know, may be my lack of familiarity 19 with the context, but there's a very sophisticated 20 discussion about kind of all of the inputs and 21 complexities that could go into the analysis, you 22 know, epistemic versus aleatory uncertainties and how 23 they would be treated, et cetera, but there seems to 24 be relatively little discussion of what happens to the 25 output of the analysis.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
59 1 And, you know, part of that is, okay, 2 fine, you get a distribution of whatever that then, 3 you know, goes to the decision makers the same like 4 any other probabilistic result, but the thing that I'm 5 wondering is are there dependencies between the 6 results of analyses for different components?
7 So, for instance, I don't know, a valve 8 versus a tank, or a pipe in one orientation versus a 9 pipe in another orientation that may be positively 10 correlated like they're likely to fail in the same 11 circumstances, or they may be negatively correlated.
12 The factors that would lead one to fail would make it 13 less risky for the other component.
14 And am I just thinking too far downstream 15 and this would be done for, you know, a single 16 component in isolation or do we need some guidance to 17 deal with those things as well?
18 DR. RAYNAUD: I think the answer, at least 19 in my mind, is that there's no true limitation. You 20 know, I think we're often thinking about single 21 component because that's what we've done to date and 22 what we're more familiar with, but I don't see any 23 reason why a modeling tool couldn't be developed to 24 model interactions between components --
25 MEMBER BIER: Got it.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
60 1 DR. RAYNAUD: -- and so forth.
2 MEMBER BIER: Okay, but that's not in the 3 current practice very much, and therefore doesn't 4 feature largely in this guidance, is that fair?
5 DR. RAYNAUD: I think that's true. I 6 would look to Dave to confirm.
7 MR. RUDLAND: Yeah, if you don't mind, 8 Patrick. Yeah, so, you know, as I mentioned in my 9 little intro stuff, you know, typically we -- you 10 know, the way that we've done PRAs is we've had 11 guidance on, you know, how to conduct a good PRA 12 analysis or how to conduct a good PFM analysis.
13 But then for problem-specific issues, for 14 instance if we needed to develop some kind of criteria 15 for a leak before a break, for instance, there would 16 be separate guidance on how to, you know, on what the 17 acceptance criteria would be, what the interactions 18 would be.
19 That would have to be in separate guidance 20 because this is really just focused on how to do the 21 analyses, and how you apply the analyses to any 22 specific problem would have to be in different 23 guidance.
24 MEMBER BIER: Okay, I guess the one thing 25 I would say is that there may -- if you eventually NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
61 1 envision that kind of interaction, there may need to 2 be a lot more information carried forward from the 3 details of the analysis to support that eventually.
4 MR. RUDLAND: Yeah, and I really, I 5 totally agree, and especially as you mentioned for 6 these components that really aren't even modeled in 7 PRA. There's a lot of thought that needs to go into, 8 you know, how do you apply these risk insights, you 9 know, in determining the safety of these particular 10 components, and what the interactions are, and, you 11 know, are there isolable (phonetic) systems that you 12 need to consider and things like that.
13 MEMBER BIER: Yeah, okay.
14 MR. RUDLAND: Yeah.
15 MEMBER BIER: Thank you.
16 CHAIR RICCARDELLA: Vesna, did you want to 17 ask a question?
18 MEMBER DIMITRIJEVIC: Yes, I did not ask 19 a question. I wanted to make the comment actually to 20 support, Pete, what you said.
21 Well, the PRA especially, but I have met 22 the fracture mechanics code many times in my life, you 23 know, pressurized (unintelligible), the risk-informed 24 ISI where we had the Westinghouse code using fracture 25 mechanics, and an EPRI code using deterministic NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
62 1 results, and then I worked in internal flooding, those 2 guides.
3 So, there is many places you can use 4 fracture mechanics in the PRA, but the point that Pete 5 was making is that, you know, the risk informed in 6 this case is not usually that risk.
7 What is happening here is actually what is 8 much more important. How are those results used? So, 9 let me -- I would say that case based like, say, on 10 the vessel, fracture mechanics analysis of the 11 probability to fail the vessel.
12 So, in all PRAs, the number for that was 13 less than ten to the minus six and it was not 14 considered in the PRA. Then, the PRAs started coming 15 with much lower numbers, and suddenly ten to the minus 16 six would dominate the results, then suddenly the new 17 number of ten to the minus six that's showing up is 18 not really justification for either one of those 19 numbers.
20 So, now how important it is to have a good 21 fracture mechanic model to calculate this less than 22 one in a million or less than one in hundred millions, 23 and it's very important in calculating that.
24 So, obviously it would be much easier to 25 prove that something is less than some value than NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
63 1 getting a specific number like 6.2, you know, in minus 2 seven or something, and that could be the goal for 3 this.
4 The other thing is that when you calculate 5 delta risk results, delta risk results are very -- to 6 calculate the importance of anything. This number is 7 the same in each CBI (phonetic) for a lot of -- and so 8 it doesn't really matter what it is because the delta 9 risk disappear.
10 So, this is extremely -- so I think that 11 this is extremely important. If you did not change 12 any factor reaching part of this number, but possibly 13 what is very not likely to happen, then you can change 14 factor between impact, the vessel fail probability.
15 In that case, it's completely irrelevant 16 what that number, as long as it's not challenging 17 safety goal. It will disappear in every delta risk 18 Reg Guide 1174 application.
19 Similarly to large LOCA, like if you want 20 to see like the fracture mechanics code be applicable 21 to large LOCA, then obviously the large LOCA is also 22 small, not as small as the vessel, but you have a 23 choice between using the expert opinion, right, versus 24 trying to use the fracture mechanics code. In that 25 case, you only have to evaluate where the NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
64 1 uncertainties are higher.
2 So, the use of fracture mechanics in the 3 PRA is often going to be very specific and limited.
4 It has to prove the smaller uncertain value. It has 5 to account for uncertainties, whether it's the 6 deterministic inputs, or the expert opinion, or just 7 to define the important factors that impact something.
8 So, you said one thing which is really 9 extremely true. This is so complex that it can just 10 be -- defining exact application is what may be your 11 risk-informed input.
12 When you say, you know, safety-13 significant, risk-informed input is defining what type 14 of application and what should be proved in that 15 application.
16 That's my opinion and I think Pete brought 17 that to say qualitative, but it's not always 18 qualitative. You can just prove envelope. You know, 19 it's smaller than this.
20 You can prove the factors which will not 21 impact that in the risk-informed applications, or you 22 can prove that uncertainties will lead to this is 23 higher than currently used. That was my take on some 24 of those things.
25 DR. RAYNAUD: Thank you.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
65 1 CHAIR RICCARDELLA: Like anything, there's 2 a trade off. As you expand the rigor, you expand the 3 complexity of the analysis. You extend the time and 4 the cost, and I think, you know, the two examples that 5 Ron just mentioned, FAVOR and xLPR, are an example.
6 I mean, look at the time and the cost that 7 went into developing those codes and, you know, you're 8 going to see perhaps less use to PFM because it just 9 turns out not to be cost effective to do it, but 10 that's why I'm suggesting some simpler criteria if 11 you're just looking for a qualitative result, but 12 that's just my opinion.
13 Are there any other members that want to 14 comment? We have one more presentation, but I think 15 maybe we'll take a break before that. Is there any 16 other members that have any comments on it?
17 MEMBER KIRCHNER: Pete, this is Walt. I 18 was trying to think through after listening to you and 19 Vesna in particular, and also Vicki's comments.
20 I wonder, Patrick, you and your 21 colleagues, have you thought about how this might be 22 applied to new materials through advanced reactors 23 that may be operating at higher temperatures, and 24 hence the overall, I'll just say qualitatively, the 25 probability of something like a pressure primary NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
66 1 vessel rupture or fracture could be much higher?
2 CHAIR RICCARDELLA: Or, Walt, like 3 graphite, which we just heard has to be analyzed in 4 the probabilistic method, right?
5 MEMBER KIRCHNER: Yeah, or composites. I 6 often think about graphite composites as pressure 7 vessels, or other internal structures like to house 8 control rods or boron carbide absorbers so that you 9 get the metal out of the system, but you introduce new 10 materials and new challenges.
11 So, Patrick, have you been thinking about 12 -- this morning, we had a very nice presentation on 13 Division 5 of the boiler and pressure vessel code.
14 Are you thinking along those lines to how this might 15 play out to supplement a risk-informed application for 16 an advanced high-temperature vessel?
17 DR. RAYNAUD: Yeah, so I guess what I want 18 to emphasize is that, you know, what we're thinking 19 about here is not really tied to any specific 20 technology, I mean, other than fracture mechanics.
21 But, you know, if you have the correct 22 models to model different temperature ranges, 23 different materials, different phenomena that might 24 lead to failures where fracture mechanics can be used, 25 then we believe that what we proposing here is general NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
67 1 enough.
2 And it's really focused more on 3 statistical methods and stuff much more than the 4 detail of which fracture mechanics model you're using, 5 so it is extensible to these other classes of problems 6 and materials that you've described.
7 MR. RUDLAND: Yeah, this is Dave Rudland 8 from the staff I could just comment also. You know, 9 it's really highly dependent on the information that 10 you have and the uncertainties that you've developed 11 and your confidence in those, right?
12 So, for these high-temperature materials 13 where we know there's not a lot of data quite yet, 14 right, I think it would be more difficult to apply 15 these types of techniques because you wouldn't have as 16 much confidence in the results because of the lack of 17 data.
18 You can make some approximations, and of 19 course adjust acceptance criteria based on that, but, 20 you know, as the industry develops more and more data 21 on the different materials, I think these applications 22 and the application of these types of tools will 23 become more useful.
24 CHAIR RICCARDELLA: Okay, so it's 25 approaching 3:30. We have one more presentation.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
68 1 David Dijamco, do you think you can get your 2 presentation done in a half hour or so?
3 MR. DIJAMCO: Oh, yes, I think I can, 4 Pete, so, yeah.
5 CHAIR RICCARDELLA: So, would it be okay 6 to take --
7 MR. DIJAMCO: Mine's only ten minutes or 8 so.
9 CHAIR RICCARDELLA: All right, so let's 10 take about a ten-minute break. It's 3:30. We'll 11 break until 3:40 and then we'll reconvene, okay?
12 MEMBER KIRCHNER: Could you afford 15 13 minutes, Pete --
14 CHAIR RICCARDELLA: Okay.
15 MEMBER KIRCHNER: -- for the break?
16 CHAIR RICCARDELLA: I can. We may end up 17 staying a little bit past 4:00, but that's okay.
18 MEMBER KIRCHNER: Thank you.
19 CHAIR RICCARDELLA: All right, so we're on 20 recess until, I have to keep converting the time, 21 until 3:45.
22 (Whereupon, the above-entitled matter went 23 off the record at 3:29 p.m. and resumed at 3:45 p.m.)
24 CHAIR RICCARDELLA: Okay, so it's 3:45 25 Eastern Time and we will resume the meeting. David, NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
69 1 are you ready to go?
2 MR. DIJAMCO: I'm ready to go.
3 CHAIR RICCARDELLA: Okay.
4 MR. DIJAMCO: Okay, so, good afternoon, 5 everybody, and good afternoon, Dr. Riccardella. It's 6 good to hear from you.
7 So, for those who don't know me, my name 8 is Dave Dijamco. I'm from the Office of Nuclear 9 Reactor Regulation from the Vessels and Internals 10 Branch, and I'm going to talk about the use of the PFM 11 guidance from a technical reviewer's perspective.
12 So, the NRC received a submittal from 13 Vogtle using PFM probabilistic fracture mechanics as 14 a technical basis, and so we had a chance to pilot 15 this guidance, which I want to emphasize it's still in 16 an unpublished draft and pre-decisional form, but we 17 had a chance to pilot this guidance during the review 18 of the Vogtle submittal.
19 And so the objective of this presentation 20 I would say is kind of the, is the more informal of 21 the three presentations, which is basically for me to 22 just share my thoughts on questions like how to review 23 when, where to focus, of course, in the guidance, and 24 other questions such as how is the guidance helpful or 25 what did we learn from its use.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
70 1 CHAIR RICCARDELLA: David, I take it this 2 was the operating units and Vogtle 1 and 2, not 3 and 3 4?
4 MR. DIJAMCO: That's correct. That's 5 correct, yeah.
6 CHAIR RICCARDELLA: Thank you.
7 MR. DIJAMCO: 1 and 2, yeah.
8 MEMBER BLEY: David, it's Dennis Bley.
9 You folks used the guidance. Was it available for the 10 folks at Vogtle or did they use it?
11 MR. DIJAMCO: No, I believe they were not 12 available for the Vogtle folks.
13 MEMBER BLEY: Okay.
14 MR. DIJAMCO: It was just internally 15 available to the NRC.
16 MEMBER BLEY: So, we have a case where 17 they did it on their own and then you applied the 18 guidance when you reviewed it. It would be 19 interesting --
20 (Simultaneous speaking.)
21 MR. DIJAMCO: I believe so, to some 22 extent. I believe -- I applied it to what was there.
23 MEMBER BLEY: Okay.
24 MR. RUDLAND: This is Dave Rudland. The 25 Vogtle staff had the EPRI white paper that they used.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
71 1 They followed the EPRI white paper, which was kind of 2 the basis for some of the categorizations that Patrick 3 talked about.
4 MEMBER BLEY: Okay, thanks.
5 MR. DIJAMCO: So, I just wanted to give a 6 quick background. So, EPRI initiated an NDE 7 optimization program that sought to increase the 8 inservice inspection intervals of certain class one 9 and class two components from ten to 30 years, and 10 EPRI issued a series of publicly available reports as 11 a result of this effort, and so each of these EPRI 12 reports is for a specific component.
13 For example, that first report, 14590, 14 it's for steam generator nozzles. The second report 15 is for pressurizer vessel shell wells and so forth, 16 and, of course, the centerpiece in all of this is 17 probabilistic fracture mechanics.
18 And this slide just gives you just a very, 19 very high level view of what the submittal was about.
20 It was basically an alternative request to increase 21 the ISI interval of steam generator welds and nozzles 22 from ten to 30 years, and it referenced the first of 23 that report I showed in the previous slide, that 24 14590, as the technical basis.
25 And I just provided here some basic NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
72 1 information about the submittal and the NRC safety 2 evaluation. All of these documents are available in 3 ADAMS.
4 And I just wanted to mention also that 5 there were actually three more submittals that came in 6 after Vogtle which referenced each of the other three 7 EPRI reports that are shown on the previous slide.
8 Another thing to note is that the NRC did not review 9 any of the EPRI reports for generic use.
10 So, what's this EPRI report 14590? So, 11 again, it's the reference technical basis for the 12 Vogtle submittal, and the specific components were 13 steam generator nozzle to vessel welds and nozzle 14 inside radii.
15 As expected, because it's PFM based, it 16 was extensive. It was nearly 200 pages, lots of 17 technical topics, and in fact, PFM is only one of 18 several chapters.
19 And so this is what my brain looked like 20 after going over the report. I call it my spaghetti 21 brain because of all of these crisscrossing concepts 22 and topics.
23 So, my background is in mostly 24 deterministic analysis. Coming into the review, I did 25 have a little bit of knowledge of PFM.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
73 1 So, even though some of these concepts 2 like degradation mechanisms, finite models, 3 convergence and things like that, they were familiar 4 to me, but I just wasn't sure how to approach the 5 review.
6 So, the main question in my mind as a 7 technical reviewer was how do I review this thing?
8 And, of course, the PFM guidance, which I took to mean 9 the draft guidance and the draft NUREG, they suggested 10 an approach, or maybe it's better to say that I 11 inferred an approach from looking over the guidance.
12 So, how did the guidance help? So, at the 13 time of the Vogtle review, the details of the graded 14 approach that Patrick talked about were still under 15 development.
16 And as Patrick had mentioned, they are 17 still under current internal review, but the guidance 18 at that time had enough of a framework for me to work 19 with, especially the sections on the suggested content 20 of PFM submittals.
21 And I just wanted to note here the EPRI 22 white paper that Dave Rudland mentioned earlier, and 23 that was also on PFM submittals. It's publicly 24 available and that was very helpful in crafting the 25 draft graded approach.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
74 1 And so this brings me to the first thought 2 that I wanted to share, which is that the guidance 3 helped me put a framework around various topics, and 4 thus organizing my review, and these are the topics 5 that are covered in the guidance which Patrick talked 6 about in detail in his presentation.
7 And so again, this is, you know, my 8 spaghetti brain without the guidance and into a well-9 ordered path with the guidance.
10 And this is the actual outline of the 11 safety evaluation for the Vogtle submittal. I'm not 12 going to go through each one of these. The point here 13 is just to show that many of the topics that I've kind 14 of mentioned in the previous slides made it into the 15 safety evaluation.
16 So, I think there's an observation to be 17 made here, which is that I think there was an overall 18 efficiency gained in the submittal and review process.
19 The guidance will help the industry prepare submittals 20 that have a consistent level of quality, which in turn 21 helps the NRC review the submittals.
22 And there's nothing really new in this 23 process. One example that I can think of that's 24 relevant in the (unintelligible) is the use of Reg 25 Guide 1.99, Revision 2, to determine the embrittlement NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
75 1 levels of reactor pressure vessel materials.
2 The second thought that I wanted to share 3 was that the guidance helped me understand sensitivity 4 analysis and sensitivity studies and their importance 5 in interpreting results.
6 And so the PFM results for the base cases 7 were actually pretty far from the acceptance criteria, 8 but those base cases were based on a representative 9 configuration rather than a bounding configuration.
10 So, I felt that during the review, that I 11 needed to understand the impact of the most important 12 input parameters, and the SA and SS helped me in doing 13 that, and both of these topics are covered well in the 14 draft guide and the draft NUREG.
15 The third thought that I wanted to share 16 was that the guidance supports knowledge management.
17 If you're just coming up to speed on PFM like I was, 18 and in many ways, I still am, I think the guidance is 19 a good learning tool.
20 For example, the draft chapter four of the 21 NUREG contains a compendium of methods for 22 establishing confidence in a PFM analysis, so it talks 23 about things like uncertainty, convergence, acceptance 24 methods, things like that.
25 And I think this is a boom to all NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
76 1 stakeholders, especially as we move more and more 2 towards risk-informed decision making, of course of 3 which PFM is only a part as Dave Rudland mentioned 4 during his talk, but it could be an important part.
5 MEMBER BLEY: David?
6 MR. DIJAMCO: Yes?
7 MEMBER BLEY: Coming to this without a 8 real strong background in probabilistic fracture 9 mechanics as many people probably will, were there 10 areas where you were hoping for more help than you 11 found in the guidance, or if there were some of those, 12 did they get factored into the current version of the 13 guidance?
14 MR. DIJAMCO: I think were -- so, at that 15 time, you know, that version of the guidance that was 16 available when I reviewed Vogtle and as I mentioned, 17 you know, again for me, I'm mostly deterministic, but 18 when I needed help, you know, there were others in the 19 staff that I went to, especially, and he's here right 20 now, Dave Rudland, and so I think where I lacked the 21 knowledge and the proper confidence to review it, you 22 know, I consulted with the right people.
23 MR. RUDLAND: Can I just comment also that 24 in addition to what Dave has mentioned, we also, 25 especially for this particular request, held an audit NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
77 1 of the probabilistic fracture mechanics code, which 2 was a unique code just for this application, to gain 3 further knowledge about how the code works, and the 4 quality assurance, and the validation and 5 verification.
6 So, that wasn't necessarily in the 7 submittal because we were able to hold this virtual 8 basically audit of the code to understand it a little 9 better, and that also helps us to understand it if 10 we're not familiar with it.
11 MEMBER BLEY: Thank you for that.
12 MEMBER KIRCHNER: To the two Davids, this 13 is Walt Kirchner. Are there a lot of probabilistic 14 fracture mechanics codes out there? Is each a one-off 15 case or is there, besides the NRC codes, are there, 16 quote, unquote, industry standards that most 17 applicants would turn to?
18 MR. DIJAMCO: How do you feel about that, 19 Dave? I mean --
20 MR. RUDLAND: Well, I'll tell you, you 21 know, there is the hopes of these larger codes like 22 FAVOR and xLPR, that they would be generic enough to 23 be used for a lot of applications.
24 What we're finding is that I think the 25 industry prefers to develop specific codes for their NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
78 1 particular applications. So, we're seeing that more 2 and more, that they're more, I don't want to say one-3 offs, but at least applicable to only a certain set of 4 components.
5 CHAIR RICCARDELLA: You know, Walt, I 6 think a lot of these industry codes though, they kind 7 of have a common kernel, you know, the Monte Carlo 8 sampling, and it's just that they adapt the input and 9 the output and maybe the fracture mechanics models to 10 the specific problem, but a lot of them are based on 11 a common routine. At least I know when we do that, we 12 have one common Monte Carlo sampling approach.
13 MR. RUDLAND: Yeah, and I agree with Pete, 14 and I think that was one of the reasons why we put 15 into our graded approach, you know, a previously 16 approved or approved as the use code that's been 17 modified, because a lot of times, they'll do that.
18 They'll just change a model or this, that, or the 19 other thing, and apply it to something different, so.
20 MEMBER KIRCHNER: Thank you.
21 MR. DIJAMCO: Okay, actually that was my 22 last slide. That's pretty much all I have, but I just 23 wanted to make kind of a couple of final remarks.
24 So, you know, I just shared the thoughts 25 of just one reviewer. As Dave mentioned earlier, Dave NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
79 1 Rudland, we have been getting PFM submittals recently, 2 and I think we do expect to get more and more PFM 3 submittals, and so I think having the guidance is very 4 timely.
5 And I think as these submittals come in, 6 I think it will be interesting to hear the 7 perspectives of other reviewers.
8 CHAIR RICCARDELLA: So, David, just one 9 technical question. You mentioned Reg Guide 1.99. I 10 don't see how that came into play on this Vogtle 11 submittal since it's not in a radiation zone. What 12 was the reference to Reg Guide 1.99?
13 MR. DIJAMCO: Pete, I was just referring 14 to the process of using or having a regulatory guide 15 to, you know, guidance for the industry, and really 16 for the industry and NRC to help the submittal and 17 review process. That's kind of the analogy there.
18 That's just one example of how, you know, the use of 19 reg guidance is --
20 CHAIR RICCARDELLA: Yeah.
21 MR. DIJAMCO: -- has been used in the 22 past.
23 CHAIR RICCARDELLA: Obsolete though it may 24 be.
25 MR. DIJAMCO: Correct, correct, that's NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
80 1 right, yeah.
2 CHAIR RICCARDELLA: I couldn't help but 3 get in that dig.
4 (Laughter.)
5 MR. DIJAMCO: Right, no, I understand.
6 MEMBER BALLINGER: Obsolete, but still 7 used.
8 MR. DIJAMCO: It's still used, yeah.
9 CHAIR RICCARDELLA: Okay, that's a subject 10 for another day, right?
11 (Laughter.)
12 MR. DIJAMCO: That's all I had, so.
13 CHAIR RICCARDELLA: Okay, well, thank you 14 for the presentation, David. Are there any other 15 member comments or discussion? Okay, hearing none, we 16 will turn to public comments. Is the bridge line open 17 now and unmuted?
18 MR. DASHIELL: The bridge line is open for 19 public comments.
20 CHAIR RICCARDELLA: Thank you, Thomas.
21 So, do we have any comments from the public? If so, 22 please state your name and make your comment.
23 DR. COFIE: Dr. Riccardella, this is Nat 24 Cofie here. Can you hear me?
25 CHAIR RICCARDELLA: Yes, I hear you, Nat.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
81 1 How are you doing?
2 DR. COFIE: I'm doing good, Dr. Pete.
3 Thank you for this meeting. You know, I've learned a 4 lot. I just have a simple question for Patrick and 5 maybe Dr. Rudland.
6 They mentioned, you know, codes by, you 7 know, classification, NRC-approved codes. I heard 8 something like that. What constitutes an NRC-approved 9 code? Is there a database or somewhere that one could 10 go and find out what codes for PFM have been approved 11 by the NRC? Is it a code that has gone through an 12 audit? What constitutes an NRC-approved code?
13 DR. RAYNAUD: I can take that and Dave can 14 add to it. For now, obviously NRC's own codes, so 15 FAVOR and XLPR, and we're looking specifically for 16 codes where a safety evaluation report would have been 17 written for that code or for that code in a specific 18 application.
19 CHAIR RICCARDELLA: So, that's 20 significant. If the code has been used for an 21 application and that application was approved, then 22 that constitutes and NRC-approved code?
23 DR. RAYNAUD: I wouldn't say that.
24 MR. RUDLAND: I would say as long as it's 25 used in the exact same application, then, yeah, but if NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
82 1 not, then it would have to go to the different, the 2 next step down where --
3 CHAIR RICCARDELLA: It would be one of 4 those modified categories.
5 MR. RUDLAND: Yeah.
6 DR. RAYNAUD: Exactly.
7 MR. RUDLAND: Because again, the first 8 application of any code, we're going to do like we did 9 with the Vogtle and have an audit or something like 10 that to really understand the workings of the code, so 11 I think the bulk is going to happen in the first types 12 of review.
13 MR. MOORE: Mr. Chairman, does the court 14 reporter need the caller's name?
15 CHAIR RICCARDELLA: Yes, Nat, do you want 16 to give your name and spell it? I can do it.
17 DR. COFIE: Oh, my name?
18 CHAIR RICCARDELLA: It's Nathaniel Cofie, 19 C-O-F-I-E.
20 MR. MOORE: Thank you.
21 CHAIR RICCARDELLA: Are there any other 22 comments?
23 DR. HARRINGTON: Pete, this is Craig 24 Harrington.
25 CHAIR RICCARDELLA: Oh, hi, Craig, how are NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
83 1 you?
2 DR. HARRINGTON: I'm fine. How are you?
3 I just wanted to add onto the comments that you and 4 Dave made a minute ago about other codes. The 5 distinction between like an xLPR or the more one-off 6 codes like were used for Vogtle or some of these other 7 activities.
8 I think to the extent that industry can 9 use an existing code like an xLPR for a new 10 application, they're certainly going to do that 11 because of the benefits of it having some degree of 12 preapproval and, you know, all of that.
13 But back to that first category that 14 Patrick talked about of planning the analysis and the 15 steps to ensure that your quantities of interest are 16 adequately addressed by the code, these probabilistic 17 fracture mechanics analyses are complex enough and 18 they're nuanced enough that very often even with the 19 range of flexibility that we tried to build into xLPR, 20 it still doesn't quite get to exactly the place that 21 you might want it or need it to go.
22 And so I think that drives these one-off 23 codes, not a preference by industry, but just that 24 it's that complex of a problem that it's very hard to 25 solve every probabilistic fracture mechanics problem NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
84 1 with a broad brush tool.
2 And even with xLPR, for any of the 3 analyses that we're doing, we end up tweaking little 4 things or making small adjustments possibly to tailor 5 it to the specific need at hand, so I just wanted to 6 make that clarification.
7 CHAIR RICCARDELLA: Thank you very much for 8 your input, Craig. Is there anybody else out there 9 that would like to make a comment?
10 Well, with that and no further comments 11 from the members, I thank everyone for a very 12 interesting series of presentations and a very 13 informative session, and the meeting is adjourned.
14 (Whereupon, the above-entitled matter went 15 off the record at 4:07 p.m.)
16 17 18 19 20 21 22 23 24 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
Need for Probabilistic Fracture Mechanics Guidance Remarks by David L. Rudland, Ph.D.
Senior Level Advisor for Materials Division of New and Renewed Licenses Office of Nuclear Reactor Regulations Advisory Committee on Reactor Safeguards Meeting of the Subcommittee on Metallurgy & Reactor Fuels July 20, 2021
Integrated Decision Making
- Objective is integrated decision making
- Key is risk informed not risk based
- Use of risk insights for passive component integrity 07/20/2021 PFM Guidance l ACRS Sub-Committee Meeting 2
Risk-Informed Licensing Other licensing actions Guidance/Process Under development 07/20/2021 PFM Guidance l ACRS Sub-Committee Meeting 3
Use of Probabilistic Fracture Mechanics
- Inputs to PRA
- Change in Inspection Schedule - eliminate inspections?
- Estimate LOCA Frequency
- Leak-before-break Probabilistic Fracture Mechanics (PFM) is being used Requests for LOCA Inspection Relief Frequencies LBB Others?
07/20/2021 PFM Guidance l ACRS Sub-Committee Meeting 4
Licensing Reviews and Emergent Issues Process defined Process under development Risk-Informed LIC-206, others License Basis Changes (RD 1.174/RG 1.200 Process defined Risk/PRA Need adequate, consistent Requests TSTF-505 Risk-informed LIC-504 information and confidence in results with PFM Completion times Acceptability NFPA-805 Risk-informed Fire Protection Risk information Emergent submitted outside of the 10CFR 50.69 issues?
formal risk-informed licensing SSC Categorization Relief basis change process Requests 5b Risk-informed Surveillance Frequencies Deterministic Risk-informed Reviews Inservice Inspection Qualitative Reliance on Licensee Risk Information Quantitative 07/20/2021 PFM Guidance l ACRS Sub-Committee Meeting 5
PFM - Past Regulatory Issues
- Difficult for NRC staff to reproduce or verify PFM calculations o Complex regulatory review o Black Box codes with insufficient vetting of inputs and code: low confidence in outputs
- Challenges where PFM was proposed as regulatory basis for long term inspection programs under NRC rulemaking 07/20/2021 PFM Guidance l ACRS Sub-Committee Meeting 6
PFM Hurdles and Path Forward
- PFM submittals and level of acceptability have been inconsistent
- Risk-informed decision making is still in the process of being implemented throughout all agency activities Change Defense meets in Depth
- PFM results are only one part of risk-informed Regs decision making Safety Monitor Margin
- Some license processes under development
- Use of PFM for licensing is increasing rapidly Increase in Risk
- PFM Regulatory Guide will provide for consistent and adequate information for licensing application reviews 07/20/2021 PFM Guidance l ACRS Sub-Committee Meeting 7
RES Perspectives on PFM Analyses and on a Graded Approach for PFM Remarks by Patrick A. C. Raynaud, Ph.D.
Senior Materials Engineer Division of Engineering Office of Nuclear Regulatory Research Advisory Committee on Reactor Safeguards Meeting of the Subcommittee on Metallurgy & Reactor Fuels July 20, 2021
Objectives and Disclaimers The goals of this presentation are:
- 1. Describe NRC/RES current perspectives on the analytical steps that could be used to create a high-confidence PFM demonstration for the purpose of demonstrating an adequate solution to a technical problem
- 2. Describe what a graded approach for PFM analyses and documentation might look like when considering the safety significance and the complexity of a given problem Disclaimers:
- 1. All content shown in these slides is pre-decisional and does not represent an official position of the NRC
- 2. None of the ideas presented here are final
- 3. This content is not intended to be guidance on what constitutes an acceptable approach for PFM submittals to the NRC 07/20/2021 Probabilistic Fracture Mechanics l ACRS Sub-Committee Meeting 2
Outline
- Part 1: Analytical steps in a probabilistic fracture mechanics demonstration
- Part 2: Detailed thoughts on a graded approach for PFM 07/20/2021 Probabilistic Fracture Mechanics l ACRS Sub-Committee Meeting 3
Analytical steps in a probabilistic fracture mechanics demonstration Step Action
- Define the regulatory context
- 1. Translate regulatory *Define the QoI and how it relates to the model output and acceptance criteria Plan requirements into an
- Determine the suitability of the PFM code for the application analysis plan
- Identify key elements of the problem that impact analysis choices Analyze 2. Characterize model input uncertainty
- Identify uncertain model inputs
- Specify probability distributions on uncertain inputs
- Select a sampling scheme
- 3. Estimate QoIs and *Assess sampling uncertainty their associated Analysis uncertainty *Conduct sensitivity analysis results *Conduct output uncertainty analysis inconclusive, refinements 4. Conduct sensitivity studies to assess the *Determine a set of sensitivity studies needed.
credibility of modeling *Conduct sensitivity studies and present results assumptions
- 5. Draw conclusions *Interpret analysis results Synthesize From analysis results *Iterate on the analysis process to refine model results 07/20/2021 Probabilistic Fracture Mechanics l ACRS Sub-Committee Meeting 4
Step 1: Translation of Regulatory Requirements into an Analysis Plan
- Action 1: Define the Regulatory Context
- How will PFM analyses will be used as a technical basis for a regulatory action?
- What criteria will be used to support a proposed regulatory action?
- Action 2: Define the Quantity of Interest and How it Relates to the Model Output and Acceptance Criteria
- Map regulatory requirements onto specific model outputs
- Ensure that the model is predicting appropriate and relevant quantities
- Action 3: Determine the Suitability of the Probabilistic Fracture Mechanics Code for the Specific Application
- Determine whether a specific PFM code is suitable for the application of interest
- Identify any potential limitations of the code with regard to the application
- Action 4: Identify Key Elements of the Problem that Impact Analysis Choices
- Identify key elements of the PFM application that will determine how to conduct the analysis
- Example: simplifications to a model because of computational limitations or because the problem is inherently not complex 07/20/2021 Probabilistic Fracture Mechanics l ACRS Sub-Committee Meeting 5
Step 2: Model Input Uncertainty Characterization
- Action 1: Identify Uncertain Model Inputs
- Determine which model inputs are treated with uncertainty
- Determine the type of uncertainty (aleatory or epistemic) for each input
- Action 2: Specify Probability Distributions on Uncertain Inputs
- Uncertainty in model inputs is represented through probability distributions
- Uncertainty is propagated forward to the model outputs to estimate and quantify uncertainty in QoIs 07/20/2021 Probabilistic Fracture Mechanics l ACRS Sub-Committee Meeting 6
Step 3: Estimation of Quantity of Interest and Associated Uncertainty
- Action 1: Select a Sampling Scheme for Sampling Uncertain Model Inputs
- Select a method for propagating uncertainty in the model inputs through the model
- Goal is to estimate the QoI and the associated uncertainty
- Action 2: Assess Sampling Uncertainty: Statistical Convergence Analysis
- Assess the statistical convergence of QoI estimates from model outputs given a sampling scheme
- Action 3: Conduct Sensitivity Analyses to Determine Input Uncertainty Importance
- Identify problem drivers
- Confirm that the model is behaving as expected
- Identify inputs that may need refinement before final estimation of the QoI
- Identify assumptions may be candidates for sensitivity studies (Step 4)
- Identify candidates for more targeted sampling methods such as importance sampling
- Action 4: Conduct Output Uncertainty Analysis
- Provide a final estimate, with associated uncertainty, of the QoI
- Visualize results 07/20/2021 Probabilistic Fracture Mechanics l ACRS Sub-Committee Meeting 7
Step 4: Sensitivity Studies to Assess the Credibility of Modeling Assumptions
- Action 1: Determine a Set of Sensitivity Studies
- Identify important assumptions that merit further scrutiny
- Understand what might happen if these assumptions were changed
- Action 2: Conduct Sensitivity Studies and Present Results
- Many different options for sensitivity studies, but same common elements
- Reference realization (or baseline case) with a documentation of the QoI
- One or several modified realizations illustrating the concept that needs to be represented
- Comparison between the reference realization and the modified realization(s)
- Comparison criterion to decide whether the change is significant
- Conclusion, including potential consequences 07/20/2021 Probabilistic Fracture Mechanics l ACRS Sub-Committee Meeting 8
Step 5: Draw Conclusions from Analysis Results
- Action 1: Interpret Analysis Results
- Synthesize the information gathered in Steps 1 through 4
- Draw conclusions from the information
- May be able to directly compare PFM results with an acceptance criterion
- May need several analyses to create an evidence package
- Action 2: Iterate on the Analysis Process to Refine Model Results
- Determine whether additional analyses are required to draw informative conclusions from the modeling
- If results inconclusive, many possible paths:
- Changing or clarifying aspects of the PFM code
- Refining the input uncertainty distributions
- Choosing a different sampling scheme or increasing the number of model realizations
- Adding more sensitivity studies to address existing limitations 07/20/2021 Probabilistic Fracture Mechanics l ACRS Sub-Committee Meeting 9
Detailed thoughts on a graded approach for PFM
- PFM is complex
- Topics Covered
- Software QA and V&V
- The depth and breadth of a PFM analysis might vary - Models widely depending on several factors - Inputs
- Uncertainty Propagation
- It makes sense to take a graded approach - Convergence
- for PFM analyses themselves - Sensitivity Analyses
- for the level of detail to be presented as part of an - QoI Uncertainty Characterization evidence package - Sensitivity Studies
- General Principles
- Higher safety significance More analyses, more documentation
- Higher complexity Higher burden to create defensible and rigorous
- Higher level of novelty evidence 07/20/2021 Probabilistic Fracture Mechanics l ACRS Sub-Committee Meeting 10
Software QA and V&V
- Safety demonstrations for the NRC usually require that a QA program be in place
- Pre-submittal meetings are very useful to help ensure that everyone agrees on the graded approach path Category Description Graded Approach QV-1 NRC-approved code QV-1A Exercised within validated range Demonstrate code applicability within the validated range.
Describe features of the specific application where the code is validated and applicable (i.e., areas of known code capability).
QV-1B Exercised outside of validated range Provide evidence for the applicability of the code to the specific application with respect to the areas of unknown code capability.
Describe features of the specific application where the code has not been previously validated and applied (i.e., areas of unknown code capability).
QV-1C Modified Give an SQA summary and V&V description for modified portions of the code.
Demonstrate that the code was not broken as a result of changes.
Make detailed documentation available for further review upon request.
QV-2 Commercial off-the-shelf software designed Demonstrate code applicability.
for the specific purpose of the application Describe the software and its pedigree.
Make software and documentation available for review upon request.
QV-3 Custom code Summarize the SQA program and its implementation.
Provide a basic description of the measures for quality assurance, including V&V of the PFM analysis code as applied in the subject report.
For very simple applications, possibly provide the source code instead of standardized SQA and V&V.
Include separate deterministic fracture mechanics analyses to support other validation results, as appropriate for a given application.
07/20/2021 Probabilistic Fracture Mechanics l ACRS Sub-Committee Meeting 11
Models
- Models could be addressed with the following graded approach
- Each model in a given PFM tool could potentially be categorized differently Category Description Graded Approach M-1 Model from a code in category Reference existing documentation for that model in the NRC-approved code, demonstrate that the current range of the QV-1A or QV-1B within the same model is within the previously approved and validated range, and demonstrate that the model functions as intended in the validated range new software.
M-2 Model from a code in category See M-1, except demonstrate validity of the model for the new applicability range (document a comparison of model QV-1A or QV-1B outside the predictions for the entire new range to applicable supporting data, including quantitative goodness-of-fit analyses).
validated range M-3 Model derived from a category M-1 See M-2, and include a detailed description of changes to the M-1 or M-2 model, with justification for the validity of the new or M-2 model model.
M-4 Well-established model not Describe gaps and limitations in the code capabilities for the analysis, combined with a strategy for mitigating identified previously part of an NRC-approved gaps and communicating any remaining issues or risks.
code Describe the model(s) applied in the PFM analysis code in sufficient detail so a competent analyst familiar with the relevant subject area could independently implement the model(s) from the documentation alone. Model forms can either be theoretical, semiempirical, or empirical.
Establish a basis for all significant aspects of the model(s). This may consist of raw data or published references. Document or reference any algorithms or numerical methods (e.g., root-finding, optimization) needed to implement the model(s). Discuss any significant assumptions, approximations, and simplifications made, including their potential impacts on the analysis.
Identify important uncertainties or conservatisms.
Describe the computational expense of the model and how that might affect analysis choices.
M-5 First-of-a-kind model not yet See M-4, and perform and document model sensitivity studies to understand trends in the model, as compared to expected published in a peer-reviewed journal model behavior and to the data used to develop the model, and describe model maturity and the status of the technical basis.
07/20/2021 Probabilistic Fracture Mechanics l ACRS Sub-Committee Meeting 12
Inputs Category Graded Approach I-1D List input value.
I-1R List input distribution type and parameters. Input Category Low Knowledge of Input High Knowledge of Input If applicable, list uncertainty classification (aleatory or epistemic). Characteristics Characteristics I-2D List input value. Deterministic Uncertain Deterministic Uncertain If there is a lack of data, justify the use of expert judgment. High Importance I-4D I-4R I-3D I-3R I-2R List input distribution type and parameters. Low Importance I-2D I-2R I-1D I-1R If applicable, list uncertainty classification (aleatory or epistemic).
If there is a lack of data, justify the use of expert judgment.
I-3D List input value.
State the rationale for setting the input to a deterministic value.
For each deterministic input, give the rationale (method and data) for the selection of its numerical value, along with any known conservatisms in that numerical value and the rationale for such conservatisms.
Reference documents that contain the foundation for input choices.
Explain the correlations between inputs and how they are modeled, and verify that correlated inputs remain consistent and physically valid.
Describe any sensitivity analyses/studies performed to show that the input or its classification does not have a significant effect on the QoI.
I-3R List input distribution type and parameters.
If applicable, list uncertainty classification (aleatory or epistemic).
If relevant, classify uncertain inputs as aleatory or epistemic and give the corresponding rationale.
For each uncertain input, describe both its distribution parameter values and its distributional form. Give the rationale (method and data) for selecting each distribution, including any known conservatisms in the specified input distributions and the rationale for the conservatism. Detail the distributional fitting method, including interpolation, extrapolation, distribution truncation, and curve fitting.
Reference documents that contain the foundation for input choices.
Explain the correlations between inputs and how they are modeled, and verify that correlated inputs remain consistent and physically valid.
Describe any sensitivity analyses/studies performed to show that the input or its classification does not have a significant effect on the QoI.
I-4D See I-3D.
If there is a lack of data, justify the use of expert judgment.
I-4R See I-3R.
If there is a lack of data, justify the use of expert judgment.
07/20/2021 Probabilistic Fracture Mechanics l ACRS Sub-Committee Meeting 13
Uncertainty Propagation
- Each analysis within a larger PFM demonstration could potentially fall in a different category Category Description Graded Approach UP-1 Analysis does not Give the method for uncertainty propagation and describe the simulation framework.
employ a surrogate If Monte Carlo sampling is used, describe the finalized sampling scheme and rationale for the sampling scheme, including model sampling method, sample size, the pseudo-random number generation method, and the random seeds used.
Describe the approach for maintaining separation of aleatory and epistemic uncertainties, if applicable.
If importance sampling is used to oversample important regions of the input space, justify the choice of importance distribution.
UP-2 Analysis does employ a See UP-1 and describe the form of the surrogate model(s), any approximations or assumptions, the method used for fitting the surrogate model surrogate, and the validation process for the surrogate model.
UP-2A Surrogate model used See UP-2 and describe the features of the different surrogate models used.
for sensitivity analysis UP-2B Surrogate model is See UP-2 and quantify the magnitude of error associated with the surrogate model approximation and include as additional used for uncertainty uncertainty in the estimation of the QoI.
propagation 07/20/2021 Probabilistic Fracture Mechanics l ACRS Sub-Committee Meeting 14
Convergence
- Discretization convergence should be achieved to have valid results
- Statistical convergence graded approach for each analysis within PFM demonstration Category Description Graded Approach SC-1 [Acceptance criteria met with at least one order of No sampling uncertainty characterization recommended as long as the uncertainty is magnitude margin] AND [no importance sampling sufficiently small relative to the margin.
AND no surrogate models used]
SC-2A [Acceptance criteria met with at least one order of Describe the approach used for assessing statistical convergence, with one method magnitude margin] AND [use of importance sampling needed for sampling uncertainty characterization.
OR surrogate models OR both] Explain the approach used for characterizing sampling uncertainty.
Justify why the sampling uncertainty is small enough for the intended purpose (i.e., why statistical convergence is sufficient for the intended purpose).
Describe how sampling uncertainty is used in the interpretation of the results.
SC-2B [Acceptance criteria met with at least one order of See SC-2A, and distinguish between epistemic and aleatory means and standard magnitude margin] AND [use of importance sampling deviations.
OR surrogate models OR both)) AND [separation of aleatory and epistemic uncertainties is implemented in the PFM code]
SC-3A [Acceptance criteria met with less than one order of See SC-2A, and provide two different methods for sampling uncertainty characterization.
magnitude margin]
SC-3B [Acceptance criteria met with less than one order of See SC-3A and give a sample size convergence analysis for both the aleatory and magnitude margin] AND [separation of aleatory and epistemic sample sizes.
epistemic uncertainties is implemented in the PFM code]
07/20/2021 Probabilistic Fracture Mechanics l ACRS Sub-Committee Meeting 15
Sensitivity Analyses Sensitivity Category Description Analysis Graded Approach Needed?
SA-1 Previously approved code (QV-1, QV- No Describe important input and measure of input importance from previous use.
1A) with same QoI and same inputs SA-2 Previously approved code (QV-1, QV- Yes Explain the methods used for sensitivity analysis, including any initial screening and model approximations 1A) with different QoI and assumptions.
State whether a local or global sensitivity analysis approach is used.
Give the QoI used for the sensitivity analysis.
For a global sensitivity analysis, describe the sampling scheme along with the rationale for selection, including the sampling technique, number of model realizations, and random seed for the model realizations.
Provide the results of the sensitivity analysis, including the most important model inputs identified; a measure of the input importance, such as the variance explained by the most important inputs; and relevant graphical summaries of the sensitivity analysis results.
SA-3 Modified approved code with limited Yes Describe analyses, important input, and measure of input importance.
independent variables (e.g., <5, determined on a case-by-case basis)
SA-4 Modified approved code with many Yes See SA-2.
independent variables (e.g., >5, determined on a case-by-case basis)
SA-5 First-of-a-kind code with limited Yes Describe the analyses, important input, and measure of input importance and include additional independent variables (e.g., <5, documentation.
determined on a case-by-case basis)
SA-6 First-of-a-kind code with many Yes, with See SA-2.
independent variables (e.g., >5, submodel SA as Indicate how the sensitivity analysis results informed future uncertainty propagation for estimation of the determined on a case-by-case basis) appropriate QoI and associated uncertainty.
State whether the results of the sensitivity analysis are consistent with the expected important inputs based on expert judgment.
07/20/2021 Probabilistic Fracture Mechanics l ACRS Sub-Committee Meeting 16
QoI Uncertainty Characterization Category Description Graded Approach O-1 Acceptance criteria met with at least one order of Give a measure of the best estimate and uncertainty in the QoI.
magnitude margin Include a graphical display of the output uncertainty.
Describe how the best estimate and its uncertainty were calculated, including a clear description of the types of uncertainty (e.g., input, sampling, epistemic) being summarized.
Summarize key uncertainties considered in the analysis and any major assumptions, conservatisms, or simplifications that were included and assess (qualitative or quantitative) their effect on the analysis conclusions.
O-2A Acceptance criteria met with less than one order of See O-1 and provide the reasoning behind a strong basis.
magnitude margin and a strong basis for input distributions and uncertainty classification O-2B Acceptance criteria met with less than one order of See O-1.
magnitude margin and no strong basis for input Include a sensitivity analysis (if important inputs are unknown) and sensitivity studies for any inputs that do not distributions or uncertainty classification, or both have a strong basis.
O-3 O-1, O-2A, or O-2B and potential unknowns See O-1, and provide the reasoning behind a strong basis.
Describe potential unknowns and their possible effect on analysis results.
OR Include a sensitivity analysis (if important inputs are unknown) and sensitivity studies for any inputs that do not have a strong basis.
07/20/2021 Probabilistic Fracture Mechanics l ACRS Sub-Committee Meeting 17
Sensitivity Studies Category Description Sensitivity Study Needed? Graded Approach SS-1 Category QV-1A code with same QoI No Summarize sensitivity studies conducted in prior approval.
SS-2 Category QV-1A code with different QoI Limited, focused on inputs Summarize past sensitivity studies conducted in prior approval and current sensitivity related to QoI studies.
SS-3 Category QV-1B or QV-1C code with limited Limited, focused on impact of Summarize past and current sensitivity studies.
independent variables (e.g., <5, determined modification on a case-by-case basis)
SS-4 Category QV-1B or QV-1C code with many Yes, focused on inputs related Summarize past and current sensitivity studies.
independent variables (e.g., >5, determined to QoI List the uncertain assumptions that are considered for sensitivity studies.
on a case-by-case basis)
State the impact and conclusion of each sensitivity study.
Give the rationale for why certain assumptions were or were not considered for sensitivity studies.
Provide the specific question(s) each sensitivity study is attempting to answer.
Describe a reference realization.
Describe how each sensitivity study is translated into model realizations, and compare the study and the reference realization.
List changes to the code and the QA procedure used.
SS-5 Category QV-2 or QV-3 code with limited Yes See SS-4.
independent variables (e.g., <5, determined on a case by case basis)
SS-6 Category QV-2 or QV-3 code with many Yes, model and input studies See SS-4.
independent variables (e.g., >5, determined on a case by case basis) 07/20/2021 Probabilistic Fracture Mechanics l ACRS Sub-Committee Meeting 18
Next Steps
- NRC will publish draft regulatory guidance and an accompanying draft technical basis NUREG, both for public comments
- NRC will gather public comments on the draft guidance and draft NUREG
- NRC will address all comments received
- Public meetings may take place, if needed
- At least one more ACRS briefing on the topic of PFM and PFM regulatory guidance is expected in the future 07/20/2021 Probabilistic Fracture Mechanics l ACRS Sub-Committee Meeting 19
Use of PFM Guidance A technical reviewers perspective July 20, 2021 David Dijamco Office of Nuclear Reactor Regulation Vessels and Internals Branch
Objective The NRC received a submittal from Vogtle using PFM as technical basis; pilot the PFM guidance (unpublished, draft, pre-decisional) during the review of this submittal.
Objective of this presentation is to share thoughts on:
- How the review went (with a focus on the draft guidance).
- How was the draft guidance helpful? What did we learn from its use?
7/20/2021 PFM Guidance l ACRS Sub-Committee Meeting 2
=
Background===
EPRI initiated an NDE optimization program that sought to increase the inservice inspection (ISI) intervals of certain Class 1/Class 2 components from 10 to 30 years. Issued a series of publicly available reports as a result of this effort:
- EPRI report 14590 (ML19347B107) - steam generator nozzles
- EPRI report 15905 (ML21021A271) - pressurizer vessel shell welds
- EPRI report 15906 (ML20225A141) - steam generator vessel shell welds
- EPRI report 18473 (ML21091A014) - BWR RHR heat exchanger welds and nozzles Center piece is probabilistic fracture mechanics (PFM).
7/20/2021 PFM Guidance l ACRS Sub-Committee Meeting 3
Vogtle Units 1 and 2 submittal
- Alternative request to increase the ISI interval of steam generator welds and nozzles from 10 to 30 years. Referenced EPRI report 14590 as technical basis.
- Submittal:
ML19347B105 - Dec. 2019 ML20253A311 - revised/supplemented Sept. 2020
- NRC safety evaluation:
ML20352A155 - Jan. 2021 Note: The NRC did not review EPRI report 14590 and the other EPRI reports for generic use.
7/20/2021 PFM Guidance l ACRS Sub-Committee Meeting 4
EPRI Report 14590
- Referenced technical basis for the Vogtle submittal for steam generator nozzle-to-vessel welds and nozzle inside radii
- Extensive report, nearly 200 pages, lots of technical topics (PFM is only one of several chapters) 7/20/2021 PFM Guidance l ACRS Sub-Committee Meeting 5
My brain after going over EPRI Report 14590 (spaghetti brain)
Examination coverage Stress intensity factors Sensitivity studies Selection of transient loads 7/20/2021 PFM Guidance l ACRS Sub-Committee Meeting 6
Main question as a technical reviewer was:
How do I review the submittal???
PFM Guidance Draft Regulatory Guide + Draft NUREG-7278 (both unpublished, still under internal review) 7/20/2021 PFM Guidance l ACRS Sub-Committee Meeting 7
How did the guidance help?
- At the time of the Vogtle review, details of graded approach were still under development (and are currently under internal review)
- But it had enough of a framework for me to work with, especially the sections on the suggested content of PFM submittals.
(Note: EPRI white paper on PFM submittals (ML19241A545, publicly available) was very helpful in crafting the draft graded approach.)
7/20/2021 PFM Guidance l ACRS Sub-Committee Meeting 8
- 1. The (draft, pre-decisional) PFM guidance helped me put a framework around the various topics, thus organizing my review.
Topics covered in the guidance
- Models
- Inputs
- Uncertainties
- Sensitivity analyses
- Sensitivity studies
- Convergence
- Software V& V 7/20/2021 PFM Guidance l ACRS Sub-Committee Meeting 9
Without PFM Guidance Examination coverage Stress intensity factors Sensitivity studies Selection of transient loads 7/20/2021 PFM Guidance l ACRS Sub-Committee Meeting 10
With PFM Guidance Degradation mechanisms Deterministic fracture mechanics Fatigue crack growth Stress intensity factors Fracture toughness Component configuration selection Probabilistic fracture mechanics Examination coverage Stress analysis Sensitivity studies Selection of transients Sensitivity analysis Finite element models Sampling method Pressure stress analysis Convergence Thermal stress analysis Software V&V 7/20/2021 PFM Guidance l ACRS Sub-Committee Meeting 11
Outline of safety evaluation of the Vogtle submittal
- 3.8.1 Overall PFM Approach (acceptance criteria, sampling method, etc.)
- 3.8.2 Parameters most significant to PFM results
- 3.8.3 Stress analysis
- 3.8.4 Fracture toughness
- 3.8.5 Flaw density
- 3.8.6 Fatigue crack growth rate
- 3.8.7 ISI & examination coverage
- 3.8.8 Other considerations (probability of detection, SIF models, V&V, etc.)
- 3.8.9 PFM results relevant to Vogtles request 7/20/2021 PFM Guidance l ACRS Sub-Committee Meeting 12
Overall efficiency gained in submittal/review process The (draft, pre-decisional) PFM guidance
- helps the industry prepare submittals that have a consistent level of quality
- helps the NRC review the submittals 7/20/2021 PFM Guidance l ACRS Sub-Committee Meeting 13
- 2. The (draft, pre-decisional) PFM guidance helped me understand sensitivity analysis (SA) and sensitivity studies (SS) and their importance in the interpreting the PFM results.
Both the (draft, pre-decisional) regulatory guide and (draft, unpublished)
NUREG-7278 cover these topics.
7/20/2021 PFM Guidance l ACRS Sub-Committee Meeting 14
- 3. The (draft, pre-decisional) PFM guidance supports knowledge management.
- Draft Chapter 4 of NUREG-7278 contains a compendium of methods for establishing confidence in a PFM analysis
- A boon to all stakeholders, especially as we move more and more toward RIDM of which PFM is only a part, but its an important part.
7/20/2021 PFM Guidance l ACRS Sub-Committee Meeting 15