ML21223A043
| ML21223A043 | |
| Person / Time | |
|---|---|
| Issue date: | 07/20/2021 |
| From: | Advisory Committee on Reactor Safeguards |
| To: | |
| Brown, C, ACRS | |
| References | |
| NRC-1596 | |
| Download: ML21223A043 (127) | |
Text
Official Transcript of Proceedings NUCLEAR REGULATORY COMMISSION
Title:
Advisory Committee on Reactor Safeguards Metallurgy & Reactor Fuels and Structural & Seismic Analysis Subcommittees Docket Number:
(n/a)
Location:
teleconference Date:
Tuesday, July 20, 2021 Work Order No.:
NRC-1596 Pages 1-84 NEAL R. GROSS AND CO., INC.
Court Reporters and Transcribers 1323 Rhode Island Avenue, N.W.
Washington, D.C. 20005 (202) 234-4433
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 www.nealrgross.com 1
1 2
3 DISCLAIMER 4
5 6
UNITED STATES NUCLEAR REGULATORY COMMISSIONS 7
ADVISORY COMMITTEE ON REACTOR SAFEGUARDS 8
9 10 The contents of this transcript of the 11 proceeding of the United States Nuclear Regulatory 12 Commission Advisory Committee on Reactor Safeguards, 13 as reported herein, is a record of the discussions 14 recorded at the meeting.
15 16 This transcript has not been reviewed, 17 corrected, and edited, and it may contain 18 inaccuracies.
19 20 21 22 23
1 UNITED STATES OF AMERICA 1
NUCLEAR REGULATORY COMMISSION 2
+ + + + +
3 ADVISORY COMMITTEE ON REACTOR SAFEGUARDS 4
(ACRS) 5
+ + + + +
6 JOINT SUBCOMMITTEES ON METALLURGY & REACTOR FUELS 7
AND STRUCTURAL & SEISMIC ANALYSIS 8
+ + + + +
9 TUESDAY 10 JULY 20, 2021 11
+ + + + +
12 The Subcommittees met via Teleconference, 13 at 2:00 p.m. EDT, Peter Riccardella, Chair, presiding.
14 15 COMMITTEE MEMBERS:
16 PETER RICCARDELLA, Chair 17 RONALD G. BALLINGER, Member 18 VICKI M. BIER, Member 19 DENNIS BLEY, Member 20 CHARLES H. BROWN, JR. Member 21 VESNA B. DIMITRIJEVIC, Member 22 GREGORY H. HALNON, Member 23 WALTER L. KIRCHNER, Member 24 JOSE MARCH-LEUBA, Member 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
2 DAVID A. PETTI, Member 1
JOY L. REMPE, Member 2
MATTHEW W. SUNSERI, Member 3
4 ACRS CONSULTANT:
5 STEPHEN SCHULTZ 6
7 DESIGNATED FEDERAL OFFICIAL:
10 ALSO PRESENT:
11 RAJ IYENGAR, RES/DE/REB 12 DAVID RUDLAND, NRR/DNRL 13 PATRICK RAYNAUD, RES/DE/REB 14 DAVID DIJAMCO, NRR/DNRL/NVIB 15 SCOTT MOORE, ACRS 16 THOMAS DASHIELL, ACRS/PMDA 17 18 19 20 21 22 23 24 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
3 CONTENTS 1
Opening Remarks and Objectives 4
2 Staff Opening Remarks..............
6 3
Need for PFM Guidance..............
9 4
RES Perspectives on PFM Analyses and 5
on a Graded Approach for PFM
....... 17 6
NRR Presentation on the use of PFM Guidance... 69 7
Public Comments................. 80 8
Adjourn..................... 84 9
10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
4 P R O C E E D I N G S 1
(2:00 p.m.)
2 CHAIR RICCARDELLA: It is now 2 p.m.
3 Eastern Time and the meeting will come to order.
4 This is a meeting of the Metallurgy and 5
Reactor Fuels and Structural and Seismic Analysis 6
Subcommittees of the Advisory Committee on Reactor 7
Safeguards. I'm Pete Riccardella, chairman of today's 8
session.
9 ACRS members in attendance are Joy Rempe, 10 Dennis Bley, Jose March-Leuba, David Petti, Matthew 11 Sunseri, Ron Ballinger, Vesna Dimitrijevic, Walt 12 Kirchner, Vicki Bier, Gregory Halnon. Our consultant 13 Steve Schultz is also on the phone.
14 Christopher Brown of the ACRS is the 15 designated federal official for the meeting.
16 I did not call Charles Brown. Charles, 17 are you here?
18 (No response.)
19 CHAIR RICCARDELLA: Evidently not.
20 During today's meeting the subcommittee 21 will discuss NRC's perspectives on probabilistic 22 fracture mechanics analysis and on a graded approach 23 for PFM.
The joint subcommittee will hear 24 presentations by and hold discussions with the NRC 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
5 staff from NRR and Research and other interested 1
persons regarding this matter.
2 The rules for participation in all ACRS 3
meetings, including today's, were announced in the 4
Federal Register on June 13th, 2019. The ACRS section 5
of the USNRC public website provides a charter, 6
bylaws, agenda, letter reports, and old transcripts of 7
all full and subcommittee meetings, including slides 8
presented there.
9 The meeting notice and agenda for this 10 meeting were posted there. We have received no 11 written statements or requests to make an oral 12 statement from the public.
13 This subcommittee will gather information, 14 analyze relevant issues and facts, and formulate 15 proposed positions and actions as appropriate for 16 deliberation by the full committee. Lists for 17 participation in today's meeting have been announced 18 as part of the notice of this meeting previously 19 published in the Federal Register.
20 A transcript of the meeting is being kept 21 and will be made available as stated in the Federal 22 Register notice.
23 Due to the COVID pandemic, today's meeting 24 is being held over Microsoft Teams for ACRS members 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
6 and NRC staff. There is also a telephone bridge line 1
allowing participation of the public over the phone.
2 When addressing the subcommittee, the participants 3
should first identify themselves and speak with 4
sufficient clarify and volume so that they may readily 5
be heard.
6 When not speaking, we request that 7
participants mute your computer or phone.
8 We will now proceed with the meeting. And 9
I'd like to stop by calling on Raj Iyengar for opening 10 remarks.
11 Raj, are you there?
12 MR. IYENGAR: Yes, I am, thank you. Can 13 you hear me?
14 CHAIR RICCARDELLA: Yes, we can.
15 MR. IYENGAR: Okay. I really appreciate 16 this opportunity to appear in front of you members and 17 to let our staff provide a briefing on this very 18 significant topic on probabilistic fracture mechanics 19 and the application results of those for reactor 20 component safety.
21 As you may all know, this topic people 22 have been working on probabilistic fraction mechanics 23 for several decades. There have been different kinds 24 of tools available. But until recently, there has not 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
7 been a concerted effort to develop a strong basis for 1
application of these tools in order to provide 2
reasonable assurance of adequate protection for 3
components.
4 I think the NRC staff from Research as 5
well as from NRR joined forces to develop a PFM, 6
probabilistic fraction mechanics guidance in early 7
2016. And that's been published by technical letter 8
report on this topic in 2018. Since then we've gotten 9
quite bit, a lot of feedback on that.
10 And as the subject of this endeavor, the 11 staff had proposed to develop a probabilistic fraction 12 mechanics regulatory guide for to better understand 13 and provide better guidance for licensees when they 14 use probabilistic fraction mechanics tools for license 15 amendments, or relief requests, licensing actions.
16 So that I think we've done a pretty great 17 collaboration between research and NRR, with the help 18 of Sandia National Labs, to develop the current new 19 technical basis document, as well as the reg guide 20 that you'll hear being talked about.
21 And I did want to mention, throughout the 22 process we've had several public interactions with our 23 stakeholders. We've been keeping with our good 24 principles of regulation, in particular with EPRI who 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
8 provided very constructive input and developed a white 1
paper on a graded approach to PFM. Which you will see 2
that some of the aspects were reflected in the current 3
document.
4 The draft documents associated with the 5
NUREG technical basis, NUREG and the draft reg guide 6
are still under internal review, so we will not be 7
able to make it public at this time. However, the 8
staff was very open to providing some kind of 9
considerations that we have in developing this 10 regulatory guide.
11 Today we'll hear from our regulator NRR 12 about why they believe NRC needs to be of some 13 guidance and why at this time. We are going to save 14 that thunder for Dave Rudland.
15 Then Research will, staff will present our 16 latest perspectives on PFM analysis and on a graded 17 approach for PFM. Research and NRR are presenting 18 their thoughts of PFM guidance in regulatory 19 application. As you know, that is the most important 20 aspect of what we do.
21 We look forward to having a productive 22 meeting and, hopefully, answering all your insightful 23 questions to the best of our ability.
24 I actually want to commend our staff from 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
9 NRR as well as Research who really worked very hard in 1
the last several years to make this happen.
2 With that, I'd like to turn this over to 3
Dave Rudland to provide his thoughts on PFM.
4 Thank you.
5 MR. RUDLAND: Thanks, Raj. So, as Raj 6
mentioned, my name is Dave Rudland and I am a senior 7
technical advisor for materials in the Division of New 8
and Renewed Licenses in NRR. And I'm going to provide 9
today a couple of slides on the need for probabilistic 10 fraction mechanics guidance.
11 And as Raj pointed out, this has been a 12 long effort, a couple of year effort in developing 13 this guidance and understanding what we think we need 14 in our discussions with industry.
15 As the NRC has been transitioning from 16 deterministic methodology to more risk-informed 17 methodology, the staff had developed an integrated 18 decision making process that includes the results from 19 probabilistic fraction mechanics in terms of the 20 change in risk to core damage frequency or large early 21 release frequency.
And coupling that with 22 investigations in how it impacts safety margins, 23 defense in depth, whether it meets the current 24 regulations, and how performance monitoring is done 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
10 creates a integrated risk-informed solution. And it's 1
important to point out that this is risk-informed and 2
not risk-based.
3 In recent times, though, the agency has 4
been trying to expand the use of risk beyond PRA and 5
calculation of CDF by using more risk insights, and 6
especially for passive component integrity where when 7
you'd be talking about components that may not be 8
actually modeled within the PRA. So, that's kind of 9
what we're going to be focusing on today.
10 Through the course of this development in 11 risk-informed
- space, the staff has developed 12 applications, be it how to change license basis using 13 risk-informed methodology or how to categorize SPCs in 14 risk-informed space.
15 And with those, of course, they've also 16 developed specific guidance, guidance that is directly 17 related to the application in question. And also 18 they've developed generic guidance.
19 As shown here, Regulatory Guide 1-200, 20 which is one approved approach for determining if the 21 technical adequacy of probabilistic fraction mechanics 22 is sufficient to provide confidence in the result.
23 In addition to what I've shown here, 24 there's other licensing actions that are being 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
11 considered, and guidance processing tools are still 1
under development in those areas. And one of those 2
areas if probabilistic fraction mechanics.
3 I think Raj alluded to it a bit, but for 4
those that aren't familiar probabilistic fraction 5
mechanics analyses are basically probabilistic 6
analyses that leverage fraction mechanics and the 7
uncertainties in their inputs and models in developing 8
probabilities of failure. In PRA space, you know, you 9
have initiating events, scenario, development, and 10 consequence, probabilistic fraction mechanics can be 11 used in determining initiating event frequencies.
12 But, typically, PFM is used more for 13 requests for inspection relief, or determining LOCA 14 frequencies in the calculation of leaks, or breaks, or 15 other things. And it's those areas where we are, 16 where we are lacking guidance.
17 If I can talk about the licensing reviews 18 that I just mentioned and put them in more of a 19 graphical form, you can see that on this, on this 20 schematic. On the X factors it is kind of a reliance 21 on licensees' PRA information, the risk information.
22 And the Y axis is the acceptability of that PRA, those 23 PRA results.
24 The processes that I talked about earlier 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
12 are here on the right-hand side where we have high 1
reliance and high acceptability on those particular 2
processes.
3 Over on the other end of the scale where 4
we have relief requests for maybe relief from the SME 5
code, we may not rely much on PRA information. We may 6
use more qualitative risk arguments. And, therefore, 7
the acceptability of any particular PRA is not as 8
important to that review.
9 I'm going to be talking about stuff and 10 we're going to be talking about stuff today that's 11 kind of in the middle where we may be using risk 12 outside the defined processes, or outside of formal 13 risk-informed licensing basis processes, but may use 14 probabilistic fraction mechanics. Those processes are 15 still under development.
16 Recently, the staff has released LIC 206 17 which is integrated risk-informed decision making for 18 licensing reviews where some guidance is from the 19 staff on how they can try to use this outside of these 20 normal processes. But in terms of PFM, there's 21 currently no, no real guidance. And to make, to make 22 decisions on requests that use this, we need really 23 adequate and consistent information, and we need to 24 make sure that we have confidence in the result.
25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
13 Like Raj had mentioned also, PFM has been 1
around for quite some time. And there have been 2
submittals in the past, and successful submittals, 3
that have used PFM. However, typically there's a very 4
complex, lengthy review because these codes are very 5
complex. There's a lot of different models on how 6
uncertainties are being propagated that's very 7
computer code-specific. The reviews can be very 8
complex.
9 Many staff believe these codes are like 10 black boxes where input's in and output's out without 11 even knowing what's happening within the code. And 12 that may be due to insufficient documentation, or 13 insufficient vetting of the code and the input. And 14 this all leads to very low confidence in the output.
15 And this becomes particularly an issue 16 when we are, of course, using that as a basis for 17 modifying or changing things like long 18 (unintelligible) inspection programs.
19 Back when we first started this effort we 20 had a public meeting with the industry to talk about 21 PFM and kind of our thoughts and ideas and our 22 position on what should kind of go into these analyses 23 and what should be reported. And we presented this 24 table which shows some of the past EPRI reports and 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
14 categories and which we think are important to discuss 1
and to consider when using PFM in a regulatory 2
submittal. And what we did was we kind of labeled 3
what we thought -- whether we thought these things for 4
these reports met the staff's position for acceptable 5
PFM.
6 And as you can see, there's a lot of --
7 there's some yeses, some noes, a lot of partials. But 8
what's consistent in here is that there was not much 9
consistency across the reports, in a particular report 10 or across many reports in terms of how the information 11 was provided to the NRC.
12 There are a lot of hurdles still in PFM in 13 moving forward. Now, the submittals that we've had 14 with PFM are -- have been inconsistent, have been 15 complex. The level of acceptability has been mixed 16 also as we have considered PFM as a basis for a 17 regulatory position.
18 This risk-informed decision making is 19 still very much in process, in the process of being 20 implemented at the agency, especially for the passive 21 components that may not be modeled in traditional PRA.
22 The PFM is just one part of this risk-informed 23 decision making process, as I mentioned. And both the 24 industry and the staff need to keep that in mind as we 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
15 are working through these kind of processes.
1 And we're still in the development, not 2
just the PFM reg guide, but other processes in terms 3
of how to use risk-informed for these types of 4
components that are under development. But use of PFM 5
for licensing is increasing rapidly. And especially 6
recently we've been seeing more and more applications 7
of using probabilistic fraction mechanics codes and 8
unique codes in that fact to -- as a basis for trying 9
to make a regulatory change.
10 And so, because of all this we really 11 think that a PFM reg guide will help both the 12 licensees as well as the staff to determine what is 13 consistent and adequate information that needs to be 14 supplied to the NRC when using this type of code for 15 licensing applications.
16 So, that is all that I have. And with 17 that, I will transfer the control over to Patrick 18 Raynaud who will be giving some details about the 19 draft guide.
20 CHAIR RICCARDELLA: Yes, thank you, David.
21 Members, are there any questions on 22 David's presentation at this time? So let's then 23 proceed.
24 MEMBER BLEY: Yeah. Pete, this is Dennis.
25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
16 CHAIR RICCARDELLA: Hi, Dennis.
1 MEMBER BLEY: Early on your mentioned some 2
of the problems with the uncertainty treatment and 3
certain people not having a lot of confidence. Has 4
that condition changed?
5 I know we have used this in some 6
regulatory areas so far. Where do we stand right now 7
on that side of things?
8 MR. RUDLAND: Well, I think the, you know, 9
the applications in the past have been very 10 inconsistent in how they have handled uncertainties 11 and reported it to the staff. I think as the more and 12 more it's being used, the better it is.
13 I think Raj pointed out that EPRI had 14 published some thoughts on this in a white paper in 15 terms of how to present uncertainties, how to present 16 the results, what to present. And that's helped in 17 the most recent applications. But I think, you know, 18 to be consistent across the board, something like this 19 kind of guidance is needed.
20 MEMBER BLEY: If we go back to the PTS work 21 some years ago, --
22 MR. RUDLAND: Uh-huh.
23 MEMBER BLEY: -- this had been chased 24 pretty hard at that time. Was it done more rigorously 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
17 there or is it just we've looked deeper into it since 1
then?
2 MR. RUDLAND: Well, you know, I think 3
especially with PTS it was a long process that took a 4
lot of time to get to where we wanted to be. Right?
5 So, it was not a, it was not a simple, short-term 6
effort.
7 And through the multitude of iterations I 8
think we got to the right place.
9 MEMBER BLEY: Okay. You're right, it took 10 a long time.
11 MR. RUDLAND: Yeah. And there were a lot 12 of good lessons learned that we've learned from that 13 effort that set into the stuff we're going to see in 14 this, in this draft guide.
15 MEMBER BLEY: Okay, thanks.
16 CHAIR RICCARDELLA: Thank you, David.
17 Patrick, are you ready to go?
18 MR. RAYNAUD: I am. Thank you.
19 I hope you can see my slides, for those of 20 you who are on the Teams meeting. I think I got it 21 right. Let me know if that's not the case.
22 CHAIR RICCARDELLA: Some of us aren't 23 getting slides. So, I would appreciate it if you 24 would just tell which slide you're on.
25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
18 MR. RAYNAUD: Of course.
1 CHAIR RICCARDELLA: Thank you.
2 MR. RAYNAUD: So, right now I'm just on the 3
cover slides. And I want to introduce myself.
4 I'm Patrick Raynaud. I'm a senior 5
materials engineer in the Reactor Engineering Branch 6
in the Division of Engineering and Research. And I've 7
been aiding the project to develop the PFM guidance 8
for the NRC side since late 2016 or somewhere about 9
there.
10 And today is our first time speaking to 11 the ACRS. And I'm going to present some of the 12 research perspectives on PFM analyses and some 13 detailed thoughts on a graded approach for PFM.
14 I'm on Slide 2. I just want to highlight 15 the objectives today.
16 First, you know, we want to describe, as 17 I said, our perspectives on the analytical steps that 18 could be used to create high-performance PFM 19 demonstrations for regulatory purposes, and to 20 demonstrate that you have an adequate solution to a 21 technical problem that would be relevant for a 22 regulatory application.
23 And then we want to describe what a graded 24 approach for PFM analysis and documentation could look 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
19 like when you consider a number of factors such as the 1
safety significance of a particular problem that you 2
were trying to solve, and the complexity of that 3
problem, as well as of the tools that you're using to 4
solve that problem.
5 Now, a few things that -- a few 6
disclaimers for today is that, you know, the slides 7
I'm showing today are all pre-decisional content 8
that's not yet an official position of the NRC. And 9
nothing presented here is completely final since 10 they're still under internal review and could change.
11 And so this content is not intended to be guidance, 12 final guidance on what constitutes an acceptable 13 approach for PFM submittals to the NRC. Hopefully, 14 we'll get there soon.
15 So, my presentation has two parts today.
16 I'm on Slide 3. First I'm going to present the 17 analytical steps in a PFM demonstration. And then I'm 18 going to focus on this graded approach that we have 19 been discussing so far.
20 Slide No. 4.
21 I'll start with describing at a high level 22 what we think are the necessary analytical steps in a 23 probabilistic fraction mechanics demonstration. And 24 really if you look at that in the chart that we have 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
20 here, we have three big parts.
1 We have a planning stage, an analysis 2
stage, and a synthesis stage. And there are five 3
steps in total.
4 So, the first part, planning, only has one 5
step. And I'm going to go into each one of these five 6
steps in the subsequent slides in more detail.
7 But the major step there is to translate 8
your regulatory requirements into an analysis plan and 9
decide what you need to do and how before you launch 10 into it.
11 The second major part is to perform an 12 analysis. And here we have three main steps.
13 First, you need to characterize the model 14 input uncertainty. And that is essentially how you 15 are going to represent your inputs that might have 16 random values.
17 The second big step is to estimate your 18 quantities of interest -- you'll see this acronym 19 often, Q of I's, throughout my slides -- and the 20 uncertainty that is associated with those quantities 21 of interest.
22 And the last step, which you may or may 23 not need, depending on your situation, is to conduct 24 sensitivity studies to assess the credibility of your 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
21 modeling assumptions and to better understand how your 1
problem changes depending on those assumptions.
2 The last part of the analysis would be to 3
synthesize your results and to draw conclusions from 4
your analysis.
5 And one thing that I really want to 6
emphasize here is that we believe that this is a 7
recursive process, you know, or an iterative process 8
where, you know, you might have some thoughts, initial 9
thoughts on an analysis and perform those analyses.
10 And then based on your results you may often have to 11 revisit some of your initial assumptions or initial 12 modeling choices to refine your results and get a 13 result that is more defensible, perhaps, for whatever 14 you're trying to prove or demonstrate as part of that 15 regulatory analysis.
16 So now I'm moving on to Slide 5. And I'm 17 going to go into each one of these five steps in some 18 more detail.
19 So, the first big step is, as I mentioned, 20 to translate the regulatory requirements into an 21 analysis plan. And here we have come up with four 22 actions that you may have to perform.
23 First you need to look at your regulatory 24 context and, you know, how your PFM analysis is going 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
22 to be used to support a decision on a regulatory 1
action. And by that we mean how is the technical 2
basis using PFM going to be input to the regulatory 3
decision.
4 And, specifically, here what might be 5
important is what criteria are going to be used to 6
support a proposed regulatory action. There may be 7
cases where the criteria are well-known. If it's a 8
specific regulation that you're trying to demonstrate 9
that you meet, or maybe the criteria are not so well-10 defined and then, you know, the criteria would need to 11 be described and justified.
12 The second action here would be to define 13 the quantity of interest and how it relates to what 14 our model or PFM code can provide as output, and then 15 to compare that also to the acceptance criteria. So, 16 specifically we're looking at mapping what quantities 17 are used to show that a regulatory requirement might 18 be satisfied, how they're mapped onto specific outputs 19 of our PFM code or model.
20 And we want to make sure that our model is 21 able to predict the needed and relevant quantities.
22 The third action would be to -- and it's 23 related to the third one -- to the second one, is to 24 determine the suitability of the probabilistic 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
23 fraction mechanics code to the specific application.
1 What we mean here is that you want to be sure you have 2
a code that is able to model the relevant physics for 3
the problem that you're trying to solve, in the 4
appropriate ranges, and that it has all the necessary 5
physics and numerical and computational features that 6
are needed to solve your problem.
7 And the fourth and final action of this 8
first big step is to identify the key elements of the 9
problem that might impact your analysis choices. For 10 example, if your model is very computationally 11 extensive, what simplifications may you need to take 12 in order to be able to solve your problem in a 13 reasonable time or in a way that doesn't require, you 14 know, a supercomputer.
15 So, now I'm on Slide 6.
16 And the second big step, which is the 17 first step of the analysis portion of our grand scheme 18 here, is to characterize the model input uncertainty.
19 So, first of all two actions here. The 20 first action is to identify which model inputs are 21 going to be characterized as uncertain or random. And 22 then, also, if you are separating aleatory and 23 epistemic uncertainty, how are you going to categorize 24 the uncertainty for each one of these inputs?
25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
24 And the second action here is to specify 1
probability distributions for those uncertain or 2
random inputs in order to be able to then represent 3
them in a numerical way and propagate those 4
uncertainties through your model.
5 And so, typically this is done by choosing 6
probability distributions of which there are a great 7
number, and setting their parameters, perhaps getting 8
into data and so, and then to propagate that 9
uncertainty to estimate your quantities of interest 10 for your problem.
11 The third step is to actually estimate the 12 quantity of interest and the associated uncertainty.
13 Four main actions here.
14 The first one is to select the sampling 15 scheme that you're going to use to propagate your 16 uncertainty through your model. There are many 17 choices there. The simplest, or the simplest in terms 18 of numerical implementation might be simple random 19 sampling using Monte Carlo methods. But there are 20 many different ways to sample an input space.
21 And the goal here is, obviously, to get a 22 good estimate of your quantity of interest and be able 23 to characterize the uncertainty associated with that 24 quantity of interest.
25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
25 The second action is to assess the 1
sampling uncertainty, which is also performing 2
statistical conversions analysis. You want to assess 3
that the quantity of interest that you've calculated 4
by propagating uncertainties through your model is 5
sufficiently converged to give a solution to your 6
problem given the sampling scheme that you've chosen.
7 And you may, as part of that iterative 8
process that I alluded to earlier, you may need to 9
adjust that sampling scheme if initially you don't 10 have a converged enough solution. That might be 11 adding samples, looking at different random seeds, or 12 completely changing your sampling strategies, maybe 13 using more advanced sampling strategies if you're 14 dealing with, for example, very low probability, if 15 that's where you need to sample more in the tails of 16 distributions to see certain things happen and have 17 more reliable outputs.
18 The third action, which may or may not be 19 needed depending on your case, is to conduct 20 sensitivity analyses to determine the input 21 uncertainty importance. Sensitivity analyses are 22 analyses that correlate the uncertainty of inputs with 23 the uncertainty of outputs and help you understand 24 which inputs drive most of the uncertainty in your 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
26 outputs of interest. And so this is essentially akin 1
to identifying the problem drivers.
2 This can also help you confirm that your 3
model is behaving as expected. You may find that some 4
input distributions may need to be refined to get 5
better convergence on your final estimation of the 6
quantities of interest that you need.
7 You may also be able to identify which 8
assumptions in your modeling assumptions may be 9
candidates for sensitivity studies, so you're making 10 a distinction between sensitivity analyses, which is 11 understanding how uncertainty propagations are 12 modeled, and sensitivity studies which are more like 13 "what if" analyses if you change your model or if you 14 change a distribution.
15 And also you can, by identifying the 16 problem drivers, you can identify which are the best 17 candidates for more targeted, more advanced sampling 18 methods like importance sampling, for example, for 19 those that drive the problem.
20 And the final action in this Step 3 is to 21 conduct output uncertainty analysis. And by the way, 22 I think there was a typo in one of my slides, so that 23 title might be different than what you have from the 24 slides that we sent earlier.
25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
27 So, the title here is conduct output 1
uncertainty analysis. And the goal is to provide a 2
final estimate and the associated uncertainty of the 3
quantities of interest. And, as always, it's very 4
helpful to visualize results when possible from a 5
reader perspective.
6 The fourth step, which is the last step of 7
the analysis portion in our scheme, is to perform 8
sensitivity studies, meaning "what if" analyses to 9
assess the credibility of some of the modeling 10 assumptions that you may have made.
11 And there are two actions here. First of 12 all, to choose which sensitivity studies are going to 13 be performed. And that means identifying important 14 assumptions that need further scrutiny and need to be 15 better understood, and to understand what happens if 16 we change these importance assumptions.
17 And the second action is to actually 18 conduct those sensitivity studies and summarize and 19 present the results.
20 There are many ways to perform sensitivity 21 studies. But in any case, there are common elements 22 to pretty much any method you use. You need a 23 reference case, maybe your baseline case. And then 24 you're going to have perturbations from there, maybe 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
28 change a model, change some input distributions. And 1
then you're going to compare the output of these new 2
realizations of your model, where you've changed 3
something, to the reference case and understand the 4
differences, see if there are differences or not, and 5
whether those changes are significant.
6 And that can help build confidence in 7
showing that perhaps the overall model is behaving as 8
expected, or to cover the various options that you may 9
need to cover to represent operational occurrences and 10 such.
11 The final step is to draw conclusions from 12 our analyses.
13 MEMBER KIRCHNER: Patrick, this is Walt 14 Kirchner.
15 MR. RAYNAUD: Yes.
16 MEMBER KIRCHNER: Before you go on could 17 you just, you know, you had sensitivity analyses 18 previously when we were looking at estimating, you 19 know, your quantity of interest and such. What are 20 the difference, what's the difference in the 21 sensitivity analyses done before this step? I guess 22 it was in Step 3.
23 MR. RAYNAUD: Uh-huh.
24 MEMBER KIRCHNER: Just is it at a micro-25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
29 level so to speak? Or in Step 3 you also were doing 1
sensitivity analyses. So, this looks like a more 2
macro look at functions.
3 CHAIR RICCARDELLA: You may mean the 4
difference between sensitivity analyses and 5
sensitivity studies. Maybe you could clarify.
6 MR. RAYNAUD: Correct. Correct. And I was 7
going to reinforce that. Perhaps I didn't make the 8
point enough.
9 There is a difference between sensitivity 10 analysis and sensitivity study. And I know for me it 11 took me a long time to realize that there was a 12 difference. But sensitivity analysis really is about 13 understanding what input uncertainty has a large 14 influence on the output uncertainty.
15 So, essentially it's studying the 16 relationship between input uncertainty and output 17 uncertainty.
18 Whereas sensitivity studies is, you know, 19 given that relationship, if I change something how 20 does the output change?
21 So, again, it's studying the relationship 22 between uncertainties versus doing "what if" analyses, 23 you know, what if I change a model, what if I change 24 a distribution, what if I change values for this 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
30 parameter or that one.
1 So, they're two different things in our 2
scheme here. Two different types of analyses.
3 MEMBER KIRCHNER: Okay, thank you. Yes, in 4
my simplistic world the first one is more a micro at 5
separate model levels, or material selections, or 6
whatever you're looking at. And this last step to me 7
is more at a macro level: you know, how does it impact 8
the overall results.
9 But I thank you for the clarification.
10 MR. RAYNAUD: You're welcome.
11 And they're definitely intimately related.
12 But they are, down in the detail of how you perform 13 these, they are different and they provide different 14 insights.
15 Okay, so unless you have more questions, 16 I'll continue.
17 So, we were on Slide No. 9 and our fifth 18 step of our scheme to perform PFM analyses.
19 And this last step is to draw conclusions 20 from analysis results. And, you know, two actions 21 here.
22 First, we're going to interpret the 23 results we have by synthesizing the information that 24 was gathered in the previous steps. And then we're 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
31 going to try to draw conclusion. We may be able to 1
draw final conclusions by comparing to our acceptance 2
criteria, or we may realize that we need additional 3
analyses or to refine our analyses to create a more 4
complete evidence package to be able to draw our 5
conclusions.
6 And the second actions, which is very much 7
related to what I just said, is to iterate on the 8
analysis process and to refine the model results. You 9
know, in many cases you will need to provide more than 10 one final analysis because PFM is very complex and 11 there are so many moving parts that, you know, this 12 iterative process and providing a more complete 13 picture of how the model behaves and how the results 14 change for different assumptions can be very important 15 in PFM space.
16 So, I'll move on now to the second part of 17 my presentation which has to do with thoughts on our 18 graded approach. That's Slide No. 10.
19 And, first, before I dive into any details 20 I want to highlight some overarching, higher level 21 concepts.
22 And the first thing I want to really 23 emphasize, and Dave before me alluded to that, is that 24 PFM is generally complex. You know, if you could 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
32 solve problems deterministically, that is, obviously, 1
much simpler and perhaps the first thing that everyone 2
tries, and that we have been doing for quite some 3
time.
4 So, when you're dealing with something as 5
complex or potentially as complex as probabilistic 6
fraction mechanics, the depth and the breadth of the 7
analyses that you're going to perform, and perhaps how 8
much you're going to have to document those things, 9
can depend on a number of factors, which is why it 10 does make sense to take a graded approach.
11 And, you know, I think this was discussed 12 in previous public meetings as far as maybe around 13 2018 or so, in particular with EPRI who brought some 14 of these concepts up. And so I want to credit our 15 stakeholder feedback for putting us onto this path.
16 So, the graded approach not only applies 17 to how you perform your PFM analyses, but of course 18 how you document them to present an evidence package 19 in a regulatory context that is something that is 20 going to be reviewed and scrutinized by the NRC.
21 The general principles that sort of guided 22 our thoughts here are that if you have higher safety 23 significant of the application at hand, higher 24 complexity, or higher degree of novelty associated 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
33 with, you know, the regulatory request for application 1
or the codes and tools that are used to perform the 2
analyses, then that probably is going to translate 3
into having to perform more analyses, have a more 4
complete documentation package. And, ultimately, it 5
will mean a higher burden to create a more defensible 6
and rigorous evidence package.
7 And so what we've done to try to develop 8
a graded approach or some thoughts on a graded 9
approach is come up with all the main topics that we 10 think need to be addressed in a probabilistic fraction 11 mechanics analysis and submittal context. And we came 12 up with eight of those, which I'll list in a second.
13 And then for each, we came up with some 14 categorization schemes and recommendations or thoughts 15 on what might be needed for analysis or documentation.
16 So, the eight broad topics that we came up 17 with are:
18 First, software QA and V&V; then models; 19 then inputs; then uncertainty propagation; then 20 convergence; then sensitivity analyses; then quantity 21 of interest uncertainty characterization, and; 22 finally, sensitivity studies.
23 So, I'm going to go into some detail for 24 each one of these. I have some pretty detailed tables 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
34 so that once you have all the slides, you have all the 1
information. But I may not cover everything in detail 2
because I think it would just be a lot of reading off 3
and talking. So, I'll try to highlight some of the 4
higher level distinctions we made.
5 CHAIR RICCARDELLA: Just a general comment 6
for my colleagues from someone who's been involved in 7
a lot of probabilistic fraction mechanics analyses.
8 And one of the things that's unique, I think, about 9
fraction mechanics that makes it so amenable to this 10 technique is we generally see much more scatter, much 11 more input scatter in the material properties in 12 fraction mechanics than we do in other strength of 13 materials type applications.
14 And I think that, you know, growth rates, 15 for example, you could see orders, an order of 16 magnitude difference, you know, in essentially the 17 same property from different samples that are tested.
18 And maybe not quite an order of magnitude, but on the 19 fracture toughness you see much more scatter in that 20 property than you would see in, for example, the 21 tensile test, yield strength or tensile strength.
22 So that's one of the things, I think, that 23 has led to probabilistic fraction mechanics being used 24 so much.
25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
35 MR. RAYNAUD: Thank you for that.
1 So, I'm going to go to my Slide 11. And 2
I'm going to go through each one of these eight topics 3
and present our current thoughts which are, again, 4
decisional not final, can still change, but our 5
current thoughts on our categorization and our graded 6
approach for these different topics.
7 Start with on Slide No. 11 our thoughts on 8
a graded approach for software quality assurance and 9
verification and validation.
10 First of all, let me just state that in 11 general safety demonstrations for the NRC usually 12 require that a QA program be in place. So, that's 13 sort of a prerequisite thought. And we also believe 14 that because of the complexity of probabilistic 15 fraction mechanics and all the intricacies of this 16 potential graded approach, if that's where we end up, 17 presubmittal meetings and discussions between 18 applicants and the NRC would probably be very useful 19 to make sure everybody's sort of on the same page as 20 to, you know, where we fall within this, you know, 21 rather complex graded approach.
22 So, when it comes to quality assurance and 23 V&V our categories are labeled QV-1, which stands for 24 QA and V&V-1, through QV-3. And, essentially, our 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
36 criteria here to determine which category you are in 1
are still with how familiar we are with the code. And 2
when I say "we," I mean the NRC in this case.
3 So, you know, the base case is if you have 4
an NRC-approved code that's exercised within its 5
validated range, previously approved, et cetera, then, 6
you know, the path in terms of documentation of the 7
code might be pretty simple and pretty basic. Just 8
show that your -- that the code works for the 9
application basically.
10 And then as you move away from that, for 11 example if you use the code outside of its previously 12 validated and accepted range, then you may have to 13 provide additional evidence that, you know, the code 14 is still applicable for what you are doing, and so on.
15 If you then modify a previously NRC-16 approved code, then you know there would be probably 17 some questions and some documentation that would be 18 helpful about what's changed and what kind of V&V and 19 QA did you perform on those changes, and what 20 documentation is available for that.
21 The second category for QA and V&V is QV-22 2, would be for commercial off-the-shelf software 23 designed specifically for the purpose of the 24 application. I don't think there are many such PFM 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
37 codes that are specifically, you know, off-the-shelf 1
available for purchase. I can't think of any, but 2
there may be some.
3 And the final category, QV-3, is if you 4
have a completely new and custom code. And then, 5
obviously, you know, much more documentation and 6
description of the QA and V&V would probably be needed 7
to instill confidence and show that the code behaves 8
as expected and so on.
9 So, the next big topic that we have is 10 models. And for this we have categories that we 11 labeled M-1 through M-5. And I really want to 12 emphasize that it's possible that in a large PFM 13 software tool different models could belong to 14 different categories. Because we sort of tied these 15 categories to the QA and V&V categories that I just 16 discussed previously.
17 So, for example, the easiest category, M-18 1, would be if you used a model from an approved NRC 19 code within its validated range, then, you know, 20 presumably we're familiar with it, we've reviewed it, 21 we've accepted. So, the path is pretty easy to show 22 that, to give us confidence that this works.
23 Conversely, if you start using that same 24 model outside of its validated range, then we'd want 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
38 to perhaps understand, you know, a demonstration of 1
why these models are still applicable in this new 2
range of application, and so on.
3 If you take a model from an NRC code but 4
you change it then, obviously, more description of the 5
changes would be needed.
6 If you take a well-established model that, 7
but that's not from an NRC code, maybe it's a 8
published model or something that others have used 9
extensively in other codes, or maybe something from 10 the ASME code, something that's widely accepted but it 11 hasn't been previously put into an approved software, 12 then, you know, obviously we'd probably want to 13 understand exactly how that was done and see some QA 14 and V&V trace of how that -- what data was used and 15 how well the model predicts and such. So, obviously, 16 the documentation starts to increase.
17 And, of course, if it's a first-of-a-kind 18 model, which is our final category, M-5, that, you 19 know, brand new, never published, just something 20 completely new for a new phenomenon that has occurred 21 then, obviously, really detailed documentation and 22 perhaps sensitivity studies and analyses to show how 23 the model behaves, and such, may be needed to instill 24 the right level of confidence.
25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
39 So, again, quite a wide range. And you 1
can see this progression from what we're most familiar 2
with to something completely novel with, you know, an 3
increased emphasis on more documentation and more 4
analyses and data analysis to show that the model is 5
behaving correctly and that we can have confidence in 6
that software.
7 Topic inputs in this one is it's a little 8
bit more complicated. So, I'm on Slide 13. And here 9
we're looking to determine our categories we thought 10 about what types of inputs we might have. And some 11 inputs in a code might be deterministic, meaning a 12 single value was provided. And others might be random 13 inputs or uncertainty inputs.
14 So, we made that distinction in our 15 categories when we numbered them I-1, for input one, 16 through I-4, for input four. But then we have a D for 17 deterministic and an R for random after those. And 18 depending on whether you have a deterministic or a 19 random input, obviously, you know, the type of 20 documentation you may need to provide may be different 21 because those models are inherently different -- or 22 those inputs are inherently different in how they are 23 characterized.
24 And then we focused on how important is 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
40 this input with regards to the output that we're 1
interested in? And then how well do we know the 2
characteristics of that input?
3 So, in our I-1 category, you know, it's 4
something that you know very well and, on top of that, 5
has low importance with regards to the output. And in 6
that case, you know, you may need very little 7
documentation. Just list your values and that's it, 8
or your distributions.
9 But as you increase the importance that 10 that input has on the output, and to know that by the 11 way, that's where sensitivity analysis may come into 12 play. And so, the more important the input is and the 13 less knowledge about that input you have, then the 14 more uncertainty you have.
15 So, if you have a lot of uncertainty on 16 something that's really important with regards to your 17 quantities of interest and your output, obviously, you 18 know, more scrutiny would be placed on such an input 19 value. And so more and more documentation and 20 analyses to show that the input representation covers, 21 you know, all the possibility -- well, "all" is a big 22 word -- but, you know, the relevant possibilities 23 would definitely be of interest in that case.
24 And, yes, one other important thing that 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
41 I want to say here is that every input you have may 1
fall in a different category, much like every model 2
you have in your overall code might fall in a 3
different category. So, there's a real detailed work 4
of categorization here that has to happen so that you 5
make sure that you're doing, you know, what you should 6
do but not too much or too little. And so it really 7
takes some care. And that's why I think, you know, a 8
lot of discussion, interactions with the NRC perhaps 9
would be helpful.
10 The next big topic is uncertainty 11 propagation. I'm on Slide 14.
12 And here, you know, we alluded to this 13 iterative process in your PFM analysis overall where 14 you might have to run several analyses within the 15 scope of your entire demonstration. Well, here, for 16 instance, your propagation may be different analyses 17 that you're going to run using different sampling 18 schemes, different features of how you model the 19 uncertainty and propagate it, might mean that 20 different analyses in your overall package and 21 demonstration might fall into different categories.
22 So, here we have categories UP, sensor 23 uncertainty propagation 1 through 2-Bravo. And, you 24 know, here we sort of left the QA and V&V aspects 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
42 aside and we focused more about on what the specifics 1
of that analysis might be, and what you might have to 2
document in those cases.
3 So, if you just run a very straightforward 4
Monte Carlo analysis, you know, obviously it would be 5
relevant to describe what methods specifically you 6
used for uncertainty propagation. You know, was it 7
simple random sampling? Did you use Latin hypercube 8
sampling? You know, you might want to describe your 9
sample sizes, your random seeds, what type of pseudo-10 random number generator you used, and so on. If you 11 separated aleatory and epistemic uncertainties, how 12 did you do that? If you used importance sampling, how 13 did you do that, and what regions of the input space 14 and why?
15 But then, you know, in some cases if your 16 model is very computationally extensive, you may only 17 run a handful of realizations and then fit a model 18 through that output, essentially create a surrogate 19 model. Some people believe this is machine learning.
20 And in that case, you know, you'd be bumped into the 21 next category because now you're using a more advanced 22 feature, something a little bit more complex that adds 23 maybe some uncertainty to the overall methodology.
24 And so, you know, we'd want to hear about 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
43 that surrogate model and how you implemented it, and 1
what are the assumptions, approximations, and how did 2
you validate it?
3 If you use a
surrogate model for 4
sensitivity analyses specifically, you know, once 5
again we would want to hear about that because you 6
could use a surrogate model for the overall 7
uncertainty propagation, or maybe only, for instance, 8
to the analyses or sensitivity studies. So, it 9
depends on what you used the surrogate model for, and 10 we'd be looking for descriptions of what was done 11 specifically.
12 A broad topic -- I'm on Slide 15 -- is 13 convergence. And, first, let me say that here I'm 14 really talking about statistical and probabilistic 15 convergence.
16 And so, you know, we assume that temporal 17 and spatial conversions, if you have finite element 18 model, for example, as part of your fraction mechanics 19 calculation, that that part of the convergence, the 20 discretization convergence, pretty much has to be 21 achieved for you to have valid results.
22 So that's, that's sort of a prerequisite, 23 if you want to think of it that way. Otherwise, your 24 results really can't be valid. But once you have 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
44 valid discretization convergence, and good enough, 1
then you want to looks at statistical convergence.
2 And here is really what we focused on when we looked 3
at our graded approach.
4 And once again, much like for uncertainty 5
propagation, you know, you could fall into different 6
categories for each one of the analyses you do as part 7
of your overall evidence package.
8 And here the criteria we used to come up 9
with our categories focused on how close you might be 10 to an acceptance criterion so, you know, how much 11 margin you have essentially, and how simple your 12 analysis is over all.
13 So that category SC, which stands for 14 statistical convergence, SC-1, which is the lowest 15 category would mean, you know, you meet your 16 acceptance criteria with plenty of margin, and you 17 haven't done anything fancy: no importance sampling, 18 no surrogate models. In that case, you know, what 19 you've done is relatively simple and there are some 20 pretty simple methods, perhaps, to show that you've 21 achieved statistical convergence.
22 Or, you know, if you really have a ton of 23 margin, maybe you don't even really need to focus too 24 much on statistical convergence at all.
25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
45 But then as you get closer to your 1
acceptance criteria, or if you use some more advanced 2
methods, like importance sampling or surrogate models, 3
or if you separate aleatory and epistemic uncertainty, 4
then, you know, we definitely want to hear more about 5
how you assessed your statistical convergence. There 6
are many ways to do this.
7 You know, if you're still relatively far 8
from your acceptance criteria that your analyses are 9
using more advanced methods, you could demonstrate 10 statistical convergence just using one method of your 11 choice. There are many to choose from.
12 But as you get closer to your acceptance 13 criteria, it would be a good idea to use more than one 14 method to assess your statistical convergence.
15 So, that's kind of the gist of how we came 16 up with our categories here in our graded approach.
17 The next big topic is sensitivity 18 analyses. So, again, this is understanding the 19 relationship between the uncertainty and the inputs, 20 and the uncertainty and the outputs. And here for 21 sensitivity analysis we have six categories labeled 22 SA-1 through SA-6.
23 And these categories are very much tied to 24 which QA and V&V category your software falls in from 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
46 the very first category I discussed. So, you know, if 1
you're in the category of an approved code with, you 2
- know, within the previously validated
- ranges, 3
presumably we know that code well and you may not even 4
need sensitivity analysis in some cases.
5 However, as you start getting further away 6
from that, you know, if you have a code that's a 7
modified previously approved code, or a first-of-a-8 kind code, and also as you improve the number of 9
independence input variables that go into your 10 software, that means you're essentially increasing the 11 complexity, as well as getting further away from what 12 we previously approved and what we know well.
13 So then, you know, we'd want to hear a lot 14 more about how you did your sensitivity analysis. Was 15 it local, global? What were the quantities of 16 interest that you looked at? You know, what sampling 17 scheme were your sensitivity analyses done for? And 18 what were the results, you know, what were the most 19 important inputs that you identified and how much of 20 the variance could be explained by those, and so on.
21 If you separated aleatory and epistemic, 22 obviously you would, you know, maybe want to study the 23 sensitivity analyses for both things. And the more 24 variables you have, independent variables as inputs to 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
47 your model, the more relevant and important the 1
analyses might be to narrow down, you know, which 2
inputs you really need to study and reply to the 3
solutions for, and so on.
4 And so, those are the kinds of things that 5
I think are important features of an analysis and that 6
really need to be explained so that you can show how 7
you built more and more evidence and confidence into 8
your final result.
9 The next big topic is quantity of interest 10 uncertainty characterization. And we often refer to 11 this as output uncertainty characterization. So our 12 categories are labeled O-1 through O-3. "O" stands 13 for output.
14 And here we focus on, you know, how far 15 are you from the acceptance criteria? And then if 16 you're close, you know, how strong is your basis and 17 your confidence on your input distributions and 18 uncertainty classification?
19 And, you know, I would say the less margin 20 you have and the less certainty you have about your 21 inputs, the less knowledge you have about your inputs, 22 the more you're going to want to study the output 23 uncertainty.
24 I'm hearing a lot of echo in the line.
25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
48 So, anyway, you know, in any case for at 1
the lowest level you're going to want to give a best 2
estimate of your quantity of interest and the 3
uncertainty that you calculated, perhaps represent 4
that graphically. How you calculated your best 5
estimate is important.
6 You know, did you separate, for example, 7
aleatory and epistemic uncertainty?
8 And what is the fraction in each of those 9
uncertainties?
10 And, you know, try to summarize what the 11 major assumptions were that went into the output that 12 you got and into the uncertainty, and what effect did 13 each of those assumptions have on the uncertainty?
14 And, you know, again, once you get closer 15 and closer to your acceptance criteria we're going to 16 want to have a stronger explanation, if you will, of 17 why your uncertainty that you calculated, you know, 18 you know it well enough to be confident that you're 19 not going to exceed your criteria.
20 The last topic now that we have is 21 sensitivity studies. And, again, this is studies 22 where you're going to change something in reference to 23 a base analysis that you performed, perhaps change 24 some models, change property of uncertainty, change 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
49 how you represent some distributions, and so on, and 1
look at the outcome, the result on -- look at how that 2
changes your results.
3 And here we have six categories from SS-1 4
to SS-6. SS stands for sensitivity studies. And 5
these categories are very much tied to which category 6
of QA and V&V you were in earlier on.
7 And if you're in, you know, category QV-1 8
where we know your code really well, you know, you may 9
not even need to do sensitivity studies. You may be 10 able to summarize what's been done previously.
11 However, as you move away from that, if 12 you have an approved code but the quantities of 13 interest that you're looking at were not previously 14 documented or studied or approved in any way, then, 15 you know, you want to provide more information on what 16 sensitivity studies you might have done.
17 If you have a very limited number of 18 independent variables in your code, you may not need 19 as many sensitivity studies or you may not need them 20 at all. It depends on exactly the specifics, you 21 know, how different is your code from a previously 22 approved code.
23 However, if you have a code that's newer 24 and you have a great number of independent variables, 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
50 then you're going to definitely want to perform and 1
document some more sensitivity studies. And the goal 2
here is to, you know, explain what your assumptions 3
were, what the impacts they had, or what impacts they 4
had on the final results of your study. And try to 5
describe, you know, what questions might arise, what 6
"what ifs" might arise with regards to your specific 7
problem, and how you address those via sensitivity 8
studies.
9 So, that covers the eight major categories 10 of PFM analysis that we envision. And, honestly, 11 there's a lot of intricate detail in these graded 12 approaches that we tried to come up with. So, you 13 know, it's a lot to deliver in the short amount of 14 time that I share, but we still have some time for 15 questions.
16 I'll just go to my last slide really quick 17 to highlight some of our next steps before maybe we 18 can have some discussion.
19 Our next steps with regards to this 20 project overall would be to publish some draft 21 regulatory guidance, as well as a draft technical 22 basis NUREG for public comment. And, of course, 23 gather and address the comments that we receive 24 If we feel like public meetings might be 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
51 needed to, you know, to gather additional feedback or 1
have discussions with our stakeholders, of course we 2
will organize some. And, you know, we expect that 3
perhaps once these documents actually are out and 4
finalized at the draft level, that maybe ACRS might 5
want to be able to discuss the various specifics of 6
those documents. So, of course, we're always 7
available for that.
8 And that ends my presentation for today.
9 Hello?
10 CHAIR RICCARDELLA: Yes, this is Pete. I'm 11 sorry. I had my microphone on mute.
12 I have sort of a general comment/question.
13 There seems to me that in a graded approach something 14 should depend, the amount of detail you get into and 15 the amount of validation, it depends on the results 16 and how the results are going to be used.
17 And I think there's a difference in my 18 experience between if you want to make a quantitative 19 statement of comparison to some acceptance criteria, 20 like 10 to the minus 5th, or a delta risk of 10 to the 21 minus 6th, you know, sort of the Reg Guide 1.174 type 22 of criteria, versus more of a qualitative use of PFM 23 results such as comparing two actions, action A versus 24 action B. It might be hydro test versus some form of 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
52 inspection. Or it might be different types of 1
inspections.
2 And that qualitative use of PFM results 3
is, to me, a useful, risk-informed approach that 4
should be considered, you know. If A, if option A 5
turns out to be an order of magnitude better than 6
option B, say, and then you do sensitivity studies and 7
you change some of the input assumptions, and maybe 8
the absolute probabilistic results change by an order 9
of magnitude, but option A always comes out better 10 than option B, you've demonstrated something that's 11 pretty useful that I think that kind of a use of PFM 12 should be included somehow in a graded approach.
13 MR. RAYNAUD: I do have some thoughts on 14 that.
15 I think, first of all, I agree. But I 16 think, ultimately, you know, my understanding of what 17 we're trying to do or show here is that we need to 18 have confidence in the results and the codes used.
19 And to do that, to be able to believe, you know, the 20 types of arguments that you've just described, even if 21 they're qualitative in nature, we need to have high 22 confidence that, indeed, the code that was used 23 produced results that we believe.
24 And I think that's a big focus of what 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
53 we're trying to do here. It is very rigorous and 1
perhaps demanding, but it really has the ultimate goal 2
of removing this feeling of the black box that has 3
been described, and instilling confidence in the 4
reviewers and the public that, yes, these results that 5
show these qualitative differences can be believed.
6 CHAIR RICCARDELLA: Uh-huh.
7 MEMBER BALLINGER: Are you all set, Pete?
8 CHAIR RICCARDELLA: Yeah.
9 MEMBER BALLINGER: Yeah, this is Ron 10 Ballinger. In reading through this, it was sort of 11 like reading the old sections of the ASME code before 12 it was computerized and you had a hard copy, and you 13 had to have 27 fingers to locate yourself into various 14 places in the document to feed things through.
15 This thing kind of cries out for a set of 16 examples that could be included which would give 17 somebody that wants to use this a little bit of 18 initial direction.
19 Maybe I'm not using the right terminology, 20 but it just seems like there should be examples in an 21 appendix or something like that that says, you know, 22 this is a simple one and this is a complex one, that 23 kind of illustrates where the nub of the problem is 24 likely to be.
25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
54 DR. RAYNAUD: Yeah, I understand that.
1 We, in fact, have drafted two reports that illustrate 2
some of the concepts that we have here. You know, 3
we're limited to the tools that we have, which are NRC 4
tools, at least the ones we're familiar with that we 5
can really use to the full extent that we want to, and 6
so it's a little bit hard to go through the graded 7
approach, but we do have some draft reports in the 8
works that are pretty much ready to go.
9 When the guidance is issued for public 10 comment, we'll post those as well, that at least 11 illustrate concrete examples with a fictional problem 12 of how you solve a PFM analysis and, you know, what 13 the documentation might look like and so on, and I 14 know you haven't seen that yet.
15 MEMBER BALLINGER: Yeah, so that would be 16 very instructive, and these are referenced 17 specifically in 1382?
18 DR. RAYNAUD: 13 --
19 MEMBER BALLINGER: Or whatever the NUREG 20 turns out to be?
21 DR. RAYNAUD: Oh, I don't know if there 22 are specific references yet because nothing being 23 public, you know, it's hard to reference nonpublic 24 things, but, yes, we can certainly add that in. We do 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
55 have the reports in a draft version.
1 MR. RUDLAND: And Ron, this is Dave 2
Rudland. If we do, you know, have a public meeting 3
once these are released, it might be a good idea for 4
us to go through those at the public meeting so that 5
you can see the examples and how they work.
6 MEMBER BALLINGER: Yeah, that's probably 7
an excellent idea. I mean, I remember a workshop that 8
Pete and I went to on xLPR, enough said.
9 DR. RAYNAUD: I was actually going to 10 mention xLPR, which of course is one of the NRC's main 11 probabilistic fracture mechanics codes for piping.
12 There have been a great number of 13 publications related to that code, including a lot on 14 sensitivity studies, sensitivity analyses, and so on 15 that could also be a basis for drawing examples.
16 MEMBER BALLINGER: Yeah, those, I'm 17 assuming that in this NUREG, there will be a set of, 18 what do you want to call it, references, not just 19 references, but a bibliography of sources that a user 20 could go to which would include some of the things 21 that you just mentioned.
22 DR. RAYNAUD: We can make sure that that's 23 there.
24 MR. RUDLAND: But realize, Ron, that xLPR, 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
56 and FAVOR for that matter, you know, the staff are 1
very, very familiar with those codes, and very, very 2
familiar with, you know, the bases, and so it would 3
not --
4 It might not be as useful to use those 5
codes as examples of how to step through this because, 6
you know, the basis is really for new or unique codes, 7
you know, where it might be more complex to use the 8
regulatory guide.
9 MEMBER BALLINGER: Yeah, I would think of 10 xLPR or FAVOR as what I would call a dead ringer 11 example, and then another example would be, you know, 12 I don't know, use another acronym, the killer example, 13 the one that's the most complicated.
14 CHAIR RICCARDELLA: Well, I would guess 15 xLPR and FAVOR are all in the one category --
16 (Simultaneous speaking.)
17 MR. RUDLAND: That's correct, yeah.
18 DR. RAYNAUD: So, you know, a lot of the 19 documentation, the QA and V&V documentation that 20 exists for both of those codes, but especially for 21 xLPR because it's a more modern code, and as a result, 22 the standards that were used in developing all of the 23 documentation are more up to today's standards, you 24 can see that that documentation is extensive.
25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
57 You know, we're talking I don't even know 1
how many reports and thousands of pages. Obviously, 2
that's not really something that is realistically 3
reviewable, you know, within a licensing submittal.
4 It's just too much, but I think that type 5
of rigor that was performed there and that exists, it 6
would be nice if that sort of thing existed for very 7
complex new codes that we've never heard of such that 8
if we have specific questions or needed to see 9
specific parts of the V&V and QA, that that paper 10 trace exists, and I think that's been one of the 11 things that's been very hard to get to in past PFM 12 reviews.
13 MR. MOORE: Chairman Riccardella, Members 14 Bier and Dimitrijevic have their hands raised.
15 CHAIR RICCARDELLA: Oh, okay, I'm sorry.
16 I didn't see that. Vicki, would you want to --
17 MEMBER BIER: Sure, I can go first. I 18 have a couple of questions that are mainly kind of to 19 understand the context for this.
20 You mentioned that some of these 21 components that would be analyzed this way might be 22 passive components that wouldn't even appear in the 23 PRA, but just kind of a what if this failed, you know, 24 how likely is that and do we need to worry about it.
25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
58 Could this also be used for components 1
that are in there, like for, you know, getting 2
fragilities for seismic PRA or would you imagine that 3
would be a different process?
4 DR. RAYNAUD: Dave, do you want to try and 5
take that one?
6 MR. RUDLAND: We're talking specifically 7
here about fracture mechanics types of codes, but 8
there's no reason why the process that we have here 9
couldn't be applied to other probabilistic style 10 codes.
11 MEMBER BIER: Okay.
12 MR. RUDLAND: Yeah, I don't see why not, 13 but this particular regulatory guide is really focused 14 just on the probabilistic fracture mechanics codes, 15 yeah.
16 MEMBER BIER: So, I guess the question I 17 have which, you know, may be my lack of familiarity 18 with the context, but there's a very sophisticated 19 discussion about kind of all of the inputs and 20 complexities that could go into the analysis, you 21 know, epistemic versus aleatory uncertainties and how 22 they would be treated, et cetera, but there seems to 23 be relatively little discussion of what happens to the 24 output of the analysis.
25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
59 And, you know, part of that is, okay, 1
fine, you get a distribution of whatever that then, 2
you know, goes to the decision makers the same like 3
any other probabilistic result, but the thing that I'm 4
wondering is are there dependencies between the 5
results of analyses for different components?
6 So, for instance, I don't know, a valve 7
versus a tank, or a pipe in one orientation versus a 8
pipe in another orientation that may be positively 9
correlated like they're likely to fail in the same 10 circumstances, or they may be negatively correlated.
11 The factors that would lead one to fail would make it 12 less risky for the other component.
13 And am I just thinking too far downstream 14 and this would be done for, you know, a single 15 component in isolation or do we need some guidance to 16 deal with those things as well?
17 DR. RAYNAUD: I think the answer, at least 18 in my mind, is that there's no true limitation. You 19 know, I think we're often thinking about single 20 component because that's what we've done to date and 21 what we're more familiar with, but I don't see any 22 reason why a modeling tool couldn't be developed to 23 model interactions between components --
24 MEMBER BIER: Got it.
25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
60 DR. RAYNAUD: -- and so forth.
1 MEMBER BIER: Okay, but that's not in the 2
current practice very much, and therefore doesn't 3
feature largely in this guidance, is that fair?
4 DR. RAYNAUD: I think that's true. I 5
would look to Dave to confirm.
6 MR. RUDLAND: Yeah, if you don't mind, 7
Patrick. Yeah, so, you know, as I mentioned in my 8
little intro stuff, you know, typically we -- you 9
know, the way that we've done PRAs is we've had 10 guidance on, you know, how to conduct a good PRA 11 analysis or how to conduct a good PFM analysis.
12 But then for problem-specific issues, for 13 instance if we needed to develop some kind of criteria 14 for a leak before a break, for instance, there would 15 be separate guidance on how to, you know, on what the 16 acceptance criteria would be, what the interactions 17 would be.
18 That would have to be in separate guidance 19 because this is really just focused on how to do the 20 analyses, and how you apply the analyses to any 21 specific problem would have to be in different 22 guidance.
23 MEMBER BIER: Okay, I guess the one thing 24 I would say is that there may -- if you eventually 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
61 envision that kind of interaction, there may need to 1
be a lot more information carried forward from the 2
details of the analysis to support that eventually.
3 MR. RUDLAND: Yeah, and I really, I 4
totally agree, and especially as you mentioned for 5
these components that really aren't even modeled in 6
PRA. There's a lot of thought that needs to go into, 7
you know, how do you apply these risk insights, you 8
know, in determining the safety of these particular 9
components, and what the interactions are, and, you 10 know, are there isolable (phonetic) systems that you 11 need to consider and things like that.
12 MEMBER BIER: Yeah, okay.
13 MR. RUDLAND: Yeah.
14 MEMBER BIER: Thank you.
15 CHAIR RICCARDELLA: Vesna, did you want to 16 ask a question?
17 MEMBER DIMITRIJEVIC: Yes, I did not ask 18 a question. I wanted to make the comment actually to 19 support, Pete, what you said.
20 Well, the PRA especially, but I have met 21 the fracture mechanics code many times in my life, you 22 know, pressurized (unintelligible), the risk-informed 23 ISI where we had the Westinghouse code using fracture 24 mechanics, and an EPRI code using deterministic 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
62 results, and then I worked in internal flooding, those 1
guides.
2 So, there is many places you can use 3
fracture mechanics in the PRA, but the point that Pete 4
was making is that, you know, the risk informed in 5
this case is not usually that risk.
6 What is happening here is actually what is 7
much more important. How are those results used? So, 8
let me -- I would say that case based like, say, on 9
the vessel, fracture mechanics analysis of the 10 probability to fail the vessel.
11 So, in all PRAs, the number for that was 12 less than ten to the minus six and it was not 13 considered in the PRA. Then, the PRAs started coming 14 with much lower numbers, and suddenly ten to the minus 15 six would dominate the results, then suddenly the new 16 number of ten to the minus six that's showing up is 17 not really justification for either one of those 18 numbers.
19 So, now how important it is to have a good 20 fracture mechanic model to calculate this less than 21 one in a million or less than one in hundred millions, 22 and it's very important in calculating that.
23 So, obviously it would be much easier to 24 prove that something is less than some value than 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
63 getting a specific number like 6.2, you know, in minus 1
seven or something, and that could be the goal for 2
this.
3 The other thing is that when you calculate 4
delta risk results, delta risk results are very -- to 5
calculate the importance of anything. This number is 6
the same in each CBI (phonetic) for a lot of -- and so 7
it doesn't really matter what it is because the delta 8
risk disappear.
9 So, this is extremely -- so I think that 10 this is extremely important. If you did not change 11 any factor reaching part of this number, but possibly 12 what is very not likely to happen, then you can change 13 factor between impact, the vessel fail probability.
14 In that case, it's completely irrelevant 15 what that number, as long as it's not challenging 16 safety goal. It will disappear in every delta risk 17 Reg Guide 1174 application.
18 Similarly to large LOCA, like if you want 19 to see like the fracture mechanics code be applicable 20 to large LOCA, then obviously the large LOCA is also 21 small, not as small as the vessel, but you have a 22 choice between using the expert opinion, right, versus 23 trying to use the fracture mechanics code. In that 24
- case, you only have to evaluate where the 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
64 uncertainties are higher.
1 So, the use of fracture mechanics in the 2
PRA is often going to be very specific and limited.
3 It has to prove the smaller uncertain value. It has 4
to account for uncertainties, whether it's the 5
deterministic inputs, or the expert opinion, or just 6
to define the important factors that impact something.
7 So, you said one thing which is really 8
extremely true. This is so complex that it can just 9
be -- defining exact application is what may be your 10 risk-informed input.
11 When you
- say, you
- know, safety-12 significant, risk-informed input is defining what type 13 of application and what should be proved in that 14 application.
15 That's my opinion and I think Pete brought 16 that to say qualitative, but it's not always 17 qualitative. You can just prove envelope. You know, 18 it's smaller than this.
19 You can prove the factors which will not 20 impact that in the risk-informed applications, or you 21 can prove that uncertainties will lead to this is 22 higher than currently used. That was my take on some 23 of those things.
24 DR. RAYNAUD: Thank you.
25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
65 CHAIR RICCARDELLA: Like anything, there's 1
a trade off. As you expand the rigor, you expand the 2
complexity of the analysis. You extend the time and 3
the cost, and I think, you know, the two examples that 4
Ron just mentioned, FAVOR and xLPR, are an example.
5 I mean, look at the time and the cost that 6
went into developing those codes and, you know, you're 7
going to see perhaps less use to PFM because it just 8
turns out not to be cost effective to do it, but 9
that's why I'm suggesting some simpler criteria if 10 you're just looking for a qualitative result, but 11 that's just my opinion.
12 Are there any other members that want to 13 comment? We have one more presentation, but I think 14 maybe we'll take a break before that. Is there any 15 other members that have any comments on it?
16 MEMBER KIRCHNER: Pete, this is Walt. I 17 was trying to think through after listening to you and 18 Vesna in particular, and also Vicki's comments.
19 I
- wonder, Patrick, you and your 20 colleagues, have you thought about how this might be 21 applied to new materials through advanced reactors 22 that may be operating at higher temperatures, and 23 hence the overall, I'll just say qualitatively, the 24 probability of something like a pressure primary 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
66 vessel rupture or fracture could be much higher?
1 CHAIR RICCARDELLA: Or, Walt, like 2
graphite, which we just heard has to be analyzed in 3
the probabilistic method, right?
4 MEMBER KIRCHNER: Yeah, or composites. I 5
often think about graphite composites as pressure 6
vessels, or other internal structures like to house 7
control rods or boron carbide absorbers so that you 8
get the metal out of the system, but you introduce new 9
materials and new challenges.
10 So, Patrick, have you been thinking about 11
-- this morning, we had a very nice presentation on 12 Division 5 of the boiler and pressure vessel code.
13 Are you thinking along those lines to how this might 14 play out to supplement a risk-informed application for 15 an advanced high-temperature vessel?
16 DR. RAYNAUD: Yeah, so I guess what I want 17 to emphasize is that, you know, what we're thinking 18 about here is not really tied to any specific 19 technology, I mean, other than fracture mechanics.
20 But, you know, if you have the correct 21 models to model different temperature
- ranges, 22 different materials, different phenomena that might 23 lead to failures where fracture mechanics can be used, 24 then we believe that what we proposing here is general 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
67 enough.
1 And it's really focused more on 2
statistical methods and stuff much more than the 3
detail of which fracture mechanics model you're using, 4
so it is extensible to these other classes of problems 5
and materials that you've described.
6 MR. RUDLAND: Yeah, this is Dave Rudland 7
from the staff I could just comment also. You know, 8
it's really highly dependent on the information that 9
you have and the uncertainties that you've developed 10 and your confidence in those, right?
11 So, for these high-temperature materials 12 where we know there's not a lot of data quite yet, 13 right, I think it would be more difficult to apply 14 these types of techniques because you wouldn't have as 15 much confidence in the results because of the lack of 16 data.
17 You can make some approximations, and of 18 course adjust acceptance criteria based on that, but, 19 you know, as the industry develops more and more data 20 on the different materials, I think these applications 21 and the application of these types of tools will 22 become more useful.
23 CHAIR RICCARDELLA:
- Okay, so it's 24 approaching 3:30. We have one more presentation.
25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
68 David Dijamco, do you think you can get your 1
presentation done in a half hour or so?
2 MR. DIJAMCO: Oh, yes, I think I can, 3
Pete, so, yeah.
4 CHAIR RICCARDELLA: So, would it be okay 5
to take --
6 MR. DIJAMCO: Mine's only ten minutes or 7
so.
8 CHAIR RICCARDELLA: All right, so let's 9
take about a ten-minute break. It's 3:30. We'll 10 break until 3:40 and then we'll reconvene, okay?
11 MEMBER KIRCHNER: Could you afford 15 12 minutes, Pete --
13 CHAIR RICCARDELLA: Okay.
14 MEMBER KIRCHNER: -- for the break?
15 CHAIR RICCARDELLA: I can. We may end up 16 staying a little bit past 4:00, but that's okay.
17 MEMBER KIRCHNER: Thank you.
18 CHAIR RICCARDELLA: All right, so we're on 19 recess until, I have to keep converting the time, 20 until 3:45.
21 (Whereupon, the above-entitled matter went 22 off the record at 3:29 p.m. and resumed at 3:45 p.m.)
23 CHAIR RICCARDELLA: Okay, so it's 3:45 24 Eastern Time and we will resume the meeting. David, 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
69 are you ready to go?
1 MR. DIJAMCO: I'm ready to go.
2 CHAIR RICCARDELLA: Okay.
3 MR. DIJAMCO: Okay, so, good afternoon, 4
everybody, and good afternoon, Dr. Riccardella. It's 5
good to hear from you.
6 So, for those who don't know me, my name 7
is Dave Dijamco. I'm from the Office of Nuclear 8
Reactor Regulation from the Vessels and Internals 9
Branch, and I'm going to talk about the use of the PFM 10 guidance from a technical reviewer's perspective.
11 So, the NRC received a submittal from 12 Vogtle using PFM probabilistic fracture mechanics as 13 a technical basis, and so we had a chance to pilot 14 this guidance, which I want to emphasize it's still in 15 an unpublished draft and pre-decisional form, but we 16 had a chance to pilot this guidance during the review 17 of the Vogtle submittal.
18 And so the objective of this presentation 19 I would say is kind of the, is the more informal of 20 the three presentations, which is basically for me to 21 just share my thoughts on questions like how to review 22 when, where to focus, of course, in the guidance, and 23 other questions such as how is the guidance helpful or 24 what did we learn from its use.
25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
70 CHAIR RICCARDELLA: David, I take it this 1
was the operating units and Vogtle 1 and 2, not 3 and 2
4?
3 MR. DIJAMCO: That's correct. That's 4
correct, yeah.
5 CHAIR RICCARDELLA: Thank you.
6 MR. DIJAMCO: 1 and 2, yeah.
7 MEMBER BLEY: David, it's Dennis Bley.
8 You folks used the guidance. Was it available for the 9
folks at Vogtle or did they use it?
10 MR. DIJAMCO: No, I believe they were not 11 available for the Vogtle folks.
12 MEMBER BLEY: Okay.
13 MR. DIJAMCO: It was just internally 14 available to the NRC.
15 MEMBER BLEY: So, we have a case where 16 they did it on their own and then you applied the 17 guidance when you reviewed it. It would be 18 interesting --
19 (Simultaneous speaking.)
20 MR. DIJAMCO: I believe so, to some 21 extent. I believe -- I applied it to what was there.
22 MEMBER BLEY: Okay.
23 MR. RUDLAND: This is Dave Rudland. The 24 Vogtle staff had the EPRI white paper that they used.
25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
71 They followed the EPRI white paper, which was kind of 1
the basis for some of the categorizations that Patrick 2
talked about.
3 MEMBER BLEY: Okay, thanks.
4 MR. DIJAMCO: So, I just wanted to give a 5
quick background. So, EPRI initiated an NDE 6
optimization program that sought to increase the 7
inservice inspection intervals of certain class one 8
and class two components from ten to 30 years, and 9
EPRI issued a series of publicly available reports as 10 a result of this effort, and so each of these EPRI 11 reports is for a specific component.
12 For example, that first report, 14590, 13 it's for steam generator nozzles. The second report 14 is for pressurizer vessel shell wells and so forth, 15 and, of course, the centerpiece in all of this is 16 probabilistic fracture mechanics.
17 And this slide just gives you just a very, 18 very high level view of what the submittal was about.
19 It was basically an alternative request to increase 20 the ISI interval of steam generator welds and nozzles 21 from ten to 30 years, and it referenced the first of 22 that report I showed in the previous slide, that 23 14590, as the technical basis.
24 And I just provided here some basic 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
72 information about the submittal and the NRC safety 1
evaluation. All of these documents are available in 2
3 And I just wanted to mention also that 4
there were actually three more submittals that came in 5
after Vogtle which referenced each of the other three 6
EPRI reports that are shown on the previous slide.
7 Another thing to note is that the NRC did not review 8
any of the EPRI reports for generic use.
9 So, what's this EPRI report 14590? So, 10 again, it's the reference technical basis for the 11 Vogtle submittal, and the specific components were 12 steam generator nozzle to vessel welds and nozzle 13 inside radii.
14 As expected, because it's PFM based, it 15 was extensive. It was nearly 200 pages, lots of 16 technical topics, and in fact, PFM is only one of 17 several chapters.
18 And so this is what my brain looked like 19 after going over the report. I call it my spaghetti 20 brain because of all of these crisscrossing concepts 21 and topics.
22 So, my background is in mostly 23 deterministic analysis. Coming into the review, I did 24 have a little bit of knowledge of PFM.
25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
73 So, even though some of these concepts 1
like degradation mechanisms, finite
- models, 2
convergence and things like that, they were familiar 3
to me, but I just wasn't sure how to approach the 4
review.
5 So, the main question in my mind as a 6
technical reviewer was how do I review this thing?
7 And, of course, the PFM guidance, which I took to mean 8
the draft guidance and the draft NUREG, they suggested 9
an approach, or maybe it's better to say that I 10 inferred an approach from looking over the guidance.
11 So, how did the guidance help? So, at the 12 time of the Vogtle review, the details of the graded 13 approach that Patrick talked about were still under 14 development.
15 And as Patrick had mentioned, they are 16 still under current internal review, but the guidance 17 at that time had enough of a framework for me to work 18 with, especially the sections on the suggested content 19 of PFM submittals.
20 And I just wanted to note here the EPRI 21 white paper that Dave Rudland mentioned earlier, and 22 that was also on PFM submittals. It's publicly 23 available and that was very helpful in crafting the 24 draft graded approach.
25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
74 And so this brings me to the first thought 1
that I wanted to share, which is that the guidance 2
helped me put a framework around various topics, and 3
thus organizing my review, and these are the topics 4
that are covered in the guidance which Patrick talked 5
about in detail in his presentation.
6 And so again, this is, you know, my 7
spaghetti brain without the guidance and into a well-8 ordered path with the guidance.
9 And this is the actual outline of the 10 safety evaluation for the Vogtle submittal. I'm not 11 going to go through each one of these. The point here 12 is just to show that many of the topics that I've kind 13 of mentioned in the previous slides made it into the 14 safety evaluation.
15 So, I think there's an observation to be 16 made here, which is that I think there was an overall 17 efficiency gained in the submittal and review process.
18 The guidance will help the industry prepare submittals 19 that have a consistent level of quality, which in turn 20 helps the NRC review the submittals.
21 And there's nothing really new in this 22 process. One example that I can think of that's 23 relevant in the (unintelligible) is the use of Reg 24 Guide 1.99, Revision 2, to determine the embrittlement 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
75 levels of reactor pressure vessel materials.
1 The second thought that I wanted to share 2
was that the guidance helped me understand sensitivity 3
analysis and sensitivity studies and their importance 4
in interpreting results.
5 And so the PFM results for the base cases 6
were actually pretty far from the acceptance criteria, 7
but those base cases were based on a representative 8
configuration rather than a bounding configuration.
9 So, I felt that during the review, that I 10 needed to understand the impact of the most important 11 input parameters, and the SA and SS helped me in doing 12 that, and both of these topics are covered well in the 13 draft guide and the draft NUREG.
14 The third thought that I wanted to share 15 was that the guidance supports knowledge management.
16 If you're just coming up to speed on PFM like I was, 17 and in many ways, I still am, I think the guidance is 18 a good learning tool.
19 For example, the draft chapter four of the 20 NUREG contains a
compendium of methods for 21 establishing confidence in a PFM analysis, so it talks 22 about things like uncertainty, convergence, acceptance 23 methods, things like that.
24 And I think this is a boom to all 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
76 stakeholders, especially as we move more and more 1
towards risk-informed decision making, of course of 2
which PFM is only a part as Dave Rudland mentioned 3
during his talk, but it could be an important part.
4 MEMBER BLEY: David?
5 MR. DIJAMCO: Yes?
6 MEMBER BLEY: Coming to this without a 7
real strong background in probabilistic fracture 8
mechanics as many people probably will, were there 9
areas where you were hoping for more help than you 10 found in the guidance, or if there were some of those, 11 did they get factored into the current version of the 12 guidance?
13 MR. DIJAMCO: I think were -- so, at that 14 time, you know, that version of the guidance that was 15 available when I reviewed Vogtle and as I mentioned, 16 you know, again for me, I'm mostly deterministic, but 17 when I needed help, you know, there were others in the 18 staff that I went to, especially, and he's here right 19 now, Dave Rudland, and so I think where I lacked the 20 knowledge and the proper confidence to review it, you 21 know, I consulted with the right people.
22 MR. RUDLAND: Can I just comment also that 23 in addition to what Dave has mentioned, we also, 24 especially for this particular request, held an audit 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
77 of the probabilistic fracture mechanics code, which 1
was a unique code just for this application, to gain 2
further knowledge about how the code works, and the 3
quality assurance, and the validation and 4
verification.
5 So, that wasn't necessarily in the 6
submittal because we were able to hold this virtual 7
basically audit of the code to understand it a little 8
better, and that also helps us to understand it if 9
we're not familiar with it.
10 MEMBER BLEY: Thank you for that.
11 MEMBER KIRCHNER: To the two Davids, this 12 is Walt Kirchner. Are there a lot of probabilistic 13 fracture mechanics codes out there? Is each a one-off 14 case or is there, besides the NRC codes, are there, 15
- quote, unquote, industry standards that most 16 applicants would turn to?
17 MR. DIJAMCO: How do you feel about that, 18 Dave? I mean --
19 MR. RUDLAND: Well, I'll tell you, you 20 know, there is the hopes of these larger codes like 21 FAVOR and xLPR, that they would be generic enough to 22 be used for a lot of applications.
23 What we're finding is that I think the 24 industry prefers to develop specific codes for their 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
78 particular applications. So, we're seeing that more 1
and more, that they're more, I don't want to say one-2 offs, but at least applicable to only a certain set of 3
components.
4 CHAIR RICCARDELLA: You know, Walt, I 5
think a lot of these industry codes though, they kind 6
of have a common kernel, you know, the Monte Carlo 7
sampling, and it's just that they adapt the input and 8
the output and maybe the fracture mechanics models to 9
the specific problem, but a lot of them are based on 10 a common routine. At least I know when we do that, we 11 have one common Monte Carlo sampling approach.
12 MR. RUDLAND: Yeah, and I agree with Pete, 13 and I think that was one of the reasons why we put 14 into our graded approach, you know, a previously 15 approved or approved as the use code that's been 16 modified, because a lot of times, they'll do that.
17 They'll just change a model or this, that, or the 18 other thing, and apply it to something different, so.
19 MEMBER KIRCHNER: Thank you.
20 MR. DIJAMCO: Okay, actually that was my 21 last slide. That's pretty much all I have, but I just 22 wanted to make kind of a couple of final remarks.
23 So, you know, I just shared the thoughts 24 of just one reviewer. As Dave mentioned earlier, Dave 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
79 Rudland, we have been getting PFM submittals recently, 1
and I think we do expect to get more and more PFM 2
submittals, and so I think having the guidance is very 3
timely.
4 And I think as these submittals come in, 5
I think it will be interesting to hear the 6
perspectives of other reviewers.
7 CHAIR RICCARDELLA: So, David, just one 8
technical question. You mentioned Reg Guide 1.99. I 9
don't see how that came into play on this Vogtle 10 submittal since it's not in a radiation zone. What 11 was the reference to Reg Guide 1.99?
12 MR. DIJAMCO: Pete, I was just referring 13 to the process of using or having a regulatory guide 14 to, you know, guidance for the industry, and really 15 for the industry and NRC to help the submittal and 16 review process. That's kind of the analogy there.
17 That's just one example of how, you know, the use of 18 reg guidance is --
19 CHAIR RICCARDELLA: Yeah.
20 MR. DIJAMCO: -- has been used in the 21 past.
22 CHAIR RICCARDELLA: Obsolete though it may 23 be.
24 MR. DIJAMCO: Correct, correct, that's 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
80 right, yeah.
1 CHAIR RICCARDELLA: I couldn't help but 2
get in that dig.
3 (Laughter.)
4 MR. DIJAMCO: Right, no, I understand.
5 MEMBER BALLINGER: Obsolete, but still 6
used.
7 MR. DIJAMCO: It's still used, yeah.
8 CHAIR RICCARDELLA: Okay, that's a subject 9
for another day, right?
10 (Laughter.)
11 MR. DIJAMCO: That's all I had, so.
12 CHAIR RICCARDELLA: Okay, well, thank you 13 for the presentation, David. Are there any other 14 member comments or discussion? Okay, hearing none, we 15 will turn to public comments. Is the bridge line open 16 now and unmuted?
17 MR. DASHIELL: The bridge line is open for 18 public comments.
19 CHAIR RICCARDELLA: Thank you, Thomas.
20 So, do we have any comments from the public? If so, 21 please state your name and make your comment.
22 DR. COFIE: Dr. Riccardella, this is Nat 23 Cofie here. Can you hear me?
24 CHAIR RICCARDELLA: Yes, I hear you, Nat.
25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
81 How are you doing?
1 DR. COFIE: I'm doing good, Dr. Pete.
2 Thank you for this meeting. You know, I've learned a 3
lot. I just have a simple question for Patrick and 4
maybe Dr. Rudland.
5 They mentioned, you know, codes by, you 6
know, classification, NRC-approved codes. I heard 7
something like that. What constitutes an NRC-approved 8
code? Is there a database or somewhere that one could 9
go and find out what codes for PFM have been approved 10 by the NRC? Is it a code that has gone through an 11 audit? What constitutes an NRC-approved code?
12 DR. RAYNAUD: I can take that and Dave can 13 add to it. For now, obviously NRC's own codes, so 14 FAVOR and XLPR, and we're looking specifically for 15 codes where a safety evaluation report would have been 16 written for that code or for that code in a specific 17 application.
18 CHAIR RICCARDELLA:
So, that's 19 significant. If the code has been used for an 20 application and that application was approved, then 21 that constitutes and NRC-approved code?
22 DR. RAYNAUD: I wouldn't say that.
23 MR. RUDLAND: I would say as long as it's 24 used in the exact same application, then, yeah, but if 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
82 not, then it would have to go to the different, the 1
next step down where --
2 CHAIR RICCARDELLA: It would be one of 3
those modified categories.
4 MR. RUDLAND: Yeah.
5 DR. RAYNAUD: Exactly.
6 MR. RUDLAND: Because again, the first 7
application of any code, we're going to do like we did 8
with the Vogtle and have an audit or something like 9
that to really understand the workings of the code, so 10 I think the bulk is going to happen in the first types 11 of review.
12 MR. MOORE: Mr. Chairman, does the court 13 reporter need the caller's name?
14 CHAIR RICCARDELLA: Yes, Nat, do you want 15 to give your name and spell it? I can do it.
16 DR. COFIE: Oh, my name?
17 CHAIR RICCARDELLA: It's Nathaniel Cofie, 18 C-O-F-I-E.
19 MR. MOORE: Thank you.
20 CHAIR RICCARDELLA: Are there any other 21 comments?
22 DR. HARRINGTON: Pete, this is Craig 23 Harrington.
24 CHAIR RICCARDELLA: Oh, hi, Craig, how are 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
83 you?
1 DR. HARRINGTON: I'm fine. How are you?
2 I just wanted to add onto the comments that you and 3
Dave made a minute ago about other codes. The 4
distinction between like an xLPR or the more one-off 5
codes like were used for Vogtle or some of these other 6
activities.
7 I think to the extent that industry can 8
use an existing code like an xLPR for a new 9
application, they're certainly going to do that 10 because of the benefits of it having some degree of 11 preapproval and, you know, all of that.
12 But back to that first category that 13 Patrick talked about of planning the analysis and the 14 steps to ensure that your quantities of interest are 15 adequately addressed by the code, these probabilistic 16 fracture mechanics analyses are complex enough and 17 they're nuanced enough that very often even with the 18 range of flexibility that we tried to build into xLPR, 19 it still doesn't quite get to exactly the place that 20 you might want it or need it to go.
21 And so I think that drives these one-off 22 codes, not a preference by industry, but just that 23 it's that complex of a problem that it's very hard to 24 solve every probabilistic fracture mechanics problem 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
84 with a broad brush tool.
1 And even with xLPR, for any of the 2
analyses that we're doing, we end up tweaking little 3
things or making small adjustments possibly to tailor 4
it to the specific need at hand, so I just wanted to 5
make that clarification.
6 CHAIR RICCARDELLA: Thank you very much for 7
your input, Craig. Is there anybody else out there 8
that would like to make a comment?
9 Well, with that and no further comments 10 from the members, I thank everyone for a very 11 interesting series of presentations and a very 12 informative session, and the meeting is adjourned.
13 (Whereupon, the above-entitled matter went 14 off the record at 4:07 p.m.)
15 16 17 18 19 20 21 22 23 24 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
Need for Probabilistic Fracture Mechanics Guidance Remarks by David L. Rudland, Ph.D.
Senior Level Advisor for Materials Division of New and Renewed Licenses Office of Nuclear Reactor Regulations Advisory Committee on Reactor Safeguards Meeting of the Subcommittee on Metallurgy & Reactor Fuels July 20, 2021
Integrated Decision Making
- Objective is integrated decision making
- Key is risk informed not risk based
- Use of risk insights for passive component integrity 07/20/2021 PFM Guidance l ACRS Sub-Committee Meeting 2
Risk-Informed Licensing 3
Other licensing actions Guidance/Process Under development 07/20/2021 PFM Guidance l ACRS Sub-Committee Meeting
Use of Probabilistic Fracture Mechanics 07/20/2021 PFM Guidance l ACRS Sub-Committee Meeting 4
- Inputs to PRA
- Change in Inspection Schedule - eliminate inspections?
- Estimate LOCA Frequency
- Leak-before-break Probabilistic Fracture Mechanics (PFM) is being used Requests for Inspection Relief LOCA Frequencies LBB Others?
Licensing Reviews and Emergent Issues 5
Risk/PRA Acceptability Risk information submitted outside of the formal risk-informed licensing basis change process Reliance on Licensee Risk Information Qualitative Quantitative Deterministic Reviews Relief Requests Requests with PFM Process under development LIC-206, others Process defined Emergent issues?
Process defined LIC-504 Risk-Informed License Basis Changes (RD 1.174/RG 1.200 TSTF-505 Risk-informed Completion times NFPA-805 Risk-informed Fire Protection 10CFR 50.69 SSC Categorization 5b Risk-informed Surveillance Frequencies Risk-informed Inservice Inspection Need adequate, consistent information and confidence in results 07/20/2021 PFM Guidance l ACRS Sub-Committee Meeting
PFM - Past Regulatory Issues 6
Difficult for NRC staff to reproduce or verify PFM calculations o
Complex regulatory review o
Black Box codes with insufficient vetting of inputs and code: low confidence in outputs Challenges where PFM was proposed as regulatory basis for long term inspection programs under NRC rulemaking 07/20/2021 PFM Guidance l ACRS Sub-Committee Meeting
PFM Hurdles and Path Forward
- PFM submittals and level of acceptability have been inconsistent
- Risk-informed decision making is still in the process of being implemented throughout all agency activities
- PFM results are only one part of risk-informed decision making
- Some license processes under development
- Use of PFM for licensing is increasing rapidly
- PFM Regulatory Guide will provide for consistent and adequate information for licensing application reviews 7
Defense in Depth Safety Margin Increase in Risk Monitor Change meets Regs 07/20/2021 PFM Guidance l ACRS Sub-Committee Meeting
RES Perspectives on PFM Analyses and on a Graded Approach for PFM Remarks by Patrick A. C. Raynaud, Ph.D.
Senior Materials Engineer Division of Engineering Office of Nuclear Regulatory Research Advisory Committee on Reactor Safeguards Meeting of the Subcommittee on Metallurgy & Reactor Fuels July 20, 2021
Objectives and Disclaimers The goals of this presentation are:
1.
Describe NRC/RES current perspectives on the analytical steps that could be used to create a high-confidence PFM demonstration for the purpose of demonstrating an adequate solution to a technical problem 2.
Describe what a graded approach for PFM analyses and documentation might look like when considering the safety significance and the complexity of a given problem Disclaimers:
1.
All content shown in these slides is pre-decisional and does not represent an official position of the NRC 2.
None of the ideas presented here are final 3.
This content is not intended to be guidance on what constitutes an acceptable approach for PFM submittals to the NRC 07/20/2021 Probabilistic Fracture Mechanics l ACRS Sub-Committee Meeting 2
Outline
- Part 1: Analytical steps in a probabilistic fracture mechanics demonstration
- Part 2: Detailed thoughts on a graded approach for PFM 07/20/2021 Probabilistic Fracture Mechanics l ACRS Sub-Committee Meeting 3
Analytical steps in a probabilistic fracture mechanics demonstration 07/20/2021 Probabilistic Fracture Mechanics l ACRS Sub-Committee Meeting 4
- Define the regulatory context
- Define the QoI and how it relates to the model output and acceptance criteria
- Determine the suitability of the PFM code for the application
- Identify key elements of the problem that impact analysis choices
- 1. Translate regulatory requirements into an analysis plan
- Identify uncertain model inputs
- Specify probability distributions on uncertain inputs
- 2. Characterize model input uncertainty
- Select a sampling scheme
- Assess sampling uncertainty
- Conduct sensitivity analysis
- Conduct output uncertainty analysis
- 3. Estimate QoIs and their associated uncertainty
- Determine a set of sensitivity studies
- Conduct sensitivity studies and present results
- 4. Conduct sensitivity studies to assess the credibility of modeling assumptions
- Interpret analysis results
- Iterate on the analysis process to refine model results
- 5. Draw conclusions From analysis results Plan Analyze Synthesize Step Action Analysis results inconclusive, refinements needed.
Step 1: Translation of Regulatory Requirements into an Analysis Plan
- Action 1: Define the Regulatory Context
- How will PFM analyses will be used as a technical basis for a regulatory action?
- What criteria will be used to support a proposed regulatory action?
- Action 2: Define the Quantity of Interest and How it Relates to the Model Output and Acceptance Criteria
- Map regulatory requirements onto specific model outputs
- Ensure that the model is predicting appropriate and relevant quantities
- Action 3: Determine the Suitability of the Probabilistic Fracture Mechanics Code for the Specific Application
- Determine whether a specific PFM code is suitable for the application of interest
- Identify any potential limitations of the code with regard to the application
- Action 4: Identify Key Elements of the Problem that Impact Analysis Choices
- Identify key elements of the PFM application that will determine how to conduct the analysis
- Example: simplifications to a model because of computational limitations or because the problem is inherently not complex 07/20/2021 Probabilistic Fracture Mechanics l ACRS Sub-Committee Meeting 5
Step 2: Model Input Uncertainty Characterization
- Action 1: Identify Uncertain Model Inputs
- Determine which model inputs are treated with uncertainty
- Determine the type of uncertainty (aleatory or epistemic) for each input
- Action 2: Specify Probability Distributions on Uncertain Inputs
- Uncertainty in model inputs is represented through probability distributions
- Uncertainty is propagated forward to the model outputs to estimate and quantify uncertainty in QoIs 07/20/2021 Probabilistic Fracture Mechanics l ACRS Sub-Committee Meeting 6
Step 3: Estimation of Quantity of Interest and Associated Uncertainty
- Action 1: Select a Sampling Scheme for Sampling Uncertain Model Inputs
- Select a method for propagating uncertainty in the model inputs through the model
- Goal is to estimate the QoI and the associated uncertainty
- Action 2: Assess Sampling Uncertainty: Statistical Convergence Analysis
- Assess the statistical convergence of QoI estimates from model outputs given a sampling scheme
- Action 3: Conduct Sensitivity Analyses to Determine Input Uncertainty Importance
- Identify problem drivers
- Confirm that the model is behaving as expected
- Identify inputs that may need refinement before final estimation of the QoI
- Identify assumptions may be candidates for sensitivity studies (Step 4)
- Identify candidates for more targeted sampling methods such as importance sampling
- Action 4: Conduct Output Uncertainty Analysis
- Provide a final estimate, with associated uncertainty, of the QoI
- Visualize results 07/20/2021 Probabilistic Fracture Mechanics l ACRS Sub-Committee Meeting 7
Step 4: Sensitivity Studies to Assess the Credibility of Modeling Assumptions
- Action 1: Determine a Set of Sensitivity Studies
- Identify important assumptions that merit further scrutiny
- Understand what might happen if these assumptions were changed
- Action 2: Conduct Sensitivity Studies and Present Results
- Many different options for sensitivity studies, but same common elements
- Reference realization (or baseline case) with a documentation of the QoI
- One or several modified realizations illustrating the concept that needs to be represented
- Comparison between the reference realization and the modified realization(s)
- Comparison criterion to decide whether the change is significant
- Conclusion, including potential consequences 07/20/2021 Probabilistic Fracture Mechanics l ACRS Sub-Committee Meeting 8
Step 5: Draw Conclusions from Analysis Results
- Action 1: Interpret Analysis Results
- Synthesize the information gathered in Steps 1 through 4
- Draw conclusions from the information
- May be able to directly compare PFM results with an acceptance criterion
- May need several analyses to create an evidence package
- Action 2: Iterate on the Analysis Process to Refine Model Results
- Determine whether additional analyses are required to draw informative conclusions from the modeling
- If results inconclusive, many possible paths:
- Changing or clarifying aspects of the PFM code
- Refining the input uncertainty distributions
- Choosing a different sampling scheme or increasing the number of model realizations
- Adding more sensitivity studies to address existing limitations 07/20/2021 Probabilistic Fracture Mechanics l ACRS Sub-Committee Meeting 9
Detailed thoughts on a graded approach for PFM
- PFM is complex
- The depth and breadth of a PFM analysis might vary widely depending on several factors
- It makes sense to take a graded approach
- for PFM analyses themselves
- for the level of detail to be presented as part of an evidence package
- General Principles
- Higher safety significance
- Higher complexity
- Higher level of novelty
- Topics Covered
- Models
- Inputs
- Uncertainty Propagation
- Convergence
- Sensitivity Analyses
- QoI Uncertainty Characterization
- Sensitivity Studies More analyses, more documentation Higher burden to create defensible and rigorous evidence 07/20/2021 Probabilistic Fracture Mechanics l ACRS Sub-Committee Meeting 10
Software QA and V&V Safety demonstrations for the NRC usually require that a QA program be in place Pre-submittal meetings are very useful to help ensure that everyone agrees on the graded approach path 07/20/2021 Probabilistic Fracture Mechanics l ACRS Sub-Committee Meeting 11 Category Description Graded Approach QV-1 NRC-approved code QV-1A Exercised within validated range Demonstrate code applicability within the validated range.
Describe features of the specific application where the code is validated and applicable (i.e., areas of known code capability).
QV-1B Exercised outside of validated range Provide evidence for the applicability of the code to the specific application with respect to the areas of unknown code capability.
Describe features of the specific application where the code has not been previously validated and applied (i.e., areas of unknown code capability).
QV-1C Modified Give an SQA summary and V&V description for modified portions of the code.
Demonstrate that the code was not broken as a result of changes.
Make detailed documentation available for further review upon request.
QV-2 Commercial off-the-shelf software designed for the specific purpose of the application Demonstrate code applicability.
Describe the software and its pedigree.
Make software and documentation available for review upon request.
QV-3 Custom code Summarize the SQA program and its implementation.
Provide a basic description of the measures for quality assurance, including V&V of the PFM analysis code as applied in the subject report.
For very simple applications, possibly provide the source code instead of standardized SQA and V&V.
Include separate deterministic fracture mechanics analyses to support other validation results, as appropriate for a given application.
Models 07/20/2021 Probabilistic Fracture Mechanics l ACRS Sub-Committee Meeting 12 Models could be addressed with the following graded approach Each model in a given PFM tool could potentially be categorized differently Category Description Graded Approach M-1 Model from a code in category QV-1A or QV-1B within the same validated range Reference existing documentation for that model in the NRC-approved code, demonstrate that the current range of the model is within the previously approved and validated range, and demonstrate that the model functions as intended in the new software.
M-2 Model from a code in category QV-1A or QV-1B outside the validated range See M-1, except demonstrate validity of the model for the new applicability range (document a comparison of model predictions for the entire new range to applicable supporting data, including quantitative goodness-of-fit analyses).
M-3 Model derived from a category M-1 or M-2 model See M-2, and include a detailed description of changes to the M-1 or M-2 model, with justification for the validity of the new model.
M-4 Well-established model not previously part of an NRC-approved code Describe gaps and limitations in the code capabilities for the analysis, combined with a strategy for mitigating identified gaps and communicating any remaining issues or risks.
Describe the model(s) applied in the PFM analysis code in sufficient detail so a competent analyst familiar with the relevant subject area could independently implement the model(s) from the documentation alone. Model forms can either be theoretical, semiempirical, or empirical.
Establish a basis for all significant aspects of the model(s). This may consist of raw data or published references. Document or reference any algorithms or numerical methods (e.g., root-finding, optimization) needed to implement the model(s). Discuss any significant assumptions, approximations, and simplifications made, including their potential impacts on the analysis.
Identify important uncertainties or conservatisms.
Describe the computational expense of the model and how that might affect analysis choices.
M-5 First-of-a-kind model not yet published in a peer-reviewed journal See M-4, and perform and document model sensitivity studies to understand trends in the model, as compared to expected model behavior and to the data used to develop the model, and describe model maturity and the status of the technical basis.
Inputs 07/20/2021 Probabilistic Fracture Mechanics l ACRS Sub-Committee Meeting 13 Category Graded Approach I-1D List input value.
I-1R List input distribution type and parameters.
If applicable, list uncertainty classification (aleatory or epistemic).
I-2D List input value.
If there is a lack of data, justify the use of expert judgment.
I-2R List input distribution type and parameters.
If applicable, list uncertainty classification (aleatory or epistemic).
If there is a lack of data, justify the use of expert judgment.
I-3D List input value.
State the rationale for setting the input to a deterministic value.
For each deterministic input, give the rationale (method and data) for the selection of its numerical value, along with any known conservatisms in that numerical value and the rationale for such conservatisms.
Reference documents that contain the foundation for input choices.
Explain the correlations between inputs and how they are modeled, and verify that correlated inputs remain consistent and physically valid.
Describe any sensitivity analyses/studies performed to show that the input or its classification does not have a significant effect on the QoI.
I-3R List input distribution type and parameters.
If applicable, list uncertainty classification (aleatory or epistemic).
If relevant, classify uncertain inputs as aleatory or epistemic and give the corresponding rationale.
For each uncertain input, describe both its distribution parameter values and its distributional form. Give the rationale (method and data) for selecting each distribution, including any known conservatisms in the specified input distributions and the rationale for the conservatism. Detail the distributional fitting method, including interpolation, extrapolation, distribution truncation, and curve fitting.
Reference documents that contain the foundation for input choices.
Explain the correlations between inputs and how they are modeled, and verify that correlated inputs remain consistent and physically valid.
Describe any sensitivity analyses/studies performed to show that the input or its classification does not have a significant effect on the QoI.
I-4D See I-3D.
If there is a lack of data, justify the use of expert judgment.
I-4R See I-3R.
If there is a lack of data, justify the use of expert judgment.
Input Category Low Knowledge of Input Characteristics High Knowledge of Input Characteristics Deterministic Uncertain Deterministic Uncertain High Importance I-4D I-4R I-3D I-3R Low Importance I-2D I-2R I-1D I-1R
Uncertainty Propagation Each analysis within a larger PFM demonstration could potentially fall in a different category 07/20/2021 Probabilistic Fracture Mechanics l ACRS Sub-Committee Meeting 14 Category Description Graded Approach UP-1 Analysis does not employ a surrogate model Give the method for uncertainty propagation and describe the simulation framework.
If Monte Carlo sampling is used, describe the finalized sampling scheme and rationale for the sampling scheme, including sampling method, sample size, the pseudo-random number generation method, and the random seeds used.
Describe the approach for maintaining separation of aleatory and epistemic uncertainties, if applicable.
If importance sampling is used to oversample important regions of the input space, justify the choice of importance distribution.
UP-2 Analysis does employ a surrogate model See UP-1 and describe the form of the surrogate model(s), any approximations or assumptions, the method used for fitting the surrogate, and the validation process for the surrogate model.
UP-2A Surrogate model used for sensitivity analysis See UP-2 and describe the features of the different surrogate models used.
UP-2B Surrogate model is used for uncertainty propagation See UP-2 and quantify the magnitude of error associated with the surrogate model approximation and include as additional uncertainty in the estimation of the QoI.
Convergence Discretization convergence should be achieved to have valid results Statistical convergence graded approach for each analysis within PFM demonstration 07/20/2021 Probabilistic Fracture Mechanics l ACRS Sub-Committee Meeting 15 Category Description Graded Approach SC-1
[Acceptance criteria met with at least one order of magnitude margin] AND [no importance sampling AND no surrogate models used]
No sampling uncertainty characterization recommended as long as the uncertainty is sufficiently small relative to the margin.
SC-2A
[Acceptance criteria met with at least one order of magnitude margin] AND [use of importance sampling OR surrogate models OR both]
Describe the approach used for assessing statistical convergence, with one method needed for sampling uncertainty characterization.
Explain the approach used for characterizing sampling uncertainty.
Justify why the sampling uncertainty is small enough for the intended purpose (i.e., why statistical convergence is sufficient for the intended purpose).
Describe how sampling uncertainty is used in the interpretation of the results.
SC-2B
[Acceptance criteria met with at least one order of magnitude margin] AND [use of importance sampling OR surrogate models OR both)) AND [separation of aleatory and epistemic uncertainties is implemented in the PFM code]
See SC-2A, and distinguish between epistemic and aleatory means and standard deviations.
SC-3A
[Acceptance criteria met with less than one order of magnitude margin]
See SC-2A, and provide two different methods for sampling uncertainty characterization.
SC-3B
[Acceptance criteria met with less than one order of magnitude margin] AND [separation of aleatory and epistemic uncertainties is implemented in the PFM code]
See SC-3A and give a sample size convergence analysis for both the aleatory and epistemic sample sizes.
Sensitivity Analyses 07/20/2021 Probabilistic Fracture Mechanics l ACRS Sub-Committee Meeting 16 Category Description Sensitivity Analysis Needed?
Graded Approach SA-1 Previously approved code (QV-1, QV-1A) with same QoI and same inputs No Describe important input and measure of input importance from previous use.
SA-2 Previously approved code (QV-1, QV-1A) with different QoI Yes Explain the methods used for sensitivity analysis, including any initial screening and model approximations and assumptions.
State whether a local or global sensitivity analysis approach is used.
Give the QoI used for the sensitivity analysis.
For a global sensitivity analysis, describe the sampling scheme along with the rationale for selection, including the sampling technique, number of model realizations, and random seed for the model realizations.
Provide the results of the sensitivity analysis, including the most important model inputs identified; a measure of the input importance, such as the variance explained by the most important inputs; and relevant graphical summaries of the sensitivity analysis results.
SA-3 Modified approved code with limited independent variables (e.g., <5, determined on a case-by-case basis)
Yes Describe analyses, important input, and measure of input importance.
SA-4 Modified approved code with many independent variables (e.g., >5, determined on a case-by-case basis)
Yes See SA-2.
SA-5 First-of-a-kind code with limited independent variables (e.g., <5, determined on a case-by-case basis)
Yes Describe the analyses, important input, and measure of input importance and include additional documentation.
SA-6 First-of-a-kind code with many independent variables (e.g., >5, determined on a case-by-case basis)
Yes, with submodel SA as appropriate See SA-2.
Indicate how the sensitivity analysis results informed future uncertainty propagation for estimation of the QoI and associated uncertainty.
State whether the results of the sensitivity analysis are consistent with the expected important inputs based on expert judgment.
QoI Uncertainty Characterization 07/20/2021 Probabilistic Fracture Mechanics l ACRS Sub-Committee Meeting 17 Category Description Graded Approach O-1 Acceptance criteria met with at least one order of magnitude margin Give a measure of the best estimate and uncertainty in the QoI.
Include a graphical display of the output uncertainty.
Describe how the best estimate and its uncertainty were calculated, including a clear description of the types of uncertainty (e.g., input, sampling, epistemic) being summarized.
Summarize key uncertainties considered in the analysis and any major assumptions, conservatisms, or simplifications that were included and assess (qualitative or quantitative) their effect on the analysis conclusions.
O-2A Acceptance criteria met with less than one order of magnitude margin and a strong basis for input distributions and uncertainty classification See O-1 and provide the reasoning behind a strong basis.
O-2B Acceptance criteria met with less than one order of magnitude margin and no strong basis for input distributions or uncertainty classification, or both See O-1.
Include a sensitivity analysis (if important inputs are unknown) and sensitivity studies for any inputs that do not have a strong basis.
O-3 O-1, O-2A, or O-2B and potential unknowns See O-1, and provide the reasoning behind a strong basis.
Describe potential unknowns and their possible effect on analysis results.
OR Include a sensitivity analysis (if important inputs are unknown) and sensitivity studies for any inputs that do not have a strong basis.
Sensitivity Studies 07/20/2021 Probabilistic Fracture Mechanics l ACRS Sub-Committee Meeting 18 Category Description Sensitivity Study Needed?
Graded Approach SS-1 Category QV-1A code with same QoI No Summarize sensitivity studies conducted in prior approval.
SS-2 Category QV-1A code with different QoI Limited, focused on inputs related to QoI Summarize past sensitivity studies conducted in prior approval and current sensitivity studies.
SS-3 Category QV-1B or QV-1C code with limited independent variables (e.g., <5, determined on a case-by-case basis)
Limited, focused on impact of modification Summarize past and current sensitivity studies.
SS-4 Category QV-1B or QV-1C code with many independent variables (e.g., >5, determined on a case-by-case basis)
Yes, focused on inputs related to QoI Summarize past and current sensitivity studies.
List the uncertain assumptions that are considered for sensitivity studies.
State the impact and conclusion of each sensitivity study.
Give the rationale for why certain assumptions were or were not considered for sensitivity studies.
Provide the specific question(s) each sensitivity study is attempting to answer.
Describe a reference realization.
Describe how each sensitivity study is translated into model realizations, and compare the study and the reference realization.
List changes to the code and the QA procedure used.
SS-5 Category QV-2 or QV-3 code with limited independent variables (e.g., <5, determined on a case by case basis)
Yes See SS-4.
SS-6 Category QV-2 or QV-3 code with many independent variables (e.g., >5, determined on a case by case basis)
Yes, model and input studies See SS-4.
Next Steps NRC will publish draft regulatory guidance and an accompanying draft technical basis NUREG, both for public comments NRC will gather public comments on the draft guidance and draft NUREG NRC will address all comments received Public meetings may take place, if needed At least one more ACRS briefing on the topic of PFM and PFM regulatory guidance is expected in the future 07/20/2021 Probabilistic Fracture Mechanics l ACRS Sub-Committee Meeting 19
Use of PFM Guidance A technical reviewers perspective July 20, 2021 David Dijamco Office of Nuclear Reactor Regulation Vessels and Internals Branch
Objective The NRC received a submittal from Vogtle using PFM as technical basis; pilot the PFM guidance (unpublished, draft, pre-decisional) during the review of this submittal.
Objective of this presentation is to share thoughts on:
- How the review went (with a focus on the draft guidance).
- How was the draft guidance helpful? What did we learn from its use?
7/20/2021 PFM Guidance l ACRS Sub-Committee Meeting 2
=
Background===
EPRI initiated an NDE optimization program that sought to increase the inservice inspection (ISI) intervals of certain Class 1/Class 2 components from 10 to 30 years. Issued a series of publicly available reports as a result of this effort:
- EPRI report 14590 (ML19347B107) - steam generator nozzles
- EPRI report 15905 (ML21021A271) - pressurizer vessel shell welds
- EPRI report 15906 (ML20225A141) - steam generator vessel shell welds
- EPRI report 18473 (ML21091A014) - BWR RHR heat exchanger welds and nozzles 7/20/2021 PFM Guidance l ACRS Sub-Committee Meeting 3
Center piece is probabilistic fracture mechanics (PFM).
Vogtle Units 1 and 2 submittal
- Alternative request to increase the ISI interval of steam generator welds and nozzles from 10 to 30 years. Referenced EPRI report 14590 as technical basis.
- Submittal:
ML19347B105 - Dec. 2019 ML20253A311 - revised/supplemented Sept. 2020
- NRC safety evaluation:
ML20352A155 - Jan. 2021 Note: The NRC did not review EPRI report 14590 and the other EPRI reports for generic use.
7/20/2021 PFM Guidance l ACRS Sub-Committee Meeting 4
EPRI Report 14590
- Referenced technical basis for the Vogtle submittal for steam generator nozzle-to-vessel welds and nozzle inside radii
- Extensive report, nearly 200 pages, lots of technical topics (PFM is only one of several chapters) 7/20/2021 PFM Guidance l ACRS Sub-Committee Meeting 5
My brain after going over EPRI Report 14590 (spaghetti brain) 7/20/2021 PFM Guidance l ACRS Sub-Committee Meeting 6
Selection of transient loads Stress intensity factors Sensitivity studies Examination coverage
Main question as a technical reviewer was:
How do I review the submittal???
7/20/2021 PFM Guidance l ACRS Sub-Committee Meeting 7
PFM Guidance Draft Regulatory Guide + Draft NUREG-7278 (both unpublished, still under internal review)
How did the guidance help?
- At the time of the Vogtle review, details of graded approach were still under development (and are currently under internal review)
- But it had enough of a framework for me to work with, especially the sections on the suggested content of PFM submittals.
(Note: EPRI white paper on PFM submittals (ML19241A545, publicly available) was very helpful in crafting the draft graded approach.)
7/20/2021 PFM Guidance l ACRS Sub-Committee Meeting 8
- 1. The (draft, pre-decisional) PFM guidance helped me put a framework around the various topics, thus organizing my review.
Topics covered in the guidance
- Models
- Inputs
- Uncertainties
- Sensitivity analyses
- Sensitivity studies
- Convergence
- Software V& V 7/20/2021 PFM Guidance l ACRS Sub-Committee Meeting 9
Without PFM Guidance 7/20/2021 PFM Guidance l ACRS Sub-Committee Meeting 10 Selection of transient loads Stress intensity factors Sensitivity studies Examination coverage
With PFM Guidance 7/20/2021 PFM Guidance l ACRS Sub-Committee Meeting 11 Degradation mechanisms Fatigue crack growth Probabilistic fracture mechanics Examination coverage Sensitivity studies Sensitivity analysis Sampling method Convergence Component configuration selection Deterministic fracture mechanics Stress intensity factors Fracture toughness Software V&V Stress analysis Selection of transients Finite element models Pressure stress analysis Thermal stress analysis
Outline of safety evaluation of the Vogtle submittal
- 3.8.1 Overall PFM Approach (acceptance criteria, sampling method, etc.)
- 3.8.2 Parameters most significant to PFM results
- 3.8.3 Stress analysis
- 3.8.4 Fracture toughness
- 3.8.5 Flaw density
- 3.8.6 Fatigue crack growth rate
- 3.8.7 ISI & examination coverage
- 3.8.9 PFM results relevant to Vogtles request 7/20/2021 PFM Guidance l ACRS Sub-Committee Meeting 12
Overall efficiency gained in submittal/review process The (draft, pre-decisional) PFM guidance
- helps the industry prepare submittals that have a consistent level of quality
- helps the NRC review the submittals 7/20/2021 PFM Guidance l ACRS Sub-Committee Meeting 13
- 2. The (draft, pre-decisional) PFM guidance helped me understand sensitivity analysis (SA) and sensitivity studies (SS) and their importance in the interpreting the PFM results.
Both the (draft, pre-decisional) regulatory guide and (draft, unpublished)
NUREG-7278 cover these topics.
7/20/2021 PFM Guidance l ACRS Sub-Committee Meeting 14
- 3. The (draft, pre-decisional) PFM guidance supports knowledge management.
- Draft Chapter 4 of NUREG-7278 contains a compendium of methods for establishing confidence in a PFM analysis
- A boon to all stakeholders, especially as we move more and more toward RIDM of which PFM is only a part, but its an important part.
7/20/2021 PFM Guidance l ACRS Sub-Committee Meeting 15