ML22140A225
| ML22140A225 | |
| Person / Time | |
|---|---|
| Issue date: | 03/08/2022 |
| From: | Office of Nuclear Reactor Regulation |
| To: | |
| References | |
| Download: ML22140A225 (76) | |
Text
1 UNITED STATES OF AMERICA NUCLEAR REGULATORY COMMISSION
+ + + + +
34TH REGULATORY INFORMATION CONFERENCE (RIC)
+ + + + +
TECHNICAL SESSION - T8 MAKING EFFECTIVE DECISIONS IN USING DEFENSE IN DEPTH, SAFETY MARGINS, AND RISK!
+ + + + +
TUESDAY, MARCH 8, 2022
+ + + + +
The Technical Session met via Video-Teleconference, at 3:00 p.m. EST, Andrea Kock, Deputy Office Director for Engineering, Office of Nuclear Reactor Regulation, presiding.
PRESENT:
ANDREA KOCK, Deputy Office Director for Engineering, NRR/NRC DOUG TRUE, Chief Nuclear Officer and Senior Vice President, Nuclear Energy Institute SMAIN YALAOUI, Senior PSA Technical Specialist, Directorate of Assessment and Analysis, NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
2 Canadian Nuclear Safety Commission MARK THAGGARD, Director, Division of Risk Analysis, RES/NRC MIKE FRANOVICH, Division of Risk Assessment, NRR/NRC LUNDY PRESSLEY, Reliability and Risk Analyst, PRA Oversight Branch, Division of Risk Assessment, NRR/NRC NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
3 TABLE OF CONTENTS PAGE Introductory Remarks Andrea Kock .......................................4 Risk-informed Decision-making: Greater Than the Sum of its Parts Doug True .........................................9 CNSC Risk Informed Decision Making Smain Yalaoui ....................................20 Polling Questions ................................32 Safety Marker Study Mark Thaggard ....................................35 Polling Question .................................44 Safety Improvements Using Risk Insights Mike Franovich....................................45 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
4 P-R-O-C-E-E-D-I-N-G-S MS. KOCK: Good afternoon, everybody.
Welcome to the NRC's RIC technical session on making effective decisions in using defense-in-depth safety margins in risk. This session is a joint effort by the NRC's Office of Nuclear Regulatory Research and the Office of Nuclear Reactor Regulation.
Before I start, I just want to acknowledge and support our friends in the Ukraine and their valiant efforts to defend their country.
We want you to know that we stand with you in this time of challenge. Next slide, please.
I just wanted to go over the agenda quickly. I'm going to start with some quick introductions and then our panel members will present and discuss the effective application of risk-informed decision making.
They're going to talk about some practical examples, best practices, and lessons learned, successes, challenges, and other considerations in how risk has helped us keep our focus on safety.
This morning, Chairman Hanson touched on NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
5 the importance of risk in decisions in a wide range of views. This session will facilitate understanding of what drives risk and the differences of perspectives on risk.
For example, we'll cover misconceptions about the role of risk considerations in defense-in-depth and how we integrate the use of risk-informed decision making with deterministic reviews for effective decision making.
These are historically some of the most challenging issues, and today you're lucky because you have the experts here to assist with untangling the ins and outs of risk-informed decision making.
Additionally, the NRC and our external stakeholders will highlight both the positive benefits and potential pitfalls of using risk-informed decision making and it will provide you a great opportunity to engage with the panelists who are experts in this very important area on the issues during the panel discussion.
We have about an hour and a half for our discussion and we plan to spend about half of that time in discussion, so we ask that you hold your questions until all of the speakers have finished NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
6 with their presentations.
Just a few acknowledgments before we start. I want to thank everyone for their preparation and participation in this session. In particular, I want to thank the session speakers for agreeing to participate in this session and for their effort taken to prepare for the session.
And a special thanks to Lundy Pressley and Matt Humberstone of the NRC for their work in coordinating this session, and, of course, our AV staff that's making sure that everything goes very smoothly. Thank you.
A quick overview, and I think this has already been said before, but the journey to become a more modern risk-informed regulator is a really important one and it will help us to more effectively accomplish our mission to protect people and the environment by helping us focus on safety-significant aspects of our work.
This is an important topic as risk-informed decision making is at the heart of sound regulatory practices, and for the NRC, it's a critical part of our regulatory transformation.
The importance of this topic, I think, is NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
7 reflected in the number of people who registered for this session. At last count, I think it was over 800. That's quite impressive.
I just want to start by introducing our panelists. First, we're going to hear from Doug True. He's the Nuclear Energy Institute's chief nuclear officer and senior vice president of generation and suppliers at the NEI.
He has more than 40 years of nuclear industry experience in nuclear safety, and prior to joining NEI, he contributed to many of the major milestones in risk-informed regulation and he was responsible for one of the largest specialty engineering organizations in the nuclear industry, including being president of the largest nuclear PRA firm in the world.
Secondly, we're going to hear from Smain Yalaoui. He's a senior probabilistic safety assessment technical specialist with the Canadian Nuclear Safety Commission.
Mr. Yalaoui has a Master's in Nuclear Engineering. He specialized in probabilistic risk assessment, and he joined the Canadian Nuclear Safety Commission in 2008.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
8 Internationally, Mr. Yalaoui contributed to the development of IAEA safety report series, and he is a member of the NEA working group on risk. He specifically participated in the IAEA and WGRISK projects on multi-unit and site-level PSA. Mr.
Yalaoui took part in the International Seismic Probabilistic Seismic Assessment peer reviews.
Our third presenter will be Mark Thaggard. He serves at the deputy director of the Division of Risk Analysis here at the NRC in the Office of Nuclear Reactor Research.
Mr. Thaggard joined the NRC in 1989, and in 2009, he was selected for the Senior Executive Service, serving in the Office of Federal and State Materials and Environmental Management Office and also in the Office of Nuclear Security and Incident Response, and most recently in the Office of Research as a director of the Division of Risk Analysis.
Our last presenter will be Mike Franovich and he is the director of the Division of Risk Assessment in the Office of Nuclear Reactor Regulation.
He has over 30 years of nuclear experience. Mr. Franovich is a member of the Senior NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
9 Executive Service and he previously served as deputy and acting director of the Fukushima Lessons-Learned Division, enhancing defenses against extreme natural events.
He currently leads the Division of Risk Assessment, which conducts probabilistic risk analysis and establishes regulatory standards for risk-informed nuclear reactor licensing oversight, accident consequence analysis, health physics, and fire protection engineering.
And without further ado, I'm going to turn the panel session over to our first presenter, Mr. Doug True.
MR. TRUE: Thanks, Andrea. I'm glad to be here today and thanks for inviting me. I look forward to the discussion today. I titled my presentation today Risk-Informed Decision-Making:
Greater Than the Sum of its Parts, because I really believe that that's true.
In fact, I think it was over 25 years ago I wrote a paper that talked about how, when you use PRA information, it's important to consider defense-in-depth and safety margins and the performance of equipment in those considerations, and that was two NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
10 years even before Reg. Guide 1174 was invented and the term risk-informed was coined.
So, this is a subject that's near and dear to my heart and I really believe that there are some misconceptions and some misunderstandings about this side, but I'm going to try to talk our way through.
And I want to start with a discussion on safety versus risk. A lot of times, we think of those as being different things. I actually believe they are intimately related, so let's go to the next slide.
So, as we all know, our regulations for the current plants are based on deterministic requirements to provide that foundation for ensuring the safety of our nuclear power plants.
Risk analysis gives us a tool that allows us to assess the risks that remain when the regulations are met, and that risk is never zero.
The risk analysis not only gives us a chance to estimate those residual risks or a level of safety, quantitative level of safety, but it also lets us understand what contributes to them, and through that, we can understand also chances in NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
11 requirements, either additional requirements or relaxations in requirements, and how they impact that risk, that residual risk, and in the case of the STP process under the ROP, what non-compliances look like and what the significance of those are.
This means that really what we do in the safety side is tied directly to what we measure on the risk, and I want to turn in the next slide to an example of that.
So, as we think about the relationship between risk and safety, you can have different levels of safety requirements across the bottom of this graph and different levels of risk on the left axis.
Here, we're looking at redundancy. You can have no safety systems and very high risk, one, or two, or three, or four safety systems and have lower and lower risk.
You can add a whole bunch of safety systems, but there will always be a residual risk, and what we're trying to do in understanding what that residual risk looks like is to understanding what contributes to it, how in this case redundancy applies, but it works in all kinds of different NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
12 directions, not just in the sense of redundancy.
You could talk about how much safety margin we need in our containment, how much shielding you need for radiation safety. It's understanding that residual that's left after we've decided what level of deterministic requirement we're going to have.
The benefit of that understanding of what contributes allows us to make risk-informed decisions. Let's go to the next slide.
And that understanding allows us to actually improve safety. It gives us the ability to focus on what's truly safety significant. It allows us to allocate resources in the manner that most effectively improves safety so we can focus on the things that are the biggest contributors to risk and maybe minimize the amount of effort we put on things that are less important to make sure we're staying focused on the things that are most important.
Risk informing also incentivizes licensees to focus on what's important to safety. If they understand what's important to safety, they know where they run the risk with the regulator of having a significant safety issue. As I said, it also NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
13 allows us to know where we can spend less time on things of less importance.
Overall, we've seen across the last 25 or 30 years that this focus on safety significance that risk allows us to do actually stimulates a net improvement in safety. What you focus on actually improves, and therefore risk goes down.
You've seen me and others at NEI present that curve of PF versus time that shows how we've driven risk down. That curve happens to apply just to internal events, but the same thing is true of external events. All of the work we did in NFPA 805 greatly reduced fire risk at those plants. Other enhancements we've made in plants have reduced other contributors to risk.
But what's important when we do this is that when we're using PRA as a tool is that we understand its limitations. As much as I'm a PRA practitioner and have been one for now 40-odd years, it's a tool that can only be used within its limitations and it has to be used appropriately.
It's neither omnipotent, nor omniscient.
It doesn't always come out with an answer that can be trusted as just a flat number. You've got to NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
14 understand what contributes to it.
But when I say that about PRA, the same thing applies to the deterministic approaches.
There's nothing perfect about a deterministic approach either. It has its own limitations that we've learned over the years, which is one of the reasons why we brought risk into our decision-making process. So, let's move to the next slide and talk about that.
Reg. Guide 1174 outlines the risk-informed decision-making process that's shown here on the right-hand side of this graphic. The PRA results are one input into that process and they're a product of a model that certainly contains uncertainties.
Uncertainties exist whether you're using a deterministic approach or you're using a probabilistic approach. It's just the PRA allows us to quantify those and illuminate them in a way that is very difficult to do if you're dealing with strictly a deterministic basis.
The brilliance of Reg. Guide 1174 was that it outlined an integrated decision-making process, a process that used PRA as one piece, but also asked us to consider defense-in-depth and safety NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
15 margins in that decision-making process.
And it's important that they use the term integrated. It's not that PRA is a gate, that defense-in-depth is a gate, and safety margins is a gate, that you have to pass through each of those.
They need to be considered in an integrated manner where you understand what the risk analysis is telling you, what the defense-in-depth considerations are telling you, and understand how those fit together.
Sometimes PRA isn't very good at modeling things. Sometimes it's quite good at it and that should be taken into account, and that goes to the point of uncertainties, and what's important about using PRA in a decision-making process is understanding the uncertainties that are important to that decision.
Typically, when we're looking at a particular use of PRA, we're looking at one particular slide, whether it's a piece of equipment that had failed and had a performance deficiency associated with it, or it's a change in a technical specification, or some other plant configuration change.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
16 We need to understand what the role that change makes in the overall risk profile and which uncertainties contribute to that. If we do that, then we can understand how that plays into the defense-in-depth and safety margin considerations.
And a good example of this is when we were looking at what to do after Fukushima. We could have gone after that with a strictly numerical approach and tried to devise some method to say once we get below ten to the minus X, we're good, but instead, we said there's a lot of uncertainty here.
These box one events like happened at Fukushima are rare. It's better for us to think about this in a defense-in-depth posture and that's how we ended up with FLEX.
So, PRA has its role. In certain things, it's very good. In other cases, we need to think about what the uncertainties are and put more emphasis on other pieces of this decision-making process.
And I just at the bottom there took a quote right directly out of Reg. Guide 1174 that talks about how important it is to take all of these pieces together in an integrated manner so that we NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
17 understand the relationships between the risk, the safety margins, defense-in-depth, how we can monitor that performance, and what we're talking about in terms of the regulation.
And I'll finish up with one last slide.
The next one talks about this and some examples. So, from a regulatory perspective, we've had some fairly significant decisions that have used the risk insights to drive that.
The ATWS rule for sure was driven by our understanding of ATWS events back at the time it was promulgated, the Station Blackout rule where we actually identified sort of a hole in the whole defense-in-depth process where we realized that loss of onsite power events were more likely and we needed more than just a limited amount of redundancy.
We actually needed an ability to cope, and in some cases, we needed an alternative AC power supply in order to be able to mitigate those risks.
That insight wouldn't have come about from a strictly deterministic understanding. It came about because we understood the likelihood and the consequences, and we understood the defense-in-depth we had.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
18 Another good example is in the Fukushima period where we installed severe accident vents and the water addition to BWR Mark 1 and Mark II.
In that process, we looked at a lot of different options, but we learned that without the water addition, having a severe accident vent wasn't going to help us much because the containment was ultimately going to fail, and that vent pathway that we thought was going to give us benefit would actually be compromised, and that understanding came about by understanding the nature of the events that occur in the severe accident regime and understanding how to mitigate those and the uncertainties associated with them.
And lastly, I won't go through all of these in detail, but utilities and licensees have made a lot of voluntary changes. I can't count the number of plants who have found things that were compliant with the regulations, but from a risk perspective, were driving risk results, and they made enhancements to their plants to address those.
A good example is fire protection piping, and a number of plants found a flooding risk from that piping could delay the impacts on AC and DC NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
19 power. Plants weren't comfortable with that and they made changes to mitigate that flooding risk and reduce their overall risk profile because of that.
Plants have installed non-safety equipment to perform functions that are important, made lots of procedural changes to the user systems in different ways, and fed back the insights from their PRAs into training so that the human actions are in a context that's without residual risk, so to make sure we're training our operators on the actions that are really going to be necessary in order to address some of the most important things.
So, as you can tell, I'm very bullish on the risk-informed concept. I think if we do it well, we can really enhance and get a better value out of our regulations by getting the best out of the risk insights, the defense-in-depth, and safety margin understandings that we already have.
With that, Andrea, I'll turn it back to you.
MS. KOCK: Thanks, Doug. You gave us some really good insights based on your experience, so I know I appreciated that and I'm sure the audience did too.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
20 I think it's time for our first polling question, so that's kind of exciting. So, these are going to be a couple of questions about your, the audience's experience in using risk-informed decision making and how extensively you've used it and has it benefitted your work?
So, we're going to ask these questions and we're going to save the answers to the polling questions to the end of the panel session, so you can wait in anticipation for those.
And then just a note, in order to access the polling questions, you can go to the right of your screen and toggle from the Q&A space into the polling questions so that you can answer those.
Okay, I can't see the polling questions.
Can anybody else on the panel see them? No? Okay, so I think what we'll do is maybe move onto our next speaker and maybe we'll catch up with the polling questions after the second speaker.
So, our second speaker will be Smain Yalaoui and he's going to be talking to us about risk-informed decision making in Canada, so I'm going to turn it over to Smain.
MR. YALAOUI: Thank you very much, Ms.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
21 Chair. Thank you for having me at this technical session. So, my presentation is about the risk-informed decision making in Canada. Next slide, please.
So, this is the outline of my presentation. First, I will discuss the RIDM within the CNSC regulatory framework. I will discuss the key principles of the RIDM. I'll provide a brief history of RIDM in Canada.
I will discuss shortly the staff procedure for risk evaluation, estimation and evaluation. I will talk also about the PSA use to support the RIDM. I will then elaborate on the CNSC risk handbook tool that we have developed at the CNSC.
I'll talk about the emergency mitigating equipment consideration known as FLEX in the U.S.
Next, I will discuss the benefits and pitfalls of PSA use in RIDM, and then I'll finish with a short summary of my presentation. Next slide, please.
So, for the RIDM regulatory framework, here are some highlights. We have a regulatory document, REGDOC-3.5.3, which is regulatory fundamentals. This regulatory document describes the risk-informed approach to licensing and NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
22 compliance activities. It emphasizes that the focus is on issues of higher risk.
There is also an internal RIDM procedure at the CNSC that further elaborates on situations where staff can apply a risk-informed approach for regulatory requirements and guidance, and for regulatory decisions as well.
This CNSC RIDM approach emphasizes that PSA can be used to complement the DSA and other RIDM key principles with due consideration of uncertainties. We can go to the next slide, please.
This slide shows the overall approach of risk-informed decision making which integrates insights from the deterministic safety analysis, the PSA, operating experience, and mandatory requirements.
The chart, if you just go to the next, it will show the chart that describes the different elements of the RIDM. The type of decisions and candidates for RIDM in Canada is the same as in other countries and regulatory bodies.
Here, I just named a few of them, and these include, for example, the design, siting, licensing, radiologic safety reviews and life NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
23 extension projects, and decommissioning. Next?
Yeah, this is the chart I was talking about, I was referring to showing the different elements that goes within the RIDM, and the same was presented by our first presenter, Dr. Doug True.
Next slide, please.
So, here I'm showing the RIDM key principles as introduced in the chart from the previous slide. The key principles include first the demonstration that the relevant legislation and requirements are met.
That defense-in-depth is maintained is the second key principle, and generally this principle is assessed without invoking the PSA, but we know that the PSA can offer some insights like the cutsets can inform by revealing how many failures may occur before we can get to core damage frequency or large range frequency.
The third key element or the key principle is about the safety margins should be maintained, and there are also instances where the PSA can be used to show that the safety margins are maintained.
The fourth principle is acceptable risk NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
24 impact, and here PSA can provide the calculation of incremental risk such as the delta CDF and delta LRF, and the last principle is to monitor the performance.
However, the challenge that we face with these key principles, I think, is just to find is there any balanced way for the consideration of these key principles or is there any weighting factors that we can use to consider the five elements? Next slide, please.
In this slide, I will provide a brief history of RIDM in Canada. So, traditionally decision making has heavily relied on defense-in-depth and expert judgment in the past, but in the last decades, we show increasing use of PSA in Canada, use of PSA in RIDM.
And this is mainly because back in 2005, we issued at the CNSC two regulatory documents, one on PSA and the other one is on the reliability program for nuclear power plants, and in parallel, the staff were developing a procedure, RIDM procedure based on the CSA standard which is called risk management.
In 2018, the CNSC issued this regulatory document, REGDOC-3.5.3, and in 2019, about three years ago, the CSA standards issued CSA N290.19 which NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
25 is called RIDM for NPPs, building on staff procedures.
And right now currently, CSA is conducting a survey on the use of this standard and to which extent it helps the industry and the regulators in applying this standard. Next?
Oh, I think I covered all of this bullet, yeah. I think I covered all of these bullets. Next?
Next? Next? Next? Next? Yeah, so now on this slide, I'm showing the -- can you just press the next, please? Next? So, yeah, and next, another one, another time? Okay, perfect.
So, in this slide, I'm showing the staff procedure for risk evaluation. This is based on the risk tolerability scale for determining the risk significance levels.
It's almost, I should say, deterministic, and the risk evaluation is based on using matrices, as you can see in the second chart. We chose the consequence and likelihoods, and we defined the risk significance levels dependent on the likelihood and the consequences of each.
So, this procedure, which was based on risk tolerability, was successfully used in the past NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
26 for the reclassification of what we call CANDU generic safety issues.
And I'm providing just in the blue box some examples, like for the reclassification of the generic issue of pressure tube failure coincident with moderator heat sink failure, which we call LOCA/LOMA, loss of coolant accident and loss of moderator, safety improvements for steam line breaks in multi-unit nuclear power plants, and the large LOCA reclassification for certain break sizes to beyond design basis accidents. Next, please.
So, in this slide, the PSA use to support RIDM, as a direct use of the PSA and the PSA result that we get from the licensees, CNSC staff have developed what we call the risk handbook tool which is a web-based application.
In this tool, the PSA and reliability program results and insights are used to risk inform the licensing and compliance verification activities.
Other PSA uses to support RIDM include risk management for outage planning for online maintenance, for example, what we call risk monitor or risk watch.
We use it also for life extension NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
27 projects. PSA can help identify safety improvement opportunities if a plant needs to go through the life extension.
We also use it for SAMG, severe accident management guidelines development, emergency preparedness drills and exercises because all of the programs and diagnostic analysis are done using the PSA and analysis derived from the level two PSA.
Next slide, please.
So, in this slide, I'm going to develop a little bit further on the CNSC risk handbook.
First, the purpose of this handbook is to support the regulatory compliance program, focusing primarily on applications for inspections, which means that this is mainly developed to support site inspectors.
This tool is used to optimize the inspection planning and improve efficiency. For example, we get requests for inspections of, let's say inspections for electrical, mechanical, or whatever system qualification inspections, then we use the PSA insights and results to provide a focused set of equipment, or human actions, or specific hazard information for the inspection purposes.
We do also use the PSA to evaluate NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
28 inspection results. For example, the site inspector may have a sense of the risk, incremental risk if a piece of equipment is taken out of service.
It also helps in the determination of safety significance of operational events. If some event happened, to have some sort of idea what is the incremental core damage frequency or large range frequency. Next slide, please.
This slide is about the emergency mitigating equipment credits in the PSA. Emergency mitigating equipment once again is known as FLEX in the U.S.
Emergency mitigating equipment functions that we have is first to prevent a severe accident.
The second objective is to repower instrumentation and monitoring of critical safety parameters.
The third objective is to prevent severe core damage. The fourth one is the in-vessel retention of collapsed core, and the fifth objective is to repower containment supporting functions.
Next? Next?
So, if we can see in the graph that I show here, the three first objectives, they are applicable to level one PSA, and the two last NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
29 objectives, they are applicable to level two PSA.
Next slide, please.
So, the prerequisites for EME credits in PSA, of course, as everything that appears in the PSA, there should be a clear guidance to deploy emergency mitigating equipment and decisions are also made within the main control room or secondary control room by authorized staff, because when the operation shifts to the emergency operating center, probably to deal with the human actions is more cumbersome in this situation.
The challenges that we faced with crediting the EMEs in the PSA is the first one is the use of PSA models, with or without EME, for different applications. Let's say if we want to compare against safety goals, do we consider the improvements from EMEs, yes or no?
The second question which is also discussed now at the CNSC and with the industry is the identification and classification of systems important to safety. So, if we consider emergency mitigating improvement as systems important to safety with all reliability program that should be applied to it, yes or no.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
30 We have also the issue of EME credits in multi-unit PSA, sequential EMEs or the FLEX equipment for different units going into accidents, and there is also the challenge of surveillance requirements for EME because most of the time they are just industry-grade equipment. Next slide, please.
In this slide, I will discuss the benefits and pitfalls of PSA. I think Dr. Doug True has provided very insightful about the benefits of PSA.
The benefits, as we know, that PSA can provide a rigorous and reproducible assessment of incremental risk, delta CDF and delta LRF, compared to risk significance levels which are based on subjective judgment. Let's say we think that consequences are high and the likelihood is low. PSA will provide a very good quantitative assessment of the incremental risk.
I'm just giving an example here. For example, in our operating policies and procedures, which is the same as tech specs in the U.S., we have, in the traditional deterministic approach, if the system redundancy is reduced, the repairs shall be made promptly or other actions taken to ensure NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
31 adequate system reliability and capability.
So, if we want to interpret this requirement, what do we mean by promptly? Does it mean minutes, hours, days? So, whereas the PSA, we can calculate, if we have the time at risk or the completion time, we can calculate exactly what's the incremental risk and compare it to the guidelines.
The pitfalls of the PSA is most of the PSA, sometimes there is a lack of cause and effect relationship in some cases. For example, if an emergency core cooling valve is passing, this is not modeled in the PSA.
If we want to change the methodology for trip set points determination, the PSA may not be a useful tool. For piping inspections, you need to do some work on PSA to help you use PSA.
There is also the issue of uncertainties in the PSA, and we always caution about the over reliance on PSA to address all safety concerns. PSA is not the solution for every issue that we may have.
Next slide, please?
This is the summary of my presentation.
I think in summary I would say that guidance is needed on how to assess the impacts on defense-in-depth and NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
32 safety margins, as well as on benefit cost analysis.
If we don't have clear guidance on how to assess or how PSA can help assess the impacts on defense-in-depth and safety margins, we may not have a clear and reproducible scheme for RIDM.
I think we all understand that PSA is a valuable tool to complement deterministic and expert judgment in the RIDM process.
Uncertainties, of course, should be accounted for, especially when the PSA is also close to the acceptance guidelines.
And I believe that development of RIDM guidance will allow a transparent and reproducible process for regulatory decisions. Next?
So, this is the end of my presentation.
Thank you for your attention.
MS. KOCK: Thank you, Smain. Okay, I think now we are going to go to polling questions, so we're going to do -- we have a total of four questions and we're going to do the first two now to kind of catch up.
So, the first question is about your experiences in using risk-informed decision making.
How much have you used it and how has it benefitted NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
33 your work? So, I'd really like to hear from you and maybe join the discussion at the end. We can get some feedback on this.
So, oh, there's the results already. How do you use risk-informed decision making as part of an official technical process? The vast majority of you have and just a few, 13 percent, have not.
So, it would be interesting to hear experiences for those of you who have, how you got to the point of using those risk-informed decision-making tools that you've talked about, and if you haven't, maybe we'll hear a little bit in the discussion session about why some folks have not used it, so that's kind of an interesting result there and I see it's fluctuating a little bit.
We can go onto polling question two. So, polling question two is going to have to do with how risk-informed decision making has benefitted your work. And I think we've already heard that there are differences of opinions on the benefit of risk-informed decision making, so it will be interesting to see the results here.
Okay, so we're still fluctuating a little bit, but it looks like most of the folks in the NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
34 audience have the opinion that risk-informed decision making has benefitted their work, and again, a smaller percentage says it hasn't.
So, it will be interesting to hear as part of the discussion at the end where perhaps it has significantly benefitted our work, and if it hasn't, why not, or maybe there are certain areas where it's not as beneficial, so that could be an interesting result.
We're going to move on now to the second polling questions, so there's an A and B aspect to those as well. The next two questions have to do with your opinion about how both the industry and the NRC or other regulatory agencies are using risk-informed decision making. 2A here is a question about whether industry is doing enough with regard to risk-informed decision making.
Okay, and the results are fluctuating just a little bit at this point. So, it looks like the majority opinion here is that there's more to do on risk-informed decision making, which really isn't surprising because I think we've already heard it's a complex topic, and there's about 20 percent of you who think that industry has done just the right amount NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
35 there.
Okay, and then the last polling question for now is going to have to do with how regulators like the NRC use risk-informed decision making and whether there's been enough there.
So, kind of interesting, the same result here for regulatory agencies as we saw for the industry. The vast consensus seems to be that there's more to do, and just to me personally, that's not surprising.
I think we've made great progress, but, you know, given some of the uncertainties and challenges that both Doug and Smain just talked about, it's not really surprising to hear there's more to do.
Okay, so that's interesting and maybe we can build off the answers to some of those questions as we move along in the presentations, but at this point, we're going to go to our next speaker who is Mark Thaggard, who is going to be talking about the safety marker study. Mark?
MR. THAGGARD: Good afternoon. The NRC has a long history of incorporating risk insights into its decision making. Some of that you're going NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
36 to hear about from Mike in the following presentation. However, the focus of my presentation is looking at the potential impacts of safety from the use of risk-informed decision making.
I plan to go over some insights from a fairly comprehensive assessment conducted by the NRC staff recently to look at safety trends within the U.S. nuclear industry.
The staff looked at a broad range of measures and markers to see if safety was trending in a positive direction, negative direction, remaining steady, or were indeterminate.
Keep in mind that this was a trending assessment and that the focus was in looking for trends. There was no attempt to make a comparison against a particular performance metric.
The staff also made no attempt to ascertain or infer compliance or noncompliance with our regulations. Next slide.
NRC maintains numerous safety activities and programs to monitor, assess, and reinforce safety. For the staff assessment, as noted, the staff looked at a broad range of these activities to get a comprehensive safety picture of U.S. commercial NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
37 nuclear power plants over the last 20 to 30 years, a time period that included the issuance of the Commission's probabilistic assessment policy statement issued in the 1990s.
Some of the measures and activities reviewed included looking at the number of scrams, the number of license event reports, inspection findings, performance indicators, risk measures, and insights from studies such as the State-of-the-Art Consequence Analysis or SOARCA, just to name a few.
The staff assessment focused only on safety and did not look at other possible trends such as economics or electric output. Next slide.
A key consideration in the staff's assessment is that there are several aspects of trends that can affect the overall conclusion. One clear example is the time frame you consider. Some performance measures might show improvements over the last 30 years, but may not show an improvement if you only looked at a 20-year period.
Another consideration is the advancements in our technical and modeling capabilities. Some performance measure improvements might be a result of our advancement in these areas NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
38 and not an actual safety improvement.
Considerations such as these required care in making a definitive conclusion on the trending direction of some activities. Accordingly, for a number of activities, the conclusion was indeterminate even when the staff could see a trend.
For a number of the safety measures, measure and performance came down to a collective engineering judgment. Next slide.
One of the considerations discussed in the last slide is the time period chosen impacts the overall conclusions. Looking at the period between 1988 to 2000, the staff saw several performance measures that clearly showed safety improvements.
This period saw many changes in the U.S.
nuclear industry, including implementing the Station Blackout rule, the maintenance rule, and the individual plant examination program. Next slide.
This slide highlights an example of a performance measure that showed clear safety improvements during the 1988 to 2000 time period. If you look at the chart on the left, it shows scrams while critical per unit over time, which shows a clear positive trend over this time period. Please note NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
39 that the reduction in the number of scrams over time is viewed as positive in this context.
The chart on the right reflects the difficulty in ascertaining safety improvements even when the data clearly indicate that things are moving in the right direction. It shows significant events per unit over time, which also shows a clear positive trend over this period. Again, a reduction in the number of events over time is viewed as positive.
Even with the apparent positive trend, the staff couldn't definitively determine that this was reflective of safety improvements because the Agency changed its reportability guidance during the period which could have affected the trend.
However, taken together, the two graphs help support an overall conclusion that nuclear plant safety improved during this period. For both of these performance measures, the positive trend after 2000 is not as clear. Next slide.
As stated, when looking at the last 20-year time period, the performance measures do not show the same level of safety improvements. There is still a trend in the right direction, but we do not see the same level as seen during the '90s.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
40 There are several possible reasons for this. One is the safety improvements from the '90s may have made further safety impacts less obvious, that is the law of diminishing returns.
This last 20-year period does include several changes that have a positive safety impact on the industry, including the reactor oversight program, the B.5.b security compensatory measure requirements, use of FLEX, and design enhancements such as the reactor cooling pump shutdown seals.
Therefore, another possible reason that we see less improvements in the last 20-year time period is that safety significant changes may not be fully realized. Next slide.
There are several different ways to look at all of the performance measures. These are six categories of performance measures used by staff. As you can see from the different category activities, some were more easy to ascertain a quantitative trend. Next slide.
Overall, the staff looked at roughly 50 performance measures. These roughly 50 performance measures, they clearly show a positive trend. Let's look at each of these.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
41 The number of scrams has dropped to historically low levels. The total number of scrams and scrams per unit showed a 20-year decrease in trends, while the number of plants with zero scrams showed a positive 20-year trend.
The number of reactor inspections has been trending steadily downward when looking at two different time periods. The current rate of all precursors exhibits a statistically significant decrease in trend for the 2000 to 2020 time period.
The collection radiation dose per unit shows a 20-year decrease in trend. The number of greater than green findings show a decrease in trend since 2014.
A micro level fleet average internal event core damage frequency or CDF based upon the Agency's risk models is trending downward.
Installation of the reactor coolant pump seals in Westinghouse PWR nuclear power plants has resulted in a reduction in both CDF and large early release alerts in our risk models.
Lastly, there is a highly statistically significant decrease in trend in the frequency of overall loss of offsite power.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
42 Only one performance measure showed a clearly negative trend, which is loss of offsite power recovery time. This marker showed a statistically significant increase in trend. Next slide.
Of the roughly 40-some remaining measures looked at by staff, nine were viewed as positive, but the assessment was more qualitative. In the interests of time, I won't go over the nine which are listed on the slide.
The remaining 30-plus measures were either steady or staff could not say definitively one way or the other the trending direction. An example where staff could not make a determination was the number of license event reports even though it showed a positive trend as I previously noted.
There were a number of the 30-plus performance measures like this that the staff concluded as indeterminate. Next slide.
Several points noted by the staff in their assessment were that there is a reduction in risk over the 30-year time period as measured by calculated average core damage frequency.
However, they noted this only considers NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
43 the contribution of risks from internal events.
External event hazards could in some cases significantly add to plant risk. The staff also noted a reduction in performance issues.
Based upon the staff's assessment, there is some margin between calculated risk and the NRC's safety goals. However, it is important to note that both uncertainty and external hazards need to be considered in determining the amount of the margin.
Next slide.
In conclusion, based on the performance measures looked at by the staff, the use of risk insights in decision making at the Agency is having an overall positive impact on the safety of the industry.
Notwithstanding the need for consideration of the influence of external hazards in our assessment and accounting for uncertainties, there have been clear safety improvements.
The performance measures show a more prominent trend in the 1990s, but a more gradual trend after 2000. Next slide.
This concludes my presentation. We'll save questions for the end, all right. I'll turn it NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
44 back to you, Andrea.
MS. KOCK: Thank you, Mark. I just want to put a plug in for everybody to get their questions in. We have one more speaker to go, but obviously you have a little bit of time, but if you have questions, go ahead and submit those and we'll cover those during the discussion period.
Now we're going to go to our third polling question which also has two subparts, and these questions have to do with your opinion on the state of industry safety, nuclear safety today versus ten years ago and the role of PRA.
So, the first question, in your opinion, is the nuclear industry safer than it was ten years ago?
Okay, it looks like there's less fluctuation now in the numbers, but the vast majority of you feel like the nuclear industry is safer than it was ten years ago, and then there's a small percentage of folks who are either in the no or depends category.
Okay, and then 3B, polling question 3B is about the role of PRA with regard to industry safety and whether PRA has played a role. So, in your NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
45 opinion, has the use of PRA and risk-informed decision making made the nuclear industry safer today than it was ten years ago?
Okay, those results look like they've kind of stopped fluctuating. So, it looks like these results follow closely the question right before this on the state of nuclear safety in that the vast majority of you feel that PRA and risk-informed decision making has contributed to safety, and then there's a smaller percentage who either answered no or maybe. Okay, so that was interesting.
We're going to move onto our last speaker, Mike Franovich, and he's going to talk about safety improvements using risk insights, so onto you, Mike.
MR. FRANOVICH: Thank you, Andrea. If we can have my first slide, please? Next slide? All right, I'll try to get us back on track a little bit.
We're running a little bit behind schedule here.
So, what I'm going to cover in my discussion is to give some context and a little bit of historical background of issues or policies that are active today and still affecting and shaping the way we do our business.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
46 And I also want to give you a few tangible, more contemporary examples of where advancements in risk assessment and other engineering analysis has actually helped enable us to make better decisions in some additional work going on, in particular with weather events, HEAF events, and also in new reactors, but more so what I'm going to talk about is how it's shaping our work in the operating reactor side, in particular, licensing.
And then a shameless plug here for an Agency initiative that I am a very strong proponent of, and that is the Be Risk Smart framework. It's an initiative that is looking to try to unify and provide more uniform application of risk concepts in all of our work, not just the technical work, but some of the more corporate support work, and using the risk triplet, and looking at the rewards that may come out of doing various projects, just not the downsides of them. So, if we could go to the next slide?
Okay, meeting the challenge of becoming a more modern risk-informed regulator, we could spend a lot of time on that topic alone. The journey is continuing.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
47 A couple of insights I want to share with you is that the technology in terms of risk assessment does continue to mature. Risks are dynamic. We do have to have effective risk management by maintaining our vigilance and assessment of operating experience.
The technology and insights do complement the traditional defense-in-depth framework that we have used for many, many decades to achieve an acceptable level of residual risk, not zero risk, but residual risk, and this journey does continue. So, if we can go to the next slide?
I mentioned the Be Risk Smart framework and a couple of other things that are going on in the Agency. Really what we've seen is a great push here in the last few years to try to come up with a more uniform application of guidance that crosscuts different offices and business lines.
Really, looking at the risk proposition, not just the negative side of what are the risks of certain decisions and not just all of the plants themselves, but also different projects, but what are the rewards?
What are the gains that we may gain by pursuing certain activities in the Agency? And try NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
48 doing them in a way that shows that we are getting a good return on our investment.
And if we are applying these approaches, in particular, the Be Risk Smart framework, it should enable us to become more agile and adaptive when we look at different projects within an office or across offices in the NRC.
So, I'm going to give you a flavor of a few of those things that are going on that might provide more insight into what's going on inside the NRC. You can go to the next slide.
Now, Mark mentioned that I would talk about some of the major policies that are affecting our thinking in the Agency. My point of this slide is not to go into a deep dive of these particular policies, but to share with you that they are active.
They are live. They do shape the way our work is going on today.
And periodically we need to go back and refresh our memories as to why we have these policies, in particular, starting in the '80s with the severe accident policy statement on how we treat severe accidents, also some anchoring guidance and expectations out of the safety goal policy statement NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
49 in terms of the qualitative safety goals, as well as the QHOs.
And then moving forward in time to the PRA policy statement where there's one particular aspect I want to highlight out of that policy statement that's still alive and well, and that is we should be using PRA technology to the extent it increased in all regulatory matters as supports by the state-of-the-art, and it goes onto say to complement that of the defense-in-depth philosophy that we use in the Agency.
And my next few examples are going to illustrate a little bit more how the state-of-the-art or what we would say today, state-of-the-practice, has actually advanced quite a bit.
And then lastly, I want to note a very important Commission decision that was made in 1999 based on a Commission paper, a White Paper sent up in 1998, and that is an overarching framework for how we are to use risk-informed, performance-based regulation, not just in operating reactors.
If you take a close look at the paper and the SRM, it actually speaks to our work in the materials area as well as the fuel facilities area.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
50 That paper is alive and well and is germane to a topic I'll talk here about when I get toward the end of my discussion. Next slide. If you can advance to the next slide, please?
Okay, my screen is frozen, so I don't know if you all are seeing the next slide, but if not, I will try to use my local copy. Okay, it looks like we have a little bit of a technical problem going on at the moment, that all the slides are frozen.
So, I'm just going to speak to what would be on the next slide that you would be seeing and that is the integrative decision making principles that Doug spoke of, and that there are five key principles.
I'm not really intending to go into depth of each of those principles of defense-in-depth, and safety margins, and performance monitoring. I will highlight that the objective also is that the risk changes for facilities should be small and within the safety goals.
But really, to me, it's something more important about how we go about making these integrated decisions. You can take each one of those decision-making principles and do them in isolation.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
51 And that's been a real challenge in the Agency to try to break down some of the silos and do more integrating through what we call integrated review teams, where you start a project with the principles in mind, not trying to bring them together through a long review process, and seeing how they all fit, and that they really are not mutually inclusive of each other.
To me, the integrated decision-making process really is a place for critical thinking.
What's going on in each one of those decision-making boxes in terms of defense-in-depth?
The plants are not static. They have changed. Many capital improvements have been made.
Where are the margins, both in physical margins and analytical margins? So, it's important that we have a culture that drives for having those conversations.
And I apologize for whatever the technical difficulties are. But we're not -- I'm not able to see the slides either on my end so I will be speaking verbally. I hope you all can still hear me.
I will just speak to describing my -- for sake of time.
There's another process. Oh, here we go.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
52 Thank you very much. Looks like the technical problem has been resolved. If we could go to the next slide.
And these integrated decision-making principles out of Reg. Guide 1174 -- we're on Rev. 3, by the way, even though this Reg. Guide came out in the 1990s -- we do use it to influence and help guide a number of other processes we have and one of those processes I'm going to talk to you is about LIC-504.
That is an agency or NRR process.
If you'll recall, the Davis-Bessie event from 2002 -- we're actually, I believe, close to the 20th anniversary -- important lessons learned. You need to have a process for not only making integrated decisions but also documenting the basis for your decisions.
So we had a number of corrective actions in the agency we took to create such guidance. This particular guidance is used very much in the agency today. It's a mature process.
We're currently on Revision 5. In the latest revision, we did add other considerations.
For example, if there are actions that are warranted what would be some of the risk offsets -- for example, NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
53 any additional exposure to radiation workers. Those should be active considerations and what we are looking at in terms of options.
And also, we included a section on risk decision or, rather, decision-making biases. There are some areas where you can get into groupthink, for example, and we need to be conscious about those when you're in that decision-making box or curve that you're not susceptible to those biases or you try to minimize them.
It is a two-step process where we look at immediate safety issues, first, if we need to take prompt regulatory action, and then if not we'll look at the longer-term actions in any second step process.
And we are using the Be riskSMART framework, which is in a NUREG that came out, NUREG/KM-0016. I would encourage you all to read it.
It's a very easy read. Again, it's not unique to PRA but it's promoting more of the risk triplet across the work that we do in the agency.
Can we go to the next slide?
All right. The first example I want to give is back in August 10th of 2020, we had a Derecho NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
54 event that hit the Duane Arnold plant, a very powerful line of wind, a storm that came through the plant and the plant performed as expected, and so to the operators this was very good news.
But we did identify a few unexpected conditions that happened. Even though there was loss of outside power, which is not unexpected, there was an issue of potential combined event and that the central service water system did have some degradation from debris that was on the river that came into the intake system.
So we did use the LIC-504 process. A few important takeaways came out of that. Looking at similarly situated plants and those similarly situated plants had done some upgrades, for example, alternate cooling for diesel generators. If they're dependent on a central service water they have alternate means.
We also found now with greater capability to quantify the benefits of flex that that is actually quite a difference maker in terms of risk impact.
Depending on the nature of the plant and its location and its site-specific hazards, it is quite influential on the risk results.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
55 And we applied the Be riskSMART framework, and the big takeaway there was even though we recommended a generic communication in terms of information notice, we also used the T in the teaching elements to get the word out and conduct a webinar and a multi-organizational panel to discuss the insights that came out of this study of various sites that we did take a look at that have some susceptibility to Derecho.
We did also find that the risks were not trivial but nor were they in a matter of a level where we needed to do some type of mandatory backfit analysis. So they were -- they did have import and we thought it was important to share that information with industry and the rest of the NRC.
We do have significant turnover, a turnover in the industry as well as in the NRC. So the T can also be viewed as knowledge transfer.
If we could go to the next example.
Doug had mentioned a number of capital improvements that plants have made, particularly for NFP 805 implementation with alternate seal injection, backup diesel generators, and so forth. One of the hazards the plants did do additional measures for is NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
56 to provide additional protection for high-energy arcing faults.
We more recently have come to discover there's a little bit more aluminum in these electrical components than originally envisioned and that aluminum might be a little bit more reactive than copper.
And so what does that actually mean if the hazards are slightly different in terms of risks?
So we're currently undergoing a LIC-504 evaluation and in that first step, which we did complete, we used more of a defense in depth qualitative risk thought with the defense in depth in mind, looking at the plants already have a level of protection for heat but they also have protection from post-9/11 measures for dealing with large fires and explosions.
So these are other qualitative considerations we need to bring into our decisions.
The work is ongoing and there's a massive effort, actually, been going on between the NRC and EPRI for years to build more consensus models to help bring in the more advanced fire modeling capability as well as the advances in the PRA technology.
And then my last example -- if we can go NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
57 to that slide -- is on new reactors. I will not go through all this on NuScale specific. You can read it at your leisure.
We had a very important takeaway and a commission direction, which had to do with the single failure criterion and the treatment and use of risk and those kind of decisions.
But more importantly, there was an insight or direction from the commission as a gentle reminder that we are to apply risk-informed principles in our decisions and it actually looped right back to the 1990s policies that I mentioned that are still active.
So we were kindly reminded as a staff you need to continue to be applying these principles in your work and so that's what we're doing. We're continuing on that journey.
And if we can go to my last slide, I'll wrap it up.
A couple takeaways. I want to leave you with a couple of thoughts. We continue to support risk-informed decision-making through our programs.
Risk is permeating a lot of our work. It touches a lot of areas in the agency.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
58 We are also using it in a manner that still complements defense in depth and safety margins and we're also including consideration for enterprise risk management, which is a federal government wide requirement on how we manage our projects and activities.
We do also take that into consideration as well, and we're also leveraging this new Be riskSMART framework with PRA technology and also looking at other business lines to see how we can help out.
And with that, I will turn it back to Andrea.
MS. KOCK: Great, Mike. That was a really great summary of the history of risk-informed decision-making at the NRC and some other things we have going on. So thanks for that.
So I'm running a little bit short on time. So we're going to quickly go to the fourth polling questions, 4A and 4B, and then we're going to jump right into questions so we have plenty of time for the discussion section.
So 4A and B have to do with the use of PRA and risk-informed decision-making in our work, NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
59 and the first question is PRA and RIDM create work efficiencies and aid in correctly focusing priorities. True or false?
That looks pretty steady. It looks like the vast majority of you feel that PRA and risk-informed decision-making do create work efficiencies and help us focus our priority.
It'd be really interesting to hear from some of the folks who are saying maybe and depends.
I would be interested to hear about that. Maybe we can get to that during the Q&A.
Okay, and 4B. 4B is PRA and risk-informed decision-making create efficiency benefits when interacting with regulators.
So it looks like we're easing out there.
A majority of you say sometimes. It would be interesting to hear when those sometimes are and what the considerations are, and whether PRA and risk-informed decision-making create efficiency when interacting with your regulator.
That's interesting. And then there's a smaller percentage of people say always and a very small percentage say never.
Okay. So with that, I think we're going NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
60 to go right into the discussion section of the panel session. We have a little over 15 minutes left.
And so the first question I have is actually for -- I heard it was Dr. True. So first question for Dr. True.
MR. TRUE: No Dr. here.
MS. KOCK: No Dr. Okay. I thought somebody called you Dr. earlier. So just Doug.
MR. TRUE: Smain was just being very nice.
MS. KOCK: Oh, okay. Great.
Okay. So first question for Doug. In recent years, there has been an impression that PRA and risk-informed decision-making are too complicated and costly.
Do you believe the benefits of risk-informed decision-making have been exhausted in the current framework and, if so, what needs to be fixed?
And while I start with Doug, I'd also invite other panel members to jump in after Doug finishes. Thanks.
MR. TRUE: Thanks, Andrea.
I got a number of thoughts on this and I think it actually ties somewhat back to some of the NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
61 answers to the questions we were asking.
So I think people generally felt like there was more that could be done and I think that's, certainly -- I, personally, believe that's certainly true.
But, I think, also the last question, I think it was, that talked about efficiencies, sometimes we do get kind of bogged down and I have a few thoughts about that.
I think part of this is we're on a journey here as the NRC tries to become a more modern risk-informed regulator.
As Mike pointed out, there's a lot of work going on to try and bring people up to the same understanding of what risk informing really means.
This is true on the industry side as well as the NRC.
It's not unique to the NRC. That this industry was founded on sort of deterministic approaches to things but bringing risk into this is a foreign concept to a lot of people that have never been exposed to it.
I think there's more work to be done to educate people to understand what risk is and what it isn't, understand what its limitations are and what its effective uses are, and that's something that we NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
62 all need to undertake.
I do think, secondly, on this subject, that there's been a tendency sometimes to focus way too much on the numbers.
We get all bogged down on decision thresholds and I think that it really should be more about how we understand the plant from the PRA perspective. Numbers are important but there are uncertainties in the numbers. So getting too bogged down in the numbers is not helpful and can often stretch things out unnecessarily.
And third, you know, do it on some of these applications that are more ambitious. For example, risk-informed completion times require a more extensive and expansive PRA because you're asking a more difficult, broad question. That makes it a little bit more costly, and I think that's been a challenge, too.
But we still -- we still continue to see places where we think there's opportunity for more risk-informed thinking. We're in the midst of a discussion about tornado impacts on cast loadings that we think has a low safety significance and merits being dispositioned that way.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
63 We also have, you know, other issues going on in the industry where we have successfully used the low safety significance process to decision-making. So I think there's more opportunity and I think education will go a long way to help us moving forward on that.
MS. KOCK: I just invite any of the other panelists to weigh in on that question.
MR. FRANOVICH: I would say I agree with Doug completely in his assessment. I would add that I think you see a little bit of a lagging effect going on because there's an ongoing shift to move to the more advanced risk management programs.
What I'm speaking of, more specifically, it started with the surveillance frequency control program. We also have 5069 and the risk categorization of SSCs and then the risk-informed completion times.
For the surveillance frequency control program, I think we're right now at almost 100 percent. The plants have been authorized to use that program, which started with the first plan, I think
-- I believe it was Limerick in 2008 as a pilot.
But we are making significant progress in NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
64 approving the 5069 applications that are coming in -
- I think we're well over 20 now -- as well as the completion time program. And as those reviews go on, eventually they get authorized and they'll get implemented. But it's in a staggered way.
And so when they get authorized and we see more broad use of them, I think you're going to feel more of the work that's going on now that doesn't maybe resonate or feel like we're doing enough in the way of risk informing.
But there is a lot more room that can be done in a lot of other domains. One of them that we're actually trying to do a little bit is in the space of treatment of the alternate source term.
Can we use some graded approaches and thinking in that space since we have collected a lot of engineering experience, in particular, from seismic and seismic PRAs?
That's just one small area. But there's a lot of other domains where, I think, we can actually do much more, and then there's the whole area of the new and advanced reactors, which is a whole another topic area.
MS. KOCK: Okay. I'm going to move on NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
65 to the second question. The second question was for Smain but, again, I'll invite other panelists to jump in on this one.
First of all, a question -- a statement.
Great presentation, Smain. So thanks.
MR. YALAOUI: Thank you.
MR. KOCK: And then the question. In your experience, what is the fundamental difference between defense in depth and safety margins in traditional engineering and risk-informed processes?
Is it the same or is it balanced in a different way?
MR. YALAOUI: Thank you very much. I think this is a very good question.
I think we all know that defense in depth and safety margins are part of the traditional deterministic approach and these are fundamental concepts.
But is this -- how this is balanced within that idea? Once again, this is a very tricky question. I would say no, a straightforward answer to that. I think it's very difficult, I'd like to say. As in one of my slides, I say there is no waning factors to balance the different key principles of the RIDM.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
66 But once again, how this -- how this is balanced in RIDM to think that defense in depth is something that decision-makers need to consider.
It's not always easy to say, for example, to which extent or how much safety margins erosion is acceptable. These are not easy questions.
I think we go with different case -- on a case by case basis and it needs to have all the specialists from safety -- deterministic safety PSA, engineering judgment and other specialists to have a good picture for the risk decision-makers.
So I leave it to other panelists if they have any other thoughts to provide.
MS. KOCK: Okay. If not, I'm going to move on to the third question. This question was for Mark but, again, others can jump in.
How did the trends that you described, Mark, compare with overall consideration of defense in depth of the safety margin? Even if some trends go up, is defense in depth and safety margins being impacted?
MR. HAGGARD: Well, so during the training assessment, we didn't consider the defense in depth and safety margin. I think one of the things NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
67 I started -- when I started the presentation, I mean, I made a comparison to risk-informed decision-making primarily because the defense in depth and safety margins are kind of like fundamental.
They're always there. So the changes that we have enacted over the years have been primarily in instituting risk-informed decision-making.
So I don't think the trends -- I don't think that -- if I'm understanding the question correctly, I don't think the training assessment that we looked at -- I don't think it had -- took any consideration in terms of safety margin and defense in depth because those are kind of like baseline.
They're always there.
Obviously, if some of the trends go in the wrong direction, it's going to impact the amount of margin we have, you know. But we didn't fundamentally look at that with the exception of looking at risk. We did a comparison looking at the amount of margin in terms of the risk numbers that we were looking at. And, obviously, if that risk number
-- if that goes up then you would see an impact on the safety margin, if I'm understanding the question NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
68 correctly.
MS. KOCK: I'd just ask if any other panel members have anything else to add.
MR. TRUE: Yeah. I'd just add that I think, Mark, in your presentation, you talked about how the B.5.b and FLEX were considered.
Those are new levels of defense in depth that we didn't have back 10 years ago or 20 years ago, and are a reflection of actually increases in defense in depth, and safety margin is a little bit harder to measure. You know, a well done PRA can often do that but sometimes even has difficulty doing that as well.
But I think we haven't seen significant reductions in defense in depth in applications we have done. So I think it's increased net or where we were 10 or 20 years ago. My take.
MS. KOCK: Okay. Anything else on that question? If not, we're going to move on to the fourth question.
Mike, this one was for you. But I really think it applies -- I think any of the panelists may have reflections on this one. So I'll start with you and then just let everybody else add.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
69 So the fourth question is, it is true that risk-informed decision-making has expanded.
However, in recent years, it seems that either NRC or industry have pulled off in this expansion on areas like physical security and others.
Do you think there's a slowdown due to a blockage that needs to be overcome or is the low-hanging fruit no longer available?
MR. FRANOVICH: So that's a very interesting question. I think the -- I would say there's still active work going on in deploying --
reviewing and deploying, approving the programs that are coming in that I mentioned already to three advanced risk management programs.
And so it may not look like there's a lot of work going on but I would say there's a lot of production work, and when you look at what's probably taking the oxygen out of the room, on a lot of the operating reactors work it's that of the work of the advanced reactors and the licensing modernization project, which is really strong, more PRA technology dependent framework.
There's been some shift in that area.
But on the operating fleet, you see more of the NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
70 production. We have had interactions quite a bit with the owners groups, in particular, in a few projects that are striking some interest.
One of them is, in particular, is there a possibility of looking at how a licensing modernization project framework could be applied to the operating reactors -- the LMP to operating reactors -- and leveraging some of the insights that are coming out of our Level Three PRA project, which is ongoing work.
There is some interest there how that might shape and level review and work in Chapter 15 space. That's still sort of in its infancy as a new concept.
We do have other work going on, for example, the risk-informed process for evaluations -
- RIPE. We do have the first submittal in house.
That's being actively worked on right now.
You know, I think when you just start a program like that you see where it goes. But that, too, is sort of at the beginning phases. So there's still a zeal. I definitely see a zeal there.
And if I listen to my colleagues over in the security arena, they have some -- done some recent NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
71 work in terms of off-site response capability, leveraging FLEX and risk insights. There's still work there.
But I would say it may not garner the visibility that it once had. I mean, our attentions have moved. I'm not judging this one way or another.
I just observe these kind of dynamics and the shift in a lot of work with the advanced new reactor designs so it maybe just appears that things aren't as active in the risk side.
But I think they're -- in my perspective, they're very active.
MS. KOCK: Okay. Other perspectives on that one? And I think this will probably be our last question. So if others have perspectives on that --
MR. THAGGARD: Yeah, I'd just like to add one thing in terms of the insights about the security.
I think the comment about the low-hanging fruits is probably a good analogy. If you -- in order to quantify the risk of security, I do think it's a bit more challenging because trying to quantify the initiating event is, you know, difficult.
And so I think the idea that maybe it's NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
72 more difficult to look at risk from security -- from that perspective, you know, and so that's probably a good point. I would agree -- I would agree with the question -- the question that -- what they're raising.
MS. KOCK: Further perspectives on that last question?
MR. TRUE: Yeah, I'll jump in. I, largely, agree with Mike. There is opportunity.
There is still more -- a lot more activities going on. I think that the implementation of the very low safety significance issue resolution process and the RIPE process to try and help us dispense with issues of low safety significance quickly and allow our resources to be put back on the things that are more significant are really important and still in the early stages of implementation.
And there are a number of utilities that are continuing to actively pursue some of the more ambitious applications like 5069 and the risk-informed completion times, and we'll see those coming over the coming years.
On the physical security side, I agree in principle with Mark that it's very difficult and may NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
73 actually be beyond the state of the art to quantify safety or security risks and -- but that doesn't mean we can't use risk insights and a lot of the things we have learned on the risk-informed completion or the risk-informed activities to inform our security practices, and I think we have started that with looking at offsite response.
We have started it with looking at other dimensions of the security response that, I think, can be made more realistic with a complete understanding of the overall plant and its capabilities. So I think there's some places to get some benefit there.
And then even outside of the reactors, I think there's -- that risk-informed thinking can be beneficial.
Oh, and one last thing. We're starting a new initiative on using risk insights -- not necessarily PRA but risk insights in the aging management programs to focus on the things that are of safety significance and put less focus on those that are less safety significant in the overall aging management program, and working with the NRC on that and expect to see significant benefits there in the NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
74 long-term operations.
MS. KOCK: Okay. I'm going to move into some closing remarks. We have only one minute left.
Just want to take the opportunity to thank everybody who supported today's panel session.
I really appreciate all the panelists' time. I appreciate those of you who are in the audience and participated in the discussion. It was rich. The questions were great. So thank you for that.
Just some highlights that I picked up from the presentations and some common themes. I think we got out of today's session that risk is created if the created concept is multifaceted, and that can bring challenges.
And I think we heard that risk-informed decision-making does allow us to focus on the issues that are most important to safety. I heard today about a lot of tools that we can use to help us make these decisions.
Doug talked about PRA. Smain talked about processes and procedures and Mike talked about integrated decision-making. To me, those are all tools that we can use to help us make risk-informed NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
75 decisions and use the tools that we have in the best way possible.
I heard from your response to the polling questions and the discussion that we have made progress but there's more to do, and how can we move forward to address those issues where there's still challenges -- you know, I think building understanding through discussions and understanding different perspectives. I heard about issuing guidance to help folks understand how to make those decisions in areas that are challenging like defense in depth.
And I think we heard that overall plant safety has improved, and PRA and risk-informed decision-making have been a part of that. But, however, there are still uncertainties that exist and we need to kind of step back and look at the big picture.
So in closing, if you can bring up the contacts slide. Oh, you did. Thank you very much.
There were a few unanswered questions.
So the contact information for our session coordinators is up on the slide. So any unanswered questions will be sent to these coordinators. I NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
76 encourage you to follow up with them to get the answers to any questions that you have that were not answered today.
And with that, I'm going to close this session and tell you to have a wonderful evening and we'll see you tomorrow.
(Whereupon, the above-entitled matter went off the record.)
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com