ML21225A698
| ML21225A698 | |
| Person / Time | |
|---|---|
| Issue date: | 03/08/2021 |
| From: | Office of Nuclear Reactor Regulation |
| To: | |
| References | |
| NRC-1420 | |
| Download: ML21225A698 (54) | |
Text
Official Transcript of Proceedings NUCLEAR REGULATORY COMMISSION
Title:
33rd Regulatory Information Conference Technical Session - M7 Docket Number: (n/a)
Location: teleconference Date: Monday, March 8, 2021 Work Order No.: NRC-1420 Pages 1-62 NEAL R. GROSS AND CO., INC.
Court Reporters and Transcribers 1323 Rhode Island Avenue, N.W.
Washington, D.C. 20005 (202) 234-4433
1 UNITED STATES OF AMERICA NUCLEAR REGULATORY COMMISSION
+++++
33RD REGULATORY INFORMATION CONFERENCE (RIC)
+++++
TECHNICAL SESSION - M7 RISK-INFORMED DESCISIONMAKING ACROSS DISCIPLINES
+++++
MONDAY, MARCH 8, 2021
+++++
The RIC session convened via Video Teleconference, at 1:50 p.m. EST, Mirela Gavrilas, Director, Office of Nuclear Security and Incident Response, presiding.
PRESENT:
MIRELA GAVRILAS, Director, Office of Nuclear Security and Incident Response, NRC CANDACE DE MESSIERES, Technical Assistant, Office of Nuclear Reactor Regulation, NRC JOEL GEBBIE, Senior Vice President and Chief Nuclear Officer, American Electric Power REGINALD MITCHELL, Chief Financial Officer, U.S. Agency for International Development NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
2 PAUL SLOVIC, Professor of Psychology, University of Oregon, Decision Research NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
3 C-O-N-T-E-N-T-S Page Introductory Remarks Mirela Gavrilas ........................................................................ 4 Be riskSMART About All NRC Decisionmaking Candace de Messieres ........................................................... 8 Be riskSMART - Application in Commercial Nuclear Power Industry - Cook Nuclear Power Plant Joel Gebbie........................................................................... 15 Enterprise Risk Management (ERM) - Perspectives from the International Development Sector Reginald Mitchell .................................................................. 22 The Psychology of Risk Paul Slovic ............................................................................ 31 Question/Answer/Discussion ............................................................. 38 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
4 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
5 PROCEEDINGS (1:50 p.m.)
DR. GAVRILAS: So good afternoon. I am Mirela Gavrilas and serve as Director for Nuclear Security and Incident Response at the NRC, but for the purpose of this session I am the very proud leader of the Be riskSMART Team.
We started 18 months ago with five or six people.
Within a couple of weeks grew to what many would consider an unimaginable 13, but to me they are all the brightest, the most enthusiastic, and the most responsive people I have ever worked with.
We are now hovering near 55 when we include our wonderful Be riskSMART ambassadors. So over the past year and a half we have had many opportunities to turn philosophical as we discussed how to organize concepts such as the risk triplet, risk management, the risk heat map, and risk acceptance into a cohesive framework that can be used across disciplines.
So in the spirit of imitation is the sincerest form of flattery we sought inspiration in many places and some of the organizations from which we adopted meaningful ideas are represented on the virtual stage this morning.
So we will start this session with Dr. Candace de Messieres. She will give an overview of the Be riskSMART framework.
Candace is currently the Technical Assistant in our office of Nuclear Reactor Regulation.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
6 She has extensive expertise in the application of risk to reactors and she was instrumental in many first-of-a-kind risk-informed licensing actions.
Prior to coming to the NRC Candace worked at the Department of Energy under a prestigious fellowship from the American Association for the Advancement of Science and Technology Policy.
She received her Ph.D. from the University of Maryland at College Park, my alma mater, and has both a BS in Physics and a BA in Music from Gettysburg.
In addition to that she has bicycled 3000 miles across Europe over the span of just two months a few years ago. She is amazing.
Following Candace we have Mr. Joel Gebbie, which he will share examples about how risk insights are applied in all aspects of nuclear operations ranging from the use of probabilistic risk assessments to hiring decisions to regulatory interactions. I am asking my Be riskSMART colleagues, sound familiar?
Joel serves as Senior Vice President and Chief Nuclear Officer for the American Electric Power Cook Nuclear Power Plant. He has worked in the commercial nuclear industry for 33 years and he has held leadership positions in both engineering and plant management.
Joel also serves as the current chairman of the NEI Nuclear Strategic Issues Advisory Council. Joel has received both a NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
7 Bachelors Degree and a Master's Degree in Mechanical Engineering from Ohio State.
One of our colleagues who was a wrestler in college was delighted to find out that Joel also wrestled for Ohio State when he was a student.
Our third speaker will be Mr. Reginald Mitchell who is the Chief Financial Officer of the U.S. Agency for International Development where he provides stewardship for a 9000 FTE and $25 billion financial portfolio.
We invited Reggie to talk to us about USAID's great efforts in cataloging risk appetites for various segments of the organization. It turns out that in addition to the leadership that he provided to USAID's risk efforts Reggie is also a 26-year NRC alum.
He worked for the NRC in numerous executive positions, including as Controller and Budget Director. Reggie started his career in the nuclear Navy, and he received a Bachelor of Science degree in Business Management from National Louis University and a Master of Public Administration from American University.
As part of his job Reggie travels the world and during one such travel local residents thought Reggie was the American actor Forrest Whitaker. I can see that, especially with a cigar in his mouth as he apparently was at the time.
Our last speaker will be Professor Paul Slovic who teaches and researches psychology at the University of Oregon. Paul NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
8 will share with us some of the highlights from his extensive body of work.
He will talk about things such as risk perceptions, factors leading to acceptance or rejection of risk and decisionmaking, and like Chairman Hanson mentioned during the morning, the importance of trust.
He founded the Institute for Decision Research in 1976, which many of you will appreciate was contemporaneous with the publication of WASH-1400.
Professor Slovic was elected into both the American Academy of Arts and Sciences and the National Academy of Sciences.
Both Candace and I were awed to realize that Paul's work was cited more than 130,000 times with one paper alone receiving over 16,000 citations. Those are amazing figures.
And now I am going to pass the mic to Candace who will get us started. Candace.
DR. DE MESSIERES: Good afternoon. Thank you, Mirela and virtual participants. It is my pleasure and honor to present today on behalf of NRC's Be riskSMART transformation initiative team and community of practice.
Before I begin I would like to reiterate Chairman Hanson's description of NRC's Be riskSMART initiative during his plenary opening remarks, and that is that Be riskSMART provides a holistic, high-level framework that gives staff confidence to consistently NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
9 apply and communicate risk insights for all NRC decisions without compromising our mission.
The key consideration was empowering staff to focus their efforts on those items of greatest importance.
Next slide, please. This slide provides some additional information on the Be riskSMART Team's purpose and approach. The first bullet emphasizes our goal that Be riskSMART is applicable in all NRC programmatic areas, including technical, legal, and corporate.
The framework enables NRC staff to consider risks systematically in a simple and plain English way. I want to emphasize that we are not revisiting any exiting criteria, but increasing consistency, awareness, and usability of existing risk-informed approaches.
I would also like to mention that this framework accommodates decisions that requires simultaneous consideration of risk, including safety and security risks, programmatic risks, reputational risks, legal, or information technology, among many.
Next slide, please. This slide describes some of the obstacles the NRC staff thought about prior to and during the formulation of the Be riskSMART framework.
I will mention a few of these. With regards to guidance, you know, there were concerns that there was either a lack of guidance, limited awareness of the guidance, or potential policy flexibilities not being reflected in the guidance.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
10 From a process and treatment standpoint there were concerns about things like limited consideration of the positive attributes of a decision or risk insights being applied too late in the process.
Lastly, there were several cultural and infrastructure obstacles, such as inconsistent management support or expectations, reluctance to adapt to processes or siloed organizations.
I would mention that on this slide you see a letter, or in some cases the whole "SMART," next to the obstacle. These are the portions of the Be riskSMART framework that actually tried to address these obstacles.
Next slide, please. In the interest of time I am going to be really brief in my description, but I would like to take a little time to go over the steps of the framework, and that's the first step is perhaps the most important.
Be clear about the problem. It could be a simple binary decision that an individual make, or it could be something more complex at an organizational level, such as how to optimize a process or approach to enhance the principles of good regulation.
I want to reiterate again that we are not changing any requirements, but this step does include explicit consideration of those requirements.
The next step is step Spot. Many of you in the nuclear safety arena and security arena are probably well familiar with this step because it looks like our risk triplet, that is what can go wrong, what are NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
11 the consequences, and how likely is it.
I will draw your attention to one of the obstacles that staff had overcome in the last slide, and that is the explicit consideration of things that can also go right, and so we have added what can go right or wrong in our Spot step.
The next step manages what you can, and that's very simple and plain English. You simply manage those challenges and opportunities that you identified in the Spot step.
It could be as simple as additional public meetings, more resources, increased inspections, or a wide range of other management strategies.
In the Act step this is the portion of the process where you as a collective group of individuals come together, consider those things from the Spot and Manage steps and think about what your risk appetite as an organization is, as an individual making the decision.
It really emphasizes the collaborative part of the process and the ability to align on those items that you spot and managed.
Once you act on the decision it doesn't stop there.
You realize the decision and think about what has occurred as a result of the act action. I would reiterate this is perhaps the area where you would do performance monitoring and check in on those things that you have implemented.
And then Teach, it's very important that you teach NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
12 others what you have learned so that you can apply those things in different contexts.
And, lastly, you might notice on the first slide we have an arrow underneath our Be riskSMART and that was very intentional.
I would like to say it's not the Amazon arrow. It was very intentional.
That means that at any time during the process in any of the steps if you learn new information or see something that you would like to address, a different problem perhaps, you can iterate on any of the steps at any time.
Next slide, please. This slide I don't have time to go into the details of all of the things on this slide, but what I wanted to show you was that, you know, we are not at odds with any of the existing, very successful in many cases, processes and approaches that we have in place, such as risk-informed decisionmaking already in the reactor safety arena or medical safety in any of the areas that the NRC has domain over.
But what we have done is mapped the Be riskSMART framework onto those processes to show where it applies and how those processes could be enhanced. There are several examples in the different areas on the screen, and, again, I won't go through those, but you can see legal examples, examples in security, in reactor safety, in corporate and research areas.
Next slide, please. So it's not just enough to develop the framework, we also wanted to ensure that it was working. This is NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
13 a diagnostic tool that the team put together regarding how we track success.
And really you can see on the left-hand column side of the graph individual contributor driving. In an ideal world individual would be empowered to organically apply this process in whatever decision they were making.
Management, of course, on the bottom axis here, would have set the infrastructure to enable staff success in this area.
However, you would not expect management to be driving the decision, it would be organically at the beginning of the process by the individuals executing that decision.
Next slide, please. This next slide talks about the further tracking and our trajectory of perspectives. In the top left you'll see a 2018 survey response regarding is it appropriate, would it be appropriate for the Agency to expand the use of risk insights.
You can see that the majority of staff are favorable, but there are some staff who were neutral or unfavorable. In contrary, a similar question in 2020 posed the risk-informed initiative effectively supports NRC mission, and you can see there is a marked shift toward favorable perception.
On the right hand of the screen you'll see a diagnostic tool where we posed to our staff as part of this initiative to get a baseline of where the staff did feel they were in decisionmaking.
Again, a similar graph as I showed on the previous NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
14 slide. Do you feel that you are driving the decisionmaking or is management driving? We found that a majority of staff are certainly engaged but there is more work to be done.
And, finally, the last slide. I would like to acknowledge that, again, I am honored and grateful to be here representing this amazing team. This is just a fraction of the people that are using the framework, but there is a large team.
We span multiple disciplines and it was very humbling to see the insights and the excitement that other people bring to this process, so I am really -- Thank you for your time, I appreciate it.
Please reach out at any time if you have any questions.
And with that I will turn it back over to Mirela and our next speaker. Thank you.
DR. GAVRILAS: Thank you, Candace. Joel, I think you're next.
MR. GEBBIE: Thank you, Mirela, and thank you for the opportunity to talk about how being risk smart is a way of life in the U.S. commercial nuclear industry.
Next slide, please. So being risk smart is a way of life in our industry and there is a nexus between both probabilistic and operational risk.
All U.S. nuclear plants focus on operational risk as much as they do focus on probabilistic risk and that focus on risk awareness and risk mitigation is vital to safe and event-free NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
15 performance at our plants.
If we can go to the next slide, please. So on this slide on the left you'll see something known as the Cook Plant Culture Playbook. Now in the U.S. all plants or fleets maintain a core set of behaviors or values or beliefs.
Many of those include risk awareness and mitigation as a core belief or value. So if you look at the third belief for Cook Nuclear Plant it is recognize and mitigate risk and we list our behaviors necessary to do that.
This culture playbook is used by all plant employees every day whether they are preparing for work, they are conducting a pre-job brief, they are in the field and encounter uncertainty, this always drives our behaviors and in this case around risk management.
On the right is a tool that we use for our leadership team. It was created to help leaders understand the risk of technical or operational decisions that they may need to make in a given day, and so we've given them this seven-question challenge to help them understand the risk of the decision they are about to make.
If we can go to the next slide, please. Now probabilistic risk is still important. At the first plant-wide meeting every day at the plant, we call it our plan of the day meeting, we always understand the probabilistic risk of all scheduled or emergent work and the mitigating actions we will take for that day.
As you can see we pay particular attention to any NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
16 deviations from baseline risk. So this slide shows that on Thursday of that work week we are conducting work activity that has a higher risk than the baseline risk, so we will pay extra attention to that work.
Next slide, please. Now we also take a very deep look at operational risk. What we are looking at here is a daily plant status for our Unit 2 of Cook Plant, and I'll call your attention to a couple of areas.
So if you look near the top left it says something called "guarded equipment." We limit access to redundant safety equipment or important to operations equipment to eliminate potential operational risk to our plant.
That equipment may be important because it's part of one of our redundant trains of safety equipment. Maybe it's a piece of equipment that causes the plant to be conditionally a single point vulnerable for the work that we are doing today.
If you look down a little further it will show you the online risk for that unit on a given day, and we code our risk in terms of green, white, orange, and red, I'm sorry, green, white, yellow, and red, and so it's important we see the aggregate of our operational risk on a daily basis.
We can move forward to the next slide, please. So, again, this is another slide out of our plan of the day meeting. I apologize for how busy it is, but it contains a lot of information relative to the risk we are going to encounter that day.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
17 So over on the far right side we see something called "activity accelerators and high risk." Oftentimes we may be performing work that has negligible probabilistic risk, but there is high risk due to either operational or radiological impact.
If you zoom in to one of the activities, there on that particular day we are offloading new fuel assemblies into our spent fuel pool. Now you can't quantify that risk in terms of probabilistic risk, but everybody can understand that if we put a new assembly in the wrong position, we could introduce a reactivity management issue into the core when we reload the core during an outage.
If we go a little further down there is something called "significant work." So that's other additional work that is significant for our plant and can pose risk.
Oftentimes our operations shift manager that day will select an activity from those two areas and call that their line of sight to the core for that day. That is the work that has the highest potential to impact nuclear safety in our daily work.
Finally, if we move over to the left side of this slide under the Be Excellent Today symbol, we also look now at human performance in risk-behavior events.
So from a human error reduction perspective we can now measure behaviors that could have led to an event or an issue at our plant. In the past we were focused on events after they occurred, we now know it is valuable to understand the risk of the behaviors that NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
18 we had that didn't result in any events.
If we can go to the next slide, please. So we have talked a lot about the daily risk and how we manage that at our plants.
How about the longer term?
So what you see on this slide is what is known as our risk heat map at Cook Nuclear Plant. Every plant maintains a big picture view of the issues that pose either probabilistic or operational or business risk to the plant. This tool provides an aggregate view of that plant risk.
Now we have both quantitative and qualitative measures that are used to calculate the probability of occurrence and the severity of consequence for a given issue in terms of risk.
Any issue that rises to what is known as enterprise risk level, and that's in the very top right corner where it's very red, often required to be assessed at a higher level than just at the plant, often at the owning corporation level.
So, for example, American Electric Power owns Cook Nuclear Plant. If we identify enterprise risk issues, they are elevated to our corporate risk executive committee.
That is a committee made up of all of our executive vice presidents and our chief executive officer. So it gets a much higher level of risk in terms of the risk to the plant and the corporation and the mitigating actions that we are taking.
The other value for this enterprise risk or this risk heat NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
19 map is it informs how we do work. So how are we scoping future refueling outages, which design modifications do we select to implement in the plant, where are we allocating budget funds, so this is a great tool based on that aggregate risk to tell us how to conduct our work going forward.
So if we go to the next slide we'll see what the result of that is at both Cook Nuclear Plant and in our industry. So the result is the U.S. nuclear industry performance has continuously improved and in 2020 was at the highest level ever.
You can see that on the left of that slide from an operational risk perspective when we look at the industry scram trend and then on the right when we look at our overall probabilistic risk you can see that our risk awareness being risk smart has driven our investments in our plants also lowering our base core damage frequency risk.
We have achieved this improved performance while reducing our all-in costs to our lowest levels ever. All that means is that safety, efficiency, and economic performance are not mutually exclusive, they are complimentary.
I look forward to answering your questions during the open session. Thank you.
DR. GAVRILAS: Thank you, Joel. Reggie, can you please give your remarks?
MR. MITCHELL: Thank you, Mirela. Good afternoon NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
20 and hello, everyone. I am glad to be here to participate in this annual conference which is so important for exchanging information.
I would like to extend kudos to the RIC planners for successfully transitioning this year's event to a virtual format.
Based on my many years with the NRC I know firsthand knowledge on the importance of the Regulatory Information Conference for sharing and exchanging relevant information among its participants and today's topic is no exception given the increased risk faced by governments and private sector entities.
As the Chief Financial Officer and head of the ERM Secretary for USAID, which stands for United States Agency for International Development, I look forward to sharing our Enterprise Risk Management journey over the last several years.
So let's talk about Enterprise Risk Management and its role it plays in managing risk. I appreciate the opportunity to share with you my insight and perspective on USAID ERM journey, more specifically our journey for instituting an ERM governance framework for identifying and managing our agency risk.
I think what is most important for me to share is what worked well and the challenges we faced and how we approached them. By way of background of who we are, USAID is the federal government lead agency for international development and disaster assistance.
We accomplish our mission through partnerships and NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
21 investments that save lives, reduce poverty, strengthen democratic governance, and help people emerge from humanitarian crisis that will lead to self-sufficiency and non-reliance on others.
We are headquartered in Washington D.C. We have missions and programs in more than 80 countries with staff worldwide totaling approximately 9000, two-thirds of which are contractors.
As you can well imagine, because of the geographical dispersion of the agency staff the impediment to strengthen our risk posture was not only required but a necessary element that continues to evolve given the environments we as an agency operates.
With that said, many of you know Enterprise Risk Management was first mandated for the federal government in 2016 when the Office of Management and Budget updated Circular A123 to introduce a new requirement to integrate Enterprise Risk Management with their internal control systems with the steady goal for departments and agency team leaderships to be cognizant and manage its enterprise level risk that could prevent the organization for achieving their objectives or goals.
While USAID already had processes in place to address risk at our business unit level and strategic programming level, this new requirement challenged us to develop and implement a risk-governance construct that strengthened our risk management practices on a more broad and strategic-level scale.
I will now share at a macro level our risk management NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
22 approach and tenets that guided our transformation to a working ERM governance framework for managing organizational risk.
Let me begin by sharing what I believe were our four key factors that led to even stronger risk culture. Next slide, please. I believe there are four major reasons why our transformation was successful.
They are alignment and buy-in from top leadership, changing the mindset of what ERM entails throughout the Agency, making ERM part of an ongoing process, and continuing to encourage risk dialogue.
I will quickly cover these one by one. The first factor, buy-in from the Agency leadership. Perhaps nothing is more important to our success than the tone from the top, which was heard and cascaded down and across the entire organization.
Senior leaders across the Agency convened to discuss existing risk management practices and internal controls to better ascertain how to best incorporate them in broader ERM governance framework for managing enterprise-level risk.
This group was fundamental to and helped shape the thought process and development of the Agency Risk Appetite Statement.
The next factor changes the mindset. We needed to emphasize that ERM and the associated risk profile reporting process was not just a check-the-box exercise but needed to be explicitly NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
23 managed and accounted for in decisionmaking. This change the mindset applied to staff and management alike worldwide.
The third factor, the ongoing process. We continually convey that risk management matters. It is not a one-off event or exercise but rather a proactive ongoing process. We often say that our Agency risk profile is a living document.
This means that the contents are revisited, updated, and evaluated periodically and on an ad hoc basis as needed given we work in dynamic environments and so does risk.
The last factor encourages risk dialogue. The risk dialogue is just as important as risk itself. Having the tone from the top expressed by Agency leadership to staff conveys that conversation about risk is a good thing and their message helped.
This was further encouraged by highlighting and socializing a video by leadership emphasizing the importance of ERM in the work that we do.
People started to talk about risk, and we witnessed a change in perspective on how to assess risks and opportunities. This change helped us transform so we could increase the likelihood of achieving our objectives.
Now I want to share how we engage our staff to support a stronger risk culture. Next slide, please. Equally as important as the factors, if not more, is finding ERM champions within the organization and putting boots on the ground.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
24 For an agency with operations spread across the globe it is often more challenging to socialize the message and even more to put it into practice.
We had to employ various strategies with both --
brought awareness to ERM and made ERM part of the Agency DNA.
A key step that made this possible was the development of our Agency Risk Appetite Statement that we published in 2018.
The Risk Appetite Statement provides USAID staff a broad-based guidance on the amount and type of risk the Agency is willing to accept. It is based on an evaluation of threats and opportunities at a high-level associated with key risk categories in order to achieve the agency missions and objectives.
We thought it was essential to develop formal Agency guidance to help staff identify where it makes sense to be risk forward and where we needed to have a more risk-cautious approach.
To this end, the Agency leadership provided guidance about where the guardrails are. The Agency leadership promoted positive risk behavior by empowering our staff at every level of the Agency and equipping them with the knowledge and tools necessary to make informed risk-based decisions.
Cognitive resources, such as the ERM and USAID programming, allowed us to talk to those in the field about ERM by linking it to processes they were already familiar with.
Through our ERM training, participating in internal and NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
25 external conferences, and our quarterly newsletter, we developed and cultivated ERM champions as practitioners who brought ERM knowledge back to their operating units.
Communications and socialization of ERM as a value-added business practice is an ongoing process that needs to be continually reinforced for it to be sustainable.
Now I would like to share how we structure the Agency ERM governance framework with a reach to more than 80 countries.
Consistent with federal best practices we leveraged the existing internal control governance structure and added on Enterprise Risk Management.
Agency key risk and internal control came together with information sharing to leadership through our ERM governance framework. The Agency requires that our operating unit develop a risk profile to identify key risks that were faced at the operating unit level.
With the operating unit communicating risks up the chain the ERM governance framework amplified the voices of our professionals in the field, which expanded the Agency leadership knowledge and awareness of the organizational risk.
With a three-part ERM governance framework we focused on financial risks through our senior assessment team which existed prior to the OMB mandate, programmatic risks through our Risk Management Council and recommendation on risk to the Agency administrator through the Executive Management Council on Risk and NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
26 internal controls. The latter two councils were established based on the OMB mandate on ERM.
Through the interactions of these three bodies we developed and maintain our Agency risk profile and force a cross collaboration to tackle some of the Agency's greatest challenges.
Some recent examples of challenges include risk of COVID-19 related ventilators being diverted and not reaching their intended recipients during the pandemic, risks of program fraud perpetrated by nefarious actors in remote locations, and the risk of political turmoil in countries where we work.
It is important to understand that USAID ERM governance bodies do not own the risk, but rather work with and guide risk owners on those risks deemed to be at the enterprise level of the Agency.
The role of the governance bodies is to deliberate risk and its impact on the Agency operations and its program's objectives, vet mitigation plans, and validate progress made towards those treatment measures.
One challenge we have found with our governance body is the continuous change of leadership which creates gaps in our community of practice.
To mitigate the impact of changing leadership the ERM Secretariat is tasked with briefing new Agency leadership to maintain a strong and healthy risk culture for the goal of our ERM governance NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
27 framework is not to control or avoid risk in our organization, rather it is to ensure the Agency leadership is aware of the risk environment and to attest that appropriate attention and action are being given to management of said risk to allow the Agency to take advantage of opportunities while mitigating threats to maximize USAID likelihood of achieving its mission.
This concludes my remarks. Thank you for time.
Please feel free to reach out to me if you have any questions. Mirela.
DR. GAVRILAS: Thank you, Reggie. Paul.
DR. SLOVIC: Good. Let me know if you can't see these slides. I am assuming you can see them. Welcome, everyone.
I am going to give you a brief introduction to the psychology of risk.
First let me just maximize the screen here. Okay.
So risk has been studied systematically for more than 60 years and a few of the questions that have been addressed in those many decades are the following, how do people think about risk, what factors determine the perception of risk and the acceptance of risk, what role do emotion and reason play in risk perception and decisionmaking, and what are some of the social, economic, and political implications of these risk perceptions.
DR. GAVRILAS: Paul, I'm sorry to interrupt, but we can't see your slides.
DR. SLOVIC: Oh, okay. Okay. Hmm, maybe I need to first --
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
28 DR. GAVRILAS: If you would like Brock to show them or Sam we can.
DR. SLOVIC: Let me share my screen and then try it again and then we'll go from there. Can you see them now?
DR. GAVRILAS: Yes.
DR. SLOVIC: Okay. You can see them now, good, okay. Let me just go to a full screen. Okay. So some of the key findings from the research are that risk and risk assessment are subjective and valuated.
Danger is real but when we put a measure on a danger called risk then we're getting into subjectivity and judgement in values.
Now every hazard has unique qualities that drive perceptions and acceptance of risk.
These perceptions, whether they are right or wrong, wise or foolish, have a life of their own. They have impacts and they create ripple effects of consequences and stigmatization of technologies like nuclear power.
A key element is trust, which is very critical in risk management. It's very hard to gain trust and it's very easy to lose it.
Most risk perception is determined by fast intuitive feelings, such as dread based on experiential thinking rather than by analytic deliberation.
Finally, understanding risk perception is critical for effective risk communication and risk management.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
29 Just a brief example of some of the factors that have been found to influence the acceptance of risk, so the acceptance of risk is reduced when the hazard is new or unfamiliar, when exposure to the hazard is not voluntary and, therefore, the risk is not under one's control, when the risk evokes feelings of dread that reduces acceptance, when the outcomes can be catastrophic, and when the benefits of an activity are not highly visible or not fairly or equitably distributed among those who bear the risks.
Let me just briefly give you an example of research on this perception. The earliest study that my colleagues and I did was in 1977 and we asked people to just quantify on a zero to 100 scale the risk to society from various activities like motor vehicles, pesticides, drugs, nuclear power plants, and the like.
In one of those studies we actually quantified, had people quantify, their sense of risk. We averaged those values and then ranked the activities according to perceived risk and in this case, we had a group of members of the public and then a group of people who were experts in risk assessment.
You see for the public group nuclear power plants were the highest in perceived risk, but it was 20th on the list for the experts.
You might think, well, this is a fear of radiation, but if you go down the list to medical x-rays, which was on our list, you see that it was 22nd on the list for the public and they weren't very concerned about x-rays, but it was relatively higher for the experts.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
30 So here you see important things, first the public and the experts disagreed, and they disagreed in different ways for two different technologies that involved radiation.
We further found that medical x-rays were perceived as more beneficial, more familiar, less new, and the medical profession was more trusted, and benefit was important.
Let me give you a further example of this. So we also asked people not only how risky is something but to evaluate each hazard on whether the exposure was voluntary or involuntary, if something went wrong could it be allocated a catastrophic consequence, did it evoke feelings of dread, and all these other qualities that you see. These were all evaluated for every hazard.
Now if we go back to nuclear power and x-rays, which we saw were very different in perceived risk, we can see why they were a difference in perceived risk, that nuclear power plants were seen as more or less voluntary exposure, more catastrophic, they evoke more dread, and they were perceived as less well-known and so forth.
So you see we could go beneath the surface of the disagreement between these two technologies to understand some of the factors that led to the difference.
Also, trust has been studied specifically and John C.
Starr, many of you are familiar with, he was the pioneer in leadership in the industry. He said the acceptance of risk is more dependent on public confidence in risk management, quantitative estimates of risk.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
31 If you have trusted the key to communication, well communication is smooth if you've got trust. If you don't have trust no form of phrasing or presentation of your message is likely to be successful.
There is something that is well-known to us through our experience, the asymmetry aspect of trust. It's far easier to destroy trust than to create it and negative or trust-destroying events outweigh the positive events.
We did an experimental study, a research study, about this. We gave people headlines from the local newspaper that reflected on the quality of management of a local nuclear power plant in their vicinity.
So among the trust-increasing headlines was that the nuclear plant conducted regular meetings with the public and public tours were encouraged. The plant managers lived near the plant and the health of the people near the plant was better than average for the region.
Then similarly we constructed trust-decreasing headlines, so there were no public hearings or public tours, and the officials lived far away and the health of the neighboring residents of the plant is worse than the average.
For each of these headlines people rated whether this would increase their trust in the management of the plant and by how much. So here is an example of two of those headlines having to do NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
32 with the health of people living near the plant as either being worse at the top or better in the other headline.
You see just one word was changed. Then you see if you look at the six and seven ratings how the strong impact of trust you see, for example, that if the plant, people near the plant have worse health 24 percent of the ratings were a very powerful decrease in trust.
But if it was better, the health was better than average, it was only a 2 percent increase in trust. This was true of all of our headlines. So you see in this graph here you see all the good news headlines.
You see what is plotted here as the percentage of powerful trust-increasing headlines. You see a very, a low percentage.
But the bad news, the trust-decreasing impact on the bad news headlines is far stronger, a very clear indication of this asymmetry of trust.
Another lesson from risk perception research is that people's political ideologies and world views about what kind of society they want to live in influence perception and acceptance of risk with regard to nuclear energy and all other kinds of things, and this could be a possible topic for discussion.
So sorry for this overly fast interview tour, but happy to answer questions. Thank you very much.
DR. GAVRILAS: Thank you, Paul. So an amazing range of perspectives. There are some things that to me appear a NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
33 common thread in our panel discussions as the NRC embarks on this journey to be a risk-informed regulator.
You know, some of our surveys, our recent surveys say that we are on our way there, but a lot remains to be done. I think we all believe that it will continue to take a great deal of effort from all of us and unwavering support from Agency leadership to get us to where we want to be.
So I am going to start the discussion, we've got quite a few questions, but I want to start the discussion with two sort of more philosophical questions.
The first one is in your experiences what can be the most detrimental thing that we, the NRC, can do to slow down the cultural change to being risk-informed? I am going to start the conversation with Reggie.
MR. MITCHELL: Mirela, can you hear me?
DR. GAVRILAS: Yes.
MR. MITCHELL: Okay. You were going in and out.
Can you repeat your question, please?
DR. GAVRILAS: Okay. So my question is what pitfalls should an organization like ours who is relatively new in seeking cultural change with regard to considering risk, what should we try to avoid?
MR. MITCHELL: Okay.
DR. GAVRILAS: I --
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
34 (Simultaneous speaking.)
MR. MITCHELL: Yes. Can you hear me?
DR. GAVRILAS: Yes.
MR. MITCHELL: Yes. So from my perspective that when you look at Enterprise Risk Management you're looking at a whole of an entity, whether it's an agency or a private sector organization, review of risk, not at the micro level but at the macro level, okay, and making sure that the leadership of the entity is aware of these risks that may impact their ability to achieve their goals.
One of the things that we had to fight when we stood up ERM, we at USAID, just like NRC, we dealt with risks ever since the agency came into existence, okay.
A different type of risk, nuclear power risk, nuclear material risk, in our case life and limb and fiduciary, you know, loss of resources, things of that nature.
However, we had to explain to the managers and staff alike that the leadership back in Washington wasn't trying to manage their programs and their organization from afar, that the ERM construct, the governance framework, was just wanting to ensure that leadership was aware of those risks that meets a certain threshold that require seeing leadership attention.
And so that was what we had to focus on and so I would highly encourage that whatever framework one puts together they do not make it what we call an operational framework.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
35 What I mean by operational framework, where literally they are trying to manage risk that may present itself overseas in a different organization, et cetera, where the local management and staff are dealing with those risks and mitigating those risks.
So I hope that answered your question, Mirela.
DR. GAVRILAS: I think it's a great start to the conversation. I wonder if Paul and Joel have thoughts on this topic?
DR. SLOVIC: Yes, just very briefly. I think the important thing is to recognize how complex the concept of risk is and that we have to expect that we are not all going to agree on issues related to risk, that there are not only technical issues but there are also value issues that influence our perception and acceptance of risk.
So I think the key is early in the program we have to respect others, the views of others with whom we disagree. We have to listen to them and try to understand their perspectives and tolerate and actually embrace diverse views.
MR. GEBBIE: Yes. And I would like to add two things. So first of all the first thing is the assumption that everything is important. Well, leaders have to have the courage and let go of things that don't address the core mission, you know, and so you're asking on behalf of the NRC in this case that's public safety.
You know, I think, you know, we can all agree there are probably some things that don't contribute whatsoever to that, so, you know, leadership has to have the fortitude to say we don't want to do NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
36 that anymore.
I think the second thing that's most important is as you learn to change there is more risk in the status quo than there is in change, but our people, our staffs, will perceive that there is more risk in change and so, therefore, they will be resistant to change.
And so it is very important for our leadership team to open, to be open to understanding the risk of remaining static and not doing anything versus embracing change and trying it out.
DR. GAVRILAS: Thanks. Candace, do you want to add anything?
DR. DE MESSIERES: I completely agree with what the other panelists have said. I just would mention from an NRC Be riskSMART perspective we are definitely actively looking at these exact issues and we have engaged our agency-wide culture initiative team to help us to navigate some of these complex issues.
DR. GAVRILAS: Thank you, Candace. So I have one more question because it is very important and by skimming through the questions that we have received from the audience I think it's something that is on lots of people's mind.
Paul, I am going to start with you. What do you think, you know, you already hinted at some of these, but what do you think are the most effective ways to increase trust?
DR. SLOVIC: The most effective way to increase trust is to respect the people that you are, you know, who you are NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
37 communicating with, to listen to them, to recognize that values are important and we may not all have identical values.
We have to appreciate where the other people are coming from and to see where we share the values, where we don't, but I think it's really respect and to also show that not only you care about them, you care about their health and safety, and are doing everything possible to maintain proper and adequate safety.
DR. GAVRILAS: Great answer. Same question, Reggie. Would you like to contribute to that?
MR. MITCHELL: Yes. I think I totally agree with Paul on the trust aspect and respecting differing views and background. I think also being consistent, being consistent to a set of principles.
I think organizationally that really does come out across the board because without that consistency then everybody they'll have a share set of values that at least they're going to operate by, not necessarily they have to personally share all the values, but as ways to how they operate within their organization as an organism that these are the values that we are here to, and these are the values that value as an organization.
DR. GAVRILAS: Great. Thanks. Candace?
Joel?
DR. DE MESSIERES: I'll just mention one thing to add on to that. I mean I think from an NRC perspective our principles of good regulation have served us well in that regard.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
38 I would note like, for example, the USAID Risk Appetite Statement has also been a good guiding developmental tool for our Be riskSMART initiative team and just echo that it has been helpful to us.
MR. GEBBIE: I'll just add one last thing. You know, we identify trust here at Cook as connection, competence, and clarity, and I think the one most applicable to risk is clarity.
It's really important to identify what risk you are trying to avoid and why. Risk is a very broad almost generic term and so when we just say, you know, embrace risk or don't take risks, if we are not specific with our workforce they are not going to be able to understand what we want them to do in terms of being risk smart.
So the more clarity we can add to that the more trust we will build with our workers, especially as it relates to risk.
DR. GAVRILAS: Great. Thank you. So I want to take a couple of questions from the audience and, you know, I am not going to -- I am going to read them just as they came.
So Candace this is one directed to you, of course the entire panel can help out with the answer. To what extent is the risk-informed approach governed by PSA information versus consideration of qualitative aspects such as defense in depth (audio interference) and maintain safety margins?
DR. DE MESSIERES: Sure. Yes, this is a great question. I would just maybe have a few remarks that, you know, the framework is consistent with our ongoing process.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
39 We are a risk-informed not a risk-based regulator. So we do certainly have, you know, we do leverage our formal risk-informed licensee initiative Reg Guide 1.174 process which has explicit consideration for defense in depth and safety margin.
And, again, we're not revisiting, you know, any of those criteria as part of this initiative. So I would say certainly when available we use probabilistic risk assessment information and in other cases, like in some corporate areas, you know, you may not always have that and that's okay, the framework accommodates that. So I'll just start it at that.
DR. GAVRILAS: Great answer. Paul or Joel or Reggie, anything to add with regard to the consideration of defense in depth in safety margins?
DR. SLOVIC: No.
DR. GAVRILAS: Okay. Let me move --
MR. GEBBIE: Yes, sorry, it took me a second to come off mute. Yes, defense in depth is very, very important, and one thing I will say about that is sometimes we add additional defense in depth when it adds no value to what we are trying to defend against.
So it's an important concept, but it can also be overused if we are not careful.
DR. GAVRILAS: Okay. I'm going to go to the next question which is actually directed to Joel and it comes from one of our SRAs.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
40 So, Joel, you mentioned that nuclear safety has improved over the past few years and showed a graph that supported that assertion. However, if there have been orders of magnitude improvements plant safety wouldn't we see that in the PRA?
For example, the initiating event frequencies would be dropping the T&M values for important components would be improving, et cetera, but to my knowledge as an SRA for the NRC that isn't occurring in either licensee PRA models or the NRC's SPAR models.
Can you provide some details about why you believe there have been significant safety improvements? Thank you.
MR. GEBBIE: Sure, and thanks for the question. I can't really speak to the initiating events within our PRA models or the SPAR models. What I can say though is is plants because of, and we'll go back to probabilistic risk, have chosen to invest in plant improvements that have lowered their base core damage frequency risk and their large early release frequency risk.
And so when I say we have seen an improvement in probabilistic risk that's what I mean, that you look at, you know, the probability of a core damage event now versus several years ago it has decreased.
So when I say, you know, probabilistic risk has improved that's what I mean.
DR. GAVRILAS: Thank you, Joel. Reggie, here is a NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
41 question for you. I think we know the answer for this, we, the Be riskSMART team know it, but if you don't mind sharing about it.
Are there specific documents that explicitly discuss how an agency like the NRC is supposed to implement ERM? Is this more than just being aware of what is going on that can be important for the Agency's mission?
How do we prevent ERM from becoming an eye of the beholder approach, e.g. different people interpret it differently and create their own priorities rather than work in coordination?
And, you know, it's addressed to you, but I think definitely Paul and maybe Candace and Joel as well can add to this.
Reggie?
MR. MITCHELL: Yes. So that's a good question.
You know, we set out to publish the Agency Risk Appetite Statement back in 2018 because we recognized that the importance is for all employees, both managers and staff alike, understand what the risk tolerance of the Agency is.
And, in fact, at one point in our Agency history we had a very low risk tolerance and we were criticized by our partners and our oversight committees for that.
One of the things that we take a look at when it comes to risk is we play in so many different sectors in the economy and in different countries and the risk, depending on what sector you are in, you guys are in the nuclear sector, the risk associated with that is a lot NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
42 different than the risk that developing and providing humanitarian assistance, our ventilators to countries or operating in countries that may not have the best security apparatus there, and so there we are putting our folks' lives at risk, which is a high bar.
So I think when you look at the governance structure we put together and the three governance structure, one is on finance, and I lead that as Chief Financial Officer, but the other two are the programmatic along with the statutory officers, such as -- CIO.
In these meetings these are, in fact, the risk management council are the programmatic career SES'ers, and the same for its service officers, who represent the whole of the Agency, and they report up to the EMCRIC, which is the Executive Management Council on Risk and Internal Control, who are the political appointees in the Agency.
We have about 20 of them, okay, and they are made aware of the risks that we are going to take. So there is not a one size fits all. It depends on the sector, it depends on what the risk is, and then we do communicate.
It is very transparent. We have a website and I have all the minutes. We have a website of the decisions that is made at the council level and even up and including the administrative level and then we promulgate those decisions out to the field.
Now once it hits the field how do we make sure that they are being adhered to? Well, you know, that's where trust comes NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
43 into play, right. You have to trust that the managers and the staff are doing the right thing based on the risk appetite and based on any decisions coming out of Washington Headquarters and the bureaus and the mission leadership, you know.
And, you know, it's like anything else, you don't know until something blows up and if something blows up, you know, and then we're not consistent with the risk appetite and the guidance on risk then those individuals who made those decisions will be held accountable for the decisions that they made that wasn't consistent with the risk appetite and the risk posture of the Agency.
DR. GAVRILAS: Thank you. Anyone else?
DR. SLOVIC: That was a great answer. I don't have anything to add to that.
DR. GAVRILAS: Okay. So here's one for you, Paul.
This is another great question. In nuclear risk assessment we still have a serious challenge in communicating what it actually is and how it actually matters.
In particular, dealing with one in 100 or one in a million.
It's very difficult even for senior experts in nuclear applications. What advise do you have in terms of this issue and in harmonizing it with multiple inputs, financial, reputational, et cetera?
DR. SLOVIC: Right. I think one of the most important lessons we have learned on risk communication is because of the complexity of risk concepts is that when you formulate your NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
44 questions formulate them, you know, based first on, you know, what does your target audience want to know, what are they concerned about.
But then when you begin to communicate first test messages. You know, like how do you get people to understand one in 100 or the difference between one in a million.
Often what we find is that if we are responding kind of in an intuitive way, we think about the one, if the one thing is an adverse event then one in a million scares us just as much as one in 100 because we forget about the denominator.
But you can learn whether your messages are communicating accurately by testing them and I think that's the key to good communication as well as also helping people appreciate risk in a broader sense, the fact that there are values.
We have to decide how we are going to value safety versus financial success and various sorts of things. So, you know, there is no magic to risk communication except respecting the audience and trying messages out and trying to understand whether people understand them and if not why and how to improve your messages.
DR. GAVRILAS: So I'm going to take a few of these questions and sort of make a big question out of it. I'm sorry, I apologize. I think I was muted for the last few seconds.
I was going to take a couple of the audience questions and make them into one big question. Most decisions that we make NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
45 have carried with them not just safety and security hazards, right, they carry with them legal hazards, reputational hazards, a whole gamut of hazards.
How do you deal with these multi-faceted decisions?
DR. SLOVIC: I'll make a first crack at that. Because it gets at a concept that is known as the social amplification of risk and that really means that when something goes wrong the impacts, the consequences, are not just the direct impacts.
For example, if there was a reactor malfunction and that, you know, you have a -- What was the physical damage? Were there health effects? Were there people injured or killed, and so forth?
That's kind of the inner circle of a broader rippling out of consequences and often risk assessment has stopped there, you know. It's been, you know, what is the probability of something happening and what are the consequences and, you know, that defines risk.
Well, it doesn't define risk because there are risks beyond, you know, to the company, to the industry, and, you know, much broader. So when the reactor at Three Mile Island had its problems a long time ago there wasn't anyone killed, but the impact on the industry was immense and that's because people when they hear about the event or read about it they read between the lines and they search for signals, not just, you know, how many people are harmed, but what does this imply about how well the technology is being NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
46 managed, for example, you know.
That affects the entire order of impacts, these reputational impacts that can be vast. It can far outweigh the direct damages of an incident, particularly if everything is contained and this sort of thing.
So that has to be -- Risk assessment has to include the broader impacts as well as the direct damages.
MR. GEBBIE: Yes. And I will say, you know, because you got like organizational issues and then individual issues and we use a pretty simple equation that, you know, risk is probability times consequence.
You have mentioned like five different areas of risk and we have about five different types of probabilities within that risk process that we rank one through five and then we just rank the consequence one through five and do a simple math here and, you know, on the Cook risk heat map I showed you if you have a five for consequence and a five for probability you would get up in that red area that becomes enterprise risk, and so we give people a fairly simple algorithm to calculate that.
When you get to the individual worker level one thing we tell our workers is if you are not sure just set the probability equal to 1.0 and ask yourself if you can accept the consequence and if you can't then you have no business taking the risk.
DR. GAVRILAS: Thanks, Joel. Reggie?
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
47 MR. MITCHELL: Yes. So I liked that answer, Joel, very much. We also have a methodology, it's not as sound scientifically as what Joel just articulated, but we also came up with the seven areas that we looked at as risk categories and we rate, you know, what was tolerance in each of the categories.
For example, we've got a programmatic category wherein our tolerance is high because we work with all kinds of environments, and then we got fiduciary risk and that risk is low because we do give a lot of money out and there is always a lookout because there are bad actors out there, including people that we work with, that will take advantage of any given situation.
So in looking at the risk appetite, which kind of lays out the risk categories, examples of the types of risk, and ranking the risk where we stood when we first set up our ERM and where we want it to be.
So if a risk was out of tolerance, for example let's say financial fiduciary risk was medium and our tolerance was low then, you know, we had to come up with mitigating plans to get to low as best we could.
So it does play into the operations and decisions that is made, you know, on that particular category of risk, okay, and then, you know, that category of risk permeates throughout the whole organization because every program deals with fiduciary risk, you know.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
48 All the countries that we operate deals with fiduciary risk and so, therefore, you know, even though I may be the risk owner functionally, because it's fiduciary, but the responsibility to maintain the level of risk falls on all of the staff and management and the Agency, and likewise for most of the seven risks that we have identified.
DR. GAVRILAS: Thank you, Reggie. Candace, this was selfish of me but I'm leaving you last.
DR. DE MESSIERES: That's fine. I'll be really brief.
I just have a few thoughts. I mean NRC definitely we do have guidance in place to evaluate some of these risks.
We do have, you know, our quarterly performance review, priority of risk, high risk items that are very attune to -- I would say like part of what we are doing with Be riskSMART is really trying to identify these organizational risks at an individual level, at a group level.
I think that's part of the power of it and, you know, we have, as Mirela mentioned, you know, we have a ways to go but we are definitely up to the challenge of trying to get there to provide additional clarity in a systematic way.
So, Mirela, you might have other remarks to add to that.
DR. GAVRILAS: Yes. No, you covered it perfectly.
So I know we are starting to run over but I do want to just get one more question in and I want your reactions as panelists.
It's actually -- I wish we had a poll for this session that asked exactly this question. The question basically is, you know, we NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
49 talked about risk acceptance and risk appetite and how it varies across not only individuals but organizations, so the question is a simple one.
It's what do we think -- Candace, it's directed to you, but I think, you know, I would like anybody who has experience with the Agency to answer it.
The question is where do we think the Agency is when it comes to risk appetite, are we risk averse, are we neutral, are we risk-takers? It would have been a great question for a poll.
DR. DE MESSIERES: Yes, sure. I will start with a very few brief things and then pass it on to our esteemed panel.
So, yes, I think, you know, again reiterating that we are regulators, so I do think that involves some natural culture that is risk averse, and for good reason.
And, again, reiterating that we do maintain our criteria with our mission first and foremost in mind, but, again, with, you know, the effort to focus our resources on those things that are most important so that we can do our best job in our mission, you know, that risk appetite may vary.
Like some things are immovable and others not, so it really, the Be riskSMART framework accommodates risk appetites that vary with both the decision and the organization within the NRC.
So I will leave it at that for me and then pass it on to our other --
DR. GAVRILAS: I am curious to hear Joel's answer NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
50 to this.
MR. GEBBIE: I think I know the NRC is risk averse.
Now, you know, it is acknowledged their first and foremost role is protecting the public, so they are risk averse by nature. We get that in the industry.
And I get back to my comment on clarity before, I think you can be risk averse and then use clarity over where you want to take risks that you consider acceptable and then start taking action for it, and that's the one thing I would urge the Agency to do is is to clarify the risk that is acceptable and then, you know, outline that with your staff, tell them the why, and then proceed forward to make some changes to be able to take those risks.
DR. GAVRILAS: So let me just hear you out, Joel. It is my sincere hope and that of all the people on Be riskSMART that our first line contributors are going to be the ones who are going to drive that initiative.
So let me ask Reggie, Reggie, any thoughts, and, Paul, from your experience?
MR. MITCHELL: Yes. My experience might be a little dated, but that was a good question, because I'll tell you why, as I remember, it's been seven years this month since I left the NRC, but I do remember that was a pain point.
And what I mean "pain point," yes, I want NRC to be risk averse when it comes to nuclear power, nuclear material, okay, NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
51 have a very low margin for error.
But taking that and extrapolating for all of the other areas and sectors that the NRC works in, primarily in the back office side of the house, we took that same mentality, at least back when I was at NRC, with everything that we did, and I think that's where risk appetite will come in very useful where you can set down your categories of appetite and don't take the same level of risk avoidance to every aspect of the organization and tell it more to the sector that you, or the category of risk that you are dealing with.
DR. GAVRILAS: Thanks, Reggie. Paul?
DR. SLOVIC: So 40 years ago around 1980 the NRC sponsored my colleagues and I to do a study and to write a book on the concept of acceptable risk and we were very grateful for that sponsorship and we produced a book.
You can see it here. It's called "Acceptable Risk" and it addresses this question in great, in detail. I think the essence, the most important thing that we concluded is that there is no one magic number that is acceptable across the board.
As we talked about there is a diversity here. We argued that acceptable risk is the risk that is associated with an acceptable decision, so the decision drives the risk.
If you take into account, you know, what's the context, what are the options, you know, what are the possible benefits and risks, and you make a decision, a careful decision, and the risk that is NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
52 associated with that is the acceptable risk without context.
DR. GAVRILAS: You know, we've got to end it on that note, because, Paul, that's the whole like thinking behind Be riskSMART, that the decision drives the risk and it's multi-faceted and complex, so you summed it up beautifully.
I cannot thank you guys enough. It has been -- Time flew. It's been a wonderful discussion and I thank both the panel and the audience and, again, our apologies to the audience for the delay.
Thank you everybody so much. Enjoy the rest of the RIC. Have a wonderful evening. Bye, everyone.
DR. DE MESSIERES: Thank you.
MR. GEBBIE: Bye.
DR. SLOVIC: Thank you.
MR. MITCHELL: Thank you.
(Whereupon, the above-entitled matter went off the record at 3:08 p.m.)
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
53 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433