ML21071A238

NUREG/KM-0016, Be Risksmart: Guidance for Integrating Risk Insights Into NRC Decisions

ML21071A238
Person / Time
Issue date:	03/31/2021
From:	NRC/EDO
To:
Malone, Tina
References
NUREG/KM-0016
Download: ML21071A238 (100)

Text

NUREG/KM-0016 Guidance for Integrating Risk Insights into NRC Decisions Office of the Executive Director for Operations

AVAILABILITY OF REFERENCE MATERIALS IN NRC PUBLICATIONS NRC Reference Material Non-NRC Reference Material As of November 1999, you may electronically access Documents available from public and special technical NUREG-series publications and other NRC records at the libraries include all open literature items, such as books, NRCs Library at www.nrc.gov/reading-rm.html. Publicly journal articles, transactions, Federal Register notices, released records include, to name a few, NUREG-series Federal and State legislation, and congressional reports.

publications; Federal Register notices; applicant, licensee, Such documents as theses, dissertations, foreign reports and vendor documents and correspondence; NRC and translations, and non-NRC conference proceedings correspondence and internal memoranda; bulletins and may be purchased from their sponsoring organization.

information notices; inspection and investigative reports; licensee event reports; and Commission papers and their Copies of industry codes and standards used in a attachments. substantive manner in the NRC regulatory process are maintained at NRC publications in the NUREG series, NRC regulations, The NRC Technical Library and Title 10, Energy, in the Code of Federal Regulations Two White Flint North may also be purchased from one of these two sources: 11545 Rockville Pike Rockville, MD 20852-2738

1. The Superintendent of Documents U.S. Government Publishing Office These standards are available in the library for reference Washington, DC 20402-0001 use by the public. Codes and standards are usually Internet: www.bookstore.gpo.gov copyrighted and may be purchased from the originating Telephone: (202) 512-1800 organization or, if they are American National Standards, Fax: (202) 512-2104 from American National Standards Institute

2. The National Technical Information Service 11 West 42nd Street 5301 Shawnee Road New York, NY 10036-8002 Alexandria, VA 22312-0002 Internet: www.ansi.org Internet: www.ntis.gov (212) 642-4900 1-800-553-6847 or, locally, (703) 605-6000 Legally binding regulatory requirements are stated only in A single copy of each NRC draft report for comment is laws; NRC regulations; licenses, including technical available free, to the extent of supply, upon written specifications; or orders, not in NUREG-series publications.

The views expressed in contractor prepared publications in request as follows:

this series are not necessarily those of the NRC.

Address: U.S. Nuclear Regulatory Commission The NUREG series comprises (1) technical and Office of Administration administrative reports and books prepared by the staff (NUREG-XXXX) or agency contractors (NUREG/CR-XXXX),

Division of Resource Management & Analysis (2) proceedings of conferences (NUREG/CP-XXXX),

Washington, DC 20555-0001 (3) reports resulting from international agreements E-mail: distribution.resource@nrc.gov (NUREG/IA-XXXX),(4) brochures (NUREG/BR-XXXX), and Facsimile: (301) 415-2289 (5) compilations of legal decisions and orders of the Commission and the Atomic and Safety Licensing Boards and of Directors decisions under Section 2.206 of the NRCs regulations (NUREG-0750).

Some publications in the NUREG series that are posted at the NRCs Web site address www.nrc.gov/reading-rm/ DISCLAIMER: This report was prepared as an account doc-collections/nuregs are updated periodically and may of work sponsored by an agency of the U.S. Government.

differ from the last printed version. Although references to Neither the U.S. Government nor any agency thereof, nor any employee, makes any warranty, expressed or implied, material found on a Web site bear the date the material or assumes any legal liability or responsibility for any third was accessed, the material available on the date cited partys use, or the results of such use, of any information, may subsequently be removed from the site. apparatus, product, or process disclosed in this publication, or represents that its use by such third party would not infringe privately owned rights.

NUREG/KM-0016 Guidance for Integrating Risk Insights into NRC Decisions Manuscript Completed: March 2021 Date Published: March 2021 Prepared by:

Mirela Gavrilas Eric Duncan Jason Paige Brian K. Harris Candace De Messieres Elizabeth Bowlin Michael Gartman Reed Anzalone Rebecca Richardson Office of the Executive Director for Operations

ABSTRACT To become a modern, risk-informed regulator, the NRC focused on four transformational areas:

(1) managing the workforce, (2) applying risk in decision-making, (3) generating innovative ideas to improve the way that NRC works, and (4) adopting new technologies and approaches to data analytics. The Be riskSMART framework supports the second transformation area by providing a systematic approach to making risk-informed decisions across disciplines. Be riskSMART combines traditional concepts, such as the risk triplet, risk management, the risk heat map and risk appetite, into a plain language framework that gives the staff confidence to apply and communicate risk-insights for all kinds of NRC decisions whether they are in the technical, corporate, or legal arena. This NUREG/CR provides detailed guidance on using the Be riskSMART framework and contains example case studies from across a series of disciplines.

iii

Be riskSMART: Guidance for Integrating Risk Insights into NRC Decisions TABLE OF CONTENTS ABSTRACT ................................................................................................................................. iii ABBREVIATIONS AND ACRONYMS ....................................................................................... vii 1 BACKGROUND ....................................................................................................................... 1 2 FRAMEWORK ......................................................................................................................... 3 2.1 Introduction ................................................................................................................... 3 2.2 Step 1: Beclear about the problem .......................................................................... 4 2.3 Crosswalks to Discipline-Specific Guidance................................................................. 5 2.4 Step 2a: Spotwhat can go wrong/right? ................................................................... 5 2.5 Step 2b: Spotwhat are the consequences? ............................................................. 6 2.6 Step 2c: Spothow likely is it? ................................................................................... 8 2.7 Heat Map ...................................................................................................................... 8 2.8 Step 3: Managewhat you can .................................................................................. 9 2.9 Step 4: Acton a decision ........................................................................................ 10 2.10 Risk Appetite .............................................................................................................. 11 2.11 Facilitating Discussions About Risk ............................................................................ 13 2.12 Step 5: Realizethe result ........................................................................................ 13 2.13 Step 6: Teachothers what you learned .................................................................. 14 2.14 The Arrow ................................................................................................................... 15 3 CASE STUDIES ..................................................................................................................... 17 4 ADDRESSING CHALLENGES TO RIDM ............................................................................. 19 5 PERFORMANCE MANAGEMENT OF RISK-INFORMED DECISIONMAKING ................... 21 6 RESOURCES ........................................................................................................................ 25 7 STAFF ACKNOWLEDGMENTS............................................................................................ 27 v

Be riskSMART: Guidance for Integrating Risk Insights into NRC Decisions APPENDIX A APPLICABILITY OF THE BE riskSMART FRAMEWORK TO REACTOR SAFETY DECISIONS ................................................................. A-1 APPENDIX B APPLICABILITY OF THE BE riskSMART FRAMEWORK TO MATERIAL SAFETY DECISIONS ................................................................. B-1 APPENDIX C APPLICABILITY OF THE BE riskSMART FRAMEWORK TO SECURITY, PREPAREDNESS, AND RESPONSE DECISIONS .................. C-1 APPENDIX D APPLICABILITY OF THE BE riskSMART FRAMEWORK TO RESEARCH DECISIONS............................................................................... D-1 APPENDIX E APPLICABILITY OF THE BE riskSMART FRAMEWORK TO OFFICE OF THE GENERAL COUNSEL LEGAL ADVICE ......................................... E-1 APPENDIX F APPLICABILITY OF THE BE riskSMART FRAMEWORK TO CORPORATE SUPPORT DECISIONS ..........................................................F-1 APPENDIX G BE riskSMART CASE STUDIES .................................................................... G-1 APPENDIX H FACILITATED DIAGNOSTIC TOOLMEASURING CURRENT STATE OF USING RISK INSIGHTS........................................................................... H-1 APPENDIX I 2020 DIAGNOSTIC SURVEY RESULTS ........................................................ I-1 vi

Be riskSMART: Guidance for Integrating Risk Insights into NRC Decisions ABBREVIATIONS AND ACRONYMS ADAMS Agencywide Documents Access and Management System ANO Arkansas Nuclear One ATF accident-tolerant fuel CRDM control rod drive mechanism DBT design-basis threat DOE U.S. Department of Energy DRP Division of Reactor Projects ECCS emergency core cooling system EDG emergency diesel generator EPU extended power uprate ERM enterprise risk management FBI Federal Bureau of Investigation FOF force-on-force FR Federal Register FY fiscal year GAO Government Accountability Office ISFSI independent spent fuel storage installation NRC U.S. Nuclear Regulatory Commission NSIR Nuclear Security and Incident Response OCIO Office of the Chief Information Officer OGC Office of the General Counsel OIG Office of the Inspector General OMB Office of Management and Budget PHE public health emergency PM project manager PRA probabilistic risk assessment PWR pressurized-water reactor RAPT Reasonable Assurance of Protection Time RG Regulatory Guide RFO refueling outage RES Office of Nuclear Regulatory Research RIDM risk-informed decisionmaking SBT security bounding time SCCS Safety Culture and Climate Survey SPAR standardized plant analysis risk SSC structures, systems, and components TCD thermal conductivity degradation TTC Technical Training Center vii

Be riskSMART: Guidance for Integrating Risk Insights into NRC Decisions Be riskSMART: Guidance for Integrating Risk Insights into NRC Decisions 1 BACKGROUND The U.S. Nuclear Regulatory Commission (NRC) has a long history of integrating risk insights into its decisionmaking processes. Among the most foundational policy-setting documents in this area are the Commission policy statements on safety goals 1 and the use of probabilistic risk assessment (PRA). 2 These policy statements establish goals that broadly define an acceptable level of radiological risk and formalize the NRCs commitment to increasing the use of risk assessment technology to the extent supported by the state-of-the-art methods and data and in a manner that complements the NRCs deterministic approach and supports its traditional defense-in-depth philosophy. The Commission later defined relevant terminology and reaffirmed its expectations in a white paper on the NRCs risk-informed, performance-based approach to regulatory decisionmaking. 3 Specifically, the Commission affirmed the NRC as a risk-informed regulator that embraces a philosophy whereby risk insights are considered together with other factors to establish requirements that better focus licensee and regulatory attention on design and operational issues commensurate with their importance to public health and safety. For example, in the reactor safety area, the NRC employs integrated risk-informed decisionmaking (RIDM) for several regulatory activities where quantitative risk information is considered alongside adequate defense in depth, safety margins, regulatory compliance, and performance measurement strategies. 4 This philosophy has been adopted in other areas, such as materials and security applications.

The NRC staff developed and executed several plans to implement Commission policy related to RIDM. These include the PRA Implementation Plan, the Risk-Informed Regulation Implementation Plan, and the Risk-Informed and Performance-Based Plan in 1994, 2000, and 2006, respectively. 5 More recent agencywide efforts include the development of NUREG-1860, Feasibility Study for a Risk-Informed and Performance-Based Regulatory Structure for Future Plant Licensing, issued December 2007; 6 NUREG-2150, A Proposed Risk Management Regulatory Framework, issued April 2012; 7 and recommendation 1 of the Fukushima 1 Volume 51 of the Federal Register (FR), page 30028 (51 FR 30028), Safety Goals for the Operation of Nuclear Power Plants; Policy Statement, dated August 21, 1986 (republished) (Agencywide Documents Access and Management System (ADAMS) Accession No. ML051580401).

2 60 FR 42622, Use of Probabilistic Risk Assessment Methods in Nuclear Regulatory Activities; Final Policy Statement, dated August 16, 1995.

3 SRM-SECY-98-144, Staff RequirementsSECY-98-144White Paper on Risk-Informed and Performance-Based Regulation, dated March 1, 1999 (ADAMS Accession No. ML003753601).

4 Regulatory Guide 1.174, An Approach for Using Probabilistic Risk Assessment in Risk-Informed Decisions on Plant-Specific Changes to the Licensing Basis, Revision 3, issued January 2018 (ADAMS Accession No. ML17317A256).

5 Information on these plans and risk-informed activities is available at https://www.nrc.gov/about-nrc/regulatory/risk-informed/rpp.html.

6 NUREG-1860, Feasibility Study for a Risk-Informed and Performance-Based Regulatory Structure for Future Plant Licensing, Volumes 1 and 2, issued December 2007 (ADAMS Accession Nos. ML080440170 and ML080440215).

7 NUREG-2150, A Proposed Risk Management Regulatory Framework, issued April 2012 (ADAMS Accession No. ML12109A277).

1

Be riskSMART: Guidance for Integrating Risk Insights into NRC Decisions Near-Term Task Force. 8 Under these plans and initiatives, the NRC developed key RIDM regulatory guidance documents and processes that continue to be successfully applied to advance the use of RIDM today.

Additionally, in 2016, the Office of Management and Budget (OMB) issued a requirement 9 for Federal agencies to implement enterprise risk management (ERM), a forward-looking approach that addresses the full spectrum of an organizations risks in a portfolio view. The NRC established an ERM capability coordinated with its strategic planning, performance management, and internal control processes.

The NRC can further enhance its effectiveness and efficiency by integrating risk insights into all programs and at all levels across the agency. However, several cultural and structural challenges have hindered the development and widescale adoption of an agencywide RIDM framework beyond senior leadership. 10,11,12 In June 2019, the NRC hosted the NRC Futures Jam to identify challenges and seek out new and modernized approaches to the way that the agency operates, which included the advancement of agencywide RIDM. The staff had the ability to make and respond to comments and to participate in facilitated discussions with agency leaders. Based on themes that emerged from the NRC Futures Jam, NRC leadership established a framework for transformation that encompasses a broad set of activities intended to advance the agency toward the vision of becoming a modern, risk-informed regulator. The NRC identified four key focus areas for transformation: (1) managing the workforce, (2) applying risk in decisionmaking (also referred to as Be riskSMART), (3) generating innovative ideas to improve the way that the NRC works, and (4) adopting new technologies and approaches to data analytics. It is to be noted that the scope of transformation, as presently conceived, primarily concerns how the agency conducts its work and does not envision substantive changes to existing regulatory policies.

During 2020, the NRC developed the Be riskSMART transformation initiative and associated framework to address the cultural and structural challenges to RIDM.

8 SECY-13-0132, U.S. Nuclear Regulatory Commission Staff Recommendation for the Disposition of Recommendation 1 of the Near-Term Task Force Report, dated December 6, 2013 (ADAMS Accession No. ML13277A413).

9 OMB Circular A-123, Managements Responsibility for Enterprise Risk Management and Internal Control, M-16-17, dated July 15, 2016.

10 Memorandum from Daniel H. Dorman to Margaret M. Doane, Implementing Commission Direction on Applying Risk-Informed Principles in Regulatory Decision Making, dated November 18, 2019 (ADAMS Accession No. ML19319C832).

11 SECY-17-0112, Plans for Increasing Staff Capabilities to Use Risk Information in Decision-Making Activities, dated November 13, 2017 (ADAMS Accession No. ML17270A192).

12 SECY-18-0060, Achieving Modern Risk-Informed Regulation, dated May 23, 2018 (ADAMS Accession No. ML18110A186).

2

Be riskSMART: Guidance for Integrating Risk Insights into NRC Decisions 2 FRAMEWORK 2.1 Introduction RIDM is an integrated approach to the decision process that considers risk (i.e., the possibility that events will occur (a loss or an opportunity) and affect the achievement of objectives) in addition to requirements and deterministic factors. The Be riskSMART RIDM framework is a collaborative tool to bring people together to authoritatively speak on the challenges and opportunities that exist for each type of risk. This framework can empower people to accept or reject risk, depending solely on the attributes and risk appetite. The framework is designed to give the staff confidence in applying risk insights to achieve the following:

• Ensure that all NRC decisions appropriately focus on the most significant endeavors to enhance safety and operational effectiveness.

• Remove the stigma from risk information and replace it with an understanding of the value added by considering risk.

• Express that when the NRC considers risk, it also considers benefits.

• Support interdisciplinary decisions.

• Further technological advancement in the NRCs mission delivery.

To accomplish the above design requirements, the team made the framework scalable from simple to complex issues. To accommodate staff members who are unfamiliar with RIDM and risk information, the framework uses plain language and provides a step-by-step structure to consider risk systematically, especially qualitative information. To accommodate staff members who already use a risk-informed approach, the framework does not invent new concepts but rather serves as an umbrella to increase consistency, awareness, and usability.

While the framework applies to technical, legal, and corporate decisions, it does not replace any existing RIDM approaches, such as PRA and ERM. It does not revise any of the criteria already in place for making risk-informed decisions, such as reactor safety decisions involving the significance determination process. Instead, it incorporates existing RIDM approaches.

Figure 1 illustrates the six steps of the Be riskSMART framework:

(1) Beclear about the problem (2) Spotwhat can go right or wrong? what are the consequences? and how likely is it?

(3) Managewhat you can (4) Acton a decision (5) Realizethe result (6) Teachothers what you learned The arrow below the SMART portion of the logo represents continual learning and indicates that the framework is iterative between steps and as an overall process when necessary. Together, the steps comprise the Be riskSMART acronym for the actions to be taken in using the framework.

3

Be riskSMART: Guidance for Integrating Risk Insights into NRC Decisions Figure 1 The Be riskSMART Framework The following sections provide guidance on how to apply the Be riskSMART framework. Each step is illustrated using the example problem statement: Should I commute to work using a bike to get more exercise?

2.2 Step 1: Beclear about the problem You cannot reach your destination without knowing where you are going. That is why it is essential first to have a clear picture of the problem or what you want to solve when applying the Be riskSMART framework.

With that in mind, clearly identify the issue or question that you are trying to address. Or more simply stated, Be...clear about the problem.

In applying the Be riskSMART framework, binary decisions (weighing two options, such as yes or no) are usually the most straightforward, especially if the issue is limited to one type of risk. 13 You can also use the 13 Types of risk include technical/programmatic, security, legal, financial, reputational, information technology, and human capital.

4

Be riskSMART: Guidance for Integrating Risk Insights into NRC Decisions framework to select from among several options. A more complex use of the framework would be an open-ended question, such as how to solve an issue or how to develop a new process.

The more complex the issue or types of risk involved; the more analysis is needed for each step of the framework.

Find out if the issue has already been solved. If not, then begin the framework process by asking, What is required? Part of being clear about the problem is identifying compliance requirements that the solution must meet. For help finding requirements and guidance related to an issue, consult the Be riskSMART crosswalk chart for the discipline pertaining to the issues topic (Appendices A-F).

Anyone using the Be riskSMART framework should remember that risk exists in all we do and that we should always strive to understand the risk of the problem and the options being considered for resolution.

2.3 Crosswalks to Discipline-Specific Guidance Appendices A-F provide guidance for specific topical areas, as summarized in Table 1. Within these appendices, crosswalk charts link each step of the Be riskSMART framework with existing NRC guidance. The crosswalk charts are tools to (1) demonstrate that existing agency guidance often incorporates RIDM techniques, (2) provide information resources to complete the framework steps, and (3) illustrate that the framework steps are scalable to support a broad range of issues, whether simple decisions or complex issues requiring research and indepth analysis.

In some cases, existing risk-related guidance documents provide complementary application-specific approaches for holistically implementing all of the Be riskSMART framework steps. In other cases, application-specific guidance may be limited to select steps, and the Be riskSMART framework guidance can be used to complete the RIDM process.

Table 1 Crosswalk Charts to Discipline-Specific Guidance Technical Appendix A Reactor Safety Appendix B Material Safety Appendix D Security, Preparedness, and Response Appendix C Regulatory Research Legal Appendix E Office of the General Counsel Legal Advice Corporate Appendix F Corporate Support 2.4 Step 2a: Spotwhat can go wrong/right?

After clearly defining your issue or question, the next step is to spot the risk information related to it. Spot what can go wrong or right, what are the consequences, and how likely is it. These three questions are referred to as the risk triplet.

What are the challenges (i.e., what can go wrong?) and opportunities (i.e., what can go right?)

associated with the issue or question you developed? Prepare a comprehensive list of challenges and opportunities. You will use the identified challenges and opportunities in Step 4 5

Be riskSMART: Guidance for Integrating Risk Insights into NRC Decisions of the framework, Acton a decision, weighing their importance to the decision based on the consequences and likelihood of what may occur. Your list of what can go right? and what can go wrong? can take several different forms.

For binary decisions, the consideration of what can go right and what can go wrong will lead to a list of pros (what can go right) and cons (what can go wrong). The focus should not be solely on what can go wrong because without a full appreciation of what can go right, new ideas may never be adopted.

For more complex decisions involving an open-ended question, such as solving an issue or developing a new process, the list for what can go right may appear as process objectives and the list for what can go wrong may appear as process pitfalls. In the context of all decisions, risk information supplements minimum acceptable requirements for compliance and quality.

Some issues will likely require an indepth analysis that uses risk insights to complement deterministic methods and prescriptive requirements. For example, quantitative risk information is considered alongside adequate margin, defense in depth, regulatory compliance, and performance measurement strategies. This is one reason that the framework is intended to be scalable and incorporate agency guidance on discipline-specific RIDM approaches within the Be riskSMART steps. For all complex decisions, in addition to analyzing known risk or risks, use research and discovery methods to attempt to identify unknown risks of all types.

The icon for this step in the Be riskSMART framework is a magnifying glass (see Figure 1). The magnifying glass serves as a reminder to carefully study the problem being addressed and to use the risk triplet: What can go right/wrong?, What are the consequences?, and How likely is it?

2.5 Step 2b: Spotwhat are the consequences?

What are the consequences associated with the challenges and opportunities identified in the previous step?

Consequences are the possible outcomes of opportunities (what can go right) or challenges (what can go wrong). In identifying and considering consequences, it is important to clearly separate the consequences of opportunities and challenges from the opportunities and challenges themselves.

For example, in the reactor area, the decision of whether to adopt a new, streamlined licensing review process 6

Be riskSMART: Guidance for Integrating Risk Insights into NRC Decisions that would eliminate time-consuming and onerous calculation reviews could result in the failure to identify errors in the calculations (i.e., what can go wrong). The consequence of an unidentified calculation error could be additional dose to workers resulting from safety limits being exceeded. This is an example of why existing NRC guidance is incorporated within the steps of the framework.

The severity of consequences for each opportunity and challenge must be carefully considered in Step 4, Acton a decision, in the framework. Consequences may impact people, environment, quality, time, cost (e.g., financial risk to current resources and reputational risk that could impact future congressional budget requests), and the ability to meet compliance requirements. Consequences can include both quantitative and predictable impacts, such as a calculated increase or decrease in staff hours, or qualitative and unpredictable impacts, such as an organizations morale in response to a decision.

It is also important to focus on the opportunities and challenges that are within your control. For example, streamlining a process may provide an opportunity for you or your organization to focus attention on other important issues. However, speculating on challenges or opportunities from unknown, external stakeholder actions because of the process change may not be appropriate.

When assessing consequences, do not build in any Managewhat you can actions yet.

Instead, focus solely on the raw consequences anticipated to potentially occur and their impact.

In Step 3 of the framework, you will develop Managewhat you can actions and reexamine differences in the consequences that most likely can be managed. These will better inform the Acton a decision step of the framework.

The concept of consequence adjustment can be used in Managewhat you can and Acton a decision. The consequence adjustment reflects information that may not be easily quantifiable or may not be within a Be riskSMART users control but that can amplify or diminish the consequences. Examples where a consequence adjustment, or weighting factor, could be applied include repeated poor performance by a licensee in addressing safety culture issues, the history of a licensees completion of regulatory commitments, and external stakeholder interest.

Throughout all of the steps in the Be riskSMART framework, it is important to communicate your responses to the framework questions and the basis for each response (e.g., a requirement or standard, agency guidance, proven formula). As you communicate your responses, consider the logical relationship between risk and documentation: More risk will require more documentation.

7

Be riskSMART: Guidance for Integrating Risk Insights into NRC Decisions 2.6 Step 2c: Spothow likely is it?

What is the likelihood of a consequence occurring? For some issues, this can be calculated quantitatively. However, many decisions lead to consequences whose likelihood must be assessed using qualitative considerations.

Consider all available data.

Examples of data that can be used to assign a relative likelihood of a specific consequence include survey results; hard data, including trending data (e.g., has it occurred within an industry or occupation?, has it occurred within other Federal agencies?, has it occurred in our organization? If so, how many times?); and anecdotal information.

When assessing the likelihood (i.e., low, medium, or high) of qualitative risk for a decision that will require alignment among stakeholders, it is essential that everyone has a common understanding of relative values of terms, such as low, medium, and high.

Similar to evaluating consequences, when assessing likelihood, do not build in any Managewhat you can actions at this point. After developing Managewhat you can actions, you will be able to reassess the likelihood of your identified consequences and better inform Step 4, Act...on a decision, in the process.

2.7 Heat Map Proceeding through the framework, you can visually document each risk as a point on a heat map. A heat map is a graph of risks plotted by consequence and likelihood.

On the heat map pictured to the right, the x represents the bike examples risk of falling and its high consequence and medium likelihood.

Note the difference in how challenges and opportunities in a heat map are read. The higher a consequence for a challenge, the greater the potential loss, whereas the higher the consequence for an opportunity, the greater the potential gain.

8

Be riskSMART: Guidance for Integrating Risk Insights into NRC Decisions 2.8 Step 3: Managewhat you can The objective of Step 3, Managewhat you can, in the framework is to reduce the consequences and likelihood of what can go wrong and improve the consequences and likelihood of what can go right by applying risk management techniques (mitigating, preventing, preparing for, and transferring challenges and enhancing opportunities). Risk management is an activity to control challenges to achieving an organizations objectives. Consider the agency as a whole, using techniques from ERM.

It is important to keep in mind that an action to mitigate a bad outcome may reduce the overall impact of a good outcome, especially in complex decisions. As a result, actions to manage or mitigate a bad outcome must be carefully evaluated, and more significant Managewhat you can actions may need to be set aside to provide a better final result.

At an advanced level, this step may include strategies to mitigate unknown as well as known risks.

Taken collectively for the various opportunities and challenges that are enhanced and reduced, Managewhat you can actions can serve as a powerful tool for reaching a successful outcome.

In public affairs, Managewhat you can actions commonly include elements of communications to external stakeholders, including nongovernmental organizations, other Federal and State partners, and individual members of the public. Careful preplanning and thoughtful messaging can go a long way to prevent or minimize the challenges identified in what can go wrong? when tackling an issue or problem.

As a result of managing consequences and their likelihoods, the point on the heat map can move.

In the bike example heat map pictured to the right, we reduced the consequences of the accident shown in the previous heat map by using protective gear, and then we minimized the likelihood by traveling on roads with dedicated bike lanes this moves the point on the top right (representing a high consequence with a medium likelihood) down to the point on the left (representing a medium consequence with a low likelihood).

9

Be riskSMART: Guidance for Integrating Risk Insights into NRC Decisions The icon for this step in the Be riskSMART framework is an umbrella (see Figure 1). The umbrella serves as a reminder that the actions in this step can help protect against adverse consequences.

2.9 Step 4: Acton a decision After receiving inputs, evaluating information, and documenting what you spotted and managed, you can now make a decision. Fully informed, strategic decisions require fair consideration of all views. It is also important to develop contingency plans and be prepared to make course corrections as needed.

This step involves the actual decision on what action(s) to take after the previous steps in the process have been fully and thoroughly developed and considered. One critical aspect of this step is identifying the decisionmaker. For example, in the reactor and materials areas, technical issues with potential generic implications will likely involve stakeholders in both the regional office where the issue occurred and at the NRC Headquarters office with expertise in the technical area. When this occurs, identify early in the decision process who has final decisionmaking authority. Otherwise, the absence of a clear consensus could significantly hamper the ability to make a decision.

Before the decision itself, the NRCs advisory organizations help to ensure the development of a strong and accurate basis for a decision. This includes legal advice from the Office of the General Counsel and technical advice from the Advisory Committee on Reactor Safeguards and the Advisory Committee on the Medical Uses of Isotopes.

At the NRC, issues are often decided by steering committees with members across the agency, which supports a holistic view of a decision. Decision bodies, processes, and documentation are built into the regulatory process. 14 For example, in the area of licensee oversight, the annual Agency Action Review Meeting ensures that coordinated courses of action are developed and implemented for licensees with performance issues. In the corporate support area, the Human Capital Council, Information Technology/Information Management Portfolio Executive Council, 14 How We Regulate, NRC Web site, https://www.nrc.gov/about-nrc/regulatory.html.

10

Be riskSMART: Guidance for Integrating Risk Insights into NRC Decisions and Strategic Sourcing Group make decisions about human resources, information technology, and procurement, respectively. In strategic decisions for all NRC program areas, the Executive Director for Operations Quarterly Performance Review Meetings address enterprisewide risks.

For a binary problem, Act...on a decision will simply be a decision of whether to move forward with whatever was proposed in Step 1, Beclear about the problem. It is a simple Yes or No question. If the decision is Yes, the next step is to implement the actions needed to put the decision into effect. If the decision is No, the process may end at this step. However, you could restart the Be riskSMART framework with a revised, Step 1, possibly leading to a different result.

Choosing one from a range of options is also fairly straightforward in this step as you weigh and consider the individual, Spotthe risk triplet, and Managewhat you can, actions for each option. To gain stakeholder alignment, sometimes you may adjust options or combine components of options as part of negotiations, as long as they are still supported by the underlying analysis and meet compliance requirements.

More complex problems will likely require a choice among many Managewhat you can actions developed in the previous step. The decision on which actions to use can be made by comparing the before and after consequences and likelihood of What can go right/wrong? with the Managewhat you can actions selected. The Adjust Results Based on Management Strategies heat map illustrates this choice: The results of what you have spotted may change based on what management strategy you choose. The data point may move to a different place on the heat map if you select a different risk management strategy.

The decision to move forward in a specific direction is frequently driven by the risk appetite that exists. For example, we may decide to move forward with a new process that has medium overall risk if the risk appetite is medium or high, but not move forward if the risk appetite is low.

The icon for this step in the Be riskSMART framework is a hand ready to press a START button (see Figure 1). The hand and button serve as a reminder that before any actions can be implemented and results realized, a definitive decision must be made.

2.10 Risk Appetite The amount of risk we are willing to accept in making a decision must also be carefully considered. This is referred to as a risk tolerance level or risk appetite (illustrated by the dotted lines in the figures in the preceding step, Acton a decision). In ERM, these terms have specific meaning:

• Risk appetite establishes the context for making a decision.

As determined by the entire organization, it is the amount of risk (on a broad or macro level) an organization is willing to accept in pursuit of strategic objectives and value to the enterprise.

11

Be riskSMART: Guidance for Integrating Risk Insights into NRC Decisions

• A risk tolerance level is the amount of risk acceptable for a decision. Risk tolerance translates risk appetite into meaningful terms at the operational level. In setting risk tolerance levels, consider the relative importance of the related objectives and align risk tolerance with risk appetite. 15 Naturally, acceptance varies based on the type of risk (e.g., technical, security, legal, reputational, financial, information technology) and is closely linked with an organizations mission and culture. You can gauge an organizations risk appetite by looking at its past decisions related to a type of risk. Additionally, an organization can establish a risk appetite philosophy. 16 The NRC has a safety and security mission and a strong safety culture. To determine the amount of risk acceptable in a specific area, see references such as Regulatory Guide 1.174, An Approach for Using Probabilistic Risk Assessment in Risk-Informed Decisions on Plant-Specific Changes to the Licensing Basis. Overall, any risk tolerance consideration should focus on the impact to the NRCs mission and be guided by the Principles of Good Regulation. 17 15 Playbook: Enterprise Risk Management for the U.S. Federal Government, Chief Financial Officers Council and Performance Improvement Council, Draft Revision, issued January 2021.]

16 U.S. Agency for International Development Risk Appetite Statement, issued June 2018.

17 The NRC Principles of Good Regulation are Independence, Openness, Efficiency, Clarity, and Reliability, https://www.nrc.gov/about-nrc/values.html#principles.

12

Be riskSMART: Guidance for Integrating Risk Insights into NRC Decisions 2.11 Facilitating Discussions About Risk The heat map used in the bike example can also be used as a discussion tool, as illustrated below.

For complex decisions with numerous answers to the questions of what can go right and what can go wrong, risks can be prioritized by severity. Plot each answer as a data point using consequence and likelihood. The data points can be compared (above or below) a risk appetite. It is essential for all stakeholders to be transparent about the amount of acceptable risk. A populated heat map can support discussions about different perspectives and lead to an optimal decision.

For enterprise risk management, you can plot every major challenge and opportunity that an organization has, resulting in a snapshot of all risks for management to discuss and prioritize.

2.12 Step 5: Realizethe result Step 5, Realizethe result, involves implementing the decision while managing what you can and measuring your performance and progress.

Carry out actions resulting from the decision, including any Managewhat you can actions. For example, in the enforcement area, a change to the Enforcement Manual could require training in addition to issuance of a revised manual.

It is time to ascertain efficacy. In addition to the data required to support compliance requirements, objectives and key results, the final step in the process, Teachothers what you learned, should drive what data to collect and measure since this information will inform others of the results. You can use effectiveness reviews and other similar assessment tools to measure results and the success of the decisions made. These results can then be used as part of a performance management strategy to adjust and fine tune, as necessary, the Managewhat you can, actions to optimize the results. The annual review 13

Be riskSMART: Guidance for Integrating Risk Insights into NRC Decisions and revision to the Reactor Oversight Process is one of many great examples of NRC processes that are revised after information is gathered as part of an assessment effort.

The icon for this step of the Be riskSMART framework is a lit lightbulb (see Figure 1). The glowing lightbulb should serve as a reminder that actions have been initiated and results can be seen.

2.13 Step 6: Teachothers what you learned The goal of the final step in the framework is to manage knowledge by providing lessons learned and insights to a broader audiencespread the word! In other words, share information gathered from use of the Be riskSMART framework with others who can benefit from it.

As part of Step 6, Teachothers what you learned, ensure that internal and external stakeholders affected by a decision are fully aware of it and how it will affect their work.

Each time the Be riskSMART framework is used, one or more of the following areas will likely emerge and be of interest to others:

• Use the experience to inform requirements. The results may provide valuable input to update guidance.

• Consider that the decision may impact other areas of the NRCs work.

• Inform future RIDM by sharing your experience, for example, the consideration, impact, and adoption or elimination of a large number of Managewhat you can actions before making a decision; a decision that required consideration of many areas of risk and associated risk appetites; or a risk appetite that was established in an area.

• In the legal area, decisions made by the courts that were litigated by the NRC with a consideration of risk and risk appetite would be good candidates for sharing with other interested stakeholders. The use of the Be riskSMART framework to make a decision that resulted in a hearing would also be of interest to other legal professionals.

The icon for this step of the Be riskSMART framework is a classroom (see Figure 1), and it serves as a reminder that sharing information with others improves the overall effectiveness in the use of the Be riskSMART framework, improves agency processes and actions, and improves the NRC as it strives to become a modern, risk-informed regulator.

14

Be riskSMART: Guidance for Integrating Risk Insights into NRC Decisions 2.14 The Arrow It is important to note the arrow symbol extending from the T to the S underneath the Be riskSMART logo. The arrow serves as a reminder that the framework is iterative, both overall and among the steps. Adjust for changing conditions and as you learn.

Any step may be repeated, and a decision may be revisited. Before revisiting a decision, remember that the framework is meant to be iterative but ultimately drive to a conclusion. It will be helpful to establish a threshold to meet before changes would be made.

15

Be riskSMART: Guidance for Integrating Risk Insights into NRC Decisions 3 CASE STUDIES The team applied the Be riskSMART framework to case study exercises (past and present-day decisions) to validate and refine the framework steps. Table 2 lists the case study topics.

Appendix G contains the step-by-step application of the Be riskSMART framework for each decision.

Table 2 Case Studies Applying the Be riskSMART Framework Technical 1 Subsequent License Renewal 2 Licensing of Accident Tolerant Fuel 3 Refueling Outage Inspection 4 Independent Spent Fuel Storage Installation Inspection 5 Security Bounding Time 6 Research Regulatory Readiness for Emergent Technical Issues 7 Dispositioning Emergent IssuesControl Rod Drive Mechanism 8 Generic Issue and Extended Power Uprate Licensing Action 9 Pilot Resident Inspector Site Coverage 10 Cybersecurity Onsite Inspection During a Public Health Emergency 11 Force-on-Force Exercise During a Public Health Emergency Legal 12 Timing of the Office of the General Counsels Legal Review Corporate 13 Office Restack 14 Forward Funding 15 Summer Recruiting Process 16 Use of the NRCs Collaborative Learning Environment to Generate Exams 17

Be riskSMART: Guidance for Integrating Risk Insights into NRC Decisions 4 ADDRESSING CHALLENGES TO RIDM The Be riskSMART framework was developed to address the cultural and structural challenges 18 to risk-informing decisions. Culturally, the framework serves as a shared 17F foundation to give the NRC staff confidence in applying risk insights and to empower individual contributors to drive the RIDM process. Structurally, the framework provides a consistent approach and common language to communicate risk information across disciplines.

Challenges are listed below with the corresponding framework steps. The asterisk denotes cultural challenges:

• Inconsistent management support and expectations (A, T) The Act step provides that the decisionmaker recognize the agencys risk appetite, and the Teach step reinforces expectations.

• Reluctance to adapt processes* (T)The Teach step and arrow incorporate adapting to change and making adjustments.

• Applied too late in process* (S)The Spot step ensures that risk is considered early in a decision process.

• Presence of uncertainties* (S, M)The Spot step illustrates consequences and likelihood in a heat map, and the Manage step is related dealing with the opportunities.

• Treatment of low-likelihood events* (S)The Spot step consequence multiplier addresses subjective consequences in the treatment of low-likelihood events.

• Lack of or limited awareness of guidance (T)The Teach step and arrow support the establishment of guidance, awareness of guidance, and updating the guidance as new information is obtained.

• Potential flexibilities not reflected in guidance (S); rigid interpretation of regulations*

(S)The Spot and Manage steps address potential flexibilities not reflected in guidance and rigid interpretation of regulations. You must thoroughly understand what flexibilities are available to answer the risk triplet question, What could go wrong? and how to manage that risk.

• Limited consideration of benefits* (S)The Spot risk triplet question, What can go right? helps to prevent overemphasis on the negative attributes of a decision without crediting the positive attributes.

• Siloed organizations (SMART)All steps work together to solve siloed organizations through cross-organizational common language, risk management, and decision implementation.

18 Summarized from Implementing Commission Direction on Applying Risk-Informed Principles in Regulatory Decision Making, dated November 18, 2019 (ADAMS Accession No. ML19319C832).

19

Be riskSMART: Guidance for Integrating Risk Insights into NRC Decisions 5 PERFORMANCE MANAGEMENT OF RISK-INFORMED DECISIONMAKING How do we monitor progress?

The NRC envisions that individual contributors at all levels and across the agency are empowered to drive the RIDM process (ideal state in Figure 2). In an ideal state, management will have established the infrastructure and culture. Management continues to play a key role in supporting the process, but risk-informing starts earlier in the decision process and occurs organically as part of everyday operations. The ideal state considers everyones opinion:

individual contributor, supervisor, every layer of management, and external stakeholders.

Figure 2 The Be riskSMART objective The Be riskSMART initiative was tasked with developing a tool to measure the agencys progress toward reaching this objective. Using the criteria of culture, processes, management expectations, and organization infrastructure, the team converted the diagram of the objective (Figure 2) into a facilitated diagnostic tool. The facilitated diagnostic tool can be applied at any level; from a team to an agency. It establishes a baseline state for RIDM in an organization and can be revisited to monitor progress. Additionally, the diagnostic tool describes each state and provides actions to support overcoming the challenges to move forward from the current state to the ideal state. The path to the ideal state depends on the organization, type of decisions, risk-acceptance stance, and the obstacles to overcome from the current state to the ideal state.

Appendix H provides the facilitated diagnostic tool.

Trajectory of perspectives In 2018, the Office of Nuclear Reactor Regulation (NRR) conducted surveys on perspectives about risk and RIDM within the office. From this survey, attitudes toward risk were generally positive, and only about 25 percent of the staff surveyed were unfavorable toward the idea of expanding the use of risk or risk insights in decisionmaking. However, the Office of the Inspector General (OIG) Safety Culture and Climate Survey (SCCS) in 2020 found about a 21

Be riskSMART: Guidance for Integrating Risk Insights into NRC Decisions 10-percent shift toward even more positive perspectives on risk, raising favorable responses up to 58 percent at the agency level. (For a direct comparison, NRRs favorability rate for this question from the SCCS is even higher, up to 67 percent.) Figure 3 illustrates these findings.

Figure 3 Trajectory of perspectives from 2018 to 2020 on using risk insights The 2018 NRR RIDM surveyconsistent with anecdotal evidence received over the years also identified that staff members were not comfortable with the available guidance for making risk-informed decisions in their daily work. Only a quarter of those surveyed felt comfortable with the existing guidance. The survey also found that the majority of staff perceived a gap between management expectations for the use of risk or risk insights in decisionmaking (only one favorable response on perception of an expectations gap out of more than 100 respondents). Figure 4 illustrates these data.

Figure 4 RIDM Guidance and Management Expectations 22

Be riskSMART: Guidance for Integrating Risk Insights into NRC Decisions During 2020, each NRC office and region applied the facilitated diagnostic tool to identify who in each NRC organization initiates or drives the use of risk insights in decisionmaking: the individual contributor or management. This survey captured responses from approximately 25 percent of the agency (Figure 5). Appendix I provides full results by individual offices and regions. Comparing the results of the 2018 NRR RIDM survey with the 2020 Be riskSMART Diagnostic survey, the majority of responses indicate that management and staff are equally driving the adoption of RIDM at the NRC.

Figure 5 Current State of RIDM in 2020 The conclusion: Be riskSMART meets staff members where they are.

Following the issuance of a Be riskSMART framework training course, the NRC will develop objectives and key results for RIDM for use as a baseline for comparison with a future diagnostic survey.

23

Be riskSMART: Guidance for Integrating Risk Insights into NRC Decisions 6 RESOURCES In addition to the Be riskSMART Training Course in the NRC Talent Management System and the Be riskSMART Community of Practice, the following NRC resources are relevant to the Be riskSMART initiative:

• Regulatory Guide 1.174, An Approach for Using Probabilistic Risk Assessment in Risk-Informed Decisions on Plant-Specific Changes to the Licensing Basis, Revision 3, January 2018 (ADAMS Accession No. ML17317A256).

• Management Directive 4.4, Enterprise Risk Management and Internal Control, December 14, 2017 (ADAMS Accession No. ML17312B109).

• NUREG-2122, Glossary of Risk-Related Terms in Support of Risk-Informed Decisionmaking, November 2013 (ADAMS Accession No. ML13311A353).

• NUREG-2150, A Proposed Risk Management Regulatory Framework, April 2012 (ADAMS Accession No. ML12109A277).

• Office of the Executive Director for Operations, Procedure 0960, Enterprise Risk Management Reporting Instructions, March 4, 2020 (ADAMS Accession No. ML19161A125).

• Risk-Informed Thinking Workshop, public slides, August 2, 2017 (ADAMS Accession No. ML17265A846).

• The Dynamic Futures for NRC Mission Areas, February 1, 2019 (ADAMS Accession No. ML19022A178).

25

Be riskSMART: Guidance for Integrating Risk Insights into NRC Decisions 7 STAFF ACKNOWLEDGMENTS Be riskSMART Agency Transformation Initiative Initiative Lead Mirela Gavrilas, Office of Nuclear Security and Incident Response (NSIR)

Champions Margaret M. Doane, Executive Director for Operations Darrell J. Roberts, DEDM Sponsors John B. Giessner, Region (R) III John W. Lubinski, Office of Nuclear Material Safety and Safeguards (NMSS)

L. Ben Ficks, Office of the Chief Financial Officer (OCFO)

Team Core Team:

Eric Duncan, NRR/Regions, Co-Team Lead Extended Team:

Jason Paige, NRR, Project Manager Gladys Figueroa Toledo, Office of Enforcement Brian Harris, Office of Nuclear Regulatory James Beardsley, NSIR Research (RES) Jason Piotter, NMSS Candace de Messieres, NRR June Cai, NMSS Elizabeth Bowlin, OCFO Katie McCurry, NRR Michael Gartman, Office of the General Counsel Mirabelle Shoemaker, NMSS (OGC)/Office of the Chief Human Capital Patrick Moulding, OGC Officer (OCHCO) Roger Hannah, Office of Public Affairs, RII Rebecca Richardson, NSIR/Office of Samson Lee, NRR Administration (ADM) Shilp Vasavada, NRR Reed Anzalone, NRR Stacey Prasad, NSIR Woody Machalek, ADM Ambassadors Allison Robinson, OCHCO Jake Dolecki, RI Paul Krohn, RI Amanda Marshall, NSIR Jakob Steffes, RIII Peter Davis, RIII NRAN Amy Hsu, Office of International Janice Owens, OIP Ronald Cureton, RII Programs (OIP) Jennifer Whitman, NRR Russell Chazell, Office of the Arlette Howard, OCFO Joanne Savoy, OIP Secretary Betsy Ullrich, RI Jonathan Fiske, NSIR Sarah Lopas, NMSS Brian Harris, OGC Jonathan Marcano Lozada, Shane Sandal, RII D.A. Adams, Office of NMSS Shaun Anderson, NRR Congressional Affairs (OCA) Joshua Havertape, RIII Tam Tran, NMSS Dennis Morey, NRR Julie Winslow, NRR Tania Martinez Navedo, NRR Diana Betancourt-Roldan, RIII Kathy Lyons-Burke, Office of the Tanner Robertson, OCFO NRAN Elba Sanchez Santiago, RIII Chief Information Officer Taylor Lamb, NRR Emma Hayward, OCHCO NRAN (OCIO) Trish Gallalee, OCHCO Erick Martinez Rodriguez, NSIR Lindsay Merker, RIV Wendy Reed, RES Fanta Sacko, NSIR Lynnea Wilkins, OCA William Cook, RI Geoffrey Miller, RIV Matt Whorral, OI Glen Sanabria Otero, NMSS Michael Dean, RI NRAN Patricia Walker-Webb, OCIO 27

Be riskSMART: Guidance for Integrating Risk Insights into NRC Decisions APPENDIX A APPLICABILITY OF THE BE riskSMART FRAMEWORK TO REACTOR SAFETY DECISIONS NRC Uses Its REACTOR SAFETY Guidance and Tools to Regulations Met Key Principles of Defense-in-Depth Consistency Risk-Informed Integrated Maintenance of Safety Margins Decisionmaking Risk Analysis Performance Monitoring Standard Review Plan (NUREG-0800) S M A Regulatory Guide 1.174* S M A R Licensing Modernization Project (LMP) S M A R Guidance Enhanced Safety Focused Review (ESFRA) S M A or Processes Reactor Oversight Process S M A R Backfit Process (MD 8.4, NUREG-1409) S M A Process for Emergent Issues (LIC-504) S M A Notice of Enforcement Discretion S M A

• Includes related risk-informed licensing guidance (e.g., RG 1.177, 1.201)

Key Principles of Risk-Informed Integrated Decisionmaking:

The Be riskSMART framework steps align with risk-informed decisionmaking (RIDM) for reactor safety.

Regulatory Guide (RG) 1.174, An Approach for Using Probabilistic Risk Assessment in Risk-Informed Decisions on Plant-Specific Changes to the Licensing Basis, describes one acceptable approach for assessing the nature and impact of proposed licensing basis changes using integrated RIDM. Specifically, RG 1.174 describes a framework composed of five key principles for integrated RIDM. These principles consider quantitative risk information as just one of five key principles of RIDM: (1) regulations met or exemption requested, (2) defense-in-depth consistency, (3) maintenance of safety margins, (4) risk analysis, and (5) performance monitoring strategies. The relationship between these key principles and the Be riskSMART framework steps is described below. In the context of reactor safety, probabilistic risk assessments (PRAs) provide crucial input to the implementation of the Be riskSMART framework. Senior reactor analysts in the regional offices and reliability and risk analysts at U.S. Nuclear Regulatory Commission (NRC) Headquarters serve as important resources in the development and implementation of the Be riskSMART framework.

A-1

Be riskSMART: Guidance for Integrating Risk Insights into NRC Decisions Examples of Specific Guidance and Processes:

The items listed below contain Office of Nuclear Reactor Regulation sources of information and processes that can be used to complete an evaluation using the Be riskSMART framework:

• BeKey Principle 1, regulations met (or exemption requested), helps to identify and be clear about the problem to be addressed by the Be riskSMART framework.

Ultimately, the problem is related to whether and how a regulation is being proposed to be met.

• SPOTKey Principles 2, 3, and 4, defense-in-depth consistency, 19 maintenance of safety margins, 20 and risk analysis, 21 support the Spot step of the Be riskSMART framework.

Key Principle 2, defense-in-depth consistency, provides an input to the determination of what can go right or wrong (e.g., extension of the completion time for emergency diesel generator (EDG) maintenance impacts the defense-in-depth for onsite alternating current power sources and a what can go wrong scenario of station blackout).

Key Principle 3, maintenance of safety margins, also provides an input to the determination of what can go right or wrong (e.g., material degradation, such as in pipe welds, can reduce the margin and help spot what can go wrong, what are the consequences, and how likely is it).

Key Principle 4, risk analysis, integrates the risk triplet and provides a structured approach for the Spot step. Risk assessments performed using PRAs include a broad range of challenges (what can go wrong), and both safety- and nonsafety-related structures, systems, and components (SSCs) for mitigation (what can go right and what are the consequences).

• MANAGEKey Principles 2, 3, and 4, defense-in-depth consistency, maintenance of safety margins, and risk analysis, support the Manage step of the Be riskSMART framework.

Key Principle 2, defense-in-depth consistency, provides input to manage the risk by identifying redundant, independent, and diverse SSCs and or strategies (e.g., using 19 Defense in depth is an element of the NRCs safety philosophy that involves designing and operating nuclear facilities in a manner that creates multiple independent and redundant layers of defense to compensate for potential human and mechanical failures so that no single layer, no matter how robust, is exclusively relied upon. Defense in depth includes the use of access controls, physical barriers, redundant and diverse key safety functions, and emergency response measures.

20 Safety margins refer to the extra capacity factored into the design of an SSC so that it can cope with conditions beyond the expected to compensate for uncertainty. Safety margins can be considered a part of, or complementary to, defense in depth, in that safety margins provide extra (redundant) capacity.

Incorporation of safety margins is one of the ways designers deal with the uncertainty of the challenges that the designed SSCs face.

21 A quantitative evaluation of the risk impact, including treatment of uncertainties, is used to ensure that increase in risk from the issue under consideration (e.g., a proposed change to the licensing basis) is small and consistent with the intent of the Commissions Safety Goal Policy Statement (usually using the core damage frequency and large early release frequency metrics). A quantified risk assessment (e.g., from PRA) is often used to determine the risk increase.

A-2

Be riskSMART: Guidance for Integrating Risk Insights into NRC Decisions supplemental diesel generators, cross-tying EDGs from another unit, and prestaging FLEX diesels can add defense in depth and help manage the risk of the extended EDG completion time).

Key Principle 3, maintenance of safety margins, helps to manage the risk by addressing the uncertainties in the risk through existing safety margins (e.g., available safety margin in codes and design can help manage the risk, to an extent, from an increased seismic hazard).

Key Principle 4, risk analysis, can be used to directly compare the quantified risk against established thresholds of acceptable risk (e.g., acceptance guidelines in RG 1.174). Further, risk assessments, especially PRAs, can also identify the extent of defense in depth available (i.e., how many SSCs must fail to reach an undesirable outcome) and compensatory actions to manage the risk.

Key Principle 5, effective performance monitoring, helps to manage the risk of unexpected consequences from the decision.

• ACTKey Principle 1, regulations met, ultimately provides the minimum requirements that shall be met.

• REALIZEKey Principle 5, effective performance monitoring, supports realizing the result because it can ensure the validity of assumptions that went into the decision (e.g., ensuring that a certain SSC will be tested for operability before entering an extended completion time) and provides the means to track the effectiveness of the decision (e.g., performance indicators can provide information on increased unavailability or unreliability of SSCs arising from decisions on changing their surveillance frequencies).

• TEACHRepeated application of the five key principles of RIDM for different reactor safety problems support teaching the Be riskSMART framework.

Examples of Specific Guidance and Processes:

• NUREG-0800, Standard Review Plan for the Review of Safety Analysis Reports for Nuclear Power Plants: LWR Edition

• specific risk-informed licensing initiative guidance (e.g., RG 1.177, RG 1.201)

• LIC-206, Integrated Risk-Informed Decision-Making for Licensing Reviews

• Licensing Modernization Project

• enhanced safety focused review

• Reactor Oversight Process

• backfit process (Management Directive 8.4, Management of Backfitting, Forward Fitting, Issue Finality, and Information Requests; NUREG-1409, Backfitting Guidelines)

A-3

Be riskSMART: Guidance for Integrating Risk Insights into NRC Decisions

• process for emergent issues (LIC-504, Integrated Risk-Informed Decision-Making Process for Emergent Issues)

• notice of enforcement discretion A-4

Be riskSMART: Guidance for Integrating Risk Insights into NRC Decisions APPENDIX B APPLICABILITY OF THE BE riskSMART FRAMEWORK TO MATERIAL SAFETY DECISIONS Key Principles of Risk-Informed Integrated Decisionmaking:

The Be riskSMART framework steps align with risk-informed decisionmaking (RIDM) for material safety.

Examples of Specific Guidance and Processes:

The items listed below contain Office of Nuclear Material Safety and Safeguards sources of information and processes that can be used to complete an evaluation using the Be riskSMART framework:

• SPOTRows with an S in the Spot... column address the questions in the risk triplet:

What can go right/wrong? What are the consequences? and How likely is it? by examining what the U.S. Nuclear Regulatory Commission (NRC) currently requires for particular aspects of materials licensing and inspections. By using Spot... you can determine whether efficiencies could be gained based on licensing and operational experience. For example, prelicensing interaction enhancement, integrated acceptance reviews, and the Fuel Management Licensing Risk tool can assist in assessing What can go right or wrong? and What are the consequences? for a change to these B-1

Be riskSMART: Guidance for Integrating Risk Insights into NRC Decisions activities under consideration. In addition, information from inspection and oversight are helpful to answer the question of How likely is it?

• MANAGERows with an M in the Manage...what you can column contain regulatory vehicles such as regulations, guidance, inspection procedures, and enforcement guidance. We Manage...what you can to improve the consequences and likelihood of what can go right and reduce the consequences and likelihood of what can go wrong by applying risk-management approaches (e.g., mitigating, preventing, preparing for, and transferring challenges and enhancing opportunities). We can use low safety significance guidance and the assignment and alignment process to further enhance communication on areas that need special consideration and to providing a vehicle for communication at all levels of the NRC.

• ACTRows with an A in the Act...on a decision column contain a source of information or process related to the decisionmaker and documenting the decision. After receiving inputs, evaluating information, and documenting what you spotted and managed, you can now decide on a course of action. Fully informed strategic decisions require that all views are adequately considered. It is also important to develop contingency plans and be prepared to make course corrections as needed. As examples, the graded approach to licensing; graded approach to the rulemaking; process, inspection, and oversight process improvements; and integrated safety evaluations allow for acting on the issues identified in the Spot... and Manage...what you can steps.

• REALIZERows with an R in the Realize...the result column are related to the results of managing what you can and include the implementation of actions resulting from the decision, including any Managewhat you can actions. During this step, you can make adjustments to ensure decisions remain valid and are successful.

Continuously using, evaluating, and modifying existing guidance and documenting RIDM from those activities will achieve the overall goal of including real-time risk insights to decisionmaking.

• TEACHRows with a T in the Teach...others what you learned column are examples of where you may document and share results of the action as well as identify future sources for licensing or inspection enhancements. It is well understood that all NRC staff should continually engage in sharing and improving knowledge management because it is important to future decisionmaking. Those lessons learned are conveyed through active knowledge management activities linked to specific outcomes, updated division or office instructions, updated risk-Informed standard review plans, inspector counterpart meetings, and NUREGs or regulatory guides containing specific or generalized guidance.

B-2

Be riskSMART: Guidance for Integrating Risk Insights into NRC Decisions APPENDIX C APPLICABILITY OF THE BE riskSMART FRAMEWORK TO SECURITY, PREPAREDNESS, AND RESPONSE DECISIONS Key Principles of Risk-Informed Integrated Decisionmaking:

The Be riskSMART framework steps align with risk-informed decisionmaking (RIDM) for security, preparedness, and response.

Examples of Specific Guidance and Processes:

The items listed below contain Office of Nuclear Security and Incident Response sources of information and processes that can be used to complete an evaluation using the Be riskSMART framework.

For example, a graded approach to rulemaking means to identify the most effective and efficient approach based on the complexity of the issue. That is, the level of effort and length of time needed for the development process should be commensurate with the significance of the issue and potential challenges involved.

• SPOTRows with an S in the Spot... column address the questions in the risk triplet:

What can go right/wrong? What are the consequences? and How likely is it?

C-1

Be riskSMART: Guidance for Integrating Risk Insights into NRC Decisions Documents such as regulatory guides identify consequences and possible outcomes of opportunities (what can go right) or challenges (what can go wrong) and the likelihood of these opportunities and challenges. By examining what particular aspects of licensee security, emergency preparedness, and incident response programs currently require, you can determine whether efficiencies could be gained based on industry maturity and operational experience. For example, historic U.S. Nuclear Regulatory Commission (NRC), industry, interagency, and international documents and analyses can illustrate What can go right or wrong? and What are the consequences? for a change to these programs under consideration. Trending data from the inspector community and industry are helpful to answer, How likely is it?

• MANAGERows with an M in the Manage...what you can column contain regulatory vehicles such as regulations, guidance, inspection procedures, and enforcement guidance. These items contain approaches to controlling regulatory challenges and maximizing opportunities. We Manage...what you can in order to improve the consequences and likelihood of what can go right and reduce the consequences and likelihood of what can go wrong by applying risk-management approaches (e.g., mitigating, preventing, preparing for, and transferring challenges and enhancing opportunities).

• ACTRows with an A in the Act...on a decision column contain a source of information or process related to the decisionmaker and documenting the decision. After receiving inputs, evaluating information, and documenting what you spotted and managed, you can now make a decision. Fully informed strategic decisions require that all views be adequately considered. It is also important to develop contingency plans and be prepared to make course corrections, as needed. For example, the Office of the Secretary paper process documents the basis for regulatory decisions that affect NRC licensees and any resulting Commission direction in a staff requirements memorandum, additional documentation associated with changes to the Reactor Oversight Process, public correspondence, and Statements of Consideration for associated rules. These items are helpful when researching past decisions or determining the appropriate forum for a current issue or challenge.

• REALIZERows with an R in the Realize...the result column are related to implementation of the actions determined in Act...on a decision and measuring performance and progress. Measuring processes, reports, and dashboards provide documentation for accountability and transparency. During this step, you can make adjustments to ensure decisions remain valid and on a path to success.

• TEACHRows with a T in the Teach...others what you learned column are examples of where you may chronicle and share results of the experience and provide sources for research on prior results and improvement efforts with both internal and external stakeholders. We should continually engage in sharing and improving knowledge management because it is important to future decisionmaking.

C-2

Be riskSMART: Guidance for Integrating Risk Insights into NRC Decisions APPENDIX D APPLICABILITY OF THE BE riskSMART FRAMEWORK TO RESEARCH DECISIONS Overarching Principles:

The Be riskSMART framework steps align with overarching principles for research.

The U.S. Nuclear Regulatory Commission (NRC) has a safety strategy to further risk-inform the current regulatory framework specifically in response to advances in science and technology.

The Office of Nuclear Regulatory Research (RES) conducts research and provides technical expertise to improve the agencys knowledge of where uncertainty exists, where safety margins are not well characterized, and where regulatory decisions need to be confirmed in existing or new designs and technologies.

Examples of Specific Guidance and Processes:

The items listed below contain RES sources of information and processes that can be used to complete an evaluation using the Be riskSMART framework:

• SPOTDocuments such as user need requests, research plans, and regulatory guides identify consequences and the possible outcomes of opportunities (what can go right) or challenges (what can go wrong) and the likelihood of these opportunities and D-1

Be riskSMART: Guidance for Integrating Risk Insights into NRC Decisions challenges. As an example, Future Focused Research plans identify and spot technological challenges and solutions to address knowledge gaps in areas beyond the near horizon. In addition, regulatory guidance documents can provide important insights on the current state of issues, provide acceptable methods (opportunities) and approaches to solve regulatory issues, and identify applicable regulatory regulations.

Lastly, computational tools and infrastructure such as SPAR, SOARCA, SAPHIRE, TRACE, RAMP, MELCOR, and XLPR can be used to provide insights on the likelihood of your outcome.

• MANAGEWe Manage what you can in order to improve the consequences and likelihood of what can go right and reduce the consequences and likelihood of what can go wrong by applying risk-management approaches (e.g., mitigating, preventing, preparing for, and transferring challenges and enhancing opportunities). We can use memoranda of understanding and international agreements as guidance to increase effective engagement with external organizations in collaborative research efforts and consensus standard development. We also leverage computational tools in managing uncertainties in risks of decisions. Moreover, regulatory guidance documents can be used to enhance an opportunity by managing acceptable methods and approaches to solve regulatory issues.

• ACTAfter receiving inputs, evaluating information, and documenting what you spotted and managed, you can now make a decision. Fully informed strategic decisions require that all views are adequately considered. It is also important to develop contingency plans and be prepared to make course corrections as needed. For example, after future-focused research ideas are generated, they are compiled and refined with associated funding recommendations before research can begin. However, some of the ideas that did not qualify for funding may be managed through other processes, such as user needs requests. Both of these paths strategically increase the likelihood of the ideas success.

• REALIZEThis step in the Be riskSMART framework includes the implementation of actions resulting from the decision, including any Managewhat you can actions. It also includes any actions taken to track the effectiveness of the decision. Using future-focused research as an example, ideas chosen for funding are monitored against the overall research portfolio and performance is measured and reported. During this step, adjustments are made to ensure decisions remain valid and on a path for success.

• TEACHWe convey lessons learned through seminars and NUREGs. We continually engage in international workshops and consensus standards conferences in sharing and improving knowledge management in specialized technical expertise important to future decisionmaking. We also use regulatory guides to communicate our most current regulatory decisions to our stakeholders. Lastly, we gather and analyze data from computational tools to share so that a broader audience can benefit from the knowledge and information obtained.

D-2

Be riskSMART: Guidance for Integrating Risk Insights into NRC Decisions APPENDIX E APPLICABILITY OF THE BE riskSMART FRAMEWORK TO OFFICE OF THE GENERAL COUNSEL LEGAL ADVICE Overarching Principles:

The Be riskSMART framework steps align with overarching principles for Office of the General Counsel (OGC) legal advice.

The overarching principles set forth in the chart reflect OGCs mission. OGC provides advice to assist the agency in complying with its legal obligations. The offices mission is to deliver clear, E-1

Be riskSMART: Guidance for Integrating Risk Insights into NRC Decisions candid, and sound legal advice and counsel to the agency. We advance the NRCs mission by engaging in creative problem-solving, developing viable legal options, and providing effective representation. This function includes providing legal interpretations (the meaning of statutes and regulations), identifying legally viable solutions, and, when necessary, defending the agency when its actions are challenged in litigation.

Engaging in creative problem solving inherently involves risk-informed decisionmaking. If a proposed approach is clearly legally prohibited, OGC will advise the client why it is not a viable option. Some legal issues provide room for interpretation, and when the agency may have more than one legally permissible way to accomplish its objective, some options may involve greater legal vulnerability than others. In those situations, OGC advises clients on the degrees of legal risk associated with the legally viable options.

The emphasis on communications reflects the fact that OGC is a service organization that supports the achievement of agency priorities both by providing advice that assures the legality and enhances the clarity of agency actions, and by effectively representing the agency before agency adjudicators and in administrative and Federal court litigation. Communications that lack clarity and transparency may present both legal and policy risks. Enhancing the quality of the U.S. Nuclear Regulatory Commissions (NRCs) communications is one of three focus areas highlighted by the Executive Director for Operations and also is a focus area emphasized by OGC in its daily operations.

Examples of Specific Guidance and Processes:

The NRC already incorporates risk insights into its regulatory framework in a variety of ways, such as the frequency of facility inspections, the prioritization of new rulemaking activities, regulations that set out performance safety limits for nuclear reactors, or the establishment of criteria for evaluating safety significance in inspection findings and enforcement actions. The items listed in the chart represent key sources of information and guidance that OGC uses to risk-inform its advice, consistent with the Be riskSMART framework:

• SPOTThe foundation of OGCs advice always is identifying the potential legal issues presented by a clients question or proposal. The chart captures several of the most crucial resources that OGC uses for that purpose. In addition to binding legal authorities (e.g., statutes, court cases, regulations, interagency agreements) or Commission direction, OGC also may need to examine agency guidance documents, particularly if a proposal would involve departing from an established policy or longstanding practice (and might, for example, raise backfitting, forward-fitting, or other concerns).

OGC also has developed internal guidance for assessing legal risk. This guidance helps OGC attorneys apply the risk triplet by identifying legal risk factors relevant to each component of the triplet. As an obvious example, in determining what could go wrong (or right) with a client proposal, OGC considers whether the agency could be sued, and if so, whether the agency could lose in court. Similarly, OGC considers whether taking a particular action or relying on a particular interpretation of a regulation will have consequences or create conflicts with other actions or regulations (for example, OGC would examine whether an interpretation of specific words in one regulation might affect the interpretation of other NRC regulations that use the same words).

MANAGEThe listed resources also provide insights on how we can lessen or eliminate the identified risks. For example, in examining NRC regulations, Commission E-2

Be riskSMART: Guidance for Integrating Risk Insights into NRC Decisions adjudicatory decisions, and Federal court decisions, we may find that minor changes in the wording of a proposed rule could eliminate an ambiguity that would either make the rule difficult to enforce or create an unintended conflict with other NRC regulations.

Another tool OGC uses to manage risk is its Best Practices for Legal Review of Staff Written Products. Several of the best practices identified in this document focus on effective communication with clients (to understand the clients objective) and on ensuring clarity in the documentation of the agencys reasoningwhich is typically one of the most important ways to reduce the litigative risks associated with a particular proposed action.

• ACTBecause of OGCs counseling function, its clients are often the ultimate decisionmakers. To that end, OGC employs the listed tools to efficiently assemble and consider all pertinent inputs and views and to provide timely advice for the clients use.

The chart also reflects that risk insights may, in some instances, prompt OGC to take some actions on its own initiative, such as clarifying procedures or guidance that implement OGCs functions, or proactively engaging clients on emergent issues (or on topics for which they might not routinely seek OGC input). In its representational function, OGC identifies and acts on litigation strategies to effectively represent the agency before agency adjudicators or in administrative or Federal court litigation. In its representational function, OGC also acts to craft responses to congressional and interagency requests for policy-neutral legal input.

• REALIZEThis step in the Be riskSMART framework includes the assessment of actions taken to implement a decision and any outcomes of these actions, including unforeseen consequences. This may include helping the client to track how effective the decision has proven to be in resolving the issue it was intended to address, reacting to any formal legal challenges to the decision, and monitoring whether any of the potential risks identified before the action was taken (such as favorable/unfavorable implications in inspection and enforcement space) actually arose.

• TEACHThe riskSMART framework is an iterative process that constantly seeks improvement in agency policy and processes. OGC uses knowledge-management tools to document and foster consistency in OGC advice, and it captures and conveys lessons learned through seminars, internal working groups, and interactions with its legal counterparts at other agencies. As risk insights from past actions accrue, OGC updates its internal guidance on a rolling basis and advise its clients on opportunities and obligations to revise their own guidance and procedures.

E-3

Be riskSMART: Guidance for Integrating Risk Insights into NRC Decisions APPENDIX F APPLICABILITY OF THE BE riskSMART FRAMEWORK TO CORPORATE SUPPORT DECISIONS Overarching Principles:

The Be riskSMART framework steps align with overarching principles for corporate support product lines.

Examples of Specific Guidance and Processes:

The items listed below contain corporate support sources of information and processes that can be used to complete an evaluation using the Be riskSMART framework:

• SPOTFirst, find out what is required. Federal guidance and standards are good resources for requirements. Rows with an S in the Spot column support answers to the risk triplet questions. For example, Government Accountability Office (GAO) and Office of the Inspector General reports are a good resource to answer, What can go right or wrong? The National Institute of Standards and Technology guidance helps to answer, What are the consequences? for an information system issue based on the loss of information confidentiality, integrity, and availability. Trending data from interagency working groups and professional organizations are helpful to answer, How likely is it?

F-1

Be riskSMART: Guidance for Integrating Risk Insights into NRC Decisions

• MANAGERows with an M in the Manage...what you can column contain risk management and internal control techniques, such as GAO-14-704G, Standards for Internal Control in the Federal Government, issued September 2014. These items contain approaches to controlling challenges and maximizing opportunities. To align with enterprise risk management (ERM), as you analyze an issue or decision, consider its potential impact (both challenges and opportunities) to other business lines and the agencys mission and strategic objectives. Guidance from the Office of Management and Budget and the U.S. Nuclear Regulatory Commission listed in the chart and the ERM documents referenced therein contain more information. ERM risk reports and quarterly performance review materials provide information on current and prior enterprise risks.

• ACTRows with an A in the Act...on a decision column contain sources of information or processes related to the decisionmaker and documenting a decision. For example, ERM decisions are made at the quarterly performance review meetings; information systems are good sources of quantitative information, such as financial data, transactions of all types, and decisions made in daily operations; the Office of the Secretary paper process documents the basis for high-level decisions and any resulting Commission direction in a staff requirements memorandum; and corporate support decisions are often made by steering committees listed in the crosswalk chart. These items are helpful when researching past decisions or identifying the appropriate forum to make a decision.

• REALIZERows with an R in the Realize...the result column are related to the results of managing what you can during implementation and measuring your performance and progress. These processes, reports, and dashboards provide documentation for accountability and transparency.

• TEACHRows with a T in the Teach...others what you learned column are examples of where you may chronicle and share results of the experience and provide sources for research on prior results and improvement efforts.

F-2

Be riskSMART: Guidance for Integrating Risk Insights into NRC Decisions APPENDIX G BE riskSMART CASE STUDIES Technical 1 Subsequent License Renewal 2 Licensing of Accident Tolerant Fuel 3 Refueling Outage Inspection 4 Independent Spent Fuel Storage Installation Inspection 5 Security Bounding Time 6 Research Regulatory Readiness for Emergent Technical Issues 7 Dispositioning Emergent IssuesControl Rod Drive Mechanism 8 Generic Issue and Extended Power Uprate Licensing Action 9 Pilot Resident Inspector Site Coverage 10 Cybersecurity Onsite Inspection During a Public Health Emergency 11 Force-on-Force Exercise During a Public Health Emergency Legal 12 Timing of the Office of the General Counsels Legal Review Corporate 13 Office Restack 14 Forward Funding 15 Summer Recruiting Process 16 Use of U.S. Nuclear Regulatory Commissions Collaborative Learning Environment to Generate Exams G-1

Be riskSMART: Guidance for Integrating Risk Insights into NRC Decisions TechnicalSubsequent License Renewal Should we focus operating experience audits for subsequent license clear about the Be renewals by reducing the key words search to reduce burden while problem still achieving effective results?

Reducing the key words search would allow licensees to focus resources on more important issues. A reduced scope of review for the operating experience audit would allow U.S. Nuclear Regulatory What can go Commission (NRC) staff to use fewer resources as well. The right?

reduced set of key words could be tailored to the plant, and the applicant could provide input to the relevant key words based on its knowledge of plant history.

Applicant input on key words might not encompass all risk-significant operating experience. The NRC staff may not embrace a reduction in What can go the key word search and may not agree on a standard list. Use of a wrong?

reduced list of key words tailored to each plant entails further work to develop.

Opportunities:

• Overall, reducing the key word search and allowing licensees to focus on more important issues could result in reduced negative feedback from licensee management to the NRC.

• A reduced scope could result in increased transparency and clarity on the most important items. Additionally, reducing unnecessary NRC staff resource expenditures allows the time spent during the audit to be more focused to those items of greatest safety significance for the plant.

• If applicants provide input to the list of keywords, the applicant may recommend a risk-significant keyword that the Spot staff would not have thought to include and the applicant may have missed if given a list from the staff. Applicant input increases applicant ownership of the process and What are the responsibility for overall plant safety.

consequences?

• The (actual or perceived) level of oversight could be reduced.

Challenges:

• If applicants provide input on the relevant key words, they could fail to include a relevant keyword, and the search could exclude some risk-significant operating experience.

• If staff fail to embrace a reduction in the key word search and thus do not curtail the review on an individual level, the resource reduction may not be achieved.

• Developing custom key word lists may take longer and could make preparation for the audit more difficultfor both the NRC staff and the applicant. This would be exacerbated for fleet reviews.

Opportunities:

• Licensee focus on the most risk-significant issues by How likely is it? reducing the scope of the key word search is very likely, but the impact varies from plant to plant.

G-2

Be riskSMART: Guidance for Integrating Risk Insights into NRC Decisions

• Better NRC staff focus is likely, but again varies both by plant and reviewer.

• It is very likely that the lists could be tailored to be plant specific with adequate resources allocated. It is somewhat less likely that applicants would provide adequate input into relevant operating experience; they develop the subsequent license renewal application and own the plant, so are in a good position to identify relevant operating experience.

Challenges:

• It is likely that reducing the key word search and allowing applicants to provide input would cause some operating experience to be missed, but there is probably close to the same likelihood of missing something under the current process. Applicants may be too close to their application to develop a comprehensive set of operating experience key words to search.

• It is likely that some reviewers may not substantially reduce their effort, but the impact on the review may be slight.

• The likelihood of expending more resources on a custom key word search list depends on how the program is implemented. It is very likely that the time to prepare for an audit would be increased when compared to the current process, but how much additional time is needed would depend on how aligned the plant already is with the standard list.

Steps that can be taken to manage risks:

• Communicate with licensee management regarding where additional resource savings are allocated.

• Set clear expectations regarding what is to be included in the operating experience and aging management plant audit (mostly in office, but on site if needed), provide better training/supplemental training. Ensure knowledge transfer is in the new way.

• Provide guidance and training on the process for generating the plant-specific key word list.

Manage what you can

• Make the standard list as customizable as possible while staying close to standard (e.g., instead of essential service water use essential service water or the equivalent system at the site).

• Ensure solid operating experience review and historical problem review occur during aging management plan audit and review.

• Engage in solid communication with applicants on expectations and our own process so that the applicants can provide inputs appropriately.

G-3

Be riskSMART: Guidance for Integrating Risk Insights into NRC Decisions The staff decided to pursue potential reductions in the standard Act on a decision operating experience key word search for subsequent license renewals.

The staff proposed this idea during a public meeting and considered Realize the result additional challenges and opportunities impacting the decision.

others what you The broader initiative is still currently underway and Be riskSMART Teach learned principles are being applied to evaluate emergent decisions.

G-4

Be riskSMART: Guidance for Integrating Risk Insights into NRC Decisions TechnicalLicensing of Accident Tolerant Fuel clear about the Should we take specific actions that will enable the agency to license Be problem accident-tolerant fuel (ATF)?

A1. Licensing ATF in a timely manner gives us the opportunity to enhance the NRCs reputation for being able to license new technologies.

A2. We have the opportunity to set clear expectations for new What can go ATF technologies before submitted for our review.

right?

A3. Taking actions to enable licensing of ATF allows us the opportunity to coordinate with the industry without compromising independence.

B1. Failing to take action to enable ATF licensing could damage our reputation.

B2. Failing to take actions to enable ATF licensing could lead to iterative and prolonged licensing reviews.

What can go wrong? B3. A rapidly evolving environment presents challenges and risks as we seek to take specific actions to enable ATF licensing.

B4. Internal misalignment poses a risk to efforts to enable ATF licensing.

What can go right?

A1. Enhancing the agencys reputation for licensing new technologies will engender goodwill with external Spot stakeholders.

A2. Setting clear expectations will allow us to develop a comprehensive plan, provide guidance to the industry, and focus efforts to save resources.

A3. Coordinating with the industry, provided independence is not compromised, will provide clarity between NRC staff and applicants. This would result in resource savings due to a What are the reduction in back-and-forth communications.

consequences?

What can go wrong?

B1. Damaging our reputation could result in barriers to meeting our mission.

B2. Failure to license ATF in a timely manner could result in the delayed deployment of safety-enhancing technology.

B3. If we are unable to adapt to a rapidly evolving environment, we could waste resources on unfocused, divergent efforts.

B4. Failure to align internally could result in delays and regulatory uncertainty for licensees and applicants.

How likely is it? What can go right?

G-5

Be riskSMART: Guidance for Integrating Risk Insights into NRC Decisions A1. High, because expectations of the NRC for licensing new technologies are low.

A2. Medium, because good communication and planning enables us to set clear expectations, but we are sometimes not as clear as we could be in developing guidance.

A3. High, because the industry and other external stakeholders have advocated for enhanced coordination and interaction with the NRC.

What can go wrong?

B1. Medium, because some external stakeholders (especially Congress) are paying very close attention to how we handle ATF.

B2. High, because even a small stumble could result in issues with the licensing process.

B3. High, because the industry has been iteratively designing and evaluating the economic and safety effects of ATF, resulting in a very dynamic environment.

B4. High, because of number of internal stakeholders and the significance of ATF to different stakeholders.

What can go right?

A1. Provide regular updates to external stakeholders (including Congress) on our activities, demonstrating good progress toward licensing ATF.

A2. Develop a project plan, conduct phenomena identification and ranking table exercises to identify what is needed to license ATF concepts, and establish concept-specific interim staff guidance to guide NRC technical reviewers.

A3. Establish appropriate industry/U.S. Department of Energy (DOE) communication processes (with advice from the Manage what you can NRCs Office of the General Counsel).

What can go wrong?

B1. Do a good job preparing to license ATF.

B2. Establish clear milestones for the development and execution of the project plan.

B3. Increase communication and coordination with the industry, including licensees, fuel vendors, and the DOE.

B4. Set up an internal working group and steering committee early, and routinely communicate to all internal stakeholders.

Develop an ATF project plan that lays out how the NRC plans to Act on a decision review ATF technologies.

Follow the project plan and adjust it as new information becomes Realize the result available. For example, when industry identified a need to increase G-6

Be riskSMART: Guidance for Integrating Risk Insights into NRC Decisions fuel uranium enrichments and fuel burnup to support deployment of ATF, an appendix was added to the ATF project plan to outline how the NRC will review these new considerations.

Provide regular internal and external seminars on the ATF project plan and the progress we are making toward implementing it. Also, others what you Teach document progress toward implementing the project plan through the learned documentation of phenomena identification and ranking tables, guidance documents, and similar.

G-7

Be riskSMART: Guidance for Integrating Risk Insights into NRC Decisions TechnicalRefueling Outage Inspection ChallengeRefueling outage (RFO) inspections are part of the baseline inspection program, and the inspection procedure contains a wide variety of inspection requirements. Not all the inspection requirements are of equal risk importance and some, such as containment walkdowns, require inspector attention regardless of other ongoing RFO activities. RFO durations have significantly decreased in recent years, making it challenging for a two-person resident inspector team to complete all of the baseline inspection requirements and also focus on safety-significant activities and events that may be occurring; sometime simultaneously.

Potential SolutionProvide a dedicated inspector to focus on RFO risk assessment and management for periods of elevated risk. Emphasize preparation for the inspection and the focus on the selected key safety functions that involve elevated risk. The inspector uses the outage baseline inspection procedure and other baseline procedures such as plant status and equipment alignment, as needed, to focus on the selected key safety functions. The RFO inspection also assesses a licensees response to emergent issues within the key safety function focus areas. Examples of activities generally excluded from this inspection are routine RFO inspection requirements, such as the review of the implementation of the fatigue rule, observations of startup and shutdown, and the observation of refueling operations.

clear about the Should Region III provide a dedicated inspector to focus on risk and Be problem risk management during RFOs?

The inspector is focused on the most safety-significant areas during RFOs confirming the licensee is properly assessing and managing What can go risk during periods when key safety functions are impacted by RFO right? activities. If issues exist with licensee performance, prompt inspector and licensee response can mitigate the potential for the plant incurring additional unplanned risk.

Inspection resources used for inspection during an RFO would not be available for inspection activities elsewhere or in other areas. Also, What can go some licensees may take issue with the addition of an NRC inspector wrong?

to perform inspection activities during an RFO because this is billable effort.

What can go wrong?

The consequences of dedicating inspection resources at one site at the expense of inspection resources at other sites could result in issues at other sites not being identified. However, the inspection Spot program is a sampling program, and this initiative should help ensure What are the efficient and effective use of inspection resources.

consequences?

What can go wrong?

The consequences of licensees that take issue with an additional dedicated inspector during an RFO could result in additional external stakeholder criticism, including criticism from members of Congress.

It is not likely that significant plant issues would be missed by dedicating resources for outage activities in a risk-informed manner.

Lower significance issues may not be identified, but events and higher significance issues would continue to receive priority How likely is it?

inspection through the processes under NRC Management Directive 8.3 and Inspection Manual Chapter 0309. The likelihood that some licensees will question this expenditure of resources is high.

A clear explanation to a licensee for the risk-informed basis to the Manage what you can change in practice of assigning an additional dedicated inspector G-8

Be riskSMART: Guidance for Integrating Risk Insights into NRC Decisions during an RFO could largely mitigate any programmatic concerns, particularly if the dedicated RFO inspector is only assigned during short periods during the most risk-significant timeframes of an RFO and for short-duration refueling outages when the resident inspector workload is most elevated.

Region III has begun to implement this practice at some sites.

Act on a decision Dedicated refueling outage inspectors have been successful in performing the identified activities in this initiative.

Following implementation of this new initiative, resident inspectors and dedicated RFO inspectors have given positive feedback.

Specific feedback includes the following:

• The dedicated inspector position permitted the resident office to remain cognizant of all RFO activities and the associated risk, while the dedicated inspector was able to more closely inspect specific activities and associated contingencies.

• When the RFO schedule changed, the sequencing and risk assessment of activities and their related contingencies can consume an inordinate amount of time and focus. By having the dedicated inspector, those activities were able to be assessed at a level commensurate with risk rather than at the availability of the resident inspector office.

• Having the dedicated inspector greatly reduced competing interests during a short duration, high volume of risk-related Realize the result activities.

• A dedicated inspector at Monticello identified a finding and noncited violation associated with a newly implemented RFO activity that affected the decay heat removal and inventory control key safety functions.

• To date, no negative impact on the licensee (i.e., minimal intrusiveness) or on the resident inspector staff (separate, dedicated resource) has been identified.

• Assigned inspectors have reported an increased understanding of shutdown risk principles and guidance documents.

• This effort has elevated the awareness across Region III of what it means to be a modern, risk-informed regulator.

The result of the efforts to date have been shared with senior managers in other regions and industry representatives. Insights others what you from past and future inspections will be used to propose revisions to Teach learned Inspection Procedure 71111.20 associated with inspections conducted by the resident inspectors during outages, including refueling outages.

G-9

Be riskSMART: Guidance for Integrating Risk Insights into NRC Decisions TechnicalIndependent Spent Fuel Storage Installation Inspection ChallengeThe NRC staff had to decide whether to conduct an onsite inspection of the operations of the Prairie Island Nuclear Generating Plant independent spent fuel storage installation (ISFSI) during the last week of a three-cask spent fuel loading campaign during the week of May 26, 2020. This evolution is performed approximately every 2 years, and it has been 2 years since our last inspection. The inspection is not required to be completed until 2021. However, if the inspection is postponed, we will miss our only opportunity to evaluate license performance during cask-loading operations (most risk significant activities at an ISFSI) before the inspection due date.

Potential SolutionA preliminary discussion identified a number of potential inspection options.

For this decision, we used the Be riskSMART framework to determine the best available option.

During the ongoing Coronavirus Disease 2019 (COVID-19) public health emergency (PHE), which of the following inspection options should be selected for an upcoming inspection of a Prairie Island ISFSI campaign?

Option 1: Conduct Routine Onsite Inspection Option 2: Conduct Onsite Inspection with Mitigation Strategies Option 3: Conduct 100-Percent Remote Inspection Option 4: Conduct Remote Inspection with Onsite Followup Option 5: Cancel Inspection These options include the following major elements:

Option 1: Conduct Routine Onsite Inspection

• Use two inspectors qualified to inspect ISFSI operations including cask loading and transport to a qualified/approved ISFSI pad.

• Inspection includes onsite observations in close contact with licensee staff.

clear about the Be problem

• Inspection includes onsite administrative reviews.

Option 2: Conduct Onsite Inspection with Mitigation Strategies

• Use single inspector for onsite activities.

• Inspection includes only minimally required onsite activities.

• All inspections that can be performed off site completed at temporary lodging.

Option 3: Conduct 100-Percent Remote Inspection

• Obtain access to licensee cameras/remote monitoring equipment with live feed.

• Conduct remote review of dry-cask operations, including loading and transport. Perform a remote inspection to the maximum extent practicable using a combination of live video feeds (2-second delay) from the facility and teleconference lines to monitor operations.

G-10

Be riskSMART: Guidance for Integrating Risk Insights into NRC Decisions

• Perform administrative reviews remotely.

Option 4: Conduct Remote Inspection with Onsite Followup

• Include all elements of Option 3.

• Conduct followup inspection at a later date. All document reviews would be performed remotely, and the inspection would remain open until an inspector could safely report to the site to evaluate the ISFSI material and radiological conditions.

• Requirements outlined in the ISFSI-related inspection manual chapter permits a partial inspection during the loading campaign, with inspection completion at a later date.

Option 5: Cancel Inspection

• Risk of contracting/spreading COVID-19 too high to conduct onsite inspection.

• Remote inspection not a viable alternative.

• Remote administrative reviews insufficient to credit inspection completion.

• Inspectors perform an inspection and identify and prevent a significant safety issue or noncompliance concern.

What can go right?

• Inspectors perform a remote inspection that provides reasonable assurance that operations were conducted safely.

• An inspector reports to the area and contracts COVID-19.

• The inspector is a carrier of COVID-19 and exposes the licensee and surrounding community to additional COVID-19 risks.

What can go

• Adequate lodging and support are unavailable in the wrong?

surrounding community.

Spot

• An inspector is unable to fully evaluate licensee performance and cannot conclude whether operations were conducted safely and within regulatory requirements.

• Inspector or licensee contracts potential life-threatening illness.

• Licensee has a significant safety (injured personnel, fuel damage, or radiological release) or compliance event that What are the could have been mitigated by onsite NRC presence.

consequences?

• Stakeholders have concerns with dispatching an inspector from a relatively high-impact area of the country (Chicago, IL) to an area with minimal impact (Goodhue County, MN).

G-11

Be riskSMART: Guidance for Integrating Risk Insights into NRC Decisions

• Stakeholders have concerns with the NRCs inability to effectively monitor risk-significant dry-cask storage operations during a PHE.

• Based on MN State data of COVID-19 impacts, there is a relatively low probability that an NRC inspector would be exposed to COVID-19 while working in and travelling to Goodhue County, MN.

• There is a higher likelihood that an inspector traveling from Chicago, IL, could be a carrier of COVID-19.

• It is unlikely that the licensee will have a significant safety How likely is it?

issue or noncompliance. The dry-cask storage system at the Prairie Island ISFSI has been successfully loaded numerous times and Prairie Island is using the same vendor it used previously during the last two loading campaigns in 2016 and 2018, both of which the NRC inspected.

• It is unlikely that an inspector will be able to fully complete the inspection remotely.

Options available to manage or mitigate the risk of the likelihood and consequences outlined above include the following:

• Assess options to complete inspection requirements and discuss with peers the ability to perform remote inspections.

[Completepeers do not believe an ISFSI inspection can be completed by remote methods alone.]

• Limit the number of inspectors that conduct onsite inspection.

Two inspectors usually perform this inspection; however, we could limit travel to a single inspector.

• Assess risk tolerance for travel in May. [Complete Assessed as low risk tolerance within the NRC.]

• Openly discuss risks with inspectors and Division of Nuclear Manage what you can Materials Safety management and assess their concerns and willingness to travel. [Both assigned inspectors volunteered to conduct onsite inspection.]

• Assess changes to dry-cask storage program at Prairie Island since the NRCs last inspection. [Completeno significant changes.]

• Obtain remote access to the licensees camera system to assess capabilities and limitations. [CompleteLive video capability with 2-second delay. If something of interest is identified, the inspector will note the time, contact the licensee in the OCC, and the licensee will capture the feed and upload it for agency review using the Certrec system.]

G-12

Be riskSMART: Guidance for Integrating Risk Insights into NRC Decisions

• Review State and local COVID-19 data just before traveling to minimize risks and remain compliant with State guidelines to the extent practicable.

DecisionOption 4: Perform the inspection using remote technology to the maximum extent practicable, then follow up on site when safe to do so to evaluate the material and radiological condition of the ISFSI and complete the inspection.

• Remotely monitor ISFSI operations using camera system.

• Relay real-time (2-second delay) camera observations to licensee.

Act on a decision

• Review other videotaped footage.

• Conduct daily discussions with licensee.

• Conduct onsite inspection of material and radiological conditions at later date.

• Remote Inspection: Conducted week of May 26, 2020.

Realize the result

• Onsite inspection: Planned for a date to be determined, commensurate with COVID-19 risk.

• Use Be riskSMART to choose the best solution among many options.

others what you Teach learned

• Consider all risk areas, both oversight and public perception.

G-13

Be riskSMART: Guidance for Integrating Risk Insights into NRC Decisions TechnicalSecurity Bounding Time ChallengeDevelop a Commission paper (see SECY-20-0070, Agencywide Documents Access and Management System (ADAMS) Accession No. ML20126G265) that includes recommendations for providing credit for a broader set of operator actions, including the use of FLEX equipment, and providing credit for response by local, State, and Federal law enforcement in our security inspection program at operating nuclear power plants (NPPs), in accordance with SRM-SECY-17-0100, Staff RequirementsSECY-17-0100Security Baseline Inspection Program Assessment Results and Recommendations for Program Efficiencies, dated October 9, 2018 (ADAMS Accession No. ML18283A072).

Potential SolutionsA risk-informed concept for protecting against the design-basis threat (DBT) that considers the range of capabilities and assets available to licensees that, taken together, provide reasonable assurance that licensees can maintain adequate physical protection of their sites against the DBT.

Two new concepts:

(1) Implement a new concept, Reasonable Assurance of Protection Time (RAPT), which recognizes the existing layers of protection available to sites along with how the safety and security of the site would evolve over time following initiation of an attack, in a revision to existing guidance.

(2) Provide options for the Commissions consideration on whether and how to implement a site-specific security bounding time (SBT).

Determine an approach for crediting a broader set of operator clear about the Be actions, including the use of FLEX equipment, and law enforcement problem response at operating NPPs in the security inspection program.

Commitment (RAPT) and recommendations (site-specific SBT) provided in this paper, if implemented, would incentivize enhanced What can go interactions and coordination between licensees and law enforcement right?

and allow licensees to use risk insights to focus their protective strategies where most needed and effective.

One example (multiple challenges are addressed throughout the What can go paper)timelines for response by recalled off-duty personnel (or law wrong?

enforcement) could be lengthy following the recognition of an attack.

Spot Sites may be challenged to use them for support because law enforcement, recalled off-duty personnel, or both would need What are the sufficient time to assess the situation, plan their actions to support the consequences?

site, and then execute those actions, including engaging adversaries and moving FLEX equipment to prevent radiological sabotage.

While the likelihood is hard to assess because timelines are site How likely is it? specific, the evaluation considered all the factors that can impact timelines by an offsite response force.

The risks associated with leveraging offsite resources are managed, in part, based on the U.S. Nuclear Regulatory Commissions coordination with the Federal Bureau of Investigation (FBI), and the understanding that FBI tactical response teams will respond in a Manage what you can timely manner to terrorist attacks at operating NPPs. Additionally, licensees that justify a site-specific SBT based on increased coordination with law enforcement realize an additional benefit because the licensees would have an incentive to ensure that G-14

Be riskSMART: Guidance for Integrating Risk Insights into NRC Decisions responders are prepared to support plant defense. Credit for law enforcement assistance would only be granted when it can be reasonably demonstrated that licensees have coordinated with law enforcement to facilitate a timely and effective response.

The staff sent a SECY paper dated July 30, 2020, to the Commission for a decision on the site-specific SBT concept; the NRC staff made a decision that RAPT could be implemented without Commission direction by updating guidance to specify that one way for licensees to meet the existing regulatory requirements (including the Act on a decision requirement to maintain the capabilities to defend against the DBT at all times) and provide reasonable assurance of adequate protection is by designing a physical protection program that allows the licensee to independently defend against the DBT for a minimum of 8 hours9.259259e-5 days <br />0.00222 hours <br />1.322751e-5 weeks <br />3.044e-6 months <br />, after which it is reasonable to expect that additional resources will be available.

The staff is awaiting Commission direction on the staffs recommendation; however, the staff is moving forward with allowing sites to implement RAPT following Regulatory Guide 5.76, Physical Protection Programs at Nuclear Power Reactors, issued November 2020. No additional requirements would be imposed on a licensee to Realize the result apply a RAPT, and the staff will assess the sites implementation of the RAPT under the security baseline inspection program to ensure that the licensees physical protection program continues to meet the general performance objective of Title 10 of the Code of Federal Regulations 73.55(b).

others what you We will conduct training for inspectors, hold public meetings with Teach learned industry, and develop/revise guidance documents.

G-15

Be riskSMART: Guidance for Integrating Risk Insights into NRC Decisions TechnicalResearch Regulatory Readiness for Emergent Technical Issues clear about the How does the agency ensure regulatory readiness for technical Be problem issues not currently addressed in business line?

What can go

• The staff identifies a balanced portfolio of future-focused right? research.

• Industry proposes an advance technology for which the NRC is not ready to provide a regulatory decision in a timely What can go manner (i.e., length of time to provide to regulatory wrong?

decision/guidance); additive manufacturing, advanced nonlight-water reactors.

What can go right?

• The staff is able to provide more efficient review of novel Spot What are the technologies.

consequences?

What can go wrong?

• The staff will not be prepared for novel technologies.

What can go right?

• Likely given the staffs current processes and guidelines.

How likely is it?

What can go wrong?

• Very likely given the industrys move to modernize plants to run more efficiently.

• Be cognizant of novel technologies and industry prioritization to implement new technologies. Begin regulatory research in Manage what you can high-priority/low-knowledge areas. Conduct systematic internal review of knowledge/capabilities.

• Select a balanced research portfolio to address advance technologies and provide opportunities to gain additional Act on a decision understanding in areas of high knowledge (reduce uncertainties).

• The staff should engage with industry on state-of-the-art and forecasted rollout date on a given periodicity commensurate Realize the result with making timely program/budget decisions. Coordinate NRC priorities of high/low-knowledge areas across offices.

• Hold Office of Nuclear Regulatory Research seminars and others what you workshops to communicate status of novel technologies and Teach learned current gap/challenges to address safety/security issues.

May need to hire more experts in a particular field.

G-16

Be riskSMART: Guidance for Integrating Risk Insights into NRC Decisions TechnicalDispositioning Emergent IssuesControl Rod Drive Mechanism Control rod drive mechanism (CRDM) thermal sleeves are subject to clear about the Be previously unknown wear and fatigue. What should our regulatory problem response be, if any?

• We have the opportunity to demonstrate that we can make What can go quick decisions on how to disposition emergent issues right?

consistent with their safety significance.

What can go

• CRDM thermal sleeve wear or failure could cause control wrong? rods to fail to insert.

What can go right?

• Improve the NRC's reputation for dispositioning emergent issues. Ensure the efficient use of staff and licensee resources.

What can go wrong?

What are the

• If sufficient numbers of control rods fail to insert in certain consequences?

areas of the core, minor plant transients could result in anticipated transient without scram (ATWS) events, and accidents could have more severe consequences. These could potentially result in the failure of fuel rods, the reactor coolant pressure boundary, or both.

Spot What can go right?

• High likelihoodWe have good tools available for making risk-informed decisions (e.g., LIC-504, Integrated Risk-Informed Decision-Making Process for Emergent Issues, which was actually used for this process). What can go wrong?

What can go wrong?

• Low likelihoodA conservative lower bound was placed on the number of control rods needed to fail to insert to cause an How likely is it?

issue, based on pressurized-water reactor (PWR) ATWS and accident analyses. The probability of this number of failures during an operating cycle was then modeled based on the available wear data from inspections. This probability was then used as an input to plant standardized plant analysis risk (SPAR) models (as a new initiator for ATWS events) to determine the consequences for plants most likely to be affected. The change in core damage frequency was found to be less than 1E-05 for the plants analyzed.

What can go right?

• Make sure we use the best tools available, ensuring we use them appropriately.

What can go wrong?

Manage what you can

• Licensees committed to measure control rod thermal sleeves to make sure that they were within specifications provided by the vendor. These measurements should identify sleeves that have experienced enough wear to potentially fail and allow them to be replaced.

Based on the low probability and consequences of the challenges, we Act on a decision decided to disposition the issue with a smart sample of affected plants.

To date, the inspection program has been successful at identifying affected CRDM thermal sleeves, and plants have developed the Realize the result means to replace them as needed. The issue continues to be ongoing, and the exact mechanism driving the wear has not yet been G-17

Be riskSMART: Guidance for Integrating Risk Insights into NRC Decisions identified. A new mechanism that could cause CRDM thermal sleeve failures was also found (fatigue cracking of the thermal sleeves);

however, the staff found that the consequences of this were bounded by the original analysis.

The staff documented the decision in a memorandum written under others what you Teach LIC-504. The staff also wrote a paper for presentation at the learned SMiRT-25 conference.

G-18

Be riskSMART: Guidance for Integrating Risk Insights into NRC Decisions TechnicalGeneric Issue and Extended Power Uprate Licensing Action The NRC identified a generic issue late in the review of the extended power uprate (EPU) for Turkey Point Nuclear Generating Station: a thermal conductivity degradation (TCD) issue that impacted the clear about the emergency core cooling system (ECCS) peak cladding temperature Be problem (10 CFR 50.46, Acceptance criteria for emergency core cooling systems for light-water nuclear power reactors). Should the NRC issue the Turkey Point EPU before the resolution of the generic TCD issue?

What can go A1. Proceed with issuing the EPUthe 2-year metric is met.

right?

B1. The Turkey Point peak cladding temperature could increase more than 50 degrees.

What can go wrong? B2. The Turkey Point peak cladding temperature could increase more than 2,200 degrees.

What can go right?

A1. Do not have to report missing the metric to Congress.

What can go wrong?

What are the B1. For significant errors (more than 50 degrees), the licensee is Spot consequences? required to provide a report to the NRC within 30 days and include a proposed schedule for providing a reanalysis with the report.

B2. Not in compliance with 10 CFR 50.46.

What can go right?

A1. High likelihood of not reporting a metric if not missed.

What can go wrong?

B1. High likelihood of submitting a report to the NRC, but the How likely is it?

timing (30 days vs. 1 year) and the licensees followup actions are dependent on the severity of the error.

B2. Dependent on Turkey Points current peak cladding temperature analysis limit .

What can go right?

A1. Issue a license condition that would need to be satisfied before implementation of the EPU.

Manage what you can What can go wrong?

B1/B2 Discipline.

The project manager (PM) had meetings with the technical staff, management, and licensee to discuss this approach. This approach Act on a decision satisfied the reviewers concerns, was acceptable to the licensee, and prevented any further delay of issuance of the application.

As a result, the NRC issued the Turkey Point EPU within 15 months, Realize the result which is 3 months less than the average review time.

The Turkey Point PM who managed the EPU was identified as the others what you TCD issue PM for resolution. The PM worked with the other plant Teach learned PMs to gather the necessary information for each site to determine next steps for resolving the TCD issue.

G-19

Be riskSMART: Guidance for Integrating Risk Insights into NRC Decisions TechnicalPilot Resident Inspector Site Coverage ChallengeTypically, two resident inspectors are assigned to each of the power reactor sites, and at least one of these inspectors is required to be able to respond to the site shortly following an event, such as a reactor trip. However, sometimes, neither resident inspector is available, particularly during weekends, due to annual leave plans or other reasons related to work-life balance. When that happens, the resident inspectors typically identify another qualified inspector who can provide site coverage.

Current practices among the Division of Reactor Projects (DRP) Branch Chiefs vary, with some requiring that an inspector who can respond to an event within a few hours be identified for each site, while others allow a single inspector to provide event response coverage at two sites.

A question was recently raised about the consistency of administrative practices between DRP branches and whether, in the interest of work-life balance, a single, qualified inspector could provide coverage for more than one site, knowing that if events were to occur simultaneously at two (or more) sites, it would not be possible for this single inspector to respond to all these sites at once, and a response to all but one of these sites would be delayed.

For example, Palisades Nuclear Generating Station and Donald C. Cook Nuclear Generating Station are within about a 45-minute drive of one another. If both resident inspectors at D.C. Cook are unavailable over a weekend, and one of the Palisades resident inspectors is also unavailable, could the Palisades senior resident inspector provide event response coverage at both Palisades and D.C. Cook?

A question of approval authority was also identified. For example, if a single inspector is to be authorized to provide event response coverage at more than one site, what level of supervision is needed for that authorization? In addition, under what specific circumstances should that authorization be elevated to higher levels of management? Further, what notification requirements to higher levels of management should be established?

If qualified inspectors are permitted to cover more than one site for event response, site response guidelines that can be consistently and confidently applied for cases when resident inspectors are known to be unavailable to provide event response coverage at their assigned site will be required. The specific question of whether a single inspector can provide site coverage at more than one site is of primary consideration.

Potential SolutionDevelop expectations for the coverage of multiple sites for event response using a single qualified inspector.

clear about the Should qualified inspectors be permitted to cover more than one site Be problem for event response?

By permitting multisite coverage with a single inspector, a greater What can go work-life balance can be ensured. Also, the burden of identifying right?

other inspectors to provide coverage is eliminated.

A site response delay could occur for simultaneous site events to which a single inspector is responding. An alternate responder would Spot What can go need to be identified following simultaneous events and, depending wrong?

on the site location, could delay a site response beyond the current 2-8-hour guideline.

What are the The consequence of simultaneous events at sites with a single consequences? inspector providing site response is a delay in this response. This G-20

Be riskSMART: Guidance for Integrating Risk Insights into NRC Decisions response delay could result in the delayed ability to independently assess and verify the licensees actions following an event. It could also adversely impact communications of the event and the licensees associated response to NRC management and staff tasked with determining an NRC response posture and making protective action recommendations to external State and local government officials.

Events, such as trips and scrams, requiring an onsite response by an inspector are relatively infrequent; typically, no more than just a How likely is it? couple times a year. Therefore, the likelihood of two or more sites experiencing simultaneous events that require an onsite response by an inspector was judged to be very low.

A number of actions can be taken. The most straightforward would be to not authorize multisite coverage under any circumstances.

Absent that action, though, other mitigating actions that provide an atmosphere more friendly to work-life balance can be taken. For example, when deciding whether to have multisite coverage, consider Manage what you can site safety performance; the potential for external events, such as adverse weather, to occur; and activities associated with elevated risk, such as critical refueling outage activities. To manage risk, consider a limitation on the number of sites and duration of the coverage. Limiting multisite coverage to only resident inspectors at their backup sites would also mitigate potential risk.

The DRP Branch Chiefs held a series of internal discussions and conducted benchmarking with regional counterparts in the other three regions. Following these discussions using the draft Be riskSMART framework and benchmarking efforts, the Branch Chiefs developed the following rules of practice, which are being implemented:

• The Branch Chiefs agreed that it was acceptable for a single resident inspector to provide event response coverage at both his or her site as well as another site nearby (e.g., backup site) over a weekend and under the ideal Act on a decision conditions of routine plant operations, clear weather, and acceptable site performance. The Branch Chiefs also agreed that approval for this coverage rested with them and that no additional notifications or approval was required.

• For other cases, the Branch Chiefs agreed that multisite coverage would be best handled on a case-by-case basis and that notification and approval (i.e., buy in) from the DRP Director or Deputy Director was prudent.

This decision was communicated and well received by the resident Realize the result inspectors.

This decision, including the use of the Be riskSMART framework to others what you Teach make the decision, was shared with the other regions so they could learned consider adopting something similar.

G-21

Be riskSMART: Guidance for Integrating Risk Insights into NRC Decisions TechnicalCybersecurity Onsite Inspection During a Public Health Emergency ChallengeThe NRC is deciding whether to conduct an onsite inspection of the cybersecurity program at Columbia Generating Station in Region IV during the COVID-19 pandemic.

Potential SolutionThe NRC will conduct the onsite inspection with appropriate precautions in place.

Given the ongoing COVID-19 PHE, should the inspection team for clear about the the Columbia Generating Station cybersecurity full implementation Be problem inspection travel to the site for the first week of the inspection as originally planned?

• Inspectors will have eyes on and knowledge gained of cyber configuration and complete inspection as scheduled.

What can go right?

• Inspectors will review safeguards material supporting the inspection.

What can go

• Licensee has an unknown weakness that goes uncorrected.

wrong?

• Team members are exposed to COVID-19.

What can go right?

Complete inspection as scheduled.

What can go right? Gain confidence in licensees cybersecurity program implementation.

What can go wrong?

What are the If the inspection is delayed, continued cyber vulnerabilities could exist consequences?

that put the plant at increased risk.

What can go wrong?

Spot Team members or licensee employees get sick (or worse).

COVID-19 exposure could lead to family separation after inspection or impair return travel.

What can go right?

It is very likely that if the team travels to the site, the inspection will be completed (nearly 100 percent).

What can go right?

If the team completes the inspection, the NRC will have confidence in the licensees cybersecurity program (100 percent)

How likely is it?

What can go wrong?

Low likelihood, especially because of defense in depth inherent to the cybersecurity program.

What can go wrong?

There is a nonzero probability that an individual is exposed to the COVID-19 virus. There is no known method to quantify this risk, but it is anticipated to be small for each individual.

• Have a team member work remotely and coordinate with team; limit the minimum number of licensee personnel Manage what you can (Region IV trainee will not travel).

G-22

Be riskSMART: Guidance for Integrating Risk Insights into NRC Decisions

• Discuss lessons learned with Region I after the inspection of the James A. FitzPatrick Nuclear Power Plant.

• Discuss lessons learned with operations branch following Arkansas Nuclear One (ANO) exam.

• Use Centers for Disease Control and Prevention safe health practices. Take precautions to protect staff and establish stop criteria.

• Monitor cases daily and reassess decision prior to travel.

Act on a decision Present framework to management for decision.

Develop detailed risk management instructions (including stop Realize the result criteria) for the team before travel and monitor local conditions for changes before travel.

Share lessons learned as desired after the inspection. Use the others what you Teach Region 1 (FitzPatrick), OLB to ANO, and this inspection to inform learned other onsite inspection decisions.

G-23

Be riskSMART: Guidance for Integrating Risk Insights into NRC Decisions TechnicalForce-on-Force Exercise During a Public Health Emergency Should the outcome of a force-on-force (FOF) exercise be removed clear about the Be from the FOF significance determination process screening during problem modified exercises occurring during the COVID-19 PHE?

Opportunities Challenges Since we are Removing the We must The public using a different FOF outcome ensure that could perceive method to removes some licensees do that inspection perform FOF of the pressure not have activities are exercises due to on exercise weakness that not as robust as COVID-19 PHE outcomes to go they were safety concerns, balance the undocumented before the What can go we can leverage artificialities during the COVID-19 wrong/right? this opportunity that will need COVID-19 PHE.

to identify new to be used to PHE.

ways to refine minimize the the inspection risk of program after COVID-19 the COVID-19 transmission.

PHE.

Application of This could Not citing a If not this concept result in licensee for an communicated would provide improved ineffective appropriately, data to show testing of exercise that this could give whether licensees could have the appearance changes to the protective resulted in a that the NRC is significance strategies, greater than doing less and determination including areas green finding not meeting its process that are not could change inspection Spot approach should normally how we assess objectives.

What are the be proposed tested, to a licensee in consequences? after the enable the action COVID-19 PHE. licensees to matrix.

Also, it will apply provide the resources in inspection team an efficient and with an added effective opportunity to manner and mitigate ensure indeterminate readiness to exercise protect the outcomes. site.

High likelihood Medium Low Medium The FOF likelihood likelihoodThe likelihoodThe inspection Fewer licensee previous or public is program is escalations of current FOF engaged in evolving, even DBT tactics cycle have had Office of before the could occur if no greater than Nuclear How likely is it? advent of the the green findings Security and COVID-19 PHE consequence as a result of an Incident (see of a finding is ineffective Response SECY-17-0100). not tied to exercise (NSIR)

Using this exercise outcome. The activities.

opportunity to outcomes. last white not screen finding was G-24

Be riskSMART: Guidance for Integrating Risk Insights into NRC Decisions exercise issued in outcomes calendar through the year 2014.

significance determination process using the modified exercises due to the COVID-19 PHE will likely provide valuable opportunities for insights that could be used to change the FOF inspection program after the COVID-19 PHE.

Ensure we Discuss this Other NSIR interacted leverage what with industry deficiencies with the public we can in the early and often identified during during a procedure being to understand the inspection meeting on used during the if modifying will be screened November 12, COVID-19 PHE enforcement in accordance 2020, to convey in order to better approaches with normal the basis for inform the would screening our planned overall FOF influence the processes that approach, Manage what you can program. licensees could result in gather input review process findings. from the public, for scenarios and discuss used to any perceived evaluate a or actual licensees consequences protective from this strategy. change.

Present pros and cons to management for decision by 10/X, with an Act on a decision update on 10/X.

Develop detailed procedural guidance that incorporates removing Realize the result findings related to exercise outcomes.

Share lessons learned from these exercises that occurred during the others what you Teach COVID-19 PHE. Use the information learned to better risk-inform the learned FOF program.

G-25

Be riskSMART: Guidance for Integrating Risk Insights into NRC Decisions LegalTiming of the Office of the General Counsels Legal Review ChallengeOn occasion, the Office of the General Counsel (OGC) devotes time and resources to the legal review of staff initiatives (such as proposals to revise staff guidance or explore policy changes) only to have those documents abandoned or significantly modified following feedback from external stakeholders that raise perspectives that were previously not contemplated.

When these developments make the legal analysis outdated or unnecessary, associated OGC time and resources have to be recommitted; this is time that could have been devoted originally to other matters.

Potential SolutionDefer formal legal review of certain staff products until after they have been discussed with external stakeholders.

Should OGC defer formal legal review of certain staff initiatives until clear about the Be after the staff has had some early engagement with external problem stakeholders?

• The staff receives an early external stakeholder perspective.

If external feedback persuades the staff that the contemplated initiative is unwise or unnecessary, the staff modifies or abandons the policy/proposal before OGC needs What can go to devote time and resources to a formal legal review.

right?

• External feedback persuades the staff to move forward with the policy/proposal, and subsequent OGC review identifies no significant legal concerns with pursuing it.

• The staff receives favorable external stakeholder feedback on What can go the policy/proposal document, and subsequent OGC review wrong?

identifies significant legal concerns.

What can go right?

• OGC resources are used efficiently, particularly if the policy/proposal is abandoned or significantly modified.

• The staff gains the benefit of early external stakeholder feedback.

Spot What can go wrong?

• There is potential for embarrassment or confusion if the proposal ultimately raises significant legal concerns, as the What are the staff may need to modify or abandon the policy/proposal after consequences?

having publicly discussed its intentions (and potentially receiving positive feedback or having stakeholders take steps in reliance on the preliminary proposal).

• A revised policy/proposal resulting from the after-the-fact legal review has the potential to indirectly reveal attorney-client privileged information, particularly if it was communicated that the policy/proposal document was preliminary pending a legal review.

• The likelihood that a policy/proposal will raise significant legal concerns is case specific and dependent on the significance How likely is it?

of the particular policy/proposal.

G-26

Be riskSMART: Guidance for Integrating Risk Insights into NRC Decisions

• Policy/proposal documents on certain topics, especially on topics that may intersect with matters in litigation, tend to increase the potential for legal concerns.

• If the external stakeholder perspective is particularly significant to the rationale for a proposal, consider opportunities for a limited and less resource-intensive fatal-flaw legal review conducted before initial external engagement.

• Pilot such reviews in scenarios without baseline legal sensitivities (i.e., matters not intersecting with active Manage what you can litigation).

• If a policy/proposal document is shared publicly and OGC later identifies significant legal issues, OGC may still be able to advise the staff on how to articulate its change in direction in a careful manner that minimizes the potential for disclosure of attorney-client privileged information.

• Proactively confirm common understanding with client offices on which proposals require OGC review and concurrence before engaging with external stakeholders.

• Promote consistent engagement with client offices to discuss when early OGC engagement with policy/proposal Act on a decision development would be most useful.

• If client objectives or timelines for public engagement are incompatible with a detailed OGC legal review, consider whether some limited fatal-flaw OGC legal review would be risk-beneficial.

• Assess whether experience with a pilot initiative indicates whether this new approach should be continued and expanded.

• Even on policy/proposal documents determined to be lower risk, the staff would still have discretion to seek normal OGC Realize the result review in advance of public engagement.

• When legal review occurs after initial stakeholder engagement, clearly communicate to stakeholders the preliminary status of the policy/proposal.

The Be riskSMART framework can be used to evaluate changes in others what you Teach the timing and scope of legal reviews to increase agency efficiency learned and effectiveness and to become a modern, risk-informed regulator.

G-27

Be riskSMART: Guidance for Integrating Risk Insights into NRC Decisions Corporate - Office Restack Challenge - To develop a time-saving approach to the office restack move process

Background

The NRC committed to release one floor of the Two White Flint North (TWFN) office building to the General Services Administration (GSA) by 09/30/20. This action is consistent with the NRCs ongoing efforts to reduce corporate support costs and to right-size our office space in support of federal-wide initiatives to reduce office space across the country. During the floor release process, the COVID-19 public health emergency (PHE) occurred, which delayed or ground all such projects to a halt. Finishing the project by the due date would require carrying out the office restack process in less time than usual and to complete clear about the several aspects of the project virtually for the first time. Therefore, it Be was necessary to determine if there were any part of the office restack problem process that could be expedited, delayed, or avoided without imposing unacceptable risk.

What is required?

• Nuclear Energy Innovation and Modernization Act (NEIMA) corporate support costs cap

• Requirements related to federal agency utilization of real estate from Reduce the Footprint (2015), GSA guidance, et al.

• Collective Bargaining Agreement (CBA), Article 34

• NRC COVID-19 PHE Task Force guidance

• Planning Phase: All processes verified and approved prior to office selection.

What can go

• Office Selection Phase: Selection goes through without delay.

right?

• Pre-move activities and Move Phase: Timely completion saves money.

• Planning and Office Selection Phases: Due to limited/reduced staff, time, access to the space itself, and the nuanced complexity of the restack, planning could be inadequate or incorrect resulting in delays in the selection process, subsequent phases, and project.

What can go

• Pre-move activities and Move Phase: Due to the logistics of wrong?

physically vacating space (packing, moving, storing, delivering and unpacking in the correct workstation, property and equipment) improper or lack of planning and execution can result in problems Spot and confusion.

Consequences of what can go wrong:

• If elements of the planning and office selection phases are missed, then CBA commitments may be impacted.

• If the 09/30/20 target date is missed, then NRC would be required to pay rent of $80K each month (unbudgeted), experience a reputational impact to future transactions with GSA, and complicate What are the GSAs efforts to backfill the space (GSA identified a backfill tenant consequences?

who anticipated to take the space in early FY 2021).

Consequences of what can go right:

• If the restack is completed timely, then the release of a floor to GSA will be timely, enhance NRCs reputation for fulfilling requirements on time, and reduce the NRCs rent and real property related costs by average of $1.2M per year.

G-28

Be riskSMART: Guidance for Integrating Risk Insights into NRC Decisions Based on results of prior office restacks and contributing factor of PHE:

• CBA commitments - the likelihood for noncompliance increases How likely is it?

with any error in the planning and office selection phases

• Schedule delay - high likelihood Manage the risk of what can go wrong and increase opportunities for what can go right:

• Planning and Office Selection Phases: Carry out the processes in accordance with requirements virtually. Risks outweigh potential benefits from attempting to save time here.

• Pre-move activities and Move Phase: The greatest benefit can be gained by managing risk in the move phase. The potential benefit of improving timeliness mitigates the high likelihood of schedule delay. Additionally, the move phase impacts several hundred employees and the date of the floor release. Manage the risks inherent to packing, moving, and storing property and equipment by Manage what you can utilizing experienced movers and technicians. Manage the risk of conducting some activities virtually for the first time by increasing communication and coordination with office points of contact.

In the standard process for office restack moves, the planning and office selection phases are completed prior to a larger-scale physical move at the end. Given that any approach to time savings could not affect the planning and office selection phases, staff developed a strategy to save time by overlapping the office selection and move phases. Office moves would be conducted incrementally as the office selections were still being completed.

Decisionmaker: It is within the purview of the Office of Administration (ADM) Project Manager and subject office points of contact (Office of Research (RES), ADM, Office of Nuclear Security and Incident Response (NSIR), Office of the Chief Financial Officer (OCFO), Office Act on a decision of the Chief Information Officer (OCIO)) to coordinate the decision of when the office physical move occurs.

Decision: During the moves, conduct the office moves incrementally instead of at one time (large-scale). For the turnover process after the moves, conduct the tour of the space virtually with GSA if agreeable.

Implementation: ADM and OCIO collaborated with the subject office point of contact (RES, ADM, NSIR, OCFO, OCIO) to conduct moves as the office selections occurred. Progress was monitored as part of performance management for the White Flint Campus Facilities Projects in the Administrative Services Product Line.

Result:

Realize the result

• Efficiency gained and substantial completion of office restack moves by 09/30/20

• Decommissioning related activities in the October timeframe

• Turnover of floor to GSA completed 11/02/20 The timely completion allowed the agency to realize nearly the full cost savings of $1.2M in FY21 plus projected savings in future years.

• Innovation established a new approach to office restack - carry out others what you Teach some activities in the office selection and move phases learned concurrently and virtually to save time G-29

Be riskSMART: Guidance for Integrating Risk Insights into NRC Decisions CorporateForward Funding ChallengeThe NRC is seeking to improve efficiency around the agencys forward funding.

Background

Forward funding levels provide an indicator of prior-year obligations available to spend. Forward funding can be expressed in months of available funds or dollars as prior-year unliquidated obligations. It is an issue across the Federal complex and not limited to the NRC.

Examining forward funding provides an opportunity to identify areas where the Federal Government can improve and maximize the use of resources. The Office of the Chief Financial Officer (OCFO) is taking clear about the a global approach to efficiency around the agencys forward funding Be problem by leveraging recent improvements in automation and budget execution processes.

What is required?

While there is no prescriptive requirement, the NRC has the following guidance in Management Directive 4.2, Administrative Control of Funds, dated June 22, 2015: As a general rule, the total available funds (expressed in months of projected expenditures) should equal the number of months remaining in the current year plus four.

Forward funding can be a useful financial tool to meet legal requirements and to support things that we could not do without, such as the following:

• legally mandated services

• firm/fixed-price contracts that require advance funding What can go right?

• research agreements that require advance funding, sometimes for multiple years

• contracts for critical goods and services

• Based on key points provided by offices during fiscal year (FY) 2019 The following challenges result in forward funding:

• timing of the availability of funds (e.g., small, incremental funding can delay progress and completion of the contract)

Spot

• expired contracts pending closeout for which funds have not What can go been deobligated wrong?

• issues that affect the spending plan (e.g., delay in responses to requests for additional information)

• Based on key points provided by offices during FY 2019 Not enough forward funding can result in the following:

• funding gap without annual appropriation or continuing resolution What are the

• stop-work situation consequences?

G-30

Be riskSMART: Guidance for Integrating Risk Insights into NRC Decisions Too much forward funding can result in the following:

• higher priority work not receiving funds (avoid large balance on lower priority work)

• possible opportunities for budgetary savings Based on the trend over FY 2017-FY 2019, forward funding will occur. During FY 2020, OCFO is working with offices to categorize the forward funding balances and provide more insight into the How likely is it?

likelihood of challenges and opportunities. Additionally, the likelihood of forward funding and its optimal level depend on the budget environment.

Manage the risk of what can go wrong and increase opportunities for what can go right:

• Leverage automation and standardization to improve transparency into the categories that comprise the forward funding balances. After we have identified the categories, we will be able to identify areas where efficiencies can be gained.

• Use the Mid-Year Resource Review and Resource Reallocation process to meet shortfalls and emergent needs Manage what you can from fact-of-life changes while remaining within authorized control points.

During the third quarter of FY 2020, in response to the COVID-19 PHE, the OCFO revisited the manage what you can step with two additional actions:

(1) June second-round funds return (2) modification to FY 2020 obligation metric (excludes funds returned by June 15)

Actions and progress in FY 2020:

OCFO worked with offices to develop a repeatable process whereby Act on a decision offices review and categorize prior-year balances. This review is updated at least quarterly.

Results:

• Identified major categories for forward funding.

• Developed the forward funding dashboard, an automated daily report (OCFO, Division of Budget, Funds Control and Analytics Branch, demonstrated the dashboard at the Corporate and Mission Support Monthly Meeting on September 17, 2020).

Realize the result Next Steps:

OCFO will continue to work with offices to categorize prior-year balances on a periodic basis (quarterly) and begin categorizing current-year balances. We plan to provide interactive dashboards for management to review balances at multiple levels, including categorization. This will allow offices to focus on those balances they have influence over and make more informed budget execution decisions in the current FY.

• Proactively manage balances as the year progresses rather others what you Teach than as a metric at the end of the year.

learned G-31

Be riskSMART: Guidance for Integrating Risk Insights into NRC Decisions

• OCFO recognizes that there is more work to be done on how budget execution influences budget formulation.

• OCFO plans to continue partnering with allowance holders to evaluate forward funding needs using a risk-informed approach and to improve automated reporting toward a goal of one-stop shop for financial management reports on full-time equivalents, budget execution, and forward funding.

G-32

Be riskSMART: Guidance for Integrating Risk Insights into NRC Decisions CorporateSummer Recruiting Process ChallengeHistorically, the NRCs summer intern recruiting process has lagged most other businesses pursuing exceptional, high-performing students for summer employment. As a result, students who otherwise have a strong interest in working for the NRC receive and accept an earlier offer and work somewhere else. In addition, the coordination of multiple offices pursuing the same summer student has, at times, been very challenging. A well-coordinated student summer intern recruiting process would provide for the selection of exceptional, high-performing students very early in the recruiting cycle.

Potential SolutionA new centralized and streamlined summer student recruiting process is being implemented that identifies and selects high-quality students for summer intern positions over a very short period early in the recruiting cycle. The process includes the use of tiger teams fully dedicated to the effort of reviewing candidate resumes, identifying students to interview, conducting these interviews, and completing a reference check for selected students.

Other important aspects of this streamlined process include completing interviews by a single interviewer, conducting interviews by phone, and requiring only a single reference check.

clear about the Should the NRC adopt a new centralized and streamlined summer Be problem recruiting process using tiger teams?

What can go A larger number of highly qualified students will be interviewed for right? summer intern positions at the NRC than under the current process.

The overall suitability of a student being interviewed may be What can go misjudged.

wrong?

Highly qualified students who may otherwise accept offers elsewhere will accept summer intern offers at the NRC. The opportunity to convert these summer interns to co-op students or full-time What are the employees also exists. An unsuitable student may be hired.

Spot consequences? Because the summer intern program is only for a short period, typically 3 months, the consequences of hiring an unsuitable student is relatively low. In addition, a decline in organizational effectiveness for the period that the unsuitable student is employed will occur.

It is likely that in some cases the revised process will result in selection of less-qualified candidates because telephone interviews How likely is it? are performed by a single interviewer with a single reference check.

The increased ability to interview exceptional students that the NRC has previously been unable to interview also exists.

Required elements of the process, which include the submission of a resume, the submission of college transcripts with minimum grade point average requirements, the performance of interviews by Manage what you can experienced and skilled NRC supervisors, and the completion of a reference check serve to mitigate the potential for hiring an unsuitable student.

The NRC implemented a new centralized and streamlined summer student recruiting process. Four tiger teams of two to three supervisors were formed to fill 84 summer intern requests in the Act on a decision functional areas of Information Technology/Cybersecurity, Mission and Corporate Support, Health Physics and Nuclear Engineering, and Other Engineering and Science. Interviews were conducted in mid-November, with offers starting in early December.

Realize the result The NRC hired 71 summer interns across the agency.

others what you The result of the effort will be shared. Insights are still being gathered Teach learned and will be used to propose revisions to the process.

G-33

Be riskSMART: Guidance for Integrating Risk Insights into NRC Decisions CorporateUse of the NRC's Collaborative Learning Environment to Generate Exams ChallengeShould the Technical Training Center (TTC) use the NRCs Collaborative Learning Environment (CLE) to generate exams for instructor-led training courses?

Background:

Due to the agency upgrade to Microsoft Windows 10 operating system, the legacy exam administration software that was used at the TTC for exam development, grading, and reporting is no longer supported. Consequently, the TTC staff had to identify a solution that was compatible with the new Windows 10 and Office of the Chief Information Officer (OCIO) requirements.

clear about the Be Specifically, the TTC was challenged to identify and develop new problem exam administration software that could be used as an exam bank and support exam development, test administration, and grading/reporting as required by HRTD operating procedures. One potential solution is to use the NRCs CLE to generate exams for instructor-led training courses. By creating CLE course pages for the applicable instructor-led courses, the CLE could be populated with the existing LXR exam banks for each course, which in turn could then be used to generate both hard copy and online exams.

• By successfully implementing plans to use the CLE for exam generation, administration, and grading, TTC staff would be able to meet all HRTD operating procedure requirements for What can go student testing and track required data associated with right? personnel qualifications. Additionally, due to the CLEs robust design, what used to take hours to draft into a course completion report could now be done in real-time by the CLE exam software suite.

• Exam configuration errors could lead to student inability to complete an exam.

• Student log-in issues may prevent an exam from being administered.

What can go wrong?

• Internet connectivity issues may prevent an exam from being Spot administered.

• Exam confusion could lead to incorrectly answering exam questions.

What can go right?

• Enhancing the agencys reputation for licensing new technologies will engender goodwill with external stakeholders.

What are the

• Setting clear expectations will allow us to develop a consequences?

comprehensive plan, provide guidance to the industry, and focus efforts to save resources.

• Coordinating with the industry, provided independence is not compromised, will provide clarity between the NRC staff and G-34

Be riskSMART: Guidance for Integrating Risk Insights into NRC Decisions applicants. This would result in resource savings due to a reduction in back-and-forth communications.

What can go wrong?

• A student may be unable to take an exam that is scheduled.

• We may receive negative student feedback related to the training provided by the TTC staff.

With adequate contingency planning (part of Managingwhat you How likely is it? can), the likelihood of experiencing the problems related to using the CLE for examinations should be relatively low.

Manage the risk of what can go wrong and increase opportunities for what can go right by taking the following measures:

• The TTC issued hardcopy exams concurrent with the online version.

• The TTC used human performance tools, such as self-checking and peer checking, in the design and testing processes before going live with the first pilot exam to ensure the exams behaved as expected.

• During the January/February 2020 offering of the R-304B, Manage what you can GE BWR Technology course, the TTC staff decided to pilot an exam taken online by the course attendees for their final exam. Based on student feedback, each student still received a hardcopy exam so that they could mark up questions as desired, but they were required to log their answers in the online version of the exam. To familiarize the students with using the CLE for exams, the TTC staff converted the daily review assignments into daily quizzes that the students could access for completion after the respective course presentations. These review assignments were structured in a similar fashion to the final exam used for the pilot.

The TTC staff successfully administered a pilot exam to 10 students enrolled in the GE Boiling-Water Reactor (BWR) Technology course, Act on a decision with the intention to fully transition to the CLE-based exam if the pilot proved to be successful.

Apart from a couple of minor system configuration issues that were corrected in real time and Internet connectivity issues, the pilot exam went very well. All students were able to successfully complete their Realize the result exam, and the test statistics were automatically generated by the CLE system, saving TTC instructors from manually generating the data.

The CLE-based exam program has been fully implemented.

Lessons learned from administration of the pilot exam using the CLE others what you will be shared with the internal TTC staff, and a CLE test Teach learned construction/administration desktop guide will be drafted for future use.

G-35

APPENDIX H FACILITATED DIAGNOSTIC TOOLMEASURING CURRENT STATE OF USING RISK INSIGHTS H-1

D. Increase communication between management and staff (e.g., agency, office, and division meetings, drop-ins, digital/electronic communications).

E. Encourage a culture that is receptive to innovation and change.

F. Promote leadership model behaviors.

G. Practice speed of trust behaviors.

H. Perform integrated reviews or activities that consider the holistic perspective.

Diagnostic Tool: Who drives Risk-Informed Decisionmaking (RIDM) in an organization?

1. Everyone uses RIDM because it is part of 2. Everyone uses RIDM, but not consistently. 3. Everyone uses RIDM, but there is NRC culture; individual contributors drive duplication of effort because trust has the process and decisions. eroded.

Management has set an infrastructure and culture Management has set an infrastructure and culture that supports RIDM. Staff at all levels of the Management has set an infrastructure and culture High that supports RIDM. Staff at all levels of the organization use Be riskSMART. However, the that supports RIDM. Staff at all levels of the organization use Be riskSMART. Leadership infrastructure and culture across organizations are organization use Be riskSMART. However, the model behaviors are fully realized not consistent and therefore staff implementation is speed of trust behaviors are not realized; therefore, (e.g., participative decisionmaking, receptivity to not uniform. management is still directly involved in all new ideas and thinking). risk-informed decisions and drives outcomes.

Primary Actions: B, D, G, H Primary Actions: A Primary Actions: F, G

4. Individual contributors sometimes use 5. Individual contributors and management 6. Individual contributors sometimes use RIDM, but not consistently. sometimes use RIDM, but not consistently. RIDM, but management drives the process and decisions.

Management has set some infrastructure and culture Medium Management has set some infrastructure and that supports RIDM, but it is not consistent. Some Management has set some infrastructure and culture culture that supports RIDM, but it is not consistent. individual contributors take initiative to use Be to support RIDM. Because management is directly Individual contributors only take initiative to use Be riskSMART, but management is sometimes driving involved in all risk-informed decisions and drives riskSMART in some cases. the risk-informed decisions and outcomes. outcomes, only some individual contributors use Be riskSMART. The staff does not always consider risk information early in the process.

Primary Actions: B, C, D, H Primary Actions: B, C, D, F, G Primary Actions: B, C, D, F, G, H

7. No one uses RIDM. 8. Management sometimes drives RIDM, but 9. Management originates and micromanages not consistently. all risk-informed decisions.

Management has set some infrastructure and Management has set some infrastructure and culture Management has set some infrastructure and culture Low culture that supports RIDM, but it is not consistent. that supports RIDM, but it is not consistent. that supports RIDM, but RIDM only occurs from the Individual contributors own Be riskSMART Individual contributors and management do not use Individual contributors have not taken the initiative to top down (i.e., management is directly involved in all Be RiskSMART. RIDM is not part of the culture, use Be riskSMART. risk-informed decisions and drives outcomes).

and there is a reluctance to change. Individual contributors do not use Be riskSMART.

Leadership model behaviors are not realized.

Primary Actions: B, C, D, F, H Primary Actions: B, C, D, F, G, H Primary Actions: B, C, D, E, F, G, H Low Medium High Management owns Be riskSMART H-2

Be riskSMART: Guidance for Integrating Risk Insights into NRC Decisions APPENDIX I 2020 DIAGNOSTIC SURVEY RESULTS During 2020, each office and region applied the facilitated diagnostic tool to identify who in each U.S. Nuclear Regulatory Commission organization initiates or drives the use of risk insights in decisionmaking: the individual contributor or management. Below are the results of the current state of each office and region for applying risk insights. The y axis represents the number of responses for each office and region. The x axis tallies how each responder ranked his or her office/region current state for using risk insights, based on the diagnostic tool definitions (boxes 1-9) in Appendix H. [Note: To obtain an accurate baseline for each organization, a survey target threshold of 5 percent or greater than 10 people, whichever is higher, was established. Only the organizations that met this threshold are represented below.]

Overview I-1

Offices and Regions I-2

I-3 I-4 NUREG/KM-0016 March 2021 Be riskSMART - Guidance for Integrating Risk Insights into NRC Decisions Mirela Gavrilas, Jason Paige, Eric Duncan, Brian Harris, Candace De Technical Messieres, Elizabeth Bowlin, Rebecca Richardson, Reed Anzalone, Michael Gartman Office of the Executive Director for Operations Be riskSMART Initiative Team U.S. Nuclear Regulatory Commission Washington, D.C. 20555-0001 To become a modern, risk-informed regulator, the NRC focused on four transformational areas: (1) managing the workforce, (2) applying risk in decision-making, (3) generating innovative ideas to improve the way that NRC works, and (4) adopting new technologies and approaches to data analytics. The Be riskSMART framework supports the second transformation area by providing a systematic approach to making risk-informed decisions across disciplines. Be riskSMART combines traditional concepts, such as the risk triplet, risk management, the risk heat map and risk appetite, into a plain language framework that gives the staff confidence to apply and communicate risk-insights for all kinds of NRC decisions whether they are in the technical, corporate, or legal arena. This NUREG/CR provides detailed guidance on using the Be riskSMART framework and contains example case studies from across a series of disciplines.

OEDO Be riskSMART Initiative Be riskSMART Applying Risk in Decision-Making RIDM Risk-informed Thinking Risk Triplet

NUREG/KM-0016 Be riskSMART: Guidance for Integrating Risk Insights into NRC Decisions March 2021