ML19263A645

LIC-206, Revision 1, Integrated Risk-Informed Decision-Making for Licensing Reviews

ML19263A645
Person / Time
Issue date:	06/26/2020
From:	Gregory Suber Office of Nuclear Reactor Regulation
To:
Lee Samson, 415-3168
References
LIC-206, Revision 1
Download: ML19263A645 (43)

Text

OFFICE OF NUCLEAR REACTOR REGULATION LIC-206, Revision 1 Integrated Risk-Informed Decision-Making for Licensing Reviews Volume 200 Licensing Support Approved By:

Gregory F. Suber Date Approved:

June 26, 2020 Effective Date:

June 26, 2020 Certification Date:

June 26, 2025 Responsible Organization:

Division of Operating Reactor Licensing Office of Nuclear Reactor Regulation Primary Contacts:

Samson S. Lee Ian H. Tseng 301-415-3168 301-415-7964 Samson.Lee@nrc.gov Ian.Tseng@nrc.gov Shilp Vasavada 301-415-1228 Shilp.Vasavada@nrc.gov Summary: A new appendix is added. The appendix is entitled, Appendix C, Use of Probabilistic, Risk, and PRA Insights for Technical Reviewers.

Training:

General training on risk-informed decision-making:

iLearn Course ID_280148, Risk-Informed Thinking Workshop Training for Appendix B to LIC-206:

iLearn Course ID_427162, Integrating Risk into Regulatory Reviews Web-based)

Training slides for level of review in Appendix C to LIC-206: ADAMS Accession No. ML17209A108 ADAMS Accession Number: ML19263A645

ML19263A645 *by e-mail OFFICE NRR/DE/EMIB NRR/DORL/LPL1/PM NRR/DRA/APLC NRR/DORL/LPL1/LA NAME ITseng SLee MReisiFard LRonewicz DATE 04/22/2020 04/22/2020 04/22/2020 04/22/2020 OFFICE NRR/DORL/DD NRR/DRA/D NRR/DEX/D NRR/DSS/DD NAME GSuber MFranovich EBenner MJRoss-Lee DATE 05/14/2020 05/28/2020 05/15/2020 05/18/2020 OFFICE NRR/DNRL/DD NRR/DANU/DD NRR/DRO/DD(A)

NRR/DRMA NAME RCaldwell BSmith MYoung TGorham DATE 05/11/2020 05/15/2020 05/18/2020 06/11/2020 OFFICE NRR/DORL NAME GSuber DATE 06/26/2020

NRR Office Instruction LIC-206, Revision 1 Page 2 of 8

1.

POLICY It is the policy of the Office of Nuclear Reactor Regulation (NRR) to advance the nuclear reactor safety program towards becoming a more effective and risk-informed regulator.

It is NRR managements expectation to promote timely and effective decision-making that appropriately prioritizes resources that are commensurate with the respective safety significance of the activity. It is also NRR managements continued expectation that all regulatory activities are conducted in a manner consistent with the U.S. Nuclear Regulatory Commission (NRC or the Commission) Principles of Good Regulation (i.e.,

independence, openness, efficiency, clarity, and reliability) and Organizational Values (i.e., integrity, service, openness, commitment, cooperation, excellence, and respect).

The NRR management supports the agencys Be riskSMART framework (i.e., Spot, Manage, Act, Realize, and Teach).

This office instruction (OI) provides guidance applicable to considering risk-informed decision-making (RIDM) in licensing-related activities to enhance process efficiency and effectiveness. Integrating risk insights with traditional engineering approaches provides better-reasoned regulatory decisions to appropriately disposition issues that arise in all regulatory matters, including licensing activities. Staff in NRR are expected to use RIDM, when appropriate, to complement and enhance deterministic approaches to ensure a sound risk-informed regulatory decision is made. Additionally, a risk-informed approach enables us to focus our resources on the more significant issues and prevents us from diverting agency and licensee attention on low safety-significant issues.

Furthermore, all NRR staff are expected to use risk insights to improve communication on the significance of issues both within their organizations and with external stakeholders.

2.

OBJECTIVES This OI, along with the guidance in the attachments, provide staff who support NRR with a basic framework for considering risk insights in licensing and other licensing-related activities.

This OI will also support and enhance NRR efficiency and effectiveness by:

providing staff and management with an improved framework to consider risk insights in licensing activities using risk insights to determine the scope and depth of licensing reviews promoting consistency in considering and applying risk insights improving internal and external communications involving risk considerations increasing technical consistency on applying RIDM for similar licensing activities applying a holistic and integrated view of safety that considers defense in depth, safety margin, engineering margin, engineering judgment, probabilistic risk assessment, and other technical information An outcome of this OI is to continue efforts to support inclusion of risk considerations in staff and management mindset and cultural changes at different levels of the organization.

NRR Office Instruction LIC-206, Revision 1 Page 3 of 8

3.

BACKGROUND In 1995, the NRC published in the Federal Register (60 FR 42622) its probabilistic risk assessment (PRA) policy, which states that an overall policy on the use of PRA methods in nuclear regulatory activities should be established so that the many potential applications of PRA can be implemented in a consistent and predictable manner. In that policy document, the Commission stated that it believes the use of PRA technology in NRC regulatory activities should be increased to the extent supported by the state of the art in PRA methods and data and in a manner that complements the NRCs deterministic approach.

Staff Requirements Memorandum (SRM) SECY-98-144, White Paper on Risk-Informed and Performance-Based Regulation, provides a Commission-endorsed definition and interpretation of RIDM. In RIDM, the NRC staff uses the best available probabilistic and deterministic information. As defined in SRM-SECY-98-144, a risk-informed approach to regulatory decision-making represents a philosophy whereby risk insights are considered with other factors to establish requirements that better focus licensee and regulatory attention on design and operational issues commensurate with their importance to public health and safety.

In SRM SECY 19-0036, Application of the Single Failure Criterion to NuScale Power LLCs Inadvertent Actuation Block Valves, the Commission stated that in any licensing review or other regulatory decision, the staff should apply risk-informed principles when strict, prescriptive application of deterministic criteria such as the single failure criterion is unnecessary to provide for reasonable assurance of adequate protection of public health and safety.

On May 11, 2017, the Commission was briefed on risk-informed regulatory activities by nuclear industry representatives and the NRC staff. The briefing included a status of the NRCs and the industrys risk-informed initiatives and an overview of successes and areas of focus for advancing risk-informed regulation. On June 26, 2017, the Commission issued SRM-M170511, Briefing on Risk-Informed Regulation, and directed staff to provide the Commission with an information paper discussing its plans for increasing staff capabilities to use risk information in decision-making activities.

On November 13, 2017, the staff responded to SRM-M170511 by SECY-17-0112, Plans for Increasing Staff Capabilities to Use Risk Information in Decision-Making Activities. SECY-17-0112 provides the staffs proposal for increasing its capability to use risk information in decision-making and describes challenges toward further progress in RIDM and measures that the staff is taking to overcome these challenges.

The staff developed the RIDM Action Plan, Action Plan, Risk-Informed Decision-Making Operating Reactor Business Line, dated November 27, 2018 (ADAMS Accession No. ML18317A117), implementing the staffs proposal contained in SECY-17-0112 and continuing the development and inclusion of risk considerations in licensing reviews.

NRR staff was assigned specific tasks from the RIDM Action Plan and also conducted a pilot program with a sample of licensing actions during a trial period in the fall of 2018.

Phases 1 and 2 of the RIDM Action Plan reports, dated June 26, 2018, and

NRR Office Instruction LIC-206, Revision 1 Page 4 of 8 January 30, 2019, respectively, are available in ADAMS (Accession No. ML18169A205 and Package Accession No. ML19007A339, respectively).

This OI documents the guidance resulting from the RIDM Action Plan efforts. The staff plans to update the guidance in this OI as more experience is gained in considering risk insights in licensing and other licensing-related activities. The staff continues to develop more applications as a result of the RIDM Action Plan, which may be inserted as additional appendices to this OI or revisions to procedures as they are finalized. Thus far, Integrated Review Team Guidance for Technical Reviewers, as proposed in Phase 2 of the RIDM Action Plan, has been developed and added to this OI. Other licensing-related topics identified in Phase 2 of the RIDM Action Plan are power uprate, backfits, Technical Specification Task Force (TSTF) travelers, and technical assistance requests.

There may be additional topics, as appropriate, such as relief requests.

4.

BASIC REQUIREMENTS The NRC has a longstanding commitment to increase the consideration of risk insights.

It is managements vision to enhance the integration of risk information into the organizations decision-making practices and processes to improve the technical basis for regulatory activities, increase efficiency, and improve effectiveness. This OI provides guidance on a graded approach for using risk insights in licensing. It also provides guidance to better integrate complementary insights from traditional engineering and risk assessment approaches to foster a broadened understanding of the benefits that RIDM can bring to the overall regulatory approach.

This OI contains three appendices. Appendix A documents the change history.

Appendix B provides general process details for implementing the Integrated Review Team (IRT) process. Appendix C provides RIDM implementation guidance to technical reviewers on an IRT. This OI will be updated, if appropriate, by inserting additional appendices to capture lessons learned from IRT.

Oversight of the process for ensuring the quality and consistency of using PRA, risk, or probabilistic information is an ongoing endeavor, as new technical issues can arise that challenge the scope of existing guidance and consistency among various technical work products in using IRTs. The cognizant branch chiefs in DORL and technical divisions are responsible for effective use of this OI. They should use insights gathered from information using this procedure to report any problems with, or possible improvements to, LIC-206 in accordance with NRR OI ADM-100, Preparing and Maintaining NRR Office Instructions.

5.

RESPONSIBILITIES AND AUTHORITIES All staff who support the nuclear safety and security program are responsible for understanding and applying the guidance in this OI. The staff is also responsible for identifying potential improvements to the guidance and submitting suggestions for such improvements to the primary contact for this OI.

The staff and management are responsible for focusing efforts on safety-significant issues and dispositioning low safety-significant issues efficiently.

NRR Office Instruction LIC-206, Revision 1 Page 5 of 8 The following describes the NRC staff and management roles and responsibilities associated with this OI.

A. Office Directors and Deputy Directors Office management is responsible for:

leading the offices implementation of the policy in this procedure; and initiating behavior changes throughout the organization toward RIDM.

B. Division Management Division management is responsible for:

providing overall management and oversight of licensing activities to incorporate RIDM; ensuring appropriate resources are made available to consider risk insights in licensing activities; promoting consistency in considering and applying risk insights; facilitating strategies to address challenges incorporating risk considerations in decision-making; maintaining awareness of industry initiatives that will result in a large volume of similar risk-informed applications; ensuring that the PMs, TRs, and risk analysts are aware of such situations and communicate with management peers through workload management meetings; ensuring the staff is sufficiently trained in the integrated review approach; and assisting in resolving differing views as they arise during the integration of risk and traditional engineering insights.

C. Branch Chiefs Branch chiefs are responsible for:

being familiar with pertinent elements of risk-informed and performance-based regulation; effectively managing use of risk information and technology, as appropriate, to enhance decision-making; encouraging staff participation in risk-informed processes, as appropriate; assigning appropriate resources, including risk analysts, to consider risk insights in licensing activities; ensuring staff attends RIDM training; promoting consistency in considering and applying risk insights; ensuring staff considers risk insights to inform the scope and depth of reviews; and ensuring staff appropriately apply and document use of risk information and/or risk insights.

NRR Office Instruction LIC-206, Revision 1 Page 6 of 8 D. Project Managers Project managers (PMs) are responsible for:

being familiar with pertinent elements of risk-informed and performance-based regulation; being familiar with using risk insights in licensing reviews, risk-informed licensing initiatives (e.g., Technical Specifications Task Force (TSTF) travelers and 10 CFR 50.69 license amendment requests), and regulatory guides (RGs) for risk-informed reviews (e.g., RG 1.174, RG 1.200, etc.);

facilitating issue resolution and escalation to management within IRT process; reflecting the use of risk insights and a risk-informed and performance-based philosophy into work products, as appropriate; coordinating consideration of risk insights in licensing activities; and appropriately engaging a risk analyst early for risk insights in licensing activities.

E. Reviewers (Technical Reviewers and Risk Analysts)

Technical reviewers (TRs) and risk analysts (sometimes collectively referred to as reviewers in this document), are responsible for:

being familiar with pertinent elements of risk-informed and performance-based regulation; being familiar with the Type 1, 2, and 3 definitions; being familiar with the application of risk in licensing reviews to assist PMs with team formation; considering risk information and insights in licensing activities; considering risk information and insights to inform the scope and depth of reviews to reach a reasonable assurance determination; considering licensee-submitted and staff-generated risk information; independently assessing the adequacy of licensee approaches considering risk; using risk information and risk insights to enhance decision-making; reflecting the use of risk insights and a risk-informed and performance-based philosophy in work products, as appropriate; working collaboratively with risk analysts in both traditionally deterministic and risk-informed licensing activities; working collaboratively with the responsible PM; and providing technical assistance to the PM to assist with team formation.

F. Risk Analysts In addition to the responsibilities above, risk analysts are responsible for:

providing risk insights on licensing activities as requested, including traditionally deterministic regulatory activities; familiarizing the staff with available risk tools and information in communicating risk insights;

NRR Office Instruction LIC-206, Revision 1 Page 7 of 8 working collaboratively with TRs in both traditionally deterministic and risk-informed licensing activities; and ensuring documentation that properly characterizes the consideration of risk information in the staffs decision.

6.

PERFORMANCE MEASURES The ongoing oversight of the process set forth in this OI (Section 4) will provide necessary feedback on how well the process is working.

7.

PRIMARY CONTACTS Samson S. Lee 301-415-3168 Samson.Lee@nrc.gov Ian H. Tseng 301-415-7964 Ian.Tseng@nrc.gov Shilp Vasavada 301-415-1228 Shilp.Vasavada@nrc.gov

8.

RESPONSIBLE ORGANIZATION Division of Operating Reactor Licensing Office of Nuclear Reactor Regulation

9.

EFFECTIVE DATE June 26, 2020

10.

CERTIFICATION DATE June 26, 2025

11.

REFERENCES

1. Use of Probabilistic Risk Assessment Methods in Nuclear Regulatory Activities; Final Policy Statement (60 Federal Register 42622; August 16, 1995).

2. U.S. Nuclear Regulatory Commission, SECY-17-0112, Plans for Increasing Staff Capabilities to Use Risk Information in Decision-Making Activities, dated November 13, 2017 (ADAMS Accession No. ML17270A192).

3. U.S. Nuclear Regulatory Commission, Staff Requirements Memorandum for SECY-98-144, White Paper on Risk-Informed and Performance-Based Regulation, dated March 1, 1999 (ADAMS Accession No. ML003753601).

NRR Office Instruction LIC-206, Revision 1 Page 8 of 8

4. U.S. Nuclear Regulatory Commission, SRM-M170511, Staff Requirements - Briefing on Risk-Informed Regulation, dated June 26, 2017 (ADAMS Accession No. ML17177A397).

5. U.S. Nuclear Regulatory Commission, Regulatory Guide 1.174, Revision 3, An Approach for Using Probabilistic Risk Assessment in Risk-Informed Decisions on Plant-Specific Changes to the Licensing Basis, dated January 2018 (ADAMS Accession No. ML17317A256).

6. U.S. Nuclear Regulatory Commission, Staff Requirements Memorandum for SECY-19-0036, Application of the Single Failure Criterion to Nuscale Power LLCs Inadvertent Actuation Block Valves, dated July 2, 2019 (ADAMS Accession No. ML19183A408).

7. U.S. Nuclear Regulatory Commission, NRR Office Instruction LIC-100, Revision 1, Control of Licensing Bases for Operating Reactors, dated January 7, 2004 (ADAMS Accession No. ML072000067)

Enclosures:

1. Appendix A: Change History

2. Appendix B: Integrated Review Team Process

3. Appendix C: Use of Probabilistic, Risk, and PRA Insights for Technical Reviewers

Appendix A - Change History Office Instruction LIC-206 Integrated Risk-Informed Decision-Making for Licensing Reviews LIC-206 Change History - Page 1 of 1 Date Description of Changes Method Used to Announce &

Distribute Training 6/06/19 This is the initial issuance of LIC-206. This office instruction provides guidance on a graded approach for using risk insights in licensing and to better integrate complementary insights from traditional engineering and risk assessment approaches to foster a broadened understanding of the benefits that risk-informed decision-making can bring to the overall regulatory approach.

E-mail to NRR Leadership Team and Technical Assistants General training on risk-informed decision-making:

iLearn Course ID_280148, Risk-Informed Thinking Workshop Training for LIC-206: TBD Training for Appendix B to LIC-206:

iLearn Course ID_427162, Integrating Risk into Regulatory Reviews (Web-based) 6/26/20 A new appendix is added. The appendix is entitled, Appendix C, Use of Probabilistic, Risk, and PRA Insights for Technical Reviewers.

The main body and Appendix B is revised.

E-mail to NRR Leadership Team and Technical Assistants Training slides for level of review in Appendix C to LIC-206: ADAMS Accession No. ML17209A108 U.S. Nuclear Regulatory Commission Office of Nuclear Reactor Regulation Appendix B Integrated Review Team Process

NRR Office Instruction LIC-206, Revision 1 Appendix B Page 1 of 11 1.0 INTEGRATED REVIEW TEAM PROCESS1 The Integrated Review Team (IRT) process describes a framework for forming teams comprised of project managers (PMs), technical reviewers (TRs), and Office of Nuclear Reactor Regulation (NRR) Division of Risk Assessment (DRA) risk analysts who work together on licensing review products. The IRT process facilitates the integration of risk and traditional insights in a technical evaluation. The use of the terms technical branch, technical reviewer (TR), or traditional engineering, refers to technical staff who are not qualified risk analysts or reliability experts.

Use of the terms risk analyst or probabilistic risk assessment (PRA) analyst refers to qualified PRA analysts or reliability experts who would typically be assigned to PRA branches in NRR/DRA. The term reviewer is sometimes used in this OI to collectively refer to both TRs and risk analysts.

An IRT refers to a multidisciplinary review team that consists of at least one PM, at least one TR, and typically includes a risk analyst. The team members are responsible for developing requests for additional information (RAIs), audit plans, and safety evaluations (SEs). An IRT may enhance integration of the reviews of various technical disciplines and the consolidation of SE and RAI input; reviewers are encouraged to work together as a team. Communication within a team and joint development of consolidated products enhance the efficiency of the licensing process. If a branch has a concurrence-only role, the inclusion of this branch in the IRT may not be necessary.

The IRT should determine whether meetings are needed on a case-by-case basis. Informal discussion within the team is also encouraged. The staff should charge time to the specific CAC/EPID corresponding to the licensee submittal when participating in an IRT for that review.

An IRT can be applied to the following general licensing actions:

routine plant-specific licensing actions; and emergent licensing actions (i.e., emergency and exigent amendments and verbal relief requests).

The IRT process is shown in Figure 1, which is divided into Submittal Type Determination, Review Team Formation, and Safety Evaluation language. Section 2.0 provides discussion of the IRT process in the context of routine plant-specific licensing actions. Precise usage of terminology can differ between documents. For a concise glossary of terms as they are used in this OI, please refer to Section 2.0 of Appendix C.

1 Staff performing licensing and review activities in the new reactor business line are encouraged to apply this guidance to the extent possible and appropriate, because certain requirements are different from those of the operating reactor business line, such as timelines and milestones.

NRR Office Instruction LIC-206, Revision 1 Appendix B Page 2 of 11 Figure 1: Integrated Review Team Process Flowchart 2.0 ROUTINE PLANT-SPECIFIC LICENSING ACTIONS The following process describes how to establish IRTs for routine plant-specific licensing action applications. The process description expands on the IRT process flowchart (Figure 1) and covers submittal type, staffing assignments, team formation, and consolidated RAI and SE development. The processes described in other relevant NRR office instructions (OIs), such as LIC-101 and LIC-109, are applicable unless specifically addressed in this OI. The term routine means that the application is not an emergency or exigent (i.e., emergent) request or a verbal relief request. (See Section 3.0 of this appendix for additional considerations applicable to emergent licensing actions.)

NRR Office Instruction LIC-206, Revision 1 Appendix B Page 3 of 11 2.1 Submittal Type Determination The following process for submittal type determination applies when the PM creates the project in the workload management system, Reactor Program System Licensing (RPS), and makes initial branch assignments. After completing the acceptance review in accordance with LIC-109, the PM will enter the type determination (i.e., Type 1, 2, or 3) in RPS as part of completion of the acceptance review milestone.

BOX 2: The PM will review the application as submitted by the licensee to determine whether it is a Type 1, 2, or 3 application based on the level of quantitative PRA information in the application (e.g., core damage frequency (CDF), large early release frequency (LERF), initiating event frequencies, references to PRAs, etc.) within the typical timeline for adding a project to RPS. For large applications, doing a find for PRA terms or asking the licensee during routine discussions can be helpful. If the PM is uncertain how to determine the type of application, the PM can consult with DRA. Use of PRA terms should not be viewed as the sole indicator of whether an application should be categorized as Type 2. A well-reasoned qualitative or quantitative discussion that relies on elements of the risk triplet (What can go wrong? How likely is it? What are the consequences?) may be representative of a Type 2 application. For Type 3 applications, the PM should consult with DRA, as needed, to determine which branches have reviewed similar applications in the past.

For the purpose of implementing this step, Type 1, 2, and 3 applications are defined as follows:

Type 1:

Applications contain little or no risk/PRA information.

Type 2:

Applications contain quantitative or qualitative risk/PRA information but are not formally submitted using the guidance in RG 1.174 and RG 1.200.2 Type 3:

Applications submitted as risk-informed applications in accordance with RG 1.1742.

The PM will assign branches in RPS as follows:

Type 1: Assign technical branches. The need for a formal review by risk analysts will be determined after discussion with the TR(s) or after the PM determines he or she need assistance with the no significant hazards consideration determination regarding frequency of accidents. The PM may assign DRA at this stage if it is already known that the technical staff will want a consideration of risk insights (e.g., it was discussed after a pre-submittal meeting) for purposes other than no significant hazards determination (e.g., use of risk insights to determine scope/depth of review).

Type 2: Assign DRA and technical branches.

Type 3: Assign DRA and technical branches. DRA will be the lead technical review division. If technical branch assistance is not needed, the reviewer(s) can be removed 2 Other related guidance documents such as Regulatory Guide (RG) 1.177 provide guidance on the acceptable approaches for using risk information in specific risk-informed applications. In this OI, referring to the use of guidance in RG 1.174 intends to encompass the use of other related guidance documents as well.

NRR Office Instruction LIC-206, Revision 1 Appendix B Page 4 of 11 from the project later. Assigning the technical branches at the start of the project will provide them an early opportunity to raise concerns with risk-informed applications.

2.2 Review Team Formation The review team formation is summarized in Boxes 3, 4, and 5 of the Figure 1 flowchart. The PM will make an initial determination of which technical disciplines to assign to a review and whether DRA resources are needed to support determining scope/depth of review or reach regulatory findings, based on the content and level of PRA or risk information in the submittal.

The PM will involve DRA in a Type 2 or 3 review. The PM should discuss with lead reviewers or the initial team to determine if additional reviewers are needed and if an IRT should be used.

The Team Formation Checklist (Section 4.0) in this appendix should be used to inform the team formation process. As part of completion of the acceptance review milestone in RPS, the PM will identify whether an IRT with or without a risk analyst is used for the review.

The PM should hold a meeting after the IRT is formed to gain insights of the risk significance of the licensing action and identify the appropriate scope of review. The risk analysts should share risk insights relevant to the particular licensing action with the IRT. The IRT may request additional information from the risk analysts or decide whether the risk analysts should continue with the licensing action.

BOX 3: The submittal contains no PRA information (e.g., CDF) or risk information (quantitative or well substantiated qualitative information that involve key elements of the risk triplet). The TRs and PM determine whether DRA or other technical review disciplines need to be involved using the Team Formation Checklist (Section 4.0). A broad understanding of the concepts of Scope of Review Checklist may also help in the determination (Appendix C, Section 4.0).

If it is determined that DRA support is needed to support the basis for the regulatory finding or to adjust the scope and depth of the level of review, proceed to Box 4.

Even if it is determined that DRA support is not needed to support the basis for the regulatory finding, the TR(s) is/are encouraged to seek probabilistic, risk, or PRA insights on their own to inform the review as discussed in Appendix C. If the TR wishes to leverage these insights in the review, proceed to Box 7; otherwise, proceed to Box 6. In the case of a Box 6 review, the review team is led by a TR and may be supported by other TRs as needed.

BOX 4: The submittal provides PRA information or significant well-reasoned discussions on qualitative and/or qualitatively that pertains to components of the risk triplet; however, the information has not been submitted following the guidance of RG 1.174.

A Box 4 IRT is led by a TR supported by DRA and other TRs, as needed.

BOX 5: The submittal is risk-informed in accordance with RG 1.174.

A Box 5 IRT is led by DRA and is supported by other TR(s) as needed.

NRR Office Instruction LIC-206, Revision 1 Appendix B Page 5 of 11 2.3 Review Process In order to integrate the reviews of submittals that have varying levels of risk information, SE template language is discussed in Appendix C, Section 7.0, in association with Box 7 of the IRT Process Flow Chart.

The staff should use consistent and commonly understood language when integrating risk information into regulatory decision-making, as defined in NUREG-2122, Glossary of Risk-Related Terms in Support of Risk-Informed Decision-Making, or in Section 2.0 of Appendix C of this OI.

BOX 6: This is a traditional engineering decision and does not leverage new risk or PRA information to determine scope/depth of review or reach regulatory findings. It is well established that conventional regulatory decisions based on traditional analyses are sufficient to provide reasonable assurance of safety. The Box 6 review involves traditional analysis, including concepts such as single failure and worst-case assumptions, which generally involve less analysis but provide more margin. This traditional analysis is used to support the regulatory decision that leads to the safety conclusion.

For a Box 6 review, LIC-101 SE guidance should be used to develop a traditional SE.

BOX 7: Probabilistic, risk, or PRA insights can be used to determine the scope and depth of the review, as well as to reach or support regulatory decisions, as discussed in the IRT Checklist in Appendix C (Section 4.0). Examples include probabilities of failures of equipment, the frequency of initiating events, Monte Carlo simulations of degradation, consideration of supporting PRA information such as CDF or LERF submitted not in accordance with RG 1.174, or other pertinent information. Risk insights can be obtained by both quantitative and qualitative analyses. Quantitative risk results from PRA calculations are typically the most complete characterization of risk, but they are generally supplemented by qualitative risk insights, which can include defense in depth, safety margin, performance monitoring, and traditional engineering analysis. These insights can be provided by the licensee in a Type 2 submittal or can be developed by the staff.

DRA risk analysts can assist TRs in crafting an SE that uses probabilistic or risk information, in addition to the typical traditional analysis. DRA risk analysts should assist in the crafting of the SE and should be included in the initial IRT assignment when PRA insights are considered.

When discussing probabilistic information, risk, or PRA inputs, technical staff have significant flexibility within the templates provided in NRR OIs such as LIC-101, License Amendment Review Procedures, to describe the technical basis for their regulatory decision. The limitation is that when describing probabilistic or risk input, the term PRA or use of quantitative CDF or LERF should not be used. Use of these terms would indicate PRA as a basis of the decision or as used to support the decision; therefore, it would be in Box 8.

RG 1.174 quantitative risk thresholds are not expected to be used as the basis for approving changes that were not submitted in accordance with RG 1.174. If PRA results are used to develop insights, such insights need not be developed by PRA models that meet the guidance of RG 1.200, but may be developed using NRC standardized plant analysis risk (SPAR) models, other PRA tools, or generic insights (i.e., results that have been learned from numerous PRAs

NRR Office Instruction LIC-206, Revision 1 Appendix B Page 6 of 11 performed in the past and from operational experience and that are applicable to a group of similar plants). The checklist in Section 4.0 of Appendix C can assist in gathering PRA insights in licensing action decision-making.

When the regulatory decision makes use of risk information, it must be clear how it was developed and used. In documenting the use of risk information, care should be taken to decompose the risk triplet as applicable and analyze the constituent information.

Integrating risk insights with traditional engineering approaches can provide better-reasoned regulatory decisions to reach a reasonable assurance of adequate protection finding.

For a Box 7 review, Appendix C SE guidance should be used in conjunction with LIC-101 SE guidance.

BOX 8: Using the results of a PRA as part of the basis of a regulatory decision involves using the five principles of the risk-informed decision-making (RIDM) framework from RG 1.174.

These features are: (1) the change meets the regulations, (2) is consistent with defense-in-depth philosophy, (3) maintains safety margins, (4) results in small increase in risk using PRA information, and (5) is monitored. For PRA information to be used in this RIDM framework, the PRA should be acceptable to support the application as described in RG 1.174.

DRA risk analysts typically lead review activities for these licensing actions, including primary development of the SE. The language used to describe PRA results as elements of a regulatory decision is documented in RG 1.174 and other RIDM regulatory guidance.

Language such as risk insights or PRA insights may be used, consistent with the guidance in RG 1.174 or RG 1.200. Therefore, existing precedents and guidance from RG 1.174 apply.

For a Box 8 review, RG 1.174 SE guidance should be used in conjunction with LIC-101 SE guidance.

3.0 EMERGENT LICENSING ACTIONS Early risk insights generated through an IRT may provide crucial efficiencies for emergent or emergency licensing actions, although the prescribed process here may be adjusted for the situation by the PM. The PM should consider early inclusion of DRA staff, especially if risk insights are being discussed by TRs. The IRT should leverage existing, completed PRA work and other risk insights to expedite the review.

The NRR staff will maintain the current processes and practices for emergent actions but apply the IRT approach to the extent practicable. For an emergent licensing action, the PM should involve the technical branches, including a branch in DRA; branch chiefs; and management, as appropriate, as soon as practical to understand the methodology for the review and who will be reviewing which aspects of the proposed changes. The review team should decide whether there is sufficient time to develop a consolidated SE at the start of the review and if individual branch SE inputs are needed or preferred. The PM or a technical coordinator should consolidate the SE inputs into LIC-101, Appendix B, or LIC-102 SE format. Therefore, the PM, with assistance from the review team, should begin developing the consolidated SE outline at the start of the review.

NRR Office Instruction LIC-206, Revision 1 Appendix B Page 7 of 11 4.0 TEAM FORMATION CHECKLIST The PM will determine which branches to initially assign to the review based on the technical content and the level of PRA and risk information in a submittal. The team determines if additional reviewers are needed and if an IRT should be used. The Team Formation Checklist will assist the teams with determining who needs to be part of the review team. This checklist (i.e., Table 1 below) can be used for Type 1, 2, and 3 submittals, including license amendments, relief requests (or proposed alternatives), and exemption requests. The checklist corresponds to the decision on whether to form an IRT based on considerations of the review or licensee submittals.

Table 1 lists decisions (i.e., integration required, integration considered, and integration not necessary) and descriptions under description of review or submittal topic. If any of the descriptions in Table 1 apply, then the corresponding decision can be selected.

Table 1: Team Formation Checklist Decision Description of Review or Submittal Topic Integrated Review Team Required Unless Justified Otherwise

1. Risk-informed review in accordance with RG 1.174

2. Submittal includes PRA or risk information, but not in accordance with RG 1.174

3. Complex submittals where risk insights could contribute to a more effective review

4. Request to adopt risk-informed Technical Specification Task Force (TSTF) travelers (e.g., consolidated line item improvement process), unless risk considerations were considered during the TSTF approval process Integrated Review Team Considered

1. Change to technical specifications where single failure and other conservatisms in Updated Final Safety Analysis Report, Chapter 15, are no longer considered as part of the assumption

2. Review that involves multiple system integration

3. Review that is related to structures, systems, and components (SSCs) that are important to safety, but risk information is not provided

NRR Office Instruction LIC-206, Revision 1 Appendix B Page 8 of 11 Table 1: Team Formation Checklist Decision Description of Review or Submittal Topic Integrated Review Team Not Necessary

1. Administrative change(s)

2. Reactor risk neutral activities

3. Other reviews where integrating the review with risk analysts may have no significant benefit The following sections provide additional discussion of the content of the above table.

4.1 Integrated Review Team Required Unless Justified Otherwise Risk-Informed Reviews in Accordance with RG 1.174 Review of Type 3 submittals requires the contribution of DRA risk analysts. These reviews should be integrated as soon as practicable to ensure that the TRs and risk analysts understand the scope of their respective portions of the review, the justification for the bases for acceptance in each of the technical and risk areas, and technique for formatting an integrated technical evaluation. PMs and TRs should also be familiar with the guidance in RG 1.174 and the principles of RIDM.

Submittals that Include Risk Information but Not in Accordance with RG 1.174 For these reviews, the licensee presents risk information to support its safety case. This information is not sufficient to form the basis of a potential NRC acceptance of the submittal because it does not meet RG 1.174. However, the proposed change may lend itself to risk evaluations, and DRA may be able to provide some quantitative or qualitative insights to the review. Risk insights should be considered to inform the scope and depth of the staff review to reach a reasonable assurance determination, reach or support regulatory findings such as enhancing confidence in the technical evaluations (see Appendix C, Sections 4.0 and 5.0).

DRA risk analysts should be engaged early in these reviews to ensure that the provided risk information is used appropriately and within the correct context.

Complex Submittals Where Risk Insights Could Contribute to a More Effective Review For complex submittals, many technical disciplines may be involved in the review. When reviews contain information that is complex but lacks a clear nexus to public safety, the staff should consider using risk insights. The IRT concept encourages reviewers from many technical disciplines to work together as a team. An IRT should consider risk insights to inform the scope and depth of the staff review, reach or support regulatory findings, or enhance confidence in the technical evaluations (see Appendix C). If the estimated level of effort for a review exceeds a few hundred hours, considerations of risk insights by the IRT may be particularly beneficial for determining the scope and depth of the review and reaching regulatory findings. The communication within a team and developing joint consolidated products enhance

NRR Office Instruction LIC-206, Revision 1 Appendix B Page 9 of 11 the efficiency of the licensing process. The IRT enhances the integration of staff review and preparation of consolidated SEs and RAIs from the many technical disciplines.

Request to Adopt Risk-Informed TSTF Travelers (e.g., Consolidated Line Item Improvement Process), Unless Risk Considerations Were Considered During the TSTF Approval Process 4.2 Integrated Review Team Considered For these reviews, there may be utility in engaging DRA risk analysts early in the process to provide supporting information or other insights to the reviews. In general, these conditions are listed below in the order of the most likely to benefit from a risk analyst to the least likely.

Changes to Technical Specifications Where Single Failure and Other Conservatisms in Updated Final Safety Analysis Report, Chapter 15, are No Longer Considered as Part of the Assumptions Safety-related SSCs and analyses often have numerous layers of protection, such as assuming a single failure or worst-case scenarios. Where these numerous layers of protection are in place, there may not be any use in applying risk insights developed by risk tools because the risk tools do not make the same assumptions (for example, regarding single failure). In cases where a safety-related analysis no longer relies on these prescriptive assumptions, it is both useful and prudent to look to risk analyses for insights. This is often the case when the single failure criterion is relaxed because a licensee has entered a technical specification action statement, or for a submittal where the licensee is requesting to not consider single failure for a period of time (i.e., when maintenance is planned). An example of considering risk insights in evaluating the assumption of single failure is the Commissions staff requirements memorandum to SECY-19-0036 on the licensing of NuScale.

Reviews that Involve Multiple System Integration PRA tools have the capability to estimate the synergistic efforts of changes that involve multiple systems or components, qualitatively or quantitatively. Therefore, reviews of submittals that impact multiple systems should be considered for discussion with risk analysts.

Reviews That are Related to Structures, Systems, and Components That are Important to Safety, but Risk Information is Not Provided SSCs that are important to safety are typically modeled in the PRA. For SSCs that are important to safety, there are typically more tools (e.g., SPAR, notebooks, etc.) available to the risk analysts to provide insights to TRs regarding the risk information related to an SSC important to safety. Discussion with risk analysts may be prudent to ensure all the functions of that equipment are considered in the deterministic review. In this case, the risk analyst may not provide risk insights but may provide connections to other technical areas that are not obviously related to the review topic.

NRR Office Instruction LIC-206, Revision 1 Appendix B Page 10 of 11 4.3 Integrated Review Team Not Necessary For these reviews, the need to engage with DRA risk analysts has not been identified. Either there is not a risk aspect to be considered, or the technical review process is mature enough such that adding risk resources would not improve the efficiency, effectiveness, or safety decision of the review.

Administrative Changes If administrative changes do not impact the operation or maintenance of the plant, there is no need to engage with DRA risk analysts to review these changes. Requests for NRC review of operational, testing, or maintenance procedure-related changes could potentially impact plant risk and should not be excluded from possible integration by categorizing these types of changes as administrative changes.

Reactor Risk Neutral Activities This includes reviews of topics that are not related to components that could have an impact on the safety of the reactor, such as the use of offsite sirens or domestic water, or topics for which risk methods are not available. If the TRs for such changes are new to the reviews, discussions with risk analysts may be prudent.

Other Reviews Where Integrating the Review with Risk Analysts May Have No Significant Benefit This includes reviews where integrating the review with risk analysts may have no significant use and including risk information is not needed to ensure an efficient and effective review.

However, the staff should consider the Scope of Review Checklist in Appendix C (Section 4.0) in performing its review.

5.0 PROCESS OVERVIEW FOR CONSOLIDATED SAFETY EVALUATION AND REQUEST FOR ADDITIONAL INFORMATION INPUT Efficiencies can be gained when members of an IRT work together to produce consolidated SEs and RAIs. The process should be considered when the staff review involves multiple technical disciplines, with or without a risk analyst. The following process steps are for the development of the consolidated SEs and RAIs, and they are in addition to those in other applicable office instructions.

5.1 Consolidated Safety Evaluation Preparation After confirming that reviewers are assigned responsibility to review each area, the PM should develop the licensing package with a draft SE outline for the IRT to use. The PM should begin filling in the following SE sections: Introduction, Regulatory Evaluation, and an outline of the Technical Evaluation with reviewer assignments. The document should be uploaded to a SharePoint library or other collaborative site location. The PM should send the draft SE outline to the team generally within 4 weeks after completion of the acceptance review. The review team should review and comment on the draft SE outline and agree on the section

NRR Office Instruction LIC-206, Revision 1 Appendix B Page 11 of 11 assignments. The PM should verify that someone is assigned to review each area in the submittal and there is no gap in the SE.

Upon receiving the draft SE outline, the review team should begin drafting its portion(s) of the Technical Evaluation, with each reviewer editing the Regulatory Evaluation as necessary.

The team should agree on the review methodology and ensure the Introduction and Regulatory Evaluation sections of the SE are accurate. The team may meet to assist in resolving complex technical issues. The PM may schedule meetings or schedule meetings by the request of the team as the review progresses.

In cases where a review is performed predominantly by a single branch or division, the reviewers from that division should lead the development of the SE for effectiveness and efficiency.

5.2 Consolidated Request for Additional Information Process The PM may structure an outline for the consolidated RAI, fill in the introduction matter, and upload it to SharePoint for TR/DRA use, as appropriate. Each reviewer will work on his or her sections of the SE as agreed, finishing an input to the draft SE with holes prior to sending RAIs and the draft SE to the PM. If no RAIs are required, reviewers should work to finalize their input to the consolidated SE. If a reviewer needs more time than others to develop RAIs, then the PM should send any approved RAIs to the licensee without delay. Also, some RAIs can stand alone, and the PM may send them separately for efficiency.

Each reviewer should enter his or her RAIs into the consolidated RAI document, as applicable.

Each reviewer is encouraged to read all draft RAIs and draft SE input for a cohesive understanding of technical concerns and to suggest edits that will improve overall RAI quality, as appropriate. Branch chiefs of TR/DRA will review their staffs final RAI input and draft SE input and send concurrence to the PM.

5.3 Finalizing the Consolidated Safety Evaluation When the RAI response is received, the team will use the information to finalize the consolidated draft SE. Reviewers branch chiefs will approve their staffs input to the consolidated SE and send their concurrence to the PM. The PM will see the licensing package through to completion, reviewing it for completeness and obtaining required approvals.

U.S. Nuclear Regulatory Commission Office of Nuclear Reactor Regulation Appendix C Use of Probabilistic, Risk, and PRA Insights for Technical Reviewers

NRR Office Instruction LIC-206, Revision 1 Appendix C Page 1 of 20

1.0 INTRODUCTION

This appendix provides technical reviewers (TRs) with the tools needed to use probabilistic, risk, and probabilistic risk assessment (PRA) insights to: 1) determine an appropriate scope or depth of review, and 2) reach or support regulatory findings, as part of the systematic approach in this office instruction. This approach is intended to leverage TRs existing experience and engineering knowledge while expanding their use of risk information and risk insights as a member of an integrated review team (IRT), described in Appendix B of this office instruction (OI). The guidance in this appendix applies to a Box 7 review, as described in Section 1.1 of Appendix B to increase flexibility and efficiency in the technical review by considering a wide range of probabilistic, risk, or PRA information. Since this guidance applies to both TRs and risk analysts, the term reviewer is sometimes used to refer to both groups collectively. Staff performing licensing and review activities in the new reactor business line are encouraged to apply this guidance to the extent possible and appropriate.

To date, much of the effort to risk-inform the U.S. Nuclear Regulatory Commission (NRC) regulatory activities has focused exclusively on the use of PRA. Risk assessment or risk analysis and PRA are often incorrectly used as synonyms. A PRA is one type of risk assessment or risk analysis. The PRA has a structured format and quantifies the ultimate consequences. This appendix provides guidance for uses of risk and probabilistic information beyond PRA results.

TRs are expected to do the following:

1. Use risk information to tailor the focus, depth, and scope of reviews.

2. Consider licensee-submitted and staff-generated risk information.

3. Independently assess the adequacy of licensee approaches considering risk.

This appendix supports the LIC-206 process by providing the following information:

Section 2.0 provides the definitions of selected risk terms in the context of the LIC-206 process. This will promote effective communication with a common understanding between staff in discussions of risk-related activities.

Section 3.0 discusses the IRT process from the perspective of the TR. This includes expectations for TRs to gather risk information prior to the initial IRT meeting.

Section 4.0 contains the IRT checklist that provides a guide to help TRs gather risk insights and determine an appropriate level of review.

Section 5.0 contains the IRT flowcharts that discuss ways to construct a risk argument, including guidance for TRs use for gaining better insights into the application of risk information.

Section 6.0 provides a sampling of risk tools.

Section 7.0 provides sample SE language with the goal of providing context and consistency for staff reviews.

NRR Office Instruction LIC-206, Revision 1 Appendix C Page 2 of 20 2.0 CONCISE GLOSSARY OF TERMS In order to promote effective communication between staff in discussions of risk-related activities, a common understanding of frequently used terms is needed. It is recommended that technical reviewers be familiar with the terms and concepts below. Extended discussion of these and related terms is available in NUREG-2122, Glossary of Risk-Related Terms in Support of Risk-Informed Decision Making.

Defense in Depth Defense in depth is an approach to designing and operating nuclear facilities that prevents and mitigates accidents that release radiation or hazardous materials. The key is creating multiple independent and redundant layers of defense to compensate for potential human and mechanical failures so that no single layer, no matter how robust, is exclusively relied upon.

Defense in depth includes the use of redundant equipment and/or diverse methods for achieving a safety function. The key concept of defense in depth is that no single layer of defense is exclusively relied upon. Defense in depth is commonly applied as an NRC requirement, independent of probabilistic results.

Deterministic/Probabilistic Approach A deterministic approach to decision-making relies solely on engineering analyses and does not directly consider probabilistic methods in reaching a decision. The likelihood of an adverse condition is not considered. The adverse condition is postulated to exist, and the resulting plant responses and consequences are analyzed. The deterministic approach involves implied but unquantified elements of probability in the selection of the specific conditions to be analyzed.

A probabilistic approach considers the likelihood of occurrence of adverse conditions. A PRA is an example of a probabilistic analysis, but a PRA also depends on deterministic analyses to support its underlying assumptions.

Performance-Based Performance-based is an approach that reaches conclusions based on measurable performance results. As described in NUREG/BR-0318, performance-based measures are measurable, calculable, or objectively observable parameters that can be leveraged to monitor performance. This term is generally used in the context of risk-informed, performance-based acceptance criteria used as part of the basis for NRC staff approval.

Probabilistic Risk Assessment (PRA)

Probabilistic risk assessment (PRA) is a systematic, numerical analysis of the plants strengths and weaknesses. Probabilistic information informs the PRA with information regarding the risk triplet, and the PRA develops a numerical result. A PRA is designed for realism, that is, prescriptive assumptions like single failure or worst-case assumptions are not used.

NRR Office Instruction LIC-206, Revision 1 Appendix C Page 3 of 20 Probabilistic, Risk, and PRA Insights Probabilistic, risk, and PRA information is supporting evidence that can be used to reach conclusions within an SE.

Probabilistic insights are supporting evidence derived from information pertaining to the likelihood of events or failures. Both risk and PRA approaches rely upon probabilistic analyses and insights.

Risk insights are supporting evidence derived from the application of the risk triplet. Risk insights can be obtained by both quantitative and qualitative investigations of risk. Qualitative risk insights may include generic results learned from PRAs performed in the past and from operational experience that is applicable to a group of similar plant designs. Quantitative risk insights must be derived from (reasonably) direct measurement(s) related to the equipment in question.

PRA insights are generally quantitative risk results from PRA calculations.

Probability and Frequency Probability is the likelihood that an event will occur and is a unitless quantity between 0 and 1.

Frequency is the expected number of occurrences per unit time. The NRC has defined some special types of frequencies that are discussed below in their PRA context:

Core Damage Frequency: The sum of the accident sequence frequencies of those accident sequences whose end state is core damage.

Initiating Event Frequency: The frequency of an event originating from an internal or external hazard that both challenges normal plant operation and requires successful mitigation.

Large Early Release Frequency: The frequency of a rapid, unmitigated release of airborne fission products from the containment to the environment that occurs before effective implementation of offsite emergency response and protective actions such that there is a potential for early health effects.

Risk Risk is a measure combining failure modes, likelihood of occurrence, and consequences. Risk is defined as the probability and consequences of an event, as expressed by the risk triplet, that is, the answer to the three questions in the risk triplet (see Risk Triplet definition).

The purpose of risk is to provide insight into the relative importance of potential failure modes.

This allows for optimizations such as tuning design/manufacture, designing inspection regimes, and/or replacement scheduling.

NRR Office Instruction LIC-206, Revision 1 Appendix C Page 4 of 20 Risk-Informed/Risk-Based Risk-informed is a term that is often used incorrectly in place of risk-based. These terms are not synonyms. A risk-informed approach combines risk information with other factors (e.g.,

engineering design features) to arrive at a decision. The term risk-based is applied to activities that rely solely on the use of risk information from PRA results. The NRC does not currently endorse risk-based approval of regulated activities.

Risk-Significant/Safety-Significant/Important to Safety/Safety-Related Risk-significant is a level of risk that exceeds a predetermined threshold.

Safety-significant is a criterion above which the potential to affect safety exists.

Important to safety refers to both safety-related and non-safety-related SSCs that have been deemed important.

Safety-related is an SSC that is relied upon to remain functional during and following design-basis events.

Risk-significant, safety-significant, important to safety, and safety-related are not synonymous.

Safety significance is not evaluated in a PRA - risk significance is. This risk significance can be used to determine the safety significance of SSCs in conjunction with information regarding the function of the SSCs, defense in depth, and safety margins. Safety-related is a specific SSC designation defined in 10 CFR 50.2 and determined during a plants original operating license review.

The term safety-significant is used to categorize nuclear power plant SSCs using the process outlined in Title 10 of the Code of Federal Regulations (10 CFR) Section 50.69.

Risk Triplet The risk triplet is a series of three questions that are addressed through risk analysis.

1. What can go wrong?

2. How likely is it to occur?

3. What are the consequences if it occurs?

Safety Margin Safety margin is extra capacity factored into the design of SSCs beyond the analyzed conditions that the SSC is expected to perform its intended function under. Margin is commonly used to bound uncertainty in loading conditions, material properties, accident conditions, analysis fidelity, etc.

NRR Office Instruction LIC-206, Revision 1 Appendix C Page 5 of 20 3.0 INTEGRATED REVIEW TEAM PROCESS FOR TECHNICAL REVIEWERS The IRT process, as described in Appendix B of LIC-206, was developed to make it easier for NRR technical staff to utilize a wider range of tools and information to improve the efficiency and effectiveness of their decision-making in licensing reviews.

Staff requirements memorandum (SRM) on SECY-19-0036 states:

In any licensing review or other regulatory decision, the staff should apply risk-informed principles when strict, prescriptive application of deterministic criteria such as the single failure criterion is unnecessary to provide for reasonable assurance of adequate protection of public health and safety.

Ultimately, the work of the staff is to develop the basis to support a regulatory decision.

Staff effort expended on a licensing review should be commensurate with the safety significance of the proposed change. Explicit PRA risk insights may provide understanding of the appropriate level of review necessary to make the reasonable assurance finding but may not provide the sole basis of determining said finding.

It is well-established that regulatory decisions based on deterministic analyses are sufficient to provide reasonable assurance of safety. Nevertheless, over four decades since publication of WASH-1400, the NRC has used PRA and risk information to supplement deterministic analysis and continue to increase safety. The deterministic analysis is based on concepts such as single failure and worst-case assumptions, which may not explicitly utilize risk triplet information. The NRC staff is expected to use additional risk information to optimize their reviews. This can be helpful especially when the proposed change does not obviously meet deterministic criteria or established precedent. In rare cases, risk information generated by the staff or the licensee may point to situations where proposed licensing changes that meet deterministic requirements do not provide reasonable assurance of public safety. In such situations, staff should engage the licensee consistent with the guidance in NUREG-0800, Standard Review Plan for the Review of Safety Analysis Reports for Nuclear Power Plants: LWR Edition (SRP),

Section 19.2, Appendix D, and within the construct of agency guidance on backfit and forward-fit in Management Directive 8.4, Management of Backfitting, Forward Fitting, Issue Finality, and Information Requests. Risk information is increasingly important where risk insights are afforded by PRA analysis. In this context, PRA risk insights are information pertaining to the likelihood of an event causing a consequence (often referred to as conditional probabilities).

The interaction of components, systems, and operator actions is modeled in PRA, providing a wider basis for assessing system responses, and ultimately in making a regulatory decision.

Because both safety-and non-safety-related SSCs, as well as human action, are modeled in PRA, the insights can provide plant-level information on risk significance, safety significance, and defense in depth. Therefore, such insights provide a broader perspective compared to a system or component-specific perspective.

The concept of probability or likelihood has been instilled within the NRC since the beginning, even before the use of PRA, regardless of efforts to quantify it or to apply the risk triplet. Risk information should primarily appear in regulatory decisions as a consideration of likelihood. The

NRR Office Instruction LIC-206, Revision 1 Appendix C Page 6 of 20 use of the term likelihood does not automatically mean a DRA risk analyst should be involved in the review. Many of the traditional analysis techniques are still used when considering likelihood. However, some of the prescriptive requirements such as single failure or consideration of worst cases may not apply when considering likelihood. Consequently, the value of PRA and risk insights is to illuminate areas where implicit risk has been over-or under-stated in deterministic evaluations. Risk information such as the risk triplet and quantitative and probabilistic information may also be used to support the regulatory decision.

For Types 1 and 2 submittals, the lead TR is expected to use risk information where possible.

Section 6.0 of this appendix includes some example tools to assess the risk associated with the requested change. Additionally, the lead TR should be ready to discuss that risk during the initial IRT meeting. The intent is to inform the level of effort estimate provided to the licensee at the acceptance review stage. Sections 4.0 and 5.0 of this appendix provide two methods that assist in determining the scope and depth of the review, as well as reaching or supporting regulatory findings. The IRT will determine which approach is suited to support the review.

These approaches intend to determine the depth and scope of the review by characterizing the safety significance of the proposed licensing action. Based on the risk insights, the staff may adjust the need for and/or the extent of verification of methods or assumptions.

Cases may arise where an improved way of doing things departs from past precedent and practices or is at odds with existing staff processes. TRs who face such a situation should promptly raise that discrepancy through their management chain for consideration and develop an appropriate path forward. Remember that the focus should remain on the quality of the regulatory decisions and outcomes and not on adherence to the process itself. Beyond the refocusing of staff effort, technical staff should also work to develop the capability to use risk information and risk tools as part of their independent assessment of the adequacy of licensee approaches.

4.0 INTEGRATED REVIEW TEAM CHECKLIST Significant gains in efficiency can be achieved by aligning the depth and scope of these reviews with the safety-significance of the issues and SSCs. One way to achieve this goal is to effectively utilize risk information to determine an appropriate level of review, tailoring both the depth and scope of the review based on specific needs to make a reasonable assurance finding.

This section provides TRs a basic framework for considering the level of review of a license amendment request or other license-related activity. The intent is to ensure that risk information is appropriately included in the review. It is not a standalone decision-making procedure.

The introduction to NUREG-0800 under the subsection, Scope of Review of License Application, emphasizes that the staffs review constitutes an independent audit of the applicants analysis to justify a reasonable assurance finding. It clarifies that the staff may emphasize or de-emphasize particular aspects of an SRP section, as appropriate, for the application being reviewed. The scope and depth of the review should be documented in the SE. These SRP concepts provide the basis and foundation for the checklist tool.

NUREG-1764, Guidance for the Review of Changes to Human Actions, is an example that delineates how risk information may be used to adjust the staff reviews.

NRR Office Instruction LIC-206, Revision 1 Appendix C Page 7 of 20 During the IRT meetings, the PMs and reviewers will need to determine whether to use risk insights, and therefore, use an IRT approach. The checklist below (the IRT checklist) provides a logical path to determine the scope and depth of review considering risk insights. The result is only guidance; flexibility is needed to allow for other considerations. Ideally, the SE would emphasize areas of the license amendment request near regulatory limits and possibly de-emphasize areas where there is significant margin to the regulatory limit. Engineering analyses and reviewers experience, as well as engineering judgement, should be used to determine that the change will be near the regulatory limit in the technical area being evaluated.

The staff may use risk tools such as standardized plant analysis risk (SPAR) models, Plant Risk Information e-Books (PRIBs), the risk triplet, event frequencies, or failure probabilities to develop these qualitative risk insights. RG 1.174 quantitative risk thresholds are not expected to be used as the basis for approving changes that were not submitted in accordance with RG 1.174. However, they can be used to provide insights on the risk impact of the proposed licensing action.

The questions in the IRT checklist provide a framework for developing risk insights related to proposed changes and assessing their safety significance to help in determining the scope and depth of engineering evaluations. Quantitative or qualitative risk or PRA risk insights are considered in this checklist. The DRA risk analyst will assist the IRT, as needed, in differentiating probabilistic information from PRA insights. Furthermore, the staff will use this checklist to determine whether and in what form probabilistic, risk, or PRA insights should be included as part of an SE. DRA risk analysts should be consulted when significant risk discussion or quantitative risk information (e.g., information related to core damage frequency (CDF) and large early release frequency (LERF)) is included in the submittal and will be responsible for reviewing PRA-related information contained in SEs. DRA risk analysts can use NRCs SPAR models or publicly-available information provided by a licensee as part of risk-informed reviews to develop these PRA insights, when possible. It is also encouraged that the IRT uses DRA expertise when generic risk insights or probabilistic information is used in the submittal or staffs evaluation.

The IRT checklist helps guide TRs to determine an appropriate level of review. The following criteria should be considered before deciding on an appropriate level of review.

IRT CHECKLIST A.

Have design-basis or licensing-basis values OR assumptions changed?

Y N

NA The staff should focus the review of a proposed amendment on the changes to the licensing or design basis of the plant. If there are no changes to the design or licensing basis in a specific technical area, and the impact on risk is negligible, the staff may de-emphasize or limit the review of the proposed amendment in that technical area.

B.

Is there a change in how regulatory requirements are met?

Y N

NA

NRR Office Instruction LIC-206, Revision 1 Appendix C Page 8 of 20 The reviewer should determine whether the applicable regulations and criteria are properly applied. The licensee or applicant should identify the regulatory criteria used to meet the applicable regulations for the proposed change. The TR should note any significant deviations from approved guidance or review standards. The staff may emphasize the review of the deviations from approved guidance.

C.

Is the change risk-significant?

Y N

NA If the response to Questions C1 - C4 provided below is no (N), the proposed change is not considered to be risk-significant. If the answer to any of those questions is yes (Y) or not available (NA), the risk analyst may perform additional analysis in response to Question C5 or develop risk insights using SPAR models or other tools to determine whether the proposed change is risk-significant.

C1: PREVENTION: Does the proposed change more than minimally increase the likelihood of events that challenge normal plant operations and require successful mitigation?

Y N

NA

1. Does the proposed change more than minimally increase the likelihood of events from internal plant causes (e.g., hardware faults, internal floods, or internal fires) that challenge normal plant operation and require successful mitigation?

2. Does the proposed change introduce new credible events from internal plant causes (e.g., hardware faults, internal floods, or internal fires) that challenge normal plant operation and require successful mitigation?

3. Does the proposed change more than minimally increase the intensity or occurrence of external plant causes (e.g., earthquakes or high winds)?

4. Does the proposed change more than minimally increase the impacts or introduce new impacts of external plant causes (e.g., seismic interactions, submergence from external floods) that challenge normal plant operation and require successful mitigation?

5. Does the proposed change introduce external plant causes (e.g.,

earthquakes or high winds) or parameters (e.g., increase in type and number of tornado-generated missile population) not previously evaluated?

C2: PROTECTION: Does the proposed change adversely affect common cause failures by increasing the likelihood of such failures?

Y N

NA

NRR Office Instruction LIC-206, Revision 1 Appendix C Page 9 of 20

1. Does the proposed change more than minimally increase the likelihood of a cause or event that could cause simultaneous multiple component failures?

2. Is an SSC that is the subject of the proposed change impacted by a more than minimal increase in likelihood of common cause failures due to the proposed change?

3. Is an SSC that is necessary for successful mitigation from events caused by internal and external plant causes impacted by the more than minimal increase in likelihood of common cause failures due to the proposed change?

4. Is the impact of common cause failures and/or their increase due to the proposed change risk-significant?

C3: MITIGATION:

Does the proposed change affect the likelihood of successful mitigation (i.e., plant response) of challenges to normal plant operation (events that would cause the plant to implement abnormal operating procedures and emergency operating procedures)?

Y N

NA

1. Is the likelihood of the affected SSCs to successfully perform their required function(s) (during a specified period) affected by the proposed change?

2. What is the likelihood of the affected SSCs being unavailable for test or maintenance? Is this unavailability of an SSC due to test and maintenance affected by the proposed change?

3. Does the proposed change affect the likelihood of restoring a function due to failure of affected SSCs?

a. Does the affected function rely on diverse/redundant SSCs?

b. What is the likelihood of restoring the affected SSCs or affected function if such a failure occurs, and does the proposed change decrease the likelihood?

c. Do the proposed compensatory measures manage risk-significant configurations occurring due to the proposed change?

C4: HUMAN ACTIONS: Does the proposed change more than minimally increase failures or unavailability of SSCs (or function) caused by human inaction or inappropriate actions or result in new credible human failure event?

Y N

NA

1. Are existing or new human actions necessary to implement the proposed change?

2. What is the likelihood of errors associated with new or affected (existing) human actions?

3. Are new human actions important to preserving layers of defense?

NRR Office Instruction LIC-206, Revision 1 Appendix C Page 10 of 20

4. What are the absolute or relative contributions of new or affected human actions to overall risk from the change?

C5: ADDITIONAL RISK INFORMATION:

1. If risk information is known, what are some readily available risk insights applicable to the proposed change?

a. What is the overall risk of the plant (i.e., baseline risk)?

b. What are the dominant risk contributors (or dominant contributors to change in risk) (e.g., at the initiating events, accident sequences, and cut sets levels)?

c. Has the NRC reviewed applications that proposed similar changes? What conclusions were made in those reviews by the NRC?

2. What are the absolute and relative contributions of the affected SSCs, collectively and individually, to overall risk?

3. What is the increase in risk if the affected SSC (or a collection of SSCs) was assumed to be failed or unavailable?

a. What is the relative contribution of the affected SSC (or a collection of SSCs) to the calculated risk?

b. What is the sensitivity of risk to the performance of the affected SSC?

D. Is the applicant using an approved precedent?

Y N

NA

1. Precedent licensing actions are those that have been completed with a similar proposed change and regulatory basis.

2. The reviewer should confirm that cited precedents are justified and used appropriately. A precedent of approval itself is not a justification for a proposed change but can facilitate the review by allowing the technical staff to make appropriate use of information from previously-approved reviews.

Although the licensee or applicant is not required to cite a precedent, the technical staff should remain cognizant of other applicable licensing information.

3. Past precedence should be used to emphasize or de-emphasize identified areas of review. LIC-101, Section 4.2, Use of Precedent and References to Topical Reports, provides additional information regarding the use of precedent.

E. Is consistency with defense-in-depth philosophy challenged?

Y N

NA The change is considered to maintain consistency with the defense-in-depth philosophy if an integrated assessment demonstrates no significant impact on a single consideration below or there is not a significant impact collectively across all considerations.

NRR Office Instruction LIC-206, Revision 1 Appendix C Page 11 of 20

1. Does the proposed change significantly increase the likelihood of an event or introduce a new event that could simultaneously challenge multiple barriers (e.g., interfacing systems loss-of-coolant accident and steam generator tube rupture)?

2. Does the balance among the layers of defense remain appropriate?

3. Does the proposed change significantly impact the containment function or SSCs supporting that function?

4. Does the proposed change significantly reduce the effectiveness of the emergency preparedness program?

5. Does the proposed change include an overreliance on programmatic activities as compensatory measures (see also item C4)? (Reliance on a programmatic activity as a compensatory measure might be considered excessive when a program is substituted for an engineered means of performing a safety function or when the failure of the programmatic activity could prevent an engineered safety feature from performing its intended function.)

6. Does the proposed change result in a decrease in redundancy, independency, or diversity of system functions impacted by the change?

Does the proposed change decrease the redundancy, independency, or diversity of system functions not impacted by the change?

F. Are safety margins challenged?

Y N

NA

1. Does the proposed change challenge meeting the codes and standards or are there alternatives approved for use by the NRC?

2. Does the proposed change challenge meeting safety analysis acceptance criteria in the licensing basis (e.g., FSAR, supporting analyses)?

5.0 CONSTRUCTING A RISK ARGUMENT Submittals may contain a probabilistic or risk argument. These arguments are often described in terms such as probabilistic approach or likelihood. Risk information concerns aspects of the risk triplet:

1. Failure modes or initiators - what can go wrong?

2. Probability or frequency of occurrence - how likely is it?

3. Consequence - what are the consequences if it occurs?

Probabilistic information is frequently encountered in applications with a time component. For example, the length of an equipment-allowed outage time, a period of degradation between inspections, or testing intervals. Alternatively, the time component may relate to the length of time a submittal is to be implemented such as for a temporary repair, an inspection delay until the next outage, or a fire-watch implemented when a particular piece of equipment is offline for testing.

NRR Office Instruction LIC-206, Revision 1 Appendix C Page 12 of 20 Risk information is best leveraged by the reviewer to support the reasonable assurance of the adequate protection portion of a review. Additional guidance for reviewing risk information is provided in Section 5.2 of this appendix.

5.1 Applying a Risk Argument The TR should leverage risk information to determine the necessary level of review to support the determination of reasonable assurance of adequate protection. The reviewer should confirm risk information in the submittal and/or leverage other sources of information (precedent, NUREGs, NRC databases, or risk tools (see Section 6.0 of this appendix)).

Consultation with PRA experts (e.g., Senior Level Advisor in PRA in NRR or RES) should be considered if there are discrepancies between the conclusion from the licensees information and the conclusions reached from information available to the staff. The contribution of this information to the final staff determination should be clearly articulated, documented, and supported. In some cases where risk information provided by a licensee becomes instrumental to the safety conclusion, the staff may consider a peer review. ADM-405, NRR Technical Work Product Quality and Consistency, provides guidance on situations that may prompt a peer review and guidance on how a peer review must be conducted.

The necessary level of review should be aligned to the risk significance of the requested change. For example, a change that impacts all safety-related diesel generators would likely require a higher standard of evidence (higher quality of inputs, more conservative acceptance criteria, more rigorous consequence analysis) than a change to a single medium risk pump.

When using risk information in a regulatory decision, the reviewer should clearly identify:

1) what risk information was used, 2) how the risk information was derived, and 3) how the risk information was reviewed. For example, a decision for which the basis includes an argument concerning occurrence frequency of an event may be approved by a reviewers use of a NUREG to support that frequency or to establish a lack of consequence.

5.2 Flowchart for Reviewing Risk Information The methodology in this section can be used generally to analyze a variety of situations. Risk arguments can be constructed using either licensee-submitted information or staff-generated information sourced from the risk tools in Section 6.0 of this appendix or other sources. Risk arguments are best analyzed from a risk triplet perspective. In some situations where staff generate risk information, staff should consider use of a peer review in accordance with guidance provided in ADM-405. In a generic sense, all risk arguments begin with initiators multiplied by associated frequencies linked to a measure of consequence to produce a final risk metric.

Completing the full risk triplet is not always necessary to produce an adequate argument, but such arguments must always be considered in the framework of a risk triplet. For example, an argument concerning a particular degradation mechanism may be adequate, without describing a full set of initiator and frequency information, if the consequences are shown to be negligible.

In PRA, the final risk metric is often CDF or LERF. The initiators are initiating events and the frequencies or probabilities are the estimates of the occurrence of an initiator and/or the likelihood of failure of various SSCs in different failure modes. The consequence is a

NRR Office Instruction LIC-206, Revision 1 Appendix C Page 13 of 20 conditional CDF or LERF for a particular initiating event. By combining the initiating event frequencies with the consequences of each initiating event, a CDF or LERF is derived.

In probabilistic fracture mechanics, the risk metric may be a level of leakage or net section wastage before identification and remediation (generally by some inspection plan.) The initiators are the credible degradation mechanisms such as stress corrosion cracking, microbiologically-induced corrosion, wear, etc. The frequencies are based on operating experience and testing for initiating and crack growth rates. This information is combined to provide assurance that an inspection plan will preclude inappropriate levels of leakage or net section wastage in-between inspections. Alternatively, this information may provide a basis for delaying remediation of an identified crack until a convenient period, such as an outage, without challenging the function of the host component.

Figure 2 illustrates the overall process for reviewing probabilistic information. Categories A-C are described below.

Figure 2: Risk Information Review Flowchart

NRR Office Instruction LIC-206, Revision 1 Appendix C Page 14 of 20 Initiators (Box A)

Figure 3: Initiator Review Flowchart To review the initiator component of a risk argument, a reviewer should first determine if the initiators presented are credible. A credible initiator is one that could plausibly relate to the safety-significant aspects of the review. Having made a determination considering all initiators presented, the reviewer should then consider if relevant credible initiators have been omitted. If the initiators presented are both credible and complete with regard to the safety-significant aspects of the review, they may support acceptance of the regulatory decision.

If the initiators are not credible and/or incomplete, the reviewer may be able to complete the review by drawing on alternate sources of information. It is good practice for reviewers to draw on existing secondary and tertiary resources when conducting reviews. If the reviewer does so, it is imperative to include citation and discussion of these resources in any evaluation, if it forms a critical part of the basis for staff conclusions.

Example A1:

It is claimed that a canned pump cannot leak through seals because it has none. This is deemed self-evidently credible, so seals are not a leak initiator.

Example A2:

The canned pump does, however, rely on a bolted connection to a pressure vessel; consequently, material degradation of the bolts is identified by the reviewer as a credible initiator for leakage.

NRR Office Instruction LIC-206, Revision 1 Appendix C Page 15 of 20 Frequency Distributions (Box B)

Figure 4: Frequency Distribution Review Flowchart To review the frequency distribution component of a risk argument, a reviewer should first determine if each frequency or probability distribution is credible. A credible frequency or probability distribution is one that is expected to plausibly describe the likelihood of a particular event or the likelihood of SSC failure. If the frequency or probability distribution is credible, then it may form part of the basis of an acceptance. If the frequency or probability distributions are found to not be credible, the reviewer should then consider if an appropriate built-in frequency or probability distribution update mechanism is presented. If a credible frequency or probability distribution update process is incorporated into the subject application, it may support acceptance of the probabilistic argument, even if the initial distributions presented are inadequate.

If the frequency or probability distributions are not credible and/or incomplete, and no credible frequency or probability distribution update process is included, then the reviewer may be able to complete the review by drawing on alternate sources of information. If the reviewer does so, it is imperative to include citation and discussion of these resources in any evaluation as it forms a critical part of the basis for staff conclusions.

NRR Office Instruction LIC-206, Revision 1 Appendix C Page 16 of 20 Example B1:

It is claimed that a A452-Y pump operates with a 99 percent reliability during its first 5 years of operation. This is based on operating experience with the essentially identical A452-X pump (which was red instead of blue). This is credible, as it is well-documented that A452-Y and A452-X pumps are identical except for their color.

Example B2:

The reviewer identifies that the A452-Y pump is to be installed with an operating lifetime of 10 years, whereas operating data on A452-X pumps spans 5 years of operation, and consequently requires a failure distribution for operation past 5 years. The reviewer is able to acquire operating experience for A452-X pumps, up to 12 years of operation, from an International Atomic Energy Agency (IAEA) conference proceeding and finds that they are of high pedigree. The reviewer uses the IAEA conference proceeding as a basis, relying on the applicants living program for pump maintenance and lifetime estimation to adjust initial expectations as operating experience for A452-Y pumps accumulates.

Consequences (Box C)

Figure 5: Consequence Review Flowchart To review the consequence component of a risk argument, a reviewer should first determine if the consequences presented or generated are credible. A credible consequence is one that plausibly describes the result if an initiator occurs and the SSC of interest fails (either individually or in combination with other failures). Having made a determination considering all consequences presented, the reviewer should then consider if relevant, credible consequences have been omitted. If the consequences presented are both credible and complete regarding the safety-significant aspects of the review, they may support a decision of the subject risk argument.

NRR Office Instruction LIC-206, Revision 1 Appendix C Page 17 of 20 If the consequences are not credible and/or incomplete, the reviewer may be able to complete the review by drawing on alternate sources of information. If the reviewer does so, it is imperative to include citation and discussion of these resources in any evaluation if it forms a critical part of the basis for staff conclusions.

Example C1:

It is claimed that an explosive relief valve will not impact adjacent sensors or systems, as there is no credible ejection of sufficient mass or energy to damage adjacent sensors or systems. The reviewer concurs that the applicant has credibly established that the consequence of this valve actuating is low in this respect.

Example C2:

The reviewer notes that the explosive relief valve actuation can damage adjacent explosive relief valves and cause a common cause failure not analyzed in the application. The reviewer can use available risk information or seek support from DRA to generate relevant risk information that can provide information about the consequences of the common cause failure originating from an explosive relief value actuation.

Drawing Conclusions (Box D)

Figure 6: Risk Information Review Flowchart To make a determination regarding whether the goal/claim presented by a risk argument is supported requires considering the credibility and completeness of the supporting initiators,

NRR Office Instruction LIC-206, Revision 1 Appendix C Page 18 of 20 frequencies or probabilities, and consequences. Not all goals/claims require Boxes A-B-C to be presented or credible; this must be considered on a case-by-case basis.

If the individual components of the risk argument are all credible and complete, then they may support the basis for a decision. In many cases, only one or two of the components may be presented (for example, an argument may be made using only the frequency of initiators), in which case only those components presented may be sufficient to make a determination.

It may be the case that while the individual components (initiators, frequencies or probabilities, and consequences) are deficient in some respects, in aggregate, they form a bounding or sufficient argument. If the reviewer determines this to be the case, the reviewer should carefully document this.

Finally, if the presented components do not support acceptance, the staff may be able to complete their review by drawing on alternate sources of information. If the reviewer does so, it is imperative to include citation and discussion of these sources in any evaluation, as it forms a critical part of the basis for staff conclusions.

It is the nature of probabilistic arguments that generally they are only presented when they support the goal/claim being made. Consequently, rejecting an argument based on presented information would likely require relying on alternate sources of information. This is in contrast, for example, to deterministic methodologies where substitution of more appropriate inputs may cause the methodology to conclusively refute the presented goal/claim.

Example D1:

The reviewer determines that the applicant failed to account for a significant number of initiators and associated consequences. The applicant did, however, assume a very conservative frequency for the initiators, leading to a net conservative evaluation. The reviewer then determines that addition of the missing initiators and associated consequences would be bounded by the risk conclusions of the applicant. The reviewer documents this in the evaluation as supporting the applicants conclusions, despite the deficiencies in the application.

6.0 RISK TOOLS This section contains a sample list of risk and PRA tools to help TRs gain insights on the risk significance of components, the risk profile of facilities, the significance and interplay between plant components, and the relative likelihood of and trends behind failure modes. The goal is to provide TRs with basic information to facilitate interactions with risk analysts, to further the understanding and use of submitted risk information, and to enable reviewers to use risk information in decision-making (e.g., by independently assessing the adequacy of licensee approaches.) Consult a risk analyst if any questions or concerns arise on the applicability or accuracy of any risk information.

Reactor Operational Experience Results and Databases: This website contains results for a variety of studies related to component and system reliability and trending studies.

(https://nrcoe.inl.gov/resultsdb/)

NRR Office Instruction LIC-206, Revision 1 Appendix C Page 19 of 20 Plant Risk Information e-Book (PRIB): Each NRC SPAR model includes a PRIB report, which provides a snapshot of the important contributors to the modeled plants baseline risk. (https://nrc.gov/res/59657)

Risk Assessment Standardization Project (RASP) Tool Box: The RASP Tool Box provides access to the tools and references for the analysis of operational events for the regulatory applications such as Phase 3 of the Significant Determination Process and Accident Sequence Precursor Program. (https://nrc.gov/res/24233)

PRA Toolkit in ADAMS (Accession No. ML17300A061 (non-public)): Staff developed this knowledge management toolkit to capture the nuclear power plant PRA-related knowledge, experience, regulatory, and technical documents, and assist staff in locating information to perform technical reviews and inspections.

NUREG/CR-6823, Handbook of Parameter Estimation for Probabilistic Risk Assessment (ADAMS Accession No. ML032900131): This handbook provides guidance on sources of information and methods for estimating the parameters used in PRA models (e.g., initiating event frequencies, component failure rates and unavailabilities, and equipment non-recovery probabilities) and for quantifying the uncertainties in the estimates.

7.0 SAMPLE SAFETY EVALUATION LANGUAGE To provide context and consistency to TRs, a collection of SE language is available in ADAMS at Accession No. ML20108E881.

8.0 REFERENCES

1. U.S. Nuclear Regulatory Commission, Action Plan: Risk-Informed Decision-Making in Licensing Reviews, dated August 11, 2017 (ADAMS Accession No. ML17219A346).

2. U.S. Nuclear Regulatory Commission, SECY-17-0112, Plans for Increasing Staff Capabilities to Use Risk Information in Decision-Making Activities, dated November 13, 2017 (ADAMS Accession No. ML17270A192).

3. U.S. Nuclear Regulatory Commission, Staff Requirements Memorandum for SECY-98-144, White Paper on Risk-Informed and Performance-Based Regulation, dated March 1, 1999 (ADAMS Accession No. ML003753601).

4. U.S. Nuclear Regulatory Commission, NUREG-2122, Glossary of Risk-Related Terms in Support of Risk-Informed Decisionmaking, dated November 2013 (ADAMS Accession No. ML13311A353).

5. U.S. Nuclear Regulatory Commission, Regulatory Guide 1.174, Revision 3, An Approach for Using Probabilistic Risk Assessment in Risk-Informed Decisions on Plant-Specific Changes to the Licensing Basis, dated January 2018 (ADAMS Accession No. ML17317A256).

6. U.S. Nuclear Regulatory Commission, NUREG-0800, Standard Review Plan for the Review of Safety Analysis Reports for Nuclear Power Plants: LWR.

7. U.S. Nuclear Regulatory Commission, Staff Requirements Memorandum for SECY-19-0036, Application of the Single Failure Criterion to Nuscale Power LLCs Inadvertent Actuation Block Valves, dated July 2, 2019 (ADAMS Accession No. ML19183A408).

NRR Office Instruction LIC-206, Revision 1 Appendix C Page 20 of 20

8. U.S. Nuclear Regulatory Commission, NUREG-0800, Chapter 19, Severe Accidents, Section 19.2, Review of Risk Information Used to Support Permanent Plant-Specific Changes to the Licensing Basis: General Guidance, dated June 2007 (ADAMS Accession No. ML071700658).

9. U.S. Nuclear Regulatory Commission, Management Directive 8.4, Management of Backfitting, Forward Fitting, Issue Finality, and Information Requests, dated September 20, 2019 (ADAMS Accession No. ML18093B087).

10. U.S. Nuclear Regulatory Commission, NUREG-1764, Guidance for Review of Changes to Human Actions, dated September 2007 (ADAMS Accession No. ML072640413).

11. U.S. Nuclear Regulatory Commission, ADM-405, Revision 3, NRR Technical Work Product Quality and Consistency, March 23, 2020, (ADAMS Accession No. ML20066J279).