ML19144A190

SDP for Rope Memo from Mtm to Mxf (Rev 0)

ML19144A190
Person / Time
Issue date:	10/11/2019
From:	Michael Montecalvo NRC/NRR/DRA/APOB
To:	Mike Franovich NRC/NRR/DRA
References
Download: ML19144A190 (24)

Text

UNITED STATES NUCLEAR REGULATORY COMMISSION WASHINGTON, D.C. 20555-0001 October 11, 2019 MEMORANDUM TO: Michael X. Franovich, Director Division of Risk Assessment Office of Nuclear Reactor Regulation FROM: Michael T. Montecalvo /RA/

Reliability and Risk Analyst PRA Oversight Branch Division of Risk Assessment Office of Nuclear Reactor Regulation

SUBJECT:

SIGNIFICANCE DETERMINATION PROCESS INPUT FOR 2018 REACTOR OVERSIGHT PROCESS ENHANCEMENT EFFORTS The purpose of this memorandum is to document the input provided for recommendations made regarding changes to the Significance Determination Process (SDP) during the Reactor Oversight Process (ROP) Enhancement effort started in 2018. Specifically, the recommendations contained in this memo are related to the Initiating Events, Mitigating Systems, and Barrier Integrity cornerstones.

In January 2018, the Executive Director for Operations established the Transformation Team.

Its objective was to identify potential transformational changes to the U.S. Nuclear Regulatory Commissions (NRCs) regulatory framework, culture, and infrastructure to further enhance the agencys effectiveness, efficiency, and agility. The team provided the Commission the results of its review in SECY-18-0060, Achieving Modern Risk-Informed Regulation, dated May 23, 2018 (Agencywide Documents Access and Management System (ADAMS) Accession No. ML18110A186).

As discussed in that paper, the team solicited extensive feedback from both internal and external stakeholders to inform its evaluation. Some of the feedback given to the team was not within the scope of its review and was referred to the responsible NRC organizations for consideration. The team received 72 suggestions for improvements to the ROP, which were provided to the Office of Nuclear Reactor Regulation (NRR) for its consideration (ADAMS Accession No. ML18292A594).

On September 19, 2018, the Nuclear Energy Institute (NEI) submitted a letter to the Director of NRR to consolidate and prioritize industry recommendations to enhance the ROP (ADAMS Accession No. ML18262A322). NEIs letter provided a perspective that the ROP remains sound, that it is an effective model for regulatory oversight, and that the fundamental structure of the ROP has played an important role in incentivizing good performance and focusing NRC CONTACT: Michael T. Montecalvo, NRR/DRA 301-415-1678

M. Franovich resources on departures from desired performance. However, NEIs letter argued that certain improvements to the ROP are warranted, because of the maturity of the U.S. nuclear reactors, improved understanding of risk and safety margins, and greater use of risk-informed decision-making principles since the inception of the ROP. The letter offered 27 recommendations grouped in the following four general areas:

(1) Assess the size and scope of the baseline inspection program and to consider giving credit for licensee self-assessments.

(2) Revise the assessment process, with a primary objective of reducing the burden associated with White inspection findings.

(3) Improve various SDP appendices and enact enhancements to increase its efficiency and effectiveness.

(4) Enhance the NRC and industry response to inspection findings, such as through improved communication protocols.

Overall, NEIs letter indicated that adopting these recommendations would promote prompt resolution of issues and returning the plant to its baseline risk profile as soon as practical, and that the recommendations are consistent with the NRCs Principles of Good Regulation.

NRRs Division of Risk Assessment (DRA) was given primary responsibility for the SDP recommendations taken from both the transformation team and the industry through the NEI letter noted above. In total there were 10 recommendations that were evaluated and dispositioned during this process. Six recommendations were closed after evaluation with no further action, two recommendations were closed to actions already underway, and two recommendations require industry actions to move forward. The enclosure to this memo contains the results of the evaluation, any actions taken, and the justification for the dispositioning of the recommendation.

Enclosure:

As stated

M. Franovich

SUBJECT:

SIGNIFICANCE DETERMINATION PROCESS INPUT FOR 2018 REACTOR OVERSIGHT PROCESS ENHANCEMENT EFFORTS DATED:

DISTRIBUTION:

PUBLIC MMontecalvo, NRR AZoulis, NRR RFelts, NRR MFranovich, NRR RidsNrrDra RidsNrrDraApob ADAMS Accession No: ML19144A190 NRR-106 OFFICE NRR/DRA/APOB NRR/DRA/APOB: BC NRR/DRA: D NAME MMontecalvo AZoulis MFranovich DATE 10/08/2019 10/08/2019 10/11/2019 OFFICIAL AGENCY RECORD

EVALUATION OF THE SIGNIFICANCE DETERMINATION PROCESS RECOMMENDATIONS FOR THE 2018 REACTOR OVERSIGHT PROCESS ENHANCEMENT EFFORT BACKGROUND:

SECY-99-007A, Recommendations for Reactor Oversight Process Improvements describes a method for assigning a probabilistic public health and safety risk characterization to licensee performance deficiencies (PDs) related to reactor safety. This risk characterization method was the first of a set of methods and tools developed that became central elements of the Significance Determination Process (SDP) to determine reactor inspection finding 1 significance consistent with the thresholds used for the risk-informed plant Performance Indicators (PIs).

This allowed inspection findings and PIs to be used consistently as inputs to the overall plant performance assessment portion of the Reactor Oversight Process (ROP) and to properly inform U.S. Nuclear Regulatory Commission (NRC) staff response to licensee performance.

Subsequently, other SDP tools were developed to characterize the significance of inspection findings associated with the cornerstones of the ROP (e.g., emergency preparedness, security),

and other areas of regulatory interest. These SDP tools either used quantitative risk evaluation methods or were risk-informed through expert judgement of the staff. As experience was gained, the SDP tools have been continuously improved and additional SDP tools were developed. A list of the current SDP tools is included as Attachment 1 of this document.

Fundamental Attributes of the SDP Tools Newly developed SDP tools and any changes to existing tools should comply with the fundamental attributes applicable to all SDPs. These fundamental attributes are introduced here and are explained in more detail in Inspection Manual Chapter (IMC) 0308 Att. 3, Significance Determination Process Technical Basis (Ref. 1).

Objectivity SDP tools should provide a decision logic or framework that remains relatively constant across applicable inspection findings. This enhances objectivity by reducing the likelihood that SDP results are influenced by different value judgments held by different individuals. The test of having achieved such objectivity is when different individuals using a given SDP decision logic or framework arrive at the same result when using the same input conditions and assumptions.

This attribute is achieved in part through peer reviews of SDP assessments to assure consistency between analysts.

Enclosure 1 Performance deficiencies that are determined to be of more than minor significance using IMC 0612 Appendix B - Issue Screening are referred to as inspection findings.

Scrutability (Openness)

The SDP should provide a clear framework to facilitate a shared understanding of each significance determination and its basis among technically knowledgeable stakeholders (both internal and external). This shared understanding allows for broad and independent validation of the staffs objectivity and most directly enhances NRC public credibility. When a quantitative risk model is used, the greatest challenge to achieving this attribute is to allow stakeholders a means to independently assess SDP result sensitivity to the most influential assumptions, to understand the basis of the assumptions, and to reveal the limitations and uncertainties of the risk model used and how these were considered by the staff in arriving at a final result. When quantitative risk insights and inputs from other factors considered for decision making are used, the bases of the significant factors influencing the decision outcome must be clearly documented in detail for scrutability and effective communication of the final risk-informed decision.

Timeliness The SDP is intended to support timely decisions to assess the safety significance of findings generally within a timeframe consistent with quarterly updates of the Action Matrix (described in IMC 0305) portion of the performance assessment component of the ROP. The SDP timeliness goal is therefore 90 days from the time the inspection finding is formally documented establishing the need for further review to determine significance. The process milestone for the end of the 90-day timeliness goal is the issuance of the final significance determination letter after timely completion of a public Regulatory Conference or review of a licensee written response.

Achieving SDP timeliness using best available information requires that NRC staff effectively receive information from a licensee, starting when a finding is identified. In addition, maintaining public credibility requires timely public notification of the existence of a potentially significant finding and identification of the staffs preliminary basis for potential significance. When appropriate, preliminary SDP results should reveal what influential information is needed from the licensee that might change the preliminary decision.

Inspection Planning The SDPs inform inspection activities and improve the effectiveness of the inspectors who directly implement the reactor inspection program. Inspectors can develop risk-informed inspection samples by reviewing information in the NRC Standardized Plant Analysis Risk (SPAR) Plant Risk Information e-Book (PRIB), and the SDP Workspace module in the Systems Analysis Programs for Hands-On Integrated Reliability Evaluations (SAPHIRE) code and through discussions with Regional Senior Reactor Analysts (SRAs).

Responsibility for Significance Determinations IMC-0308, Att. 3 outlines the expectations and responsibility for significance determinations:

Each SDP result is the sole responsibility of the NRC staff. The SDP is not a consensus process with a licensee or other parties and no staff/licensee interactions should be construed as a negotiation. The ROP requires the staff to make decisions using best available information in a timely manner and that the bases of SDP results be clear and publicly available, to the extent practical and

permitted by policy (e.g., security issues). The SDP affords licensees an opportunity to provide available information that may be useful to the staff in arriving at a best-informed decision within a reasonable time of 90 days. The staff is obligated to be clear about the basis for any SDP result and to consider licensee-provided information. The staff is not obligated to have proof of the assumptions made relative to an SDP result basis. Staff engineering or technical judgment is often required, but should be consistent with similar previous circumstances, as appropriate. The staffs technical judgment should be made objective through its use within the appropriate SDP tool used as a decision framework. However, a licensee may appeal the staffs decision if the pre-requisites of IMC 0609, Attachment 2, Process for Appealing NRC Characterization of Inspection Findings (SDP Appeal Process) are met.

Independence from Other NRC Processes The significance of inspection findings, as characterized by the SDP, is represented by a color scheme (i.e., Green, White, Yellow, Red) that is consistent with that used for the PIs. The color of an SDP result carries with it an assurance that all the specific applicable process provisions of the overall SDP have been met. Other agency programs may not have the same process attributes, definitions, or assurances, and therefore should not be characterized using the SDP color scheme. These may include severity levels of traditional enforcement and other agency probabilistic risk evaluation programs (e.g., Accident Sequence Precursor (ASP) event or condition evaluations). Keeping the SDP color scheme independent from other agency programs also aids in ensuring clear and consistent public representations that inspection findings with colors are inputs to the ROP assessment of licensee performance.

External Stakeholder Participation in SDP Development and Changes The ROP was developed with substantial involvement from both internal and external stakeholders, notably increasing openness and acceptance of the ROP. In addition, the ROP is an integrated set of tools and processes in which changes to one component may affect other components. Therefore, changes to the SDP must be carefully considered and, in some cases, it may be beneficial to engage external stakeholders prior to making substantive changes to the SDP or its component tools. Such engagement is not necessarily intended to arrive at consensus, but rather to ensure that the staff has considered possible effects which could occur from a substantive change.

Use of Computer Based Tools in the Reactor Safety Cornerstone For this discussion it will be useful to reference the process used for Initiating Events, Mitigating Systems, and Barrier Integrity Cornerstone inspection findings that are subject to IMC-0609 Appendix A, Significance Determination Process for Findings At-Power (Ref. 10). Since the initial implementation of the ROP, the at-power SDP has involved a three-phased approach.

The initial phase (Phase 1) was designed to screen findings of low risk significance to green to allow the staff to focus more resources on risk significant findings. The second phase (Phase 2) was designed to estimate the risk significance of the finding, provide an engineering understanding of the finding, and serve as an additional screening tool to identify low risk significant findings that did not screen out in the initial phase. The at-power Phase 2 process consisted of site-specific pre-solved tables and risk-informed notebooks designed using risk insights from the licensees risk model. The third phase (Phase 3) was designed to add

specificity to the Phase 2 risk evaluation if needed (e.g., provide more detailed analyses, reduce uncertainties).

Maintaining the site-specific pre-solved tables and risk-informed notebooks proved to be a challenging task. As plants implemented equipment modifications and associated revisions to the plant risk model, the accuracy of the pre-solved tables and risk-informed notebooks began to degrade. Instead of revising all the plant specific pre-solved tables and risk-informed notebooks, the agency decided to transition from the pre-solved tables and risk-informed notebooks to SAPHIRE and the site-specific SPAR models which provides an efficient and effective infrastructure that facilitates risk model fidelity and updates.

NRC Development and Use of SPAR Models The objective of the NRCs SPAR Model Program is to develop and maintain standardized risk analysis models and tools to support various regulatory activities, including the ASP Program, the Incident Investigation Program described in Management Directive (MD) 8.3, and the SDP.

The SPAR models have evolved from two sets of simplified event trees initially used to perform precursor analyses in the early 1980s. SPAR models are built with a standard modeling approach, using consistent modeling conventions that enable staff to easily use the models across a variety of U.S. nuclear power plant (NPP) designs.

Software Platform and Scope of Models As stated above, SPAR models are run on a single probabilistic risk and reliability assessment software platform, the SAPHIRE computer code. The NRC SPAR models are maintained by the Idaho National Laboratory (INL) on a website accessible to NRC users for use in various regulatory programs. They frequently support NRC users in modifications to the models required for evaluating the risk-significance of plant conditions or events. It is important to note that the SAPHIRE code was specifically designed to provide risk results in forms for ease of use in NRC regulatory applications (e.g., delta core damage frequency, conditional core damage probability).

SAPHIRE can be used to model a complex system's response to initiating events and quantify associated consequential outcome frequencies or probabilities. Specifically, for nuclear power plant applications, SAPHIRE can identify important contributors to core damage (Level 1 probabilistic risks assessment (PRA)) and containment failure during a severe accident which leads to releases (Level 2 PRA). It can be used for a PRA where the reactor is at full power, low power, or at shutdown conditions. Furthermore, it can be used to analyze both internal and external initiating events and has special features for managing models such as flooding and fire.

It can also be used in a limited manner to quantify risk, using PRA techniques, in terms of release consequences to the public and environment (Level 3 PRA).

Todays SPAR models for internal events are far more comprehensive than their predecessors.

For example, the revised SPAR models include improved loss of offsite power and Station Blackout models; an improved reactor coolant pump seal failure model; new support system initiating event models; and updated estimates of accident initiator frequencies and equipment reliability based on recent operating experience data. The NRC is continuing to develop new SPAR modules for assessing plant risk from internal fires, external hazards (e.g., high wind and seismic events), and for assessing post-core damage severe accident progression. The NRCs SPAR models are generally focused on at-power operations, however several low power/shutdown (LPSD) models have been developed and are being used to support regulatory programs.

Although the SPAR models are plant-specific models, they rely on a set of standardized modeling conventions (e.g., standardized naming conventions, standard modeling approaches, and logic structure). They employ a standard approach for event-tree development, as well as a standard approach for initiating event frequencies, equipment performance parameters, and human performance data. These input data can be modified to be more plant- and event-specific, when needed. SPAR standardization is needed to allow agency risk analysts to efficiently use SPAR models for a wide variety of NPPs without having to relearn modeling conventions and basic assumptions. Although the system fault trees contained in the SPAR models generally are not as detailed as those in licensee PRAs, in some cases SPAR models may contain more sophisticated modeling, such as for common-cause failures, support systems, and losses of offsite power.

Model Improvements and Quality Assurance (QA)

The staff initiated the Risk Assessment Standardization Project (RASP) in 2004. A primary focus of RASP was to standardize risk analyses performed in SDP, in ASP, and the NRC Incident Investigation Program completed under MD 8.3. The project enhanced SPAR models to be more plant-specific and documented consistent methods and guidelines for risk assessments of internal events during power operations; internal fires and floods; external hazards (e.g., seismic events and tornadoes); and internal events during LPSD operations. One outcome of the RASP was the development of the Risk Assessment of Operational Events Handbook (commonly referred to as the RASP Handbook) and better alignment between the event assessment processes. The RASP Handbook is a living document that undergoes periodic revision based on guidance needs when new issues regarding PRA techniques are identified.

The staff implemented a QA plan for the SPAR models in 2006. The main objective of the plan is to ensure that the SPAR models continue to represent the as-built, as-operated NPPs and continue to be of sufficient adequacy for performing event assessments of operational events in support of the staffs risk-informed activities. In addition to model development, the QA plan provides mechanisms for internal and external peer review, validation and verification, and configuration control of the SPAR models. The staff has processes in place to verify, validate, and benchmark these models according to the guidelines and standards established by the SPAR Model Program. As part of this process, the staff performs reviews of the SPAR models and results against the licensee PRA models, when applicable. The QA plan also provides a feedback process from the model users for error reporting, tracking, and resolution.

It should be noted that the SPAR models are generally used to categorize and prioritize operational events and conditions, including licensee non-compliance issues with existing regulations. Although the SPAR models are not maintained under a RG 1.200 program, the SPAR QA program and other process controls (such as internal and external reviews) provide assurance that SPAR-based analyses appropriately reflect the as-built, as-operated NPP.

Model Updates and Continuing Development Existing SPAR models for operating plants need to be updated regularly because of significant plant changes that may affect the risk profile of the plant. As SPAR models are updated, their documentation (i.e., the model report and the PRIB summary reports) is also updated to represent the latest PRA information included in each SPAR model. Comparisons between the SPAR model baseline results and licensee model results (when voluntarily submitted by the licensee) are also performed. These comparisons help ensure that SPAR models and associated risk assessments that support the SDP are of high quality and reflect the as-built, as-operated plants. The NRC is pursuing development of SPAR All- Hazard (SPAR-AHZ) models,

which contain accident scenarios from all hazard categories (including seismic, high wind, and internal fire) applicable to a given site. This initiative will allow the NRC to better understand the complete risk profile of the plants to ensure efficient regulatory decisions can be made.

Changes to the SDP after September 11, 2001 Following the events of September 11, 2001, the Commission determined that the general threat environment warranted all licensees to establish specified interim safeguards and security compensatory measures. These compensatory measures were eventually codified as 10 CFR 50.54(hh)(2). The provisions of 10 CFR 50.54(hh)(2) state, each licensee shall develop and implement guidance and strategies intended to maintain or restore core cooling, containment, and spent fuel pool cooling capabilities under the circumstances associated with loss of large areas of the plant due to explosions or fire, to include strategies in the following areas: (i)

Firefighting; (ii) Operations to mitigate fuel damage; and (iii) Actions to minimize radiological release.

All licensees submitted their proposed, site-specific mitigation strategies to the NRC. The staff reviewed these submittals and concluded that all licensees had identified a range of strategies that, if implemented as described, would be adequate to satisfy the industry-proposed license conditions. The staff issued a Temporary Instruction (TI) 2515/171, Verification of Site Specific Implementation of B.5.b Phase 2 & 3 Mitigating Strategies, (Ref. 11) to be used to verify implementation of the strategies at each site.

To deal with the unique nature of the B.5.b inspection findings, the staff recognized the need to develop a specific qualitative significance determination model based on expert judgment, focused on defense-in-depth, informed by stakeholder input. To ensure a consistent assessment of findings identified during the inspections, the staff, with input from industry stakeholders, developed a special SDP for TI 2515/171. The NRC staff incorporated this SDP into Revision 1 of the TI and issued the TI for use on July 25, 2008.

The staff committed to incorporate the lessons learned from the performance of TI 2515/171 into the ROP baseline inspection program. These actions were completed when the staff incorporated the inspection guidance contained in TI 2515/171 into the Triennial Fire Inspections and the SDP used for the inspections was issued as IMC 0609 Appendix L, Significance Determination Process for B.5.b (Ref. 12).

Changes to the SDP after the Events at Fukushima Daiichi Following the earthquake and tsunami at the Fukushima Daiichi NPP in March 2011, the NRC established a senior-level task force referred to as the Near-Term Task Force (NTTF). The NTTF conducted a systematic and methodical review of the NRC regulations and processes to determine if the agency should make safety improvements considering the events in Japan. As a result of this review and the recommendations made by the NTTF, the Commission issued three orders:

• NRC Order EA-12-049 added requirements for mitigation strategies for Beyond-Design-Basis external events.

• NRC Order EA-12-051 added requirements for spent fuel pool instrumentation, and

• NRC Order EA-13-109 added requirements to ensure that BWR Mark I and Mark II containments have reliable hardened venting capability.

The NRC staff subsequently issued TI 2515/191 Inspection of the Implementation of Mitigation Strategies and Spent Fuel Pool Instrumentation Orders and Emergency Preparedness Communication/Staffing/Multi-Unit Does Assessment Plans, (Ref. 13) which was used to verify licensee's satisfactory implementation of NRC Order EA-12-049, NRC Order EA-12-051, and the communications and staffing plans needed to respond to a large-scale event as requested in the NRC's March 12, 2012 request for information letter. Additionally, the NRC staff issued TI 2515/193 Inspection of the Implementation of EA-13-109: Order Modifying Licenses with Regard to Reliable Hardened Containment Vents Capable of Operation under Severe Accident Conditions. TI 2515/193 only applies to General Electric Boiling-Water Reactors with Mark I and II Containments and will be used to verify licensees satisfactory implementation of NRC Order EA-13-109.

IMC 0609 Appendix O, Significance Determination Process for Mitigating Strategies and Spent Fuel Pool Instrumentation (Orders EA-12-049 and EA-12-051) was designed to provide inspectors with a simplified screening framework and associated instructions for use in assessing Mitigating Strategies and Spent Fuel Pool Instrumentation inspection findings. Order EA-12-049 requires a three-phase approach for mitigating beyond-design-basis external events. The initial phase requires the use of installed equipment and resources to maintain or restore core cooling, containment pressure control/heat removal and spent fuel pool cooling capabilities. The transition phase requires providing sufficient, portable, onsite equipment and consumables to maintain or restore these functions until they can be accomplished with resources brought from off site. The final phase requires obtaining sufficient offsite resources to sustain those functions indefinitely.

Appendix O is used to screen inspection findings related to the initial and transition phases. For the final phase of the order, this SDP only applies to that portion of the licensees mitigating strategy that occurs after the licensee accepts the delivered equipment at the site from the National Safer Response Centers. Oversight of the National Safer Response Centers is performed in accordance with the NRCs Vendor Inspection Program.

This SDP also only applies to PDs that are associated with equipment (i.e., FLEX), procedures, training and programmatic aspects that are used specifically for Mitigating Strategies or Spent Fuel Pool Instrumentation as defined by Orders EA-12-049 and EA-12-051 respectively. If equipment serves a function(s) for other purposes, a more appropriate SDP tool will be used. For example, if the PD concerns installed plant equipment that is used for Mitigating Strategies (e.g.,

Phase 1 equipment), but is also used to mitigate other transients or accidents, the more appropriate SDP (e.g., IMC 0609 Appendices A, G, and H) would be used to adequately assess significance.

No specific SDP was developed for findings associated with implementation of NRC Order EA-13-109 since it was determined that the existing SDP Appendices could adequately assess the significance of these findings. Information concerning the use of existing appendices to assess the adequacy of findings associated with NRC Order EA-13-109 is contained in an NRC staff memorandum (Ref. 2) that was issued in December 2018.

Use of Qualitative Factors in the SDP In the late summer of 2002, the Executive Director for Operations (EDO) directed the formation of an NRC task group to perform an independent and objective review of the SDP. On December 13, 2002, the SDP task group finished its report and provided several recommendations, many of which were consistent with the SDP improvement initiatives already being developed by NRC staff. Some common recommendations involved the consideration of uncertainty in the SDP, the need to improve clarity of risk-informed decision-making guidance, and the importance of making

timely regulatory decisions. These common recommendations revealed the need for an alternative process to estimate the safety significance of inspection findings that are difficult to estimate using quantitative risk tools and methods (e.g., use of SAPHIRE and SPAR models).

Although previous inspection program guidance required NRC management review for findings that could not be evaluated by the SDP, a focus group was created to develop a new SDP tool, which eventually became IMC 0609, Appendix M, The Significance Determination Process Using Qualitative Criteria, issued on December 22, 2006. A subsequent revision of Appendix M in 2012 provided guidance for making regulatory decisions using a deterministic framework of a small set of qualitative factors. Based on feedback from both internal and external stakeholders and the results of an SDP Business Process Improvement initiative completed in 2014, recommendations were made to update Appendix M to: (1) clarify entry conditions, and (2) develop a framework that takes the inputs and arrives at an integrated risk-informed decision.

Revision effort at this time was further motivated by a temporary increase in the use of Appendix M (e.g., to deal with external flooding findings after increased inspections in the wake of the events at the Fukushima Daiichi NPP), which abated in subsequent years. Based on internal and external stakeholder feedback, gains made through the concurrent activities (e.g., Inspection Finding Resolution Management process), and a decrease in the usage of Appendix M, the staff ultimately opted for a targeted update of the Appendix. That more targeted update focused on adding clarity and specificity to the conditions of usage as a non-quantitative SDP tool for assessing significance of licensee PD.

As stated above Appendix M was developed as an alternative to existing SDP tools when the NRC staff encounters difficulty in determining the safety significance of inspection findings using those available methods. This difficulty may arise in exceptional situations and circumstances where the unique complexities of an inspection finding may challenge decision makers in making an objective and reliable risk-informed decision in the most efficient manner. There are two circumstances where the use of Appendix M is warranted as outlined below.

• As specifically directed by other SDP appendices - Other SDP appendices have specific instances when NRC staff are directed to use Appendix M. These cases have already been evaluated such that the use of Appendix M is appropriate to support the significance assessment of the inspection finding for a proper risk-informed decision-making outcome.

As such, the use of this entry condition does not require the approval of the Significance and Enforcement Review Panel (SERP), i.e., a Planning SERP.

• When the cognizant NRC staff determine that no other SDP appendix is compatible for use with the specific circumstances associated with the inspection finding and the associated degraded condition (e.g., readily-available information is insufficient to support a reliable and efficient evaluation), subject to confirmation by a planning SERP.

The second entry condition is to be applicable only under circumstances when the available SDP tools are not adequate to provide a preliminary significance determination in a reliable and efficient manner or the inspection finding is not amenable to quantitative assessments for risk-informed decision making. In these situations, NRC staff may evaluate development of a new SDP tool to address the specific type of inspection findings if these findings become more frequent. As a result, IMC 0609 Appendix M is the appropriate and efficient tool to use for making risk-informed decisions on these inspection findings.

DISCUSSION:

As part of this initiative to enhance the ROP, the NRC staff evaluated inputs from the Transformation Initiative along with recommendations provided by industry via Nuclear Energy Institute (NEI), as captured below:

• NEI Recommendation 3B - Combine BDB into One SDP: For Beyond Design Basis (BDB) SDP, combine all IMC 0609 appendices currently used into one SDP for BDB events.

• NEI Recommendation 3C - Stop Appendix M Revision: Stop work on IMC 0609, Appendix M and leave as-is.

• NEI Recommendation 3D - Standardize PRA Inputs to SDP: Develop a consensus methodology for PRA inputs that will align the NRC and industry on uncertainties in key variables when beginning a PRA analysis of a PD. These key variables include, among others, Human Reliability, Common Cause, and Exposure Time. Presently, PRA results differ from the NRCs SPAR models largely due to the sensitivity of these inputs and incorporation of plant changes. The NRC RASP Handbook provides very conservative initial assumptions which drive the significance higher in many cases. This causes consternation and application of intense resources, both by the NRC and industry, only to eventually come out in the end as very low safety significance. Since 2010, less than nine percent of findings actually escalated above very low significance.

• NEI Recommendation 3E - Develop Procedure to Align on PRA Inputs Early: While the consensus methodology is being developed, the NRC should institute a procedural requirement to fill out a worksheet with the three key variables and obtain alignment with the licensee on other major inputs prior to running models either in SPAR or the licensees PRA. Experience shows that when these variables are commensurate input into either the SPAR or site-specific PRA, the outcomes are nominally equivalent.

• NEI Recommendation 3F - Develop NRC Interface for Licensee PRAs: Industry and NRC should jointly develop a portal for NRC access to the licensees site-specific PRA models for the purpose of supporting SDPs. This interface would enable NRC to maintain independence of thought, assumptions, and inputs, while allowing PRA practitioners to see results from the licensee models. The portal would allow NRC to exercise the licensees PRA application without requiring NRC to obtain the training or licenses necessary for detailed manipulation of models in the licensees chosen PRA software.

• NEI Recommendation 3G - Eliminate Use of SPAR in SDP: Once the portal is implemented and proven satisfactory, NRC should eliminate the use of the SPAR model in SDPs.

• Transformation Initiative Input (840) - Eliminate the Use of SPAR Models - The NRC should rely on more realistic licensee models to determine the significance of any deficiencies. The SPAR models should be phased out. Instead, assumptions associated with the representation of a PD should be determined and discussed with the licensee prior to conducting the analysis.

• Transformation Initiative Input (619) - Yellow risk would get more significant Detailed Risk Evaluation and actions on the action matrix.

• Transformation Initiative Input (621) - Use deterministic or simpler SDP models for those items that are initially scoped to be less than yellow risk.

• Transformation Initiative Input (336) - Eliminate the use of the post-PD reevaluation risk assessment to save both licensee and NRC resources, and drive quicker decisions and provide a better assessment of actual licensee performance. Or, at a minimum, use an average of pre-PD risk and post-PD risk to determine significance; however, this does not produce nearly as much resource savings for the licensees or agency.

Stakeholder Interactions NRC staff solicited feedback from internal stakeholders to contribute to the assessment of recommendations and proposed ROP revisions. These stakeholders included, all Regions, Office of Nuclear Reactor Regulation (NRR)/(DIRS), NRR/ Division of Risk Assessment, and RES. In addition to internal stakeholders, input from external stakeholders/public was obtained through interactions during the ROP monthly meetings, including meetings dedicated to the ROP Enhancement initiative. Staff will continue to reach out to industry, the public and NGOs as further progress in made on the ROP enhancements related to the SDP.

Completed Early Opportunities NEI recommendation 3B The NRC staff was in the process of revising IMC 0609, Appendix O, Significance Determination Process for Mitigating Strategies and Spent Fuel Pool Instrumentation, prior to the start of the ROP enhancement initiative. The NRC staff will retire IMC 0609, Appendix O due to licensees extensive use of diverse and flexible coping strategies (FLEX) equipment in areas beyond compliance with Commission orders and the ability to credit this equipment within probabilistic risk assessment (PRA) models. This will improve decision making by allowing the NRC staff to rely on the quantitative results of models reflecting FLEX equipment when evaluating the risk of inspection findings. This is consistent with guidance provided by the Commission in SRM-SECY-13-0137 (Ref. 18) that the SDP should continue to place emphasis on the use of the existing quantitative measures of the change in plant risk for both operating and new reactors. A forthcoming revision of IMC 0609, Appendix A, The Significance Determination Process for Findings At Power, will incorporate lessons learned from use and the screening questions contained in Appendix O to address issues related to FLEX and spent fuel pool level instrumentation capabilities. IMC 0609, Appendix L, will remain as a separate SDP appendix because of the unique nature and security implications of findings associated with compliance with Title 10 of the Code of Federal Regulations (10 CFR) 50.54(hh)(2). This equipment has not traditionally been modeled in PRAs, therefore the NRC staff determined it is appropriate to remain in its own deterministic appendix for evaluation of potential findings. The staff intends to continue discussing ongoing and future revisions to the IMC 0609 appendices through routine ROP working group interactions.

NEI recommendation 3C The NRC staff was also revising IMC 0609 Appendix M, Significance Determination Process Using Qualitative Criteria, prior to the start of the ROP Enhancement Initiative. NEI

acknowledged that the recommendation provided on Appendix M was focused on a previous major overhaul effort. Appendix M was subsequently publicly issued in January 2019 and is available for use by the NRC staff. The NRC staff believes the stringent entry conditions combined with the use of a decision-making management body (i.e., planning SERP) to approve use of Appendix M in situations where it is not previously designated in procedures puts adequate controls in place to ensure the use of qualitative factors in the SDP is appropriate. NEI has confirmed during multiple ROP Working Group monthly public meetings that there are no major issues with the revision that was made available for use in January 2019.

Recommendations Requiring Commission Approval and Vote There are no recommendations in the SDP area that require Commission approval or vote.

Recommendations Requiring Additional Evaluation There are no recommendations in the SDP area that require additional evaluation.

Recommendations NEI Recommendation 3D The NRC staff agrees that early and open interactions with the licensee are an important component of understanding the PD and developing an accurate preliminary significance determination. A relatively new change to the ROP was the introduction of the Inspection Finding Review Board (IFRB). The purpose of the IFRB is to provide a formal framework to obtain regional staff and management agreement on the proposed PD and to effectively manage the actions needed to reach a preliminary decision on the significance of inspection findings that do not initially screen to Green. This framework aims to ensure that all involved regional managers and staff are aligned on the specific actions needed, the scope of the work to be done, and the associated schedule to reach an informed decision on licensee PDs and their preliminary significance prior to conducting a SERP. A key component of the IFRB process is contact with the licensee.

From IMC 0609, Significance Determination Process, (Ref. 3) Section 06.04:

After the IFRB approves the performance deficiency for an inspection finding that did not screen to Green, the IFRB Chair will notify licensee senior management that the NRC will be performing additional reviews and analysis to determine significance. The Chair will also communicate the desire for timely, open, and constructive dialogue using best available information, emphasizing the Chairs focal point role in the process. If the preliminary significance assessment of a finding is White, Yellow, Red, or GTG, the licensee will be given the opportunity to provide additional information and perspectives at a public Regulatory Conference or in a written response on the docket. This opportunity will be offered in the cover letter of the inspection report or in the preliminary significance determination letter.

The IFRB was piloted in 2018 and subsequently made a permanent part of the ROP in 2019.

The NRC staff believes that awareness of the recent changes and the positive effects on communications with licensees are not widespread due to the short pilot period and the low

number of findings (and licensees) that were exposed to the changes. Feedback from licensees and NRC staff that have used the process has been positive.

The recommendation mentions key variables used in assessing the significance of findings (i.e.,

human reliability, common cause, and exposure time). These variables are aspects that would be discussed between inspectors, analysts, and the licensee. Discussions with Senior Reactor Analysts (SRAs) and HQ Risk Analysts determined that, in general, open and effective communications with the licensees are taking place. There are several times during the SDP process where the lead analyst must understand and present the licensees risk results, including any areas of disagreement, to NRC management including those who are voting SERP members. The analysts understanding of areas of disagreement between the licensee and NRC risk assessment leads to in-depth discussions with members of management during meetings that are driven by processes prescribed within the ROP (i.e., SERP meetings, IFRB meetings), and individual briefings provided by NRC staff members in preparation for those meetings.

The recommendation also calls for a consensus methodology and alignment with the licensee. These factors are in direct conflict with the principle of independence and more specifically with the technical basis of the SDP. From IMC-0308 Att. 3, Significance Determination Process Technical Basis (Ref. 1):

The SDP is not a consensus process with a licensee or other parties and no staff/licensee interactions should be construed as a negotiation. The ROP requires the staff to make decisions using best available information in a timely manner and that the bases of SDP results be clear and publicly available, to the extent practical and permitted by policy (e.g., security issues). The SDP affords licensees an opportunity to provide available information that may be useful to the staff in arriving at a best-informed decision within a reasonable time of 90 days. The staff is obligated to be clear about the basis for any SDP result and to consider licensee-provided information. The staff is not obligated to have proof of the assumptions made relative to an SDP result basis. [Emphasis added]

Although the SDP is an NRC process, the licensee is encouraged to provide information to analysts during the process. However, as is explained above, alignment or consensus with the licensee is not a concept consistent with the technical basis of the SDP and would undermine the NRCs independence in implementing the process. The staff does believe enhancements could be made to the guidance at the next planned IMC revision to ensure interactions are happening early and staff (e.g., analysts, IFRB chairs) are being as open as possible with the licensee on key influential assumptions. For example, if exposure time is established and that determination is made in time to support the IFRB chairs call to licensee management, that information should be shared at that time. The NRC staff is using the ROP feedback form process to track completion of this enhancement.

The recommendation also discusses aspects of guidance contained within the RASP Handbook. The NRCs guidance for the treatment of common-cause failure (CCF) dependencies, human reliability analysis (HRA), and exposure time are established in Volume 1 (Ref. 5) of the RASP Handbook.

The NRC staff has had extensive interactions with the industry concerning CCF. In January 2018, NEI provided the NRC with revision 1 of a White Paper for Addressing Common Cause Failure Impact within the Significance Determination Process (Ref. 15), which

suggested a methodology allowing qualitative assessment of CCF for the SDP. NRC staff engaged external stakeholders during the September 2018 ROP public meeting and conducted a standalone Category 2 public meeting on December 12, 2018 to share internal stakeholder comments and to solicit feedback from external stakeholders on the proposal shared by NEI in the white paper.

Informed by the insights gained from these interactions, NRC staff proposed a three-pronged approach to address internal and external stakeholders comments about the impact of CCF on the SDP. The following approach be used on a trial-basis for one calendar year, beginning on April 1, 2019:

When conducting DREs, risk analysts and senior reactor analysts will perform sensitivity studies to evaluate the impact of CCF. This is already a common practice and is consistent with RASP guidance.

Consistent with RASP CCF Ground Rule #3, licensees can provide specific information on whether plant-specific unique CCF defense strategies warrant the adjustment of the conditional CCF values. However, the burden will be on the licensees to demonstrate that their defense strategies go above and beyond what is already reflected in the current data and therefore warrant additional credit.

A common misconception in the application of this ground rule is that success of the other components in the common cause component group (CCCG) demonstrates a unique successful defensive strategy. However - as discussed in NUREG-2225 - the lack of observed degradation on the other components in the CCCG is not - by itself -

indicative of a unique CCF defense strategy in probabilistic analysis.

Using an existing User Need Request, the NRR will ask the Office of Nuclear Regulatory Research (RES) to examine the quantitative aspect of causal alpha factors, which would categorize the effects of CCF based on the cause of the failure. To the extent practicable, APOB intends to share the results of this study with industry and other stakeholders.

After the trial period ends, NRC staff will examine the insights gained and will evaluate whether more durable changes to NRC guidance (e.g., RASP Manual) are appropriate.

Similar to the work on CCF, the NRC staff has done extensive work on HRA with involvement by external stakeholders including the industry. In December 2006, the Advisory Committee on Reactor Safeguards (ACRS) met with the Commission to discuss ACRS activities and their current focus areas. The ACRS members and the Commission discussed the availability of multiple HRA methods for use by the NRC staff. In response to this discussion, the Commission tasked the ACRS to work with the NRC staff to choose an HRA method to cover all situations or to provide guidance on what method is appropriate to use in what circumstances.

NUREG-2114, Cognitive Basis for Human Reliability Analysis, (Ref. 16) was composed in response to that direction and forms the cognitive basis for an NRC developed HRA method called the Integrated Human Events Analysis System (IDHEAS). As described in NUREG-2199, An Integrated Human Event Analysis System (IDHEAS) for Nuclear Power Plant Internal Events At-Power Application, (Ref. 17) the method incorporates the strengths of existing methods and addresses their weaknesses. The goals of developing the method were to:

• Reduce analyst-to-analyst variability.

• Add realism to HRA.

• Improve transparency and traceability of the HRA process.

• Base the method on state-of-art understanding of human cognition and behaviors.

IDHEAS improves upon existing HRA methods by providing a structured approach to task analysis through crew response diagrams, a model addressing time uncertainties, a quantification model based upon insights from cognitive psychology and human behavior literature, and a set of human error probabilities in the quantification model estimated by a structured panel of informed experts. The method addresses post-initiating, internal at-power events. It assumes that the crews being modelled are the nuclear power plant control room licensed crews that have been trained to work together within pre-defined team structures and work processes. Therefore, it models the errors made by trained crews performing required responses to plant disturbances. It is applicable to development of PRA models for internal at-power events and can be used to support risk-informed decision-making including the SDP.

The NRC built upon the development of IDHEAS with the IDHEAS general methodology (IDHEAS-G) for HRA in all nuclear applications. IDHEAS-G provides a structure to quantify human error probabilities (HEP), but it needs application specific numeric values for off the shelf HEP calculation. The NRC staff will use the combination of the outcomes of expert elicitation, data from literature, and operational experience to develop the HRA tool for all applications. To this end, the NRC staff initiated an expert elicitation project specifically focused on HRA for actions associated with the use of portable FLEX equipment and strategies. The FLEX HRA expert elicitation project used an expert panel (including internal and external stakeholders) to estimate benchmark HEPs for a representative set of FLEX actions under given scenarios and to evaluate performance shaping factors (PSF) that are pertinent to the use of these strategies.

The estimated HEPs of the FLEX actions will serve as anchors for the base HEP values in the tool, and the FLEX-specific PSFs will inform the simplification of the tool from IDHEAS-G. The results of the expert elicitation will be used to refine NRC staff modelling of FLEX equipment and strategies in the NRC SPAR models.

The NRC staff will continue to endeavor to engage with industry to address technical issues and concerns related to evaluation of PD within the ROP. These discussions can lead to additional projects and initiatives (like the CCF and HRA efforts) to ensure PRA practices used for the SDP remain state of the art, consistent with the 1995 PRA policy statement.

NEI Recommendation 3E Some of the aspects of this recommendation have been elaborated on in the discussion above on NEI recommendation 3D (e.g., communications, independence of NRC risk analysis),

however, there is one issue that needs to be addressed specifically associated with this recommendation. The SERP is the NRC decision-making body for SDP related issues. The key variables that are discussed in the recommendation generally drive the outcomes of the risk analysis that is being performed for the identified licensee deficient performance. To adequately implement this recommendation and maintain proper management oversight the NRC staff would need to engage with and obtain approval from SERP members prior to obtaining alignment with the licensee. Inserting additional process steps requiring alignment with the licensee on key variables would force interactions with the SERP at a time when all the nuances of the PD and its risk significance are not fully developed. This would also force an interaction with SERP members for findings that could eventually be evaluated as Green. The development of a risk analysis for the SDP is an iterative process and the staff believes

implementing this recommendation would hamper the efficient disposition of inspection findings.

Therefore, no further NRC staff actions are planned with respect to this recommendation.

NEI Recommendation 3F and 3G, Transformation Initiative Input 840 There have been multiple efforts to evaluate the elimination or reduced usage of the SPAR models in regulatory decision-making. Until recently these efforts were not well documented with sound rationalization for the NRC staff position. The lack of documentation was identified in an Office of Inspector General (OIG) report OIG-17-A-26, Evaluation of Proposed NRC Modifications to the Probabilistic Risk Assessment Process, dated September 21, 2017 (Ref.

6). OIG recommended formally documenting evaluation results to conclusively establish the agency position on the NRCs use of licensees PRA models, to include reliable, verifiable cost data. This action was completed and documented in an NRC memorandum to Dr. Brett M.

Baker, Response to the Office of the Inspector Generals Evaluation of Proposed NRC Modifications to the Probabilistic Risk Assessment Process (OIG-17-A-26) dated June 29, 2018 (Ref. 7).

The NRC staff uses plant-specific SPAR models, developed and maintained by INL, in several regulatory applications. As explained in the background section of this document the SPAR models utilize standardized conventions and modeling methods to improve staff efficiency. In some cases, SPAR models are more detailed than the associated licensee-maintained, non-standardized models. The OIG response memorandum describes the most recent effort that was launched in 2015 by the NRCs Risk-Informed Steering Committee (RISC). The RISC directed the NRC staff to evaluate the costs and benefits associated with using licensees PRA models in lieu of the SPAR models with the goal of eliminating the SPAR model program completely. The staff identified and evaluated several technical, regulatory, cost, and other related factors pertinent to use of licensees PRA models in lieu of the SPAR models. These included, but were not limited to, fixed and variable costs, ease of use for NRC staff (including training costs), potential legal issues (including loss of the ability to perform independent confirmatory analysis), and licensee willingness to participate. The results from the cost analysis indicated a significant cost for transition to licensee models with a potential for longer-term small cost savings once full transition was complete.

The NRC staff also worked with the NEI to gauge licensees willingness to participate, since the viability of the proposal depended upon full NRC access to licensee PRA models (which are not normally submitted to the NRC under the current regulatory framework). While some licensees were supportive of the proposal, there was considerable resistance towards allowing NRC staff full access to the licensee PRA models. Based on these considerations, the NRC staff recommended that the NRC should continue to rely on SPAR models in implementing its risk-informed regulatory activities. Based on the staffs recommendation, the RISC made the decision to conclude the evaluation and continue to use SPAR models for operating reactor oversight programs.

While the effort in 2015 was focused on complete elimination of the SPAR model program, a previous effort in 2007 was focused solely on use of licensee PRA models in the SDP. The current recommendation by NEI is somewhat different than what was under discussion during that time, but some of the same principles apply. In an October 2007 letter to NEI (Ref. 8), the

staff presented its final position and rationale regarding the topic areas discussed during several public meetings that were held regarding the topic:

Maintaining the independence of the NRC and licensees models.

The NRC staff concluded that, because the NRCs ROP is intended to provide an independent regulatory assessment of licensee performance, it would be inappropriate for licensee risk analysts to take the lead in assessing the significance of performance deficiencies at their site. Such an arrangement would also minimize the NRC staffs ability to ensure that issues are assessed in a timely manner. Maintaining the NRCs independent oversight of licensee performance is critical for effective NRC oversight and is an important aspect of upholding public confidence in the process.

Differences in SDP outcomes (NRC vs. licensee assessment methodologies).

Differences in SDP outcomes between the NRC and the licensee are driven by factors other than the baseline PRA model used for the analysis; in fact, the PRA models are often in close agreement. The staff has processes in place to verify, validate, and benchmark these models according to the guidelines and standards established by the SPAR Model Program. As part of this process, the staff performs reviews of the SPAR models and results against the licensee PRA models, when applicable. The QA plan also provides a feedback process from the model users for error reporting, tracking, and resolution. The differences noted during the SDP are more often in the way engineering assumptions, HRA, and recovery are handled within the analysis. Licensees often have unique perspectives on the event or condition under agency review. Therefore, the SDP encourages input from licensees regarding such risk insights. The recent introduction of the IFRB has further improved communication with the licensee on these matters.

Standardization of modeling and assessment techniques used.

At present, the industry has not uniformly implemented a standardized approach to performing risk analysis that would ensure uniform application across the spectrum of industry PRA models. In this regard, the NRCs use of the SPAR models, together with the ongoing development of guidance on conducting risk assessments (i.e., RASP Handbook), ensures greater uniformity in the agencys regulatory assessments. The RASP Handbook is publicly available and NRC staff will continue to engage with industry on use of best practices in risk assessment in the future.

Potential use by the NRC staff of licensees PRA models.

Another alternative considered was NRC staff use of provided licensee PRA models.

Under this option, the staff would perform the assessment of risk significance using the licensee model. At the time, NRC staff concluded that the logistical and resource needs to maintain the 70-plus industry PRA models on some four software platforms would require the diversion of NRC staff efforts and the addition of risk analysts. This was also one of the major considerations during the 2015 effort. The alternative was analyzed as not a viable option unless the industry implemented a single RG 1.200 compliant modeling approach on one analysis platform facilitating efficient use of NRC resources.

It was determined that continued improvement to the standardization of PRA modeling methods in SPAR and industry PRA models is the most effective use of resources, commensurate with the need for the staff to maintain its own methods for confirmatory and independent analysis. This determination remains as the NRC staff position today.

Implementation of the recommendations would require licensee resources to develop a portal and method for sharing the licensee PRA models with the NRC. During the November ROP

monthly meeting, the industry acknowledged that while this was a topic of consideration when providing input for the ROP Enhancement initiative, there are no current plans to create the portal associated with this recommendation. The NRC staff is open to piloting this type of approach using current SDP guidance for evaluation of external hazards. In the absence of NRC derived information regarding external hazards, the evaluation of external hazards generally relies on licensee developed models with review of the results by SRAs. Therefore, these activities could be more efficient for licensees and NRC staff with a portal established like the one described in the recommendation. However, the implementation of these recommendations relies on industry action and resources, so no further NRC staff actions are planned at this time.

Transformation Initiative Input 619 and 621 Detailed risk evaluations (DREs) are performed for PDs that do not screen to Green using the IMC 0609 screening questions. For many PDs, the initial SPAR model results determine the risk significance is Green without significant resource expenditure. However, a small percentage of PDs require changes to the SPAR models or additional information from the licensee to determine the initial risk significance. Performance of a DRE is an iterative process and guidance provided to staff is that resources expended are adjusted based on complexity of the issue and significance of the finding. The technical basis for the SDP (Ref. 1) states:

The resource burden to perform an SDP analysis is normally considered appropriate if it increases stakeholder understanding of the basis for potentially risk significant conditions, especially when an inspection finding is believed to be greater than Green. However, it is appropriate due to SDP timeliness considerations for the staff to cease further effort to refine or review an analysis, acknowledge the limitations and uncertainties, and proceed to a final determination using best available information and reasonable technical or probabilistic judgments. When making the decision to continue further review, especially when the additional review will cause an issue to be untimely, it is essential for the analysts and decision makers to keep in perspective that the purpose of the SDP assessment is to determine what action the staff should take (e.g., supplemental inspection) as a result of the inspection finding.

Since the inception of the ROP, the screening questions have been clarified and refined based on lessons learned and experience. In addition, new screening questions have been added to improve the overall effectiveness of the screening process. This is an ongoing process and it is realized that the screening questions are not an all-inclusive set. The NRC staff believes that the screening process and the iterative nature of performing DREs is adequate to adjust the resources necessary to determine the preliminary risk-significance of PDs.

Transformation Initiative Input 336 Each finding entering the SDP, regardless of the cornerstone under which it is identified, is a PD that is more than minor as prescribed in IMC 0612, Issue Screening (Ref. 9). PD that are determined to be minor are not findings and therefore do not require evaluation using the SDP.

As described in the background section, once a PD is determined to be more than minor the SDP uses screening questions and logic to expeditiously screen findings for which there is high confidence that the significance is of very low safety significance - Green. Once through that process the inspector and SRA will collaborate to further understand the PD and assess its risk

significance. Many findings are evaluated as Green during this initial collaboration, but a small subset proceed to a more comprehensive DRE.

The event analysis process (i.e., DRE) involves the modification of a SPAR model to reflect attributes of an event, solution of the modified model to estimate the risk significance of the event and documentation of the analysis and its results. The process is structured to ensure the analysis is comprehensive and traceable. A detailed review by the analyst and a subsequent independent review minimizes the likelihood of errors and enhances the quality of the risk analysis. Many tasks in the analysis process may require the analyst to repeat or consider pervious tasks several times throughout the analysis. This iterative approach will eventually result in the convergence of the PRA model that best represents the as-built, as-operated plant and the deficient licensee performance that is being represented in the model. The use of a model that either doesnt reflect the as-built, as-operated plant, or doesnt adequately capture the deficient licensee performance would be contrary to the principles of good regulation and the goals of the SDP.

As stated in the response for Transformation Initiatives 619 and 621, the NRC staff believes that the screening process and the iterative nature of performing DREs is adequate to allow for the adjustment of resources necessary to determine the preliminary risk-significance of PDs.

Alternative Views (including internal and external stakeholders)

NEI Recommendation 3B recommends combining all IMC 0609 appendices currently used for beyond design basis events into one SDP appendix. The industry has expressed in multiple ROP public meetings that this recommendation was meant to pertain to the combination of IMC 0609 Appendix L and Appendix O for B.5.b and FLEX equipment respectively. The NRC staff has evaluated this recommendation and determined that it will not affect efforts currently underway to subsume Appendix O into Appendix A and leave Appendix L as a separate standalone SDP. The industry has continued to express the alternate view that Appendix L and Appendix L should be combined. The SDP uses risk insights and other relevant information, as appropriate, to assist NRC staff in determining the safety or security significance of inspection findings. The SDP does not explicitly differentiate between design basis or beyond design basis events when estimating risk significance, although risk results may be affected by the subset of events impacted by the deficient licensee performance. The decision to subsume Appendix O screening questions and guidance into Appendix A is largely driven by how the licensees are utilizing this equipment and the ability to credit this equipment within probabilistic risk assessment (PRA) models. This is consistent with Commission direction to rely on quantitative results when available while considering other relevant information in determining risk significance. Appendix L evaluates the loss of a large area of a reactor plant due to fires or explosions initiated by a terrorist threat. The security implications associated with evaluation of inspection findings using Appendix L are unique and not currently considered in Appendix A.

The complexity of adding these aspects led the NRC staff to determine that keeping Appendix L as a separate Appendix is appropriate. Additionally, this equipment has not traditionally been modeled in PRAs, so quantitative results would not be available to inform the risk significance of inspection findings.

Staff Recommendation 621 suggests the use of a deterministic or simpler SDP for those items that are initially scoped to be a less-than-Yellow risk. This recommendation has been closed without further action after an NRC staff review of the SDP program and processes in place.

The alternative view is that the staff should further examine the SDP to determine whether efficiencies can be realized, for example, in the use of an SDP Phase 2 approach that might be

applied where either existing SDP assessment tools such as SAPHIRE might be used or more effective and efficient Phase 1 or Phase 2 screening questions might be developed. The basis for this view is the significant resources used and the time it takes the staff to evaluate greater-than-Green inspection findings, especially those determined to be White. Since the inception of the ROP, the screening questions have been clarified and refined based on lessons learned and experience. In addition, new screening questions have been added to improve the overall effectiveness of the process. These enhancements are ongoing and a part of the ROP assessment and improvement process. For findings that do not screen to Green, continued assessment is necessary. For many findings, the initial SAPHIRE results determine the risk significance is Green without significant resource expenditure. This Phase 2 assessment is generally completed by a Senior Reactor Analyst (SRA), but flexibility is provided in the process that it can be completed by an Inspector with SRA consultation. A small percentage of findings require changes to the models or additional information from the licensee to determine the initial risk significance. Performance of this more detailed assessment is an iterative process and guidance provided to staff is that resources expended should be adjusted based on significance of the finding and complexity of the issue. The NRC staff believes that the continual process of improving screening and the iterative nature of performing more detailed analysis is adequate to adjust the resources necessary to determine the preliminary risk-significance of inspection findings.

REFERENCES:

1. IMC 0308 Attachment 3, Significance Determination Process Technical Basis

2. NRC Memorandum to Greg Bowman, Interim Guidance to Clarify the Use of Inspection Manual Chapter 0609 in Dispositioning Findings Identified Under Temporary Instruction 2515/193, December 17, 2018 (ML18254A221)

3. IMC 0609, Significance Determination Process

4. IMC 0609, Attachment 5, Inspection Finding Review Board

5. Risk-Assessment of Operational Events Handbook, Volume 1, Internal Events

6. Office of the Inspector Generals Report OIG-17-A-26, Evaluation of Proposed NRC Modifications to the Probabilistic Risk Assessment Process, September 21, 2017 (ML17264A298).

7. NRC Memorandum to Dr. Brett M. Baker, Response to the Office of the Inspector Generals Evaluation of Proposed NRC Modifications to the Probabilistic Risk Assessment Process (OIG-17-A-26) (ML18173A253)

8. Reyes, Luis A., NRC, letter to Marvin S. Fertel, NEI, regarding NRC response to NEI letter on August 2 Commission Briefing, October 15, 2007 (ML072490540).

9. IMC 0612, Issue Screening

10. IMC-0609 Appendix A, Significance Determination Process for Findings At-Power

11. Temporary Instruction 2515/171, Verification of Site Specific Implementation of B.5.b Phase 2 & 3 Mitigating Strategies

12. IMC 0609 Appendix L, Significance Determination Process for B.5.b

13. Temporary Instruction 2515/191 Inspection of the Implementation of Mitigation Strategies and Spent Fuel Pool Instrumentation Orders and Emergency Preparedness Communication/Staffing/Multi-Unit Does Assessment Plans

14. IMC 0609 Appendix O, Significance Determination Process for Mitigating Strategies and Spent Fuel Pool Instrumentation (Orders EA-12-049 and EA-12-051)

15. NEI, White Paper for Addressing Common Cause Failure Impact within the Significance Determination Process, January 2018 (ML18016A125)

16. NUREG-2114, Cognitive Basis for Human Reliability Analysis, January 2016

17. NUREG-2199, An Integrated Human Event Analysis System (IDHEAS) for Nuclear Power Plant Internal Events At-Power Application, May 2016

18. SRM-SECY-13-0137, Staff Requirements - SECY-13-0137 - Recommendations for Risk-Informing the Reactor Oversight Process for New Reactors

ATTACHMENT 1 - LIST OF CURRENT SDP TOOLS (IMC-0609)

1. Appendix A, Significance Determination Process for Findings At-Power

2. Appendix B, Emergency Preparedness SDP

3. Appendix C, Occupational Radiation Safety SDP

4. Appendix D, Public Radiation Safety SDP

5. Appendix E

a. Part I, Baseline Security SDP for Power Reactors

b. Part II, Force-on-Force Security SDP for Power Reactors

c. Part III, Construction Fitness-for-Duty Significance Determination Process for New Reactors (Pilot)

d. Part IV, Cyber Security Significance Determination Process for Power Reactors

6. Appendix F, Fire Protection and Post-Fire Safe Shutdown SDP

7. Appendix G, Shutdown Safety SDP

8. Appendix H, Containment Integrity SDP

9. Appendix I, Operator Requalification, Human Performance

10. Appendix J, Steam Generator Tube Integrity SDP

11. Appendix K, Maintenance Risk Assessment and Risk Management SDP

12. Appendix L, Significance Determination Process for B.5.b

13. Appendix M, Significance Determination Process Using Qualitative Criteria

14. Appendix O, Significance Determination Process for Mitigating Strategies and Spent Fuel Pool Instrumentation (Orders EA-12-049 and EA-12-051)