SECY-18-0045, Enclosure 2 - Evaluation of the Clarity of Qualitative Factors in the Significance Determination Process
| ML18059A169 | |
| Person / Time | |
|---|---|
| Issue date: | 04/12/2018 |
| From: | Mary Anderson Division of Inspection and Regional Support |
| To: | |
| Anderson M, NRR/DIRS | |
| Shared Package | |
| ML18059A155 | List: |
| References | |
| 201100134 | |
| Download: ML18059A169 (2) | |
Text
Evaluation of the Clarity of Qualitative Factors in the Significance Determination Process On December 17, 2013, the staff provided the Commission with SECY-13-0137, Recommendations for Risk-Informing the Reactor Oversight Process for New Reactors. In that paper, the staff recommended the development of an integrated risk-informed approach that involved using qualitative measures, along with quantitative risk insights, to inform regulatory decisions in a structured manner. The staff specifically included two recommendations: (1) to develop an integrated risk-informed approach for evaluating the safety significance of inspection findings for new reactor designs, and (2) to develop appropriate performance indicators (PIs) and thresholds for new reactor applications.
The Commission issued the staff requirements memorandum (SRM) for SECY-13-0137 on June 30, 2014. With respect to new reactors, in that SRM the Commission disapproved Recommendation 1 and instead directed the staff to enhance the significance determination process (SDP) by developing a structured qualitative assessment approach for events or conditions that are not evaluated in the supporting plant risk models. The Commission further directed the staff to evaluate the need to provide additional clarity on the use of qualitative factors for operating reactors to provide more transparency and predictability to the process.
As directed by the Commission, the staff has evaluated whether additional clarity on the use of qualitative factors for operating reactors is necessary to provide more transparency and predictability. When performance deficiencies occur or are identified, they are screened through Inspection Manual Chapter (IMC) 0612, Appendix B, Issue Screening, to determine if they are more than minor. If so, IMC 0609, Significance Determination Process, is used to determine the risk significance of the issue. When evaluating the significance of an issue, IMC 0609, , Initial Characterization of Findings, is used to determine which IMC 0609 appendix is appropriate for evaluating the issue. There are currently 13 appendices to IMC 0609, each one designed to evaluate certain technical issues or plant conditions to ensure that appropriate considerations are given to the issue.
Qualitative considerations in the SDP are addressed in IMC 0609, Appendix M, Significance Determination Process Using Qualitative Criteria, which is used when probabilistic risk assessment methods and tools, including the other IMC 0609 appendices, cannot adequately address the findings complexity or provide a reasonable estimate of the significance due to modeling and other uncertainties. Entry into Appendix M is either directed by another IMC 0609 appendix or by management discretion as approved by a planning Significance and Enforcement Review Panel. Appendix M has been used to finalize roughly 13% of greater-than-Green inspection findings since it was first developed in 2006. In addition, review of past uses of Appendix M has shown that significant staff time and effort can be expended in making the determination to proceed with an Appendix M significance evaluation.
Based on its initial evaluation, the staff determined that additional clarity in Appendix M was warranted and considered large-scale changes to how and when qualitative factors are considered in the SDP, including the development of an approach for aggregating the results of the decision-making attributes contained in Appendix M. Solicitation of feedback on this approach from both internal and external stakeholders revealed strong disagreement from industry and the belief from internal stakeholders that such an update would not represent an efficient use of resources. For example, in an October 12, 2017, letter providing perspectives on SDP changes (Agencywide Documents Access and Management System (ADAMS)
Accession No. ML17338A031), the Nuclear Energy Institute stated that Appendix M is the least important of ongoing SDP revision projects, and one that should be terminated.
Enclosure 2
Given the feedback received, yet still recognizing the need for additional clarity in Appendix M, the staff is now proposing a smaller, more targeted update to Appendix M. This targeted update would take advantage of work already completed on the previous effort to more comprehensively update Appendix M and would not require significant additional resource expenditures. The targeted update would address two areas that the staff determined would increase efficiency in entering and using Appendix M:
- Clarification of existing entry conditions: Appendix M does not currently explicitly identify when the procedure should be used. The staff plans to include explicit entry criteria at the beginning of Appendix M.
- Clarification of existing decision-making attributes: Appendix M currently lists several decision-making attributes, but provides no clarifying guidance on how the attributes should be considered. The staff proposes to provide such guidance and would align the enhanced guidance with Revision 3 of Regulatory Guide 1.174, An Approach for Using Probabilistic Risk Assessment in Risk-Informed Decisions on Plant-Specific Changes to the Licensing Basis, which was issued in January 2018.
As a specific example of a targeted revision that the staff is proposing, one of the decision-making attributes in the current Appendix M revision is A reduction in safety margin can be quantified. No additional guidance is provided on what this means and how this should be considered or evaluated. The staff proposes to revise Appendix M to define the safety margin attribute as Safety margin is the extra capacity factored into the design of a structure, system, or component so that it can cope with conditions beyond the expected to compensate for uncertainty. Additional guidance would be provided to help align users on what is intended when evaluating the attribute, for example:
- Does the inspection finding identify an issue which affects the licensees ability to meet the Codes and standards or their alternatives approved for use by the U.S. Nuclear Regulatory Commission?
- Does the inspection finding identify an issue which affects meeting safety analysis acceptance criteria in the licensing basis (e.g., Update Final Safety Analysis Report, supporting analyses) or proposed revisions that provide sufficient margin to account for analysis and data uncertainty?
The two questions forming the additional guidance are consistent with considerations discussed in Regulatory Guide 1.174, Section 2.1.2, Safety Margin.
The proposed changes, in concert with upcoming recommendations from the recently completed Inspection Finding Resolution Management (IFRM) trial period, would increase the efficiency in entering and using Appendix M. An effectiveness review of the IFRM trial period is ongoing, but one aspect that has received nearly universal positive feedback is the Inspection Finding Review Board (IFRB) meeting. The IFRB ensures early alignment on performance deficiencies and plans for evaluating the significance of any potentially greater-than-Green issues. This early alignment would increase the efficiency in arriving at a decision to enter Appendix M in instances where it is not directed by another IMC 0609 appendix by ensuring that the decision and basis is agreed upon and understood up front.
The staff requested feedback on this proposed targeted Appendix M update from internal and external stakeholders through the monthly ROP Working Group public meetings; there were no expressed concerns with the staffs revised plan. Once management has approved the draft Appendix M changes and the Appendix M update is completed, the staff will use the criteria in the recently-revised Management Directive 8.13, Reactor Oversight Process, dated January 16, 2018 (ADAMS Accession No. ML17347B670), to determine whether Commission notification or approval is needed prior to its issuance. The staff notes that any revisions to Appendix M for specific new reactor issues will be addressed in a separate SECY paper, which is due to the Commission in June 2018.