ML18183A043

IMC 0609 Appendix M Significance Determination Process Using Qualitative Criteria

ML18183A043
Person / Time
Issue date:	01/10/2019
From:	Michael Montecalvo NRC/NRR/DIRS/IRAB
To:
Montecalvo M
Shared Package
ML18183A141, ML19010A015	List:
References
CN 19-002, DC 18-017
Download: ML18183A043 (12)

Text

NRC INSPECTION MANUAL APOB INSPECTION MANUAL CHAPTER 0609 APPENDIX M SIGNIFICANCE DETERMINATION PROCESS USING QUALITATIVE CRITERIA 0609M-01 PURPOSE This appendix provides guidance for assessing the significance of inspection findings in all cornerstones of the Reactor Oversight Process (ROP) to allow the NRC to apply a consistent process of using qualitative and quantitative attributes for risk-informed decision making.

Appendix M should not be used by decision makers when the results of another Significance Determination Process (SDP) appendix do not appear to be appropriate (i.e., the significance is perceived as too high or too low). In these cases, the appropriate SDP appendix should be used and a deviation from the ROP Action Matrix should be pursued in accordance with Inspection Manual Chapter (IMC) 0305, Operating Reactor Assessment Program.

0609M-02 ENTRY CONDITIONS

a. As specifically directed by other IMC 0609 appendices, or

b. When the cognizant NRC staff determine that no other SDP appendix is compatible for use with the specific circumstances associated with the inspection finding and the associated degraded condition (e.g., readily-available information is insufficient to support a reliable and efficient evaluation), subject to confirmation by a planning Significance and Enforcement Review Panel (SERP).

0609M-03 BACKGROUND Occasionally, the staff may identify challenges in conducting an efficient assessment for an inspection finding using readily-available methods. For example, there may be cases where an appropriate SDP tool does not exist to determine the risk impact of a finding. In that case, the safety significance of a finding must ultimately be determined using qualitative engineering judgment and regulatory oversight experience, which is an acceptable approach in a risk-informed process. In other cases, existing quantitative tools may not be well suited for the specific application because the finding either (a) is particularly complex or (b) involves cause and effect relationships, phenomena, or plant operations where the accident sequence modeling state-of-practice is undeveloped. All probabilistic evaluations have an inherent level of uncertainty associated with their quantitative outcomes. However, the amount of uncertainty can vary depending on how well the risk impact of the finding can be modeled using available state-of-the-art tools and other sources of information (e.g., Standardized Plant Analysis Risk (SPAR) models, SDP appendices, licensee input). In cases of high uncertainty, the risk evaluation process can take significantly more time than is necessary or reasonable for most ROP applications. In all cases, a clear and well understood inspection finding must be established in accordance with the guidance in IMC 0612, Issue Screening.

Issue Date: 01/10/19 1 0609 App M

Unless explicitly directed to use Appendix M by SDP guidance, the staff should conduct a planning SERP to determine if Appendix M is an appropriate tool for characterizing the significance of a finding. Careful consideration is warranted in considering this tool, especially if another SDP tool or method provides a suitable approach (e.g., a degraded condition may be readily modeled, uncertainties associated with an initiating event frequency or failure rate probability may be sufficiently understood). In these cases, an existing SDP tool may provide a suitable characterization of significance within the established SDP timeliness goals.

0609M-04 EVALUATION PROCESS Findings should be assessed using risk insights along with deterministic engineering judgment relying upon in-house engineering knowledge and expertise, regulatory oversight experience, and best available information.

SECY-98-144 describes a risk-informed approach to regulatory decision making as one that represents a philosophy whereby risk insights are considered together with other factors to establish requirements that better focus licensee and regulatory attention on design and operational issues commensurate with their importance to public health and safety. This philosophy was elaborated on in Regulatory Guide (RG) 1.174 to develop a risk-informed decision-making process for licensing changes. This philosophy has since been implemented in other NRC risk-informed activities. In developing the risk-informed decision-making process, the NRC defined a set of key principles in RG 1.174 to be followed for risk-informed decisions regarding plant-specific changes to the licensing basis; however, the principles are global in nature and can be generalized to all activities that are the subject of risk-informed decision-making.

• Principle 1: Current Regulations Met

• Principle 2: Consistency with Defense-in-Depth Philosophy

• Principle 3: Maintenance of Safety Margins

• Principle 4: Acceptable Risk Impact

• Principle 5: Monitor Performance The generalized approach integrates all the insights and requirements that relate to the safety or regulatory issue of concern. These insights include any deterministic and/or probabilistic analyses performed to support decision-making. The generalized approach ensures that defense-in-depth measures and safety margins are maintained. The impact of the inspection finding on Principles 2 and 3 have been evaluated using the guidance in Exhibit 2. Elements of Principle 4, to the extent information is readily available, have been considered while performing the evaluation described in Step 4.1. Aspects of Principles 1 and 5, while potentially not directly applicable, can manifest themselves via the attributes that have already been evaluated (e.g., if an inspection finding causes the plant to enter into an unanalyzed condition, the elevated risk associated with that unanalyzed condition can often be correlated to an associated degradation of safety margin or defense-in-depth).

Step 4.1 - Initial Evaluation 4.1.1 The purpose of this step is to determine if there are any significance colors (Green, White, Yellow, or Red) that can be reasonably excluded from further consideration via an initial evaluation using available quantitative and/or qualitative methods and best available information. These methods should be consistent with traditional assessment Issue Date: 01/10/19 2 0609 App M

approaches using reasonably conservative assumptions (e.g., minimal to no recovery actions, use of screening values for human error probabilities). The evaluation should not involve a detailed risk evaluation (although it may involve a simpler use of the same tools) and need not be quantitative (e.g., in the case of findings associated with the Emergency Preparedness and Radiation Protection cornerstones)1. If the evaluation shows that the finding is of very low safety significance (i.e., Green), the finding can be documented in accordance with IMC 0611, Power Reactor Inspection Reports, and the guidance provided in Step 4.4.2 of this appendix.

4.1.2 If the initial evaluation indicates that the risk significance of the finding is potentially greater than Green, document the results using Exhibit 1, Results of Initial Evaluation, of this appendix and then proceed to Step 4.2.

Step 4.2 - Attributes 4.2.1 For findings in which the risk significance is potentially greater than Green, evaluate the following attributes to determine the significance of the finding, then proceed to Step 4.3. Guidance on evaluating each attribute is contained in Exhibit 2, Considerations for Evaluation of Decision Attributes, of this appendix.

4.2.1.1 Defense-in-Depth 4.2.1.2 Safety Margin 4.2.1.3 Extent of condition 4.2.1.4 Degree of Degradation 4.2.1.5 Exposure Time 4.2.1.6 Recovery Actions 4.2.1.7 Additional Qualitative Attributes Step 4.3 - Integrated Risk-Informed Decision-Making 4.3.1 Integration of the results requires that the individual insights obtained from each element of the decision-making process be weighed and combined to reach a conclusion, in this case a decision on the significance of the finding. The staff involved with analysis of the finding (e.g., inspectors, probabilistic risk assessment (PRA) experts, engineering staff) should participate in the integration process. An example approach to integrating multiple diverse sources of information as part of decision-making can be found in LIC- 504, Integrated Risk-Informed Decision-Making Process for Emergent Issues, Appendix E, but use of those concepts should be in concert with SDP-specific decision-making guidance contained in IMC 0609 Attachment 1.

Step 4.4 - Process and Documentation 4.4.1 If the results of the Appendix M evaluation indicate a greater than Green finding, the decision-making logic should be documented using Table 1, Qualitative Decision-Making Attributes for NRC Management Review, and should be included in the SERP package as described in IMC 0609, Attachment 1, Significance and Enforcement Review Panel.

1 In cases where a qualitative approach is necessitated or appropriate, analogues can be drawn to existing relationships between a performance deficiency and significance (from the IMC 0609 appendix relevant to the performance deficiency) in order to establish a conservative estimate of the findings significance.

Issue Date: 01/10/19 3 0609 App M

4.4.2 If the results of the Appendix M evaluation indicate a Green finding, document the quantitative and/or qualitative methods used, including the results, in the inspection report.

0609M-05 REFERENCES IMC 0609, Attachment 1, Significance and Enforcement Review Panel Process IMC 0611, Power Reactor Inspection Reports IMC 0612, Issue Screening NRC Regulatory Guide 1.174, An Approach for Using Probabilistic Risk Assessment in Risk-Informed Decisions on Plant-Specific Changes to the Licensing Basis NRR Office Instruction LIC-504, Integrated Risk-Informed Decision-Making Process for Emergent Issues NRC, Staff Requirements Memorandum - SECY-98-144 - White Paper on Risk-Informed and Performance-Based Regulation, SRM-SECY-98-144, March 1, 1999.

NUREG-1855, Guidance on the Treatment of Uncertainties Associated with PRAs in Risk-Informed Decisionmaking END Issue Date: 01/10/19 4 0609 App M

EXHIBIT 1 Results of the Initial Evaluation

1. Describe the influential assumptions used in the initial evaluation.

2. Provide sensitivity results on the key influential assumptions. Given that a detailed risk evaluation is not tractable, these sensitivities might be qualitative or semi-quantitative, and should only be performed when practical to do so. These might include changes to the initiating event frequency, equipment failure rates, common cause failure probabilities, and human error probabilities. In the case of purely qualitative initial evaluations, these might include subjective evaluations of whether the significance would differ for alternative assumptions.

3. Identify any information gaps in defining the influential assumptions used in the initial evaluation.

Initial Evaluation Result: ____________________________

Issue Date: 01/10/19 E1-1 0609 App M

EXHIBIT 2 Considerations for Evaluation of Decision Attributes A. Defense-in-Depth Revision 3 of RG 1.174, An Approach for Using Probabilistic Risk Assessment in Risk-Informed Decisions on Plant-Specific Changes to the Licensing Basis, identifies and provides a discussion of seven considerations that should be used to evaluate impacts on defense in depth. While RG 1.174 provides general guidance concerning analysis of the risk associated with proposed changes in plant design and operation, the considerations and discussion of defense in depth can be applied to the evaluation of findings under the Reactor Oversight Process and in the use of this appendix. It is important to note that the focus here is on the effect of the finding on defense in depth.

The seven defense-in-depth considerations presented are not intended to define how defense in depth is implemented in a plants design, but rather to help the analyst assess the impact of the finding on defense in depth.

1. Preserve a reasonable balance among the layers of defense.

A reasonable balance of the layers of defense (i.e., minimizing challenges to the plant, preventing any events from progressing to core damage, containing the radioactive source term, and emergency preparedness) helps to ensure an apportionment of the plants capabilities between limiting disturbances to the plant and mitigating their consequences. The term reasonable balance is not meant to imply an equal apportionment of capabilities. The NRC recognizes that aspects of a plants design or operation might cause one or more of the layers of defense to be adversely affected. For these situations, the balance between the other layers of defense becomes especially important when evaluating the impact of a finding and its effect on defense in depth.

2. Preserve adequate capability of design features without an overreliance on programmatic activities as compensatory measures2.

Nuclear power plant licensees implement a number of programmatic activities, including programs for quality assurance, testing and inspection, maintenance, control of transient combustible material, foreign material exclusion, containment cleanliness, and training. In some cases, activities that are part of these programs are used as compensatory measures; that is, they are measures taken to compensate for some reduced functionality, availability, reliability, redundancy, or other feature of the plants design to ensure safety functions (e.g., reactor vessel inspections that provide assurance that reactor vessel failure is unlikely).

Other examples include hardware (e.g., skid-mounted temporary power supplies); human actions (e.g., manual system actuation); or some combination of these measures. Such compensatory measures are often associated with temporary plant configurations. The preferred approach for accomplishing safety functions is through engineered systems. Therefore, when the finding necessitates reliance on programmatic activities as compensatory measures, analysis should indicate that this reliance is not excessive (i.e., not overly reliant).

2 The term compensatory measures is used here to refer to additional measures in place during the time of the degraded condition.

Issue Date: 01/10/19 E2-1 0609 App M

The intent of this consideration is not to preclude the use of such programs as compensatory measures but to ensure that the use of such measures does not significantly reduce the capability of the design features.

3. Preserve system redundancy, independence, and diversity commensurate with the expected frequency and consequences of challenges to the system, including consideration of uncertainty.

The defense-in-depth philosophy has traditionally been applied in plant design and operation to provide multiple means to accomplish safety functions. System redundancy, independence, and diversity result in high availability and reliability of the function and also help ensure that system functions are not reliant on any single feature of the design. Redundancy provides for duplicate equipment that enables the failure or unavailability of at least one set of equipment to be tolerated without loss of function. Independence of equipment implies that the redundant equipment is separate, such that it does not rely on the same supports to function. This independence can sometimes be achieved by the use of physical separation or physical protection. Diversity is accomplished by having equipment that, while it performs the same function, relies on different attributes, such as different principles of operation, different physical variables, different conditions of operation, or production by different manufacturers, which helps reduce common-cause failure (CCF). A degraded condition might reduce the redundancy, independence, or diversity of systems. The intent of this consideration is to ensure that the ability to provide the system function is commensurate with the risk of scenarios that could be mitigated by that function.

The consideration of uncertainty, including the uncertainty inherent in the PRA, implies that the use of redundancy, independence, or diversity provides high reliability and availability and also results in the ability to tolerate failures or unanticipated events.

4. Preserve adequate defense against potential CCFs.

An important aspect of ensuring defense in depth is to guard against CCF.

Multiple components may fail to function because of a single specific cause or event that could simultaneously affect several components important to risk. The cause or event may include an installation or construction deficiency, accidental human action, extreme external environment, or an unintended cascading effect from any other operation or failure within the plant. CCFs can also result from poor design, manufacturing, or maintenance practices. Defenses can prevent the occurrence of failures from the causes and events that could allow simultaneous multiple component failures. Another aspect of guarding against CCF is to ensure that an existing defense put in place to minimize the impact of CCF is not significantly reduced; however, a reduction in one defense can be compensated for by adding another.

5. Maintain multiple fission product barriers.

Fission product barriers include the physical barriers themselves (e.g., the fuel cladding, reactor coolant system pressure boundary, and containment) and any equipment relied on to protect the barriers (e.g., containment spray). In general, these barriers are designed to perform independently so that a complete failure Issue Date: 01/10/19 E2-2 0609 App M

of one barrier does not disable the next subsequent barrier. For example, one barrier, the containment, is designed to withstand a double-ended guillotine break of the largest pipe in the reactor coolant system, another barrier. A plants licensing basis might contain events that, by their very nature, challenge multiple barriers simultaneously. Examples include interfacing-system loss-of-coolant accidents or steam generator tube rupture. Therefore, complete independence of barriers, while a goal, might not be achievable for all possible scenarios.

6. Preserve sufficient defense against human errors.

Human errors include the failure of operators to correctly and promptly perform the actions necessary to operate the plant or respond to off-normal conditions and accidents, errors committed during test and maintenance, and incorrect actions by other plant staff. Human errors can result in the degradation or failure of a system to perform its function, thereby significantly reducing the effectiveness of one of the layers of defense or one of the fission product barriers. The plant design and operation include defenses to prevent the occurrence of such errors and events. These defenses generally involve the use of procedures, training, and human engineering; however, other considerations (e.g., communication protocols) might also be important.

7. Continue to meet the intent of the plants design criteria.

For plants licensed under Title 10 of the Code of Federal Regulations Parts 50 or 52, the plants design criteria are set forth in the current licensing basis of the plant. The plants design criteria define minimum requirements that achieve aspects of the defense-in-depth philosophy. When evaluating a finding, the analysis should identify the design criteria that is challenged and how the finding impacts the design criteria.

B. Safety Margin Safety margin is the extra capacity factored into the design of a structure, system, or component (SSC) so that it can cope with conditions beyond the expected to compensate for uncertainty. The evaluation should assess whether the impact of the finding is consistent with the principle that sufficient safety margins are maintained. In evaluating this factor, the staff should use engineering analysis or engineering judgment appropriate for evaluating whether sufficient safety margins would be maintained given the finding. The evaluation should consider if the inspection finding identifies an issue which affects the licensees ability to meet the codes and standards or their alternatives approved for use by the NRC. Additionally, consider if the finding identifies an issue which affects meeting safety analysis acceptance criteria in the licensing basis (e.g.,

Update Final Safety Analysis Report, supporting analyses) or proposed revisions that provide sufficient margin to account for analysis and data uncertainty.

Issue Date: 01/10/19 E2-3 0609 App M

C. Extent of Condition If a finding is not isolated to a specific occurrence, condition, or event, its safety significance is typically greater. When a finding is capable of affecting multiple SSCs, the number of degraded conditions has the potential to be greater than a case in which a finding is isolated to a specific SSC. The identified extent of condition should have a reasonable and sound technical basis to justify the scope.

D. Degree of Degradation The magnitude and detailed circumstances of the degraded condition (or programmatic weakness) have a direct effect on the safety significance of the finding. As stated in IMC 0308, Attachment 3, Technical Basis for the SDP, the finding (i.e., more-than-minor performance deficiency) is the proximate cause of the degraded condition or programmatic weakness. Logically, the more a condition is degraded or program is weakened, the more safety significant the finding.

E. Exposure Time Generally, the longer a finding is left uncorrected the more opportunities the finding has to manifest itself (i.e., act as the proximate cause of a degraded condition or programmatic weakness). As such, the longer the exposure time the more safety significant the finding.

F. Recovery Actions Even if the extent of condition, degree of the degraded condition (or programmatic weakness), and exposure time increased the safety significance of a finding, crediting established recovery actions or mitigation strategies should be appropriately considered to determine the overall significance of the finding.

G. Additional Qualitative Attributes Depending on the situation, the previous six attributes may not capture all of the qualitative attributes that may apply to the finding. Therefore, additional qualitative circumstances, as appropriate, may be considered in the decision making process. Any additional qualitative circumstances for management consideration should have a clear and reasonable nexus to the safety significance of the finding. If additional qualitative attributes are considered, one should be particularly aware of the goal of having a scrutable and repeatable outcome, and should consider whether other decision makers would reasonably be expected to invoke the same qualitative attributes.

Issue Date: 01/10/19 E2-4 0609 App M

TABLE 1 Qualitative Decision-Making Attributes for NRC Management Review Decision Attribute Basis for Input to Decision - Provide qualitative and/or quantitative information for management review and decision making.

Defense-in-Depth Safety Margin Extent of Condition Degree of Degradation Exposure Time Recovery Actions Additional Qualitative Considerations Result of management review (COLOR):

Issue Date: 01/10/19 T-1 0609 App M

Attachment 1 Revision History IMC 0609 Appendix M Commitment Accession Number Description of Change Description of Training Comment Resolution Tracking Issue Date Required and and Closed Feedback Number Change Notice Completion Date Form Accession Number (Pre-Decisional, Non-Public Information)

N/A ML062510080 This new document has been issued to This procedure was ML063050646 12/22/06 provide guidance to NRC management and developed by involved CN 06-036 inspection staff for assessing significance of stakeholders. No inspection findings. training on the procedure recommended at this time. However, additional guidance may be developed based on experience gained.

N/A ML101550365 Provided clarification in the Scope and None N/A 04/04/12 Applicability sections to articulate the Appendix CN 12-005 M entry conditions and that Appendix M is not intended to be used to develop new models or acquire in-depth expert elicitation. In addition, ROPFF 0609M-1412 was incorporated to clarify that Appendix M applies to all the safety cornerstones of the ROP.

ML18257A025 Made public to solicit industry comment at the None N/A DRAFT October 18, 2018, ROP Public meeting.

Issue Date: 01/10/19 Att1-1 0609 App M

Commitment Accession Number Description of Change Description of Training Comment Resolution Tracking Issue Date Required and and Closed Feedback Number Change Notice Completion Date Form Accession Number (Pre-Decisional, Non-Public Information)

N/A ML18183A043 Provided clarification of the existing entry None ML18184A428 01/10/19 conditions to more clearly illustrate when 0609M-2272 Cn 19-002 Appendix M should be used. In addition, ML18226A054 provided clarification of the existing decision-making attributes to align with the enhanced guidance in Revision 3 of Regulatory Guide 1.174, An Approach for Using Probabilistic Risk Assessment in Risk-Informed Decisions on Plant-Specific Changes to the Licensing Basis, which was issued in January 2018.

Also, the description of the initial evaluation was clarified to better align with intent/practice, since the previous description inferred that (in the case of a quantitative estimate) one would use enveloping input assumptions across-the-board. Finally, ROPFF 0609M-2272 was addressed to make the guidance more useful for RP and EP findings. A Commissioners Assistant note was issued (ML18311A027) to notify the Commission of the described changes in accordance with Management Directive 8.13 and COMSECY-16-0022.

Issue Date: 01/10/19 Att1-2 0609 App M