NRC Inspection Manual 0609/Appendix M, Significance Determination Process Using Qualitative Criteria
text
Issue Date: 01/10/19 1 0609 App M
NRC INSPECTION MANUAL APOB
INSPECTION MANUAL CHAPTER 0609 APPENDIX M
SIGNIFICANCE DETERMINATION PROCESS
USING QUALITATIVE CRITERIA
0609M-01 PURPOSE
This appendix provides guidance for assessing the significance of inspection findings in all
cornerstones of the Reactor Oversight Process (ROP) to allow the NRC to apply a consistent
process of using qualitative and quantitative attributes for risk-informed decision making.
Appendix M should not be used by decision makers when the results of another Significance
Determination Process (SDP) appendix do not appear to be appropriate (i.e., the significance is
perceived as too high or too low). In these cases, the appropriate SDP appendix should be
used and a deviation from the ROP Action Matrix should be pursued in accordance with
Inspection Manual Chapter (IMC) 0305, “Operating Reactor Assessment Program.”
0609M-02 ENTRY CONDITIONS
a. As specifically directed by other IMC 0609 appendices, or
b. When the cognizant NRC staff determine that no other SDP appendix is compatible for
use with the specific circumstances associated with the inspection finding and the
associated degraded condition (e.g., readily-available information is insufficient to
support a reliable and efficient evaluation), subject to confirmation by a planning
Significance and Enforcement Review Panel (SERP).
0609M-03 BACKGROUND
Occasionally, the staff may identify challenges in conducting an efficient assessment for an
inspection finding using readily-available methods. For example, there may be cases where an
appropriate SDP tool does not exist to determine the risk impact of a finding. In that case, the
safety significance of a finding must ultimately be determined using qualitative engineering
judgment and regulatory oversight experience, which is an acceptable approach in a
risk-informed process. In other cases, existing quantitative tools may not be well suited for the
specific application because the finding either (a) is particularly complex or (b) involves cause
and effect relationships, phenomena, or plant operations where the accident sequence
modeling state-of-practice is undeveloped. All probabilistic evaluations have an inherent level of
uncertainty associated with their quantitative outcomes. However, the amount of uncertainty
can vary depending on how well the risk impact of the finding can be modeled using available
state-of-the-art tools and other sources of information (e.g., Standardized Plant Analysis Risk
(SPAR) models, SDP appendices, licensee input). In cases of high uncertainty, the risk
evaluation process can take significantly more time than is necessary or reasonable for most
ROP applications. In all cases, a clear and well understood inspection finding must be
established in accordance with the guidance in IMC 0612, “Issue Screening.”
Issue Date: 01/10/19 2 0609 App M
Unless explicitly directed to use Appendix M by SDP guidance, the staff should conduct a
planning SERP to determine if Appendix M is an appropriate tool for characterizing the
significance of a finding. Careful consideration is warranted in considering this tool, especially if
another SDP tool or method provides a suitable approach (e.g., a degraded condition may be
readily modeled, uncertainties associated with an initiating event frequency or failure rate
probability may be sufficiently understood). In these cases, an existing SDP tool may provide a
suitable characterization of significance within the established SDP timeliness goals.
0609M-04 EVALUATION PROCESS
Findings should be assessed using risk insights along with deterministic engineering judgment
relying upon in-house engineering knowledge and expertise, regulatory oversight experience,
and best available information.
SECY-98-144 describes a “risk-informed” approach to regulatory decision making as one that
represents a philosophy whereby risk insights are considered together with other factors to
establish requirements that better focus licensee and regulatory attention on design and
operational issues commensurate with their importance to public health and safety. This
philosophy was elaborated on in Regulatory Guide (RG) 1.174 to develop a risk-informed
decision-making process for licensing changes. This philosophy has since been implemented in
other NRC risk-informed activities. In developing the risk-informed decision-making process,
the NRC defined a set of key principles in RG 1.174 to be followed for risk-informed decisions
regarding plant-specific changes to the licensing basis; however, the principles are global in
nature and can be generalized to all activities that are the subject of risk-informed decisionmaking.
- Principle 1: Current Regulations Met
- Principle 2: Consistency with Defense-in-Depth Philosophy
- Principle 3: Maintenance of Safety Margins
- Principle 4: Acceptable Risk Impact
- Principle 5: Monitor Performance
The generalized approach integrates all the insights and requirements that relate to the safety
or regulatory issue of concern. These insights include any deterministic and/or probabilistic
analyses performed to support decision-making. The generalized approach ensures that
defense-in-depth measures and safety margins are maintained. The impact of the inspection
finding on Principles 2 and 3 have been evaluated using the guidance in Exhibit 2. Elements of
Principle 4, to the extent information is readily available, have been considered while performing
the evaluation described in Step 4.1. Aspects of Principles 1 and 5, while potentially not directly
applicable, can manifest themselves via the attributes that have already been evaluated (e.g., if
an inspection finding causes the plant to enter into an unanalyzed condition, the elevated risk
associated with that unanalyzed condition can often be correlated to an associated degradation
of safety margin or defense-in-depth).
Step 4.1 - Initial Evaluation
4.1.1 The purpose of this step is to determine if there are any significance colors (Green,
White, Yellow, or Red) that can be reasonably excluded from further consideration via
an initial evaluation using available quantitative and/or qualitative methods and best
available information. These methods should be consistent with traditional assessment
Issue Date: 01/10/19 3 0609 App M
approaches using reasonably conservative assumptions (e.g., minimal to no recovery
actions, use of screening values for human error probabilities). The evaluation should
not involve a detailed risk evaluation (although it may involve a simpler use of the same
tools) and need not be quantitative (e.g., in the case of findings associated with the
Emergency Preparedness and Radiation Protection cornerstones)1
. If the evaluation
shows that the finding is of very low safety significance (i.e., Green), the finding can be
documented in accordance with IMC 0611, “Power Reactor Inspection Reports,” and
the guidance provided in Step 4.4.2 of this appendix.
4.1.2 If the initial evaluation indicates that the risk significance of the finding is potentially
greater than Green, document the results using Exhibit 1, “Results of Initial Evaluation,”
of this appendix and then proceed to Step 4.2.
Step 4.2 - Attributes
4.2.1 For findings in which the risk significance is potentially greater than Green, evaluate the
following attributes to determine the significance of the finding, then proceed to
Step 4.3. Guidance on evaluating each attribute is contained in Exhibit 2,
“Considerations for Evaluation of Decision Attributes,” of this appendix.
4.2.1.1 Defense-in-Depth
4.2.1.2 Safety Margin
4.2.1.3 Extent of condition
4.2.1.4 Degree of Degradation
4.2.1.5 Exposure Time
4.2.1.6 Recovery Actions
4.2.1.7 Additional Qualitative Attributes
Step 4.3 - Integrated Risk-Informed Decision-Making
4.3.1 Integration of the results requires that the individual insights obtained from each
element of the decision-making process be weighed and combined to reach a
conclusion, in this case a decision on the significance of the finding. The staff involved
with analysis of the finding (e.g., inspectors, probabilistic risk assessment (PRA)
experts, engineering staff) should participate in the integration process. An example
approach to integrating multiple diverse sources of information as part of decisionmaking can be found in LIC- 504, “Integrated Risk-Informed Decision-Making Process
for Emergent Issues,” Appendix E, but use of those concepts should be in concert with
SDP-specific decision-making guidance contained in IMC 0609 Attachment 1.
Step 4.4 - Process and Documentation
4.4.1 If the results of the Appendix M evaluation indicate a greater than Green finding, the
decision-making logic should be documented using Table 1, ”Qualitative DecisionMaking Attributes for NRC Management Review,” and should be included in the SERP
package as described in IMC 0609, Attachment 1, “Significance and Enforcement
Review Panel.”
1
In cases where a qualitative approach is necessitated or appropriate, analogues can be drawn to existing
relationships between a performance deficiency and significance (from the IMC 0609 appendix relevant to the
performance deficiency) in order to establish a conservative estimate of the finding’s significance.
Issue Date: 01/10/19 4 0609 App M
4.4.2 If the results of the Appendix M evaluation indicate a Green finding, document the
quantitative and/or qualitative methods used, including the results, in the inspection
report.
0609M-05 REFERENCES
IMC 0609, Attachment 1, “Significance and Enforcement Review Panel Process”
IMC 0611, “Power Reactor Inspection Reports”
IMC 0612, “Issue Screening”
NRC Regulatory Guide 1.174, “An Approach for Using Probabilistic Risk Assessment in RiskInformed Decisions on Plant-Specific Changes to the Licensing Basis”
NRR Office Instruction LIC-504, “Integrated Risk-Informed Decision-Making Process for
Emergent Issues”
NRC, “Staff Requirements Memorandum - SECY-98-144 - White Paper on Risk-Informed and
Performance-Based Regulation,” SRM-SECY-98-144, March 1, 1999.
NUREG-1855, “Guidance on the Treatment of Uncertainties Associated with PRAs in RiskInformed Decisionmaking”
END
Issue Date: 01/10/19 E1-1 0609 App M
EXHIBIT 1 Results of the Initial Evaluation
1. Describe the influential assumptions used in the initial evaluation.
2. Provide sensitivity results on the key influential assumptions. Given that a detailed risk
evaluation is not tractable, these sensitivities might be qualitative or semi-quantitative,
and should only be performed when practical to do so. These might include changes to
the initiating event frequency, equipment failure rates, common cause failure
probabilities, and human error probabilities. In the case of purely qualitative initial
evaluations, these might include subjective evaluations of whether the significance
would differ for alternative assumptions.
3. Identify any information gaps in defining the influential assumptions used in the initial
evaluation.
Initial Evaluation Result: ____________________________
Issue Date: 01/10/19 E2-1 0609 App M
EXHIBIT 2 Considerations for Evaluation of Decision Attributes
A. Defense-in-Depth
Revision 3 of RG 1.174, “An Approach for Using Probabilistic Risk Assessment in RiskInformed Decisions on Plant-Specific Changes to the Licensing Basis,” identifies and
provides a discussion of seven considerations that should be used to evaluate impacts
on defense in depth. While RG 1.174 provides general guidance concerning analysis of
the risk associated with proposed changes in plant design and operation, the
considerations and discussion of defense in depth can be applied to the evaluation of
findings under the Reactor Oversight Process and in the use of this appendix. It is
important to note that the focus here is on the effect of the finding on defense in depth.
The seven defense-in-depth considerations presented are not intended to define how
defense in depth is implemented in a plant’s design, but rather to help the analyst
assess the impact of the finding on defense in depth.
1. Preserve a reasonable balance among the layers of defense.
A reasonable balance of the layers of defense (i.e., minimizing challenges to the
plant, preventing any events from progressing to core damage, containing the
radioactive source term, and emergency preparedness) helps to ensure an
apportionment of the plant’s capabilities between limiting disturbances to the
plant and mitigating their consequences. The term “reasonable balance” is not
meant to imply an equal apportionment of capabilities. The NRC recognizes that
aspects of a plant’s design or operation might cause one or more of the layers of
defense to be adversely affected. For these situations, the balance between the
other layers of defense becomes especially important when evaluating the
impact of a finding and its effect on defense in depth.
2. Preserve adequate capability of design features without an overreliance on
programmatic activities as compensatory measures2
.
Nuclear power plant licensees implement a number of programmatic activities,
including programs for quality assurance, testing and inspection, maintenance,
control of transient combustible material, foreign material exclusion, containment
cleanliness, and training. In some cases, activities that are part of these
programs are used as compensatory measures; that is, they are measures taken
to compensate for some reduced functionality, availability, reliability, redundancy,
or other feature of the plant’s design to ensure safety functions (e.g., reactor
vessel inspections that provide assurance that reactor vessel failure is unlikely).
Other examples include hardware (e.g., skid-mounted temporary power
supplies); human actions (e.g., manual system actuation); or some combination
of these measures. Such compensatory measures are often associated with
temporary plant configurations. The preferred approach for accomplishing safety
functions is through engineered systems. Therefore, when the finding
necessitates reliance on programmatic activities as compensatory measures,
analysis should indicate that this reliance is not excessive (i.e., not overly reliant).
2 The term “compensatory measures” is used here to refer to additional measures in place during the time of the
degraded condition.
Issue Date: 01/10/19 E2-2 0609 App M
The intent of this consideration is not to preclude the use of such programs as
compensatory measures but to ensure that the use of such measures does not
significantly reduce the capability of the design features.
3. Preserve system redundancy, independence, and diversity commensurate
with the expected frequency and consequences of challenges to the system,
including consideration of uncertainty.
The defense-in-depth philosophy has traditionally been applied in plant design
and operation to provide multiple means to accomplish safety functions. System
redundancy, independence, and diversity result in high availability and reliability
of the function and also help ensure that system functions are not reliant on any
single feature of the design. Redundancy provides for duplicate equipment that
enables the failure or unavailability of at least one set of equipment to be
tolerated without loss of function. Independence of equipment implies that the
redundant equipment is separate, such that it does not rely on the same supports
to function. This independence can sometimes be achieved by the use of
physical separation or physical protection. Diversity is accomplished by having
equipment that, while it performs the same function, relies on different attributes,
such as different principles of operation, different physical variables, different
conditions of operation, or production by different manufacturers, which helps
reduce common-cause failure (CCF). A degraded condition might reduce the
redundancy, independence, or diversity of systems. The intent of this
consideration is to ensure that the ability to provide the system function is
commensurate with the risk of scenarios that could be mitigated by that function.
The consideration of uncertainty, including the uncertainty inherent in the PRA,
implies that the use of redundancy, independence, or diversity provides high
reliability and availability and also results in the ability to tolerate failures or
unanticipated events.
4. Preserve adequate defense against potential CCFs.
An important aspect of ensuring defense in depth is to guard against CCF.
Multiple components may fail to function because of a single specific cause or
event that could simultaneously affect several components important to risk. The
cause or event may include an installation or construction deficiency, accidental
human action, extreme external environment, or an unintended cascading effect
from any other operation or failure within the plant. CCFs can also result from
poor design, manufacturing, or maintenance practices. Defenses can prevent
the occurrence of failures from the causes and events that could allow
simultaneous multiple component failures. Another aspect of guarding against
CCF is to ensure that an existing defense put in place to minimize the impact of
CCF is not significantly reduced; however, a reduction in one defense can be
compensated for by adding another.
5. Maintain multiple fission product barriers.
Fission product barriers include the physical barriers themselves (e.g., the fuel
cladding, reactor coolant system pressure boundary, and containment) and any
equipment relied on to protect the barriers (e.g., containment spray). In general,
these barriers are designed to perform independently so that a complete failure
Issue Date: 01/10/19 E2-3 0609 App M
of one barrier does not disable the next subsequent barrier. For example, one
barrier, the containment, is designed to withstand a double-ended guillotine
break of the largest pipe in the reactor coolant system, another barrier. A plant’s
licensing basis might contain events that, by their very nature, challenge multiple
barriers simultaneously. Examples include interfacing-system loss-of-coolant
accidents or steam generator tube rupture. Therefore, complete independence
of barriers, while a goal, might not be achievable for all possible scenarios.
6. Preserve sufficient defense against human errors.
Human errors include the failure of operators to correctly and promptly perform
the actions necessary to operate the plant or respond to off-normal conditions
and accidents, errors committed during test and maintenance, and incorrect
actions by other plant staff. Human errors can result in the degradation or failure
of a system to perform its function, thereby significantly reducing the
effectiveness of one of the layers of defense or one of the fission product
barriers. The plant design and operation include defenses to prevent the
occurrence of such errors and events. These defenses generally involve the use
of procedures, training, and human engineering; however, other considerations
(e.g., communication protocols) might also be important.
7. Continue to meet the intent of the plant’s design criteria.
For plants licensed under Title 10 of the Code of Federal Regulations Parts 50
or 52, the plant’s design criteria are set forth in the current licensing basis of the
plant. The plant’s design criteria define minimum requirements that achieve
aspects of the defense-in-depth philosophy. When evaluating a finding, the
analysis should identify the design criteria that is challenged and how the finding
impacts the design criteria.
B. Safety Margin
Safety margin is the extra capacity factored into the design of a structure, system, or
component (SSC) so that it can cope with conditions beyond the expected to
compensate for uncertainty. The evaluation should assess whether the impact of the
finding is consistent with the principle that sufficient safety margins are maintained. In
evaluating this factor, the staff should use engineering analysis or engineering judgment
appropriate for evaluating whether sufficient safety margins would be maintained given
the finding. The evaluation should consider if the inspection finding identifies an issue
which affects the licensees ability to meet the codes and standards or their alternatives
approved for use by the NRC. Additionally, consider if the finding identifies an issue
which affects meeting safety analysis acceptance criteria in the licensing basis (e.g.,
Update Final Safety Analysis Report, supporting analyses) or proposed revisions that
provide sufficient margin to account for analysis and data uncertainty.
Issue Date: 01/10/19 E2-4 0609 App M
C. Extent of Condition
If a finding is not isolated to a specific occurrence, condition, or event, its safety
significance is typically greater. When a finding is capable of affecting multiple SSCs,
the number of degraded conditions has the potential to be greater than a case in which a
finding is isolated to a specific SSC. The identified extent of condition should have a
reasonable and sound technical basis to justify the scope.
D. Degree of Degradation
The magnitude and detailed circumstances of the degraded condition (or programmatic
weakness) have a direct effect on the safety significance of the finding. As stated in IMC 0308, Attachment 3, “Technical Basis for the SDP,” the finding (i.e., more-than-minor
performance deficiency) is the proximate cause of the degraded condition or
programmatic weakness. Logically, the more a condition is degraded or program is
weakened, the more safety significant the finding.
E. Exposure Time
Generally, the longer a finding is left uncorrected the more opportunities the finding has
to manifest itself (i.e., act as the proximate cause of a degraded condition or
programmatic weakness). As such, the longer the exposure time the more safety
significant the finding.
F. Recovery Actions
Even if the extent of condition, degree of the degraded condition (or programmatic
weakness), and exposure time increased the safety significance of a finding, crediting
established recovery actions or mitigation strategies should be appropriately considered
to determine the overall significance of the finding.
G. Additional Qualitative Attributes
Depending on the situation, the previous six attributes may not capture all of the
qualitative attributes that may apply to the finding. Therefore, additional qualitative
circumstances, as appropriate, may be considered in the decision making process. Any
additional qualitative circumstances for management consideration should have a clear
and reasonable nexus to the safety significance of the finding. If additional qualitative
attributes are considered, one should be particularly aware of the goal of having a
scrutable and repeatable outcome, and should consider whether other decision makers
would reasonably be expected to invoke the same qualitative attributes.
Issue Date: 01/10/19 T-1 0609 App M
TABLE 1
Qualitative Decision-Making Attributes for NRC Management Review
Decision Attribute Basis for Input to Decision - Provide
qualitative and/or quantitative information for
management review and decision making.
Defense-in-Depth
Safety Margin
Extent of Condition
Degree of Degradation
Exposure Time
Recovery Actions
Additional Qualitative
Considerations
Result of management review (COLOR):
Issue Date: 01/10/19 Att1-1 0609 App M
Attachment 1
Revision History IMC 0609 Appendix M
Commitment
Tracking
Number
Accession Number
Issue Date
Change Notice
Description of Change Description of Training
Required and
Completion Date
Comment Resolution
and Closed Feedback
Form Accession Number
(Pre-Decisional, NonPublic Information)
N/A ML062510080
12/22/06
CN 06-036
This new document has been issued to
provide guidance to NRC management and
inspection staff for assessing significance of
inspection findings.
This procedure was
developed by involved
stakeholders. No
training on the
procedure
recommended at this
time. However,
additional guidance
may be developed
based on experience
gained.
N/A ML101550365
04/04/12
CN 12-005
Provided clarification in the Scope and
Applicability sections to articulate the Appendix
M entry conditions and that Appendix M is not
intended to be used to develop new models or
acquire in-depth expert elicitation. In addition,
ROPFF 0609M-1412 was incorporated to
clarify that Appendix M applies to all the safety
cornerstones of the ROP.
None N/A
DRAFT
Made public to solicit industry comment at the
October 18, 2018, ROP Public meeting.
None N/A
Issue Date: 01/10/19 Att1-2 0609 App M
Commitment
Tracking
Number
Accession Number
Issue Date
Change Notice
Description of Change Description of Training
Required and
Completion Date
Comment Resolution
and Closed Feedback
Form Accession Number
(Pre-Decisional, NonPublic Information)
N/A ML18183A043
01/10/19
Cn 19-002
Provided clarification of the existing entry
conditions to more clearly illustrate when
Appendix M should be used. In addition,
provided clarification of the existing decisionmaking attributes to align with the enhanced
guidance in Revision 3 of Regulatory Guide 1.174, “An Approach for Using Probabilistic
Risk Assessment in Risk-Informed Decisions
on Plant-Specific Changes to the Licensing
Basis,” which was issued in January 2018.
Also, the description of the initial evaluation
was clarified to better align with intent/practice,
since the previous description inferred that (in
the case of a quantitative estimate) one would
use enveloping input assumptions across-theboard. Finally, ROPFF 0609M-2272 was
addressed to make the guidance more useful
for RP and EP findings. A Commissioners’
Assistant note was issued (ML18311A027) to
notify the Commission of the described
changes in accordance with Management
Directive 8.13 and COMSECY-16-0022.
None ML18184A428
0609M-2272