ML20151U081
| ML20151U081 | |
| Person / Time | |
|---|---|
| Issue date: | 07/31/1998 |
| From: | NRC (Affiliation Not Assigned) |
| To: | |
| References | |
| NUREG-0800, NUREG-0800-19.0.0-R0, NUREG-800, NUREG-800-19.0.0-R, SRP-19.00.00, NUDOCS 9809100278 | |
| Download: ML20151U081 (78) | |
Text
.
1 l
l e.g\\
\\
UNITED STATES NUCLEAR REGULATORY COMMISSION O
\\,,,,.
I STANDARD REVIEW PLAN OFFICE OF NUCLEAR REACTOR REGULATION 19.0 USE OF PROBABILISTIC RISK ASSESSMENT IN PLANT-SPECIFIC, RISK-INFORMED DECISIONMAKING: GENERAL GUIDANCE INTRODUCTION This chapter of the Standard Review Plan (SRP) identifies the roles and responsibilities of organizations 1
in the U.S. Nuclear Regulatory Commission (NRC) that participate in risk-informed reviews of licensees' proposals for changes to the licensing basis (LB)' of nuclear power plants. The SRP identifies the types ofinformation that may be used in fulfilling an organization's responsibilities and provides general guidance on how the information from a probabilistic risk assessment (PRA) can be combined with other pertinent information in the process of making a regulatory decision.
The guidance in this document is a logical extension of current NRC policy on the use of PRA in regulatory activities which is documented in the Commission's PRA policy statement and implementation plan (Refs.1-3). In developing this SRP chapter, the staff considered the NRC's O
guidance on the use of PRA in risk-informed regulatory applications as documented in Regulatory
/
Guide (RG) 1,174 (Ref. 4) as well as the relevant industry guidance documented by the Electric Power ~
Research Institute (EPRI) in its "Probabilistic Safety Assessment (PSA) Applications Guide (Ref. 6).
In addition, this chapter references other SRP chapters that provide additional guidance for reviewing specific applications of PRA in regulated activities.
In the process of risk-informed decisionmaking, the NRC will rely on the approach discussed in this chapter. Above all, the design, construction, and operational practices of each plant are expected to be y
consistent with its LB. In addition, the risk evaluations performed to justify regulatory changes are
[/ l expected to realistically reflect these plant-specific design, construction, and operational practices. The I
These are modifications to a plant's design, operations or other activities that require NRC approval.
These modifications could include items such as exemption requests under 10 CFR 50.11 and license amendments under 10 CFR 50.90.
Rev. 0 - July 1998 USNRC STANDARD REVIEW PLAN Standard review plans are prepared for the guidance of the Office of Nuclear Reactor Regulation staff responsible for the Ivylew of applications to construct and operate nuclear power l
as of the Commission's policy to inform the nuclear indus lants. These documents are made swallable to the public and the general public of regulatory procedures and i
los. Standard review plans are not substitutes for regulatory guides or the Commission's regu ations and compliance j
with them is not required. The standard review plan sections are keyed to the Standard Format and Content of Safety i
lR Analysis Reports for Nuclear Power Ptar.ts. Not all sections of the Standard Format have a corresronding review plan.
Published standard review plans will be revised periodically, as appropriate, to accommodate comt sents and to reflect i
new information and experience.
Comments and suggestions for improvement will be considered and should be sent to the U.S. Nuclear Regulatory l
Commission, Office of Nuclear Reactor Regulation, Washington, D.C. 20555.
~
9809100278 980731 PDR NUREG 0000 R PDR
i PRA analyses should be as realistic as practicable and, when interpreting the results of those analy the staff should account for the impact of the most significant uncertainties. The results of these risk analyses will then form part of the input to the decisionmaking process that evaluates the margin plant capability (in both performance and redundancy / diversity). Specifically, the decisionmak process will use the results of the risk analyses in a manner that complements traditional engin approaches, supports the defense-in-depth philosophy, and preserves safety margins. Thus, risk analysis will inform, but it will not determine regulatory decisions.
REVIEW RESPONSIBILITIES The technical nature of a licensee's request will determine which technical review branch in the NRC's Office of Nuclear Reactor Regulation (NRR) will serve as the primary review branch and as such, has overall responsibility for leading the technical review, drafting the staff safety evaluation report (SER) or other appropriate regulatory document, and coordinating input from other technical review organizations. In addition, the following organizations will normally play a role in reviewing risk-informed proposals:
The Probabilistic Safety Assessment Branch (SPSB) assists the primary review branch (upon request) by reviewing the PRA information and findings submitted by the licensee. Review support includes assessing the adequacy of the scope, level of detail, and quality of the PRA used by the licensee to support the regulatory change, as well as applying risk-related acceptance guidelines to support decisionmaking.
The Reactor Systems Branch (SRXB) assists the primary review branch or SPSB (upon request) by providing support for. accident sequence modeling, including treatment of reactivity and thermaLhydraulic phenomena, system response, and the implementation of emergency and abnormal operating procedures.
The Containment and Severe Accident Branch (SCSB) holds the primary responsibility for reviewing containment response and containment integrity information submitted by the licensee in support of a request for regulatory action.
The Emergency Preparedness and Radiation Protection Branch (PERB) holds the primary responsibility for reviewing evaluations of radionuclide contamination or public health effects submitted by a licensee in support of a request for regulatory action.
The Office of Nuclear Regulatory Research (RES) assists the primary review branch (upon request) by providing technical support in areas involving all aspects of PRA, severe accident phenomenology, and engineering studies.
The Office for Analysis and Evaluation of Operational Data (AEOD) assists the primary review branch (upon request) by providing generic and plant-specific data from operating experience regarding system / component availabilities /reliabilities, frequency of initiating events, common cause failure rates, and human error events.
The Regional Offices assist the primary review branch (upon request) by providing information on licensees' operational experience in areas of system performance, operator performance, risk management practices, and management controls.
Rev. 0 - July 1998 SRP 19-2
F I.
AREAS OF REVIEW r
The NRC's PRA Implementation Plan as proposed in Ref 3 (and as updated quarterly, see for example Ref. 5) identifies a wide scope of regulatory activities for which PRA provides valuable insights. This scope includes activities that require NRC review and approval, as well as other activities that are considered internal to the NRC and affect licensees and applicants in a less direct manner (e.g., generic issue prioritization). This SRP chapter solely concerns licensing amendment requests submitted for NRC review and approval for which PRA can play an effective role in the decisionmaking process.
General review guidance for applicable activities is presented in this SRP chapter. In addition, application-specific SRP chapters are available to provide additional guidance for several activities including the following examples; a
changes to allawed outage times (AOTs) and surveillance test intervals (STIs) in plant-specific a
technical specifications (SRP Chapter 16.1)
]
changes in the scope and inquency of tests on pumps and valves in a licensee's inservice test (IST) program (SRP Chapter 3.9.7) changes in the scope and frequency of inspections in a licensee's inservice inspection (ISI) e program (draft SRP Chapter 3.9.8) i 1
RG 1.174 defines an acceptable approach for use in analyzing and evaluating proposed LB changes.
This approach supports the staff's desire to base its decisions on the results of traditional engineering l
evaluations, supported by insights (derived through the use of PRA methods) on the risk significance of A
the proposed changes. The decisionmaking process leading to the proposed change is expected to follow an integrated approach (considering traditional engineering and risk information)'and ~may build upon qualitative factors as well as quantitative analyses and information.
As discussed later in this section, the scope of the staff review of a risk-informed application will depend on the specifics of the application. However, this scope should include reviewing the four
{
elements suggested in Section 2 of RG 1.174. The areas of review for each of these elements are summarized as follows:
l:
Element 1 - Define the Proposed Change: The objective of this element is to lay the i
groundwork for evaluating the safety impacts of the proposed change. Therefore, one area of l
review would be an evaluation of the proposed change in light of the LB (i.e., evaluation of the structures, systems, and compone'its (SSCs), as well as the plant procedures and activities that are affected by the proposed change and how these SSCs, procedures or activities relate to the LB). In addition, an evaluation of the method of analysis and a study of available insights from traditional and probabilistic engineering studies that are relevant to the proposed change would l
be necessary to determine if the change can be supported.
j Element 2 - Conduct Engineering Evaluations: The licensees' decisionmaking process e
should factor in the appropriate traditional and probabilistic engineering insights. Reviewers L
should evaluate the proposed change to ensure that the defense-in-depth philosophy and sufficient safety margins are maintained, and that the calculated change in plant risk is within i
the guidelines specified in RG 1.174. Reviewers should also verify that insights from the l
engineering evaluations used tojustify a change have been used to improve operational and engineering decisions where appropriate, and not simply to climinate requirements the licensee i
SRP 19-3 Rev. 0 - July 1998 L
sees as undesirable.
Element 3 - Develop Implementation md Monitoring Strategies: Results from implementation and monitoring strategies can provide an early indication of unanticipated degradation of performance of those plant elements affected by the proposed change. These strategies are therefore important in applications where uncertainty in evaluation models and/or data used to justify the change can change the conclusions of the analysis. As such, the review scope should include provisions to ensure that the licensee has proposed an implementation and monitoring process that is adequate to (in part) account for uncertainties regarding plant l
performance under the proposed change.
l Element 4 - Document Evaluations and Submit Request: Reviewers should ensure that the submittal includes sufficient information to support conclusions regarding the acceptability of the proposed change, and that the archival documentation of the evaluation process and findings is maintained and available for staff audit and review. Reviewers should also ensure that the licensee has requested the appropriate regulatory action (for example, a license amendment, an exemption, or a change to technical specifications). Where appropriate, reviewers should ensure that the submittal has documented any licensee proposed enhancements to regulatory requirements (e.g., high risk significant SSCs not currently subject to regulatory control may be subject to requirements commensurate with their risk significance). Finally, reviewers should ensure that LB changes are appropriately included in an updated safety analysis report, as necessary.
l Application-Specific Reviews This chapter of the SRP is intended to provide guidance for reviewi.ng applications in risk-informed regulation where numerical values of risk indices play a relatively large role in the decisionmaking process and where a broad set of scenarios and plant operating modes may be affected. Where it is determined that an application could justify a review that is less than the full scope described in this document, reviewers should choose the relevant and applicable parts of this SRP chapter for guidance.
The necessary sophistication of the review of the PRA, its supporting analyses, and its results depends on the contribution the risk assessment provides to the integrated decisionmaking. Application-specific SRP chapters (where available) provide additional guidance in this area.
11.
ACCEIYTANCE CRITERIA This SRP chapter provides guidance for use in conducting staff reviews of PRA findings and risk insights in support of licensees' requests for changes to the LB of nuclear power plants (e.g., requests for license amendments under 10 CFR 50.90, and exemptions under 10 CFR 50.11). RG 1.174 sets forth guidance for licensees to use in implementing acceptable methods for conducting PRA and traditional engineering analyses to support such changes.
To evaluate licensee-initiated LB changes which are consistent ivith currently approved staff positions (e.g., regulatory guides, standard review plans, or branch technical positions), the staff normally uses traditional engineering analyses. Licensees would not be expected to submit risk information in support of such proposed changes. By contrast, to evaluate licensee-initiated LB changes which go beyond current staff positions, the staff may use traditional engineering analyses as well as the risk-informed approach set forth in this SRP chapter. In such instances, licensees may be requested to submit Rev. 0 - July 1998 SRP 19-4
l I
I supplemental risk information or traditional engineering infonnation if such information is not already m
included as part of the original submittals. If risk information on the proposed LB changes is not provided, the staff will determine if the application can be approved on the basis of the information provided using traditional methods and will either approve or reject the application based upon this information. For those licensee initiated LB changes which a licensee chooses (or is requested by the staff) to support with risk information, this SRP chapter describes the scope and content of the staff's review by considering engineering issues and applying risk insights.
Licensees submitting risk information to support changes to their LB (whether on their own initiative or at the request of the staff) should address each of the principles of risk-informed regulation discussed in RG 1.174. The staff should then determine if the licensees' selected approaches and methods (whether quantitative or qualitative, and traditional or probabilistic), data, and criteria for considering risk are appropriate for the decision to be made.
For each risk-informed application, reviewers should ensure that the proposed changes meet the following principles (Sections of this SRP chapter dealing with review guidance for each principle are identified in brackets):
1.
The proposed change meets the current regulations unless it is explicitly related to a requested exemption or rule change, i.e., a " specific exemption" tender 10 CFR 50.12 or a " petition for rulemaking" under 10 CFR 2.802. [Section III.2.1].
2.
The proposed change is consistent with the defense-in-depth philosophy [Section 111.2.1].
O 3.
The proposed change maintains sufficient safety margins [Section 111.2.1].
4.
When proposed changes result in an increase in core damage frequency or risk, the increases should be small and consistent with the intent of the Commission's Safety Goal Policy Statement [ Sections 111.2.2 and III.2.3].
5.
The impact of the proposed change should be monitored using performance measurement strategies [Section 111.3].
In demonstrating adherence to the above principles, reviewers should ensure that licensees address the following issues as part of their submittals:
All safety impacts of the proposed change are evaluated in an integrated manner as part of an overall risk management approach in which the licensee is using risk analysis to improve operational and engineering decisions broadly by identifying and taking advantage of opportunities to reduce risk, and not just to eliminate requirements the licensee sees as desirable. For those cases when risk increases are proposed, the benefits should be described and should be commensurate with the proposed risk increases. The approach used to identify changes in requirements was used to identify areas where requirements should be increased as well as where they could be reduced [Section 111,2.3].
The scope and quality of the engineering analyses (including traditional and probabilistic analyses) caducted to justify the proposed LB change are appropriate for the nature and scope
,n of the change and are based on the as-built and as-operated and maintained plant, including j
reflecting operating experience at the plant [Section III.2.2].
w SRP 19-5 Rev. 0 - July 1998
The plant-specific PRA supponing the licensee's proposals has been subjected to quality
=
controls such as an independent peer review or certification [Section 111.2.2].
Appropriate consideration of uncertainty is given in analyses and interpretation of findings, including using a program of monitoring, feedback and corrective action to address significant uncertainties [ Sections III.2.2 and 111.3].
The use of core damage frequency (CDF) and large early release frequency (LERF) as bases for probabilistic risk assessment guidelines is an acceptable approach to addressing Principle 4.
Use of the Commission's Safety Goal quantitative health objectives (QHOs) in lieu of LERF is acceptable in principle and licensees may propose their use. However, in practice, implementing such an approach would require an extension to a Level 3 PRA, in which case the methods and assumptions used in the Level 3 analysis, and associated uncertainties,.would require additional attention [Section 111.2.2].
Increases in estimated CDF and LERF resulting from proposed LB changes will be limited to small increments. The cumulative effect of such changes should be tracked and considered in the decision process [Section III.2.2].
The acceptability of the proposed changes should be evaluated by the licensee in an integrated fashion that ensures that all principles are met [Section 111.2.3].
Data, methods, and assessment criteria used to support regulatory decisionmaking must be well documented and available for public review [Section III.4].
III.
REVIEW GUIDANCE AND PROCEDURES For risk-informed applications, reviewers should ensure that licensees'submittals meet the principles specified in Section II of this SRP chapter, and address the expectations for risk-informed decisionmaking (also specified in Section II). This section provides guidance to assist reviewers in making this determination. For consistency, Sections Ill.1 through III.4 present this guidance in terms of the four elements of the approach described in Section 2 of RG 1.174.
Ill.1 Element 1: Define the Prooosed Change In this element, reviewers should verify that the submittal provides enough information to meet the staff's expectation that all potential safety impacts have been identified and evaluated. In addition, reviewers should be satisfied that, where appropriate, the licensee has identified design and operational aspects of the plant related to the change request that should be enhanced consistent with an improved understanding of their safety significance based on the methodology used to support the proposed j
relaxation in regulation. These enhancements should be appropriately reflected in changes to the l
plant's LB (e.g., technical specification, license conditions, and FSAR).
l Reviewers must also assess the proposed changes as they relate to the plant's LB, which specifies how the licensee satisfies certain basic regulatory requirements such as diversity, redundancy, defense-in-depth, and the General Design Criteria. This assessment should include reviewing the engineering (or other pertinent) analysis and data that identify the safety margins, and plant design and/or activities Rev. 0 - July 1998 SRP 19-6
i h) conducted to preserve those margins. If exemptions from regulations or other forms of relief are
[
needed to implement the licensee's proposed change, reviewers should ensure that the appropriate rk requests accompany the licensee's submittal.
i:
i Reviewers should also verify that the licensee has identified and appropriately used available information reflecting traditional engineering concepts and principles. Among the non-PRA sources of
}
information that should be examined to support the evaluation of safety significance include the safety t
L insights developed in licensing documents such as the FSAR, as well as the bases for the plant's Technical Specifications, which may include AOTs, limiting conditions for operation (LCOs), and surveillance requirements (SRs).
i
' Where available, plant-specific data and operational information should be factored into the def'inition of the proposed change. Reviewers should consider the way in which the issues at hand are reflected in operational data. Useful insights from plant-specific operating experience can also be obtained from i
inspections that follow incidents at the facility, including incident investigation and augmented team.
inspections conducted by the NRC, incident assessments documented in significant operating event reports prepared by the Institute of Nuclear Power Operations (INPO), licensee follow-up investigations, and routine inspections by NRC resident inspectors. Inspection results can provide i
valuable qualitative insights in such areas as human performance, management controls, adequacy of procedures, and root causes of events, which are often difficult to treat with precision in a PRA.
Finally, as part of the initial review of the licensing amendment, reviewers should determine if the submitta! adequately characterizes the impact of the proposed change (specifically, if the submittal j
identifies all SSCs or other plant elements affected by the proposed change) and if the analyses i
performed and submitted by the licensees have the scope and depth to adequately characterize the TN Q
impact of the change.
Licensees may submit proposals which include several individual LB changes that have been evaluated j
and will be implemented in an integrated fashion. For example, individual changes may be grouped -
i together for convenience (ease of implementation and/or review), or changes may be combined as risk l
. tradeoffs (balancing risk _ increases with risk decreases). Changes grouped in this way should normally be related, for example by affecting the same single system or activity, the same safety function, or the same accident sequence group, or by being of the same type (e.g., changes in AOT). However, this does not preclude unrelated changes from being accepted. When combined change requests are submitted, the staff should conduct a detailed assessment of the relationship between the individual changes and how they have been modeled in the risk assessment. In its review, the staff should evaluate the acceptability of the individual changes and the overall impact of the combined changes with respect to the principles and expectations dismed in Section II of this SRP chapter. Section j
111.2.3 discusses the review of combined change re usts in more detail.
III.2_ Flement 2: Candet Fneineerina Evalmtlans In order to make findings regarding the acceptability of a proposed license anwndment, the staff should j
' establish its position on the basis of an integrated assessment of traditional engineering evaluations and p
probabilistic information. Section 2.2 of Reg Guide RG 1.174 describes the specific evaluations that n
the licensee is expected to perform. The scope and quality of the engineering analyses conducted to justify a proposed change should be appropriate for the nature and scope of that change. Section 3 of RG 1.174 describes the various types of traditional engineering and probabilistic information which i
SRP 19-7 Rev. 0 - July 1998 i
L j
l should be included in submittals.
The results of this element should be reviewed to determine if the submittal satisfies the following principles for risk-informed decisionmaking: the proposed change meets current regulations (unless the I
change is explicitly related to a requested exemption or rule change); the defense-in-depth philosophy is maintained; sufficient safety margins are maintained; and proposed increases in core damage frequency and/or risk (if any) are small and are consistent with the intent of the Commission's Safety Goal Policy Statement.
111.2.1 Evaluation of Defense-in-Denth Attributes and Safety Margins Reviewers should assess the licensee's engineering evaluations to confirm that the principles identified in Section 11 are not compromised. These evaluations should include not only the traditional design basis accident (DBA) analyses, but also evaluations of the defense-in-depth attributes of the plant, safety margins, and risk assessments performed to obtain risk insights and to quantify the impact of the proposed change.
111.2. 1. 1 Defense-in-Depth Defense-in-depth is defined as a philosophy which ensures that successive measures are incorporated into the design and operating practices for nuclear plants to compensate for potential failures in protection and safety measures. In risk-informed regulation, the intent is to ensure that the defense-in-depth philosophy is maintained, not to prevent changes in the way defense-in-depth is achieved. The l
l defense-in-depth philosophy has been and continues to be an effective way to account for uncertainties in equipment and human performance. In come cases, risk analysis can help quantify the range of uncertainty; however, there will likely remain areas of large uncertainty or areas not covered by the risk analysis. Where a comprehensive risk analysxs can be performed, it can help determine the approximate extent of defense-in-depth (e.g., balance among core damage prevention, containment failure, and consequence mitigation) to ensure protection of public health and safety. However, because PRAs do not reflect all aspects of defense-in-depth, appropriate traditional defense-in-depth considerations should also be used to account for uncertainties.
Preservation of Multiple Barriers for Radioactivity Release Defense-in-depth can be evaluated on the basis of considerations involving the barriers that prevent or mitigate radioactivity release. Release of radioactive materials from the reactor to the environment is prevented by a succession of passive barriers including the fuel cladding, reactor coolant pressure boundary, and containment structure. These barriers, together with an imposed exclusion area and emergency preparedness, are the essential elements for accident consequence mitigation. Given these multiple barriers, safety is ensured through the application of deterministic safety criteria for the performance of each barrier, and through the design and operation of systems to support the functional performance of each barrier.
In maintaining consistency with the defense-in-depth philosophy, the proposed license amendment should not result in any substantial change in the effectiveness of the barriers. Consequently, reviewers should consider the following objectives to ensure that the proposed change maintains appropriate safety within the defense-in-depth philosophy:
The change does not result in a significant increase in the existing challenges to the integrity of Rev. 0 - July 1998 SRP 19-8
l q
the barriers.
1 V
The proposal does not significantly change the failure probability of any individual barrier.
=
The proposal does not introduce new or additional failure dependencies among barriers that significantly increase the likelihood of failure compared to the existing conditions.
The overall redundancy and diversity among the barriers is sufficient to ensure compatibility with the risk acceptance guidelines.
In demonstrating that the proposal fulfills the objectives listed above, the staff expects that the proposed change will meet the following guidelines:
A reasonable balance is preserved among prevention of core danwge, prevention of containment failure, and mitigation of consequences.
The proposal avoids over-reliance on programmatic activities to compensate for weaknesses in plant design.
The proposed change preserves system redondancy, independence, and diversity commensurate with the expected frequency of challenges, consequences of failure of the system, and associated uncertainties.
The proposal preseves defenses against potential common cause failures and assesses the C
T potential introductu n of new common cause failure mechanisms.
)
~
The proposed change does not degrade the independence of barriers.
=
The proposed change preserves defenses against human errors.
The proposal fulfills the intent of the General Design Criteria in 10 CFR 50, Appendix A.
Reviewers can assess fulfillment of the above guidelines by using qualitative or traditional engineering arguments or by using PRA results contained in the accident sequences or cutsets.
Role of PRA in Review of Defense-in-Depth In addition to the usual quantitative risk indices, PRAs provide important qualitative results, namely, the contributors to accident sequences. For PRAs that use the fault tree linking approach these contributors are described by the accident sequence minirnal cutsets. Each accident sequence minimal cutset is a combination of passive and active SSC failures and human errors that would cause core damage or a release of radioactivity. The cutsets therefore directly show one particular aspect of defense-in-depth, in that they reveal how many failures must occur in order for core damage or l
radiological release to occur. Thus, the minimal cutsets show the effective redundancy and diversity of the plant design. For analysis approaches that use event trees with boundary conditions, the results l
take the form of accident sequence descriptions and typically include elements representing unavailabilities of systems (or trains of systems) rather than components. However, in most cases, cutsets providing a component level decomposition of the system (or train) unavailabilities are o) provided, and an equivalence to the minimal cutset description can be established if necessary.
a SRP 19-9 Rev. 0 - July 1998 l
In most cases, events appearing in each minimal cutset are targeted by programmatic activities to ensure the reliability of the associated SSC. Specific activities that are important to maintain the reliability of a component include: IST, ISI, periodic surveillance required by Technical Specifications, quality assurance, and maintenance. Therefore, when a review of the minimal cutsets reveals areas where redundancy or diversity are already marginal, it would arguably be inappropriate to reduce the level of activities aimed at ensuring SSC performance. (The exception would arise if the licensee can l
show that the activities have little or no effect on SSC performance, or if it can be shown that uncertainties in the performance of the elements in this cutset are well understood and quantified. It is also possible that the licensee could propose compensating or alternative activities to provide assurance of SSC performance.) The objective of this review is to avoid completely relaxing the defense-in-depth posture at points at which the plant design has the least overall functional independence, redundancy, and/or diversity. On the other hand, in areas where a plant has substantial redundancy and diversity, defense-in-depth arguments used to justify relaxations should be given appropriate weight.
As part of the defense-in-depth evaluation, reviewers should consider the effects of multiple component failures and common cause failures that could potentially result from the proposed change. For example, if the licensee proposes to reduce the requirements for all events in a cutset, reviewers should ensure that the effect of the change is properly modeled and that the change does not have an adverse l
effect on defense in-depth.
1 Finally, in assessing the m.ident sequence cutsets, reviewers should devote attention to potential over-i reliance on programmatic activities or operator actions that compensate for weaknesses in the plant design. For example, proposed maintenance and surveillance activities should complement and not replace proper plant design.
l 111. 2. 1. 2 Safety Margins In the determination of the design performance characteristics of a system, safety margin represents an allowance for uncertainty in SSC performance. Current safety analysis practices incorporate consideration of margin in most areas. As examples, many engineering standards, licensing analyses, and technical specifications take margin into account.
Incorporating margin can result in over-designing of components, incorporation of extra systems or system trains, or in conservative operating requirements for systems and components. Therefore, some licensee applications will seek to reduce this margin in some areas. Such reductions should appropriately reflect the current understanding of existing uncertainties and the potential impact of the proposed change. Therefore, in evaluating a proposed change request, reviewers should establish that the proposal fulfills the following guidelines:
The proposal meets established engineering codes and standards or NRC-approved alternatives,
=
or deviations are justified.
The proposal meets the safety analysis acceptance criteria in the LB, or proposed revisions i
provide sufhcient margin to account for uncertainty in the analysis and data.
Clearly, these guidelines are closely related to the guidance provided in Section 111.2.1.3 regarding the need to maintain the LB. The thrust of the guidance in the present section is to sensitize reviewers to the implications of relaxing the margin when evaluating the acceptability of changes to the LB.
Rev. 0 - July 1998 SRP 19-10 o
l l!m The level ofjustification required for changes in margin should depend on how much uncertainty is f
)
associated with the performance parameter in question, the availability of alternatives to compensate for XJ adverse performance, and the consequences of functional failure of the affected elements. Therefore, the results derived from risk evaluations and the associated analysis of uncertainties (especially in the analysis areas and models affected by the application) will provide useful information to help in the reviewers' decisionmaking. As an example, in evaluating available safety margins, reviewers should l
consider the risk profile of the plant. If a proposed LB change creates or exacerbates a situation where j
i risk is dominated by a few elements (SSCs or human actions) or a few accident sequences, the reviewers should carefully evaluate the modeling of these elements or sequences including the modeling of uncertainties. Reviewers should consider the results from the analysis of uncertainty when determining of the acceptability of the reduction in margin from the proposed change.
In demonstrating available safety margins, licensees will, in some cases support their proposal by citing new data from plant tests or research projects, or will conduct analyses using models that are predicated on new data. The following examples illustrate situations in which data and analyses can be used effectively to support the LB change request:
It is shown that a phenomenon of concern cannot occur or is less likely to occur than originally thought.
It is shown that the count of safety margin in the design is significantly greater than that which was assumed when the requirement or position was imposed.
It is shown that time available for operator actions is greater than originally assumed.
p I
The reviewers' primary objective is to verify the relevance and acceptability of the new information with respect to the requested LB change. Data that directly apply to the original technical concern should be considered in the decision process. Depending on the circumstances, the cognizant review branch may have additional specific guidance available for use in reviewing the quality and acceptability of the data. However, the data or analyses must be clearly applicable to the plant and specific circumstances in which they are being applied.
111.2. 1. 3 Current Regulations Reviewers should ensure that the proposed change satisfies current regulations (including the General Design Criteria), unless the licensee explicitly includes a proposed exemptiori or rule change (i.e., a
" specific exemption" as allowed by 10 CFR 50.12 or a " petition for rulen:aking" in accordance with 10 CFR 2.802).
The LB also applies until the staff approves modifications to the existing basis. It is expected that some applications will seek to modify the LB in risk-informed submittals. Applications that seek to make qualitative changes to the LB (such as moving components out of the scope of a required progrm) should be reviewed in greater detail with respect to defense-in-depth and safety margins when compared to applications that seek to make parametric changes (such as incremental changes to surveillance interval).
111.2.2 Risk Assessment n
! V)
- (
For effective implementation of risk-hiformed regulatory approaches, reviewers should ensure that the SRP 19-11 Rev. 0 - July 1998
licensee has demonstrated that the plant's design and actual operating conditions and practices are properly reflected in the risk insights derived using the plant-specific PRA model. Otherwise, the risk assessment may provide inaccurate or misleading information that will require careful scrutiny before use in any regulatory decisionmaking process.
Development of a plant-specific, risk-informed program also requires the availability of information to identify the SSCs and human actions that contribute most significantly to the plant's estimated risk. In addition, it is necessary to be able to capture the impact of the proposed change on the elements of the PRA. Section 111.2.2.1 of this SRP chapter discusses the characterization of the proposed change in terms of PRA model elements. The results of this determination of the cause-effect relationships between the proposed application and the PRA models will help define the scope and level of detail required for the PRA to support the application. Sections 111.2.2.2 and 111.2.2.3 discuss these topics.
Many applications, such as those involving changes in component test intervals, allow explicit PRA modeling of the impact of the proposed change and quantification of the expected change in risk using plausible models of the impact on SSC unavailability to the extent tha: the affected components are included in the plant's PRA. For other risk-informed applications, however, it may not be feasible to explicitly model the cause-and-effect relationship because the resulting actual impact on component unavailability is not clearly understood. For such applications, the use of risk categorization techniques provides a useful way to identify groups of SSCs that are less risk important to risk and, as such, are possible candidates for a graded approach to regulatory requirements. Using such a categorization approach, however, it is still necessary to understand the potential or bounding impact of the proposed change, and to assess the risk impact through bounding evaluations. In either the detailed quantification approach or the risk categorization approach, risk results should be derived from analyses of appropriate quality. Section 111.2.2.4 and Appendix A to this SRP chapter present guidelines to help reviewers evaluate PRA quality. Finally, Appendix C to this SRP chapter discusses review issues related to the determination of risk contribution and component categorization.
111.2.2.1 Characterization of Change in Terms of PRA Model Elements Where quantitative PRA results are used as part of a risk-informed evaluation of a proposed change, the licensee should define the change in terms that are compatible with the risk analysis, i.e., the risk analysis should be able to effectively evaluate the effects of the change.
The approach to risk characterization should establish a cause-effect relationship to identify portions of the PRA affected by the issue being evaluated. This includes (i) identifying the specific PRA contributors for the particular application, (ii) assessing the portions of the model that should be modified for the application, and (iii) identifying supplemental analyses that could be used to support the application. This approach will help reviewers determine the scope and level of detail of analysis required for the remaining steps in the change process.
Table 111-1 of this SRP chapter summarizes the general guidance for use in identifying elements of the PRA model that may be affected by an application. This guidance, presented as a list of questions, will assist reviewers in establishing a cause-effect relationship between the application and the PRA model.
i The answers to these questions should be used to identify the extent to which the proposed change affects the design, operation, and maintenance of plant SSCs.
Reviewers should also verify that the effects of the proposed changes on plant elements (SSCs, operator actions, etc.) are adequately characterized in the elements of the PRA model, or by appropriate changes Rev. 0 - July 1998 SRP 19-12
to the logic model structure. For full-scale applications of the PRA, for example, this should be
.n i
i reflected in a quantification of the impact on the PRA results. For applications like component V
categorization, however, sensitivity studies on the effects of the change may be sufficient. Similarly, for other applications, it may be adequate to define the qualitative relationship of the ic. pact on the PRA elements, or it may simply be necessary to identify of which elements are impacted.
The review procedure for this element is therefore intended to verify that the submittal appropriately accounts for the effects of the changes on SSC reliability and unavailability, or on operator actions.
Where applicable, reviewers should also evaluate the modeling and quantification of the effects of the change ensure that the models are appropriate and that the results can be supported by plant and/or industry data.
111.2.2.2 Scope of Analysis The necessary scope of a PRA supporting risk-informed requests will depend on the specific application. Although the assessment of risk implications (in light of the acceptance guidelines defined in RG 1.174) requires that all plant operating modes and initiating events be addressed, it is not necessary in risk-informed regulation that licensees submit PRAs that treat all plant operating modes and all initiating events. Instead, when full-scope FRAs are not available, reviewers should ensure that the submitted findings are supportable on the basis of traditional engineering analyses or other plant operational information addressing modes and initiators not analyzed in the base PRA.
For plant modes and initiators not analyzed in the PRA (such as shutdown, seismic events, fire, floods and severe weather), the licensee should consider the effects of the change and provide the rationale for p
why additional PRA analyses are not necessary. This rationale could be addressed by assessing the
(
level of redundancy and diversity provided by the plant systems, system trains, human actions, etc. for responding to these unanalyzed initiating events. The licensee should also show that the proposed change does not introduce unanalyzed vulnerabilities and that redundancy and diversity will still exist in i
the plant response capability after the changes are implemented. This issue is addressed acceptably if the proposal fulfills any one of the following criteria:
The licensee addresses all modes and all initiator types using PRA.
=
The licensee demonstrates that the application does not unacceptably degrade plant capability and does not introduce risk vulnerabilities or remove elements of the plant response capability from programmatic activities aimed at ensuring satisfactory safety performance for plant modes and initiator types not included in the PRA.
If the proposed change impacts unanalyzed plant modes or initiator types, the licensee
{
demonstrates that a bounding analysis of the change in plant risk from the application (e.g., by j
qualitative arguments, or by use of sensitivity studies) meets guidelines that are equivalent to the acceptance guidelines specified in Section 2.2.4 of RG 1.174.
111.2. 2. 3 Level of Detail The level of detail in a PRA required to support an application should be such that the proposed
(
changes to the plant can be adequately characterized in the PRA model elements, as discussed in Section 111.2.2.1 of this SRP chapter. In addition, the PRA should be detailed enough to account for bsi important system and operator dependencies (functional, operational, and procedural) especially for d
SRP 19-13 Rev. 0 - July 1998
l those components affected by the application. A review of the licensee's failure modes and effects analysis and a review of plant operating and emergency procedures will be useful for this purpose.
The usefulness of PRA results in risk-informed regulation is dependent on the level of resolution of the modeled SSCs. A component-level resolution provides insights at the component level. However, if a PRA is performed at a system or train level, the insights of the PRA will be limited to that level unless it can be demonstrated that component-level insights can be bounded by system-or train-level effects.
The direct application of PRA results will therefore be limited to those SSCs that are explicitly modeled as part of the PRA basic events. Insights for SSCs that are implicitly modeled (i.e., screened out, assumed not important, etc.) shall only be used after additional consideration of the effects of the proposed change on PRA assumptions, screening analyses, and boundary conditions.
Specifically, the following relationships exist between the level of detail in the modeling of each SSC and the conclusions that can be drawn from the PRA:
If the SSCs are modeled at the basic event level, i.e., each SSC is represented by a basic event (or sometimes, more than one if different failure modes are modeled), risk insights from the PRA can be applied directly to the modeled component as long as the effects of the change are appropriately considered.
If the SSCs are included within the boundaries of other components (e.g., the governor and throttle valves being included in the pump boundary), or if they are included in " black boxes" or modules within the PRA model, or if they are modeled as part of the calculation of human error probabilities (HEPs) in recovery actions, risk insights from the PRA can be applied if the effects of the application can be mapped onto the events (e.g., modules, HEPs, etc.) in question. In these cases it should be noted that the mapping is relatively simple if the event is under the same "OR gate" with the other module or HEP events. However, if the logic involves "AND gates," the mapping is more complicated.
If the SSCs are omitted from the model because of inherent reliability, or if they are not modeled at all, risk insights for these components should be obtained through an integrated decisionmaking process (such as an Expert Panel) that revisits the assumptions or screening criteria used to support the initial omission.
111.2. 2.4 Quality of a PRA for Use in Risk-Informed Regulation The baseline risk profile is used to model the plant's licensing basis and operating practices that are important to safe operation. As such, the profile may provide insights into areas in which existing requirements can be relaxed without unacceptable safety consequences. It is therefore essential that the PRA adequately represent the risk profile. To complement this requirement, it is necessary to identify those elements of the plant that are responsible for reducing the risk to acceptable levels, and to adequately addres; we elements in the licensee's programmatic activities. Therefore, the following j
criteria should be sattsfied in risk-informed regulation.
A reasonable assurance exists with regard to the adequacy of the PRA. That is, the PRA model properly reflects the actual design, constru. tion, operating practices, and operating experience of the plant and its owner. This should include plant changes due to the licensee's voluntary actions, regulatory requirements, or previous changes made to the LB.
Rev 0 - July 1998 SRP 19-14 I
i 1W The results and conclusions are " robust" and, where appropriate, the licensee has conducted an analysis of uncertainties and sensitivities to show this robustness.
l Key performance elements are appropriately classified, and performance is backed up by
=
I licensee actions. PRA results are dependent on plant activities. They reflect not only inherent t
device characteristics, but also numerous programmatic activities,'such as IST, ISI, quality assurance, maintenance, etc. Use of a PRA to justify relaxation of a requirement should therefore imply a commitment to the important programmatic activities that are needed to maintain performance at the PRA-credited levels that served as the basis for the proposed relaxation.
I Review of PRA Quality J
l The submittal must demonstrate the quality of the licensee's technical analysis. Sections 2.2.3 and 2.5 l
of RG 1,174 provide specific guidance related to this area and serve as the basis for the staff's review to determine whether the PRA is of sufficient quality to support the decisionmaking process. The required PRA quality should be commensurate with the application for which it is applied and the role the PRA results play in the integrated decisionmaking process. The more emphasis that is placed on the risk insights and PRA results in the decisionmaking process, the more requirements have to be placed on the PRA in terms of how well the licensee assesses the risk and/or the change in risk.
Emphasis on the PRA review may be reduced if a proposed change to the LB decreases the risk or is risk neutral, or if proposed risk increases are calculated to be very small, or if the decision could be based largely on traditional engineering arguments, or if the licensee proposes compensating measures and/or qualitative factors (such as unquantified benefits) such that it can be convincingly argued that the change improves safety or the risk increase is very small.
L In assessing PRA quality, reviewers should evaluate the licensee's process to ensure quality. In addition, reviewers should reach specific findings regarding the quality of the PRA for each application. At a minimum, reviewers should reach these findings on the basis of a " focused-scope" evaluation that concentrates on application specific attributes of the PRA and on the assumptions and elements of the PRA model that drive the results and conclusions. Appendix A to this SRP chapter provides more detailed guidance regarding several issues that are important to the application-specific reviews of probabilistic evaluations performed as part of risk-informed regulation.
'Ihe robustness of the results can be determined by developing an understanding of the contributors and the sources of uncertainty that impact the results. For the proposed risk change, reviewers should identify the elements that increase risk and those that decrease risk, and then identify the contributors to both the risk increase and decrease. A review of the basic events, assumptions, and uncertainties involved in the increase and decrease in risk will help reviewers understand the elements that are important in determining the risk change, and thus ensure that the conclusions are robust with respect to the results obtained.
In addition to the focused-scope review, reviewers should consider the following factors in determining
' the need for a more detailed and larger scope staff review of the PRA:
The PRA results play a relatively significant role in the decisionmaking process, coupled with a
the finding that the proposed change in risk and/or the baseline risk is close to the decision guidelines as defined in Section 2.2.4 of RG 1.174.
s l
l SRP 19-15 Rev. 0 - July 1098
Staff audits of the licensee's process for conducting a PRA have identified practices that could detrimentally affect the quality of the technical analysis.
Results of the licensee's analysis submitted in support of a licensing action are in some way counter-intuitive or inconsistent with results for similar plants on similar issues.
The licensee's analysis is part of a pilot application of PRA in a regulatmy activity.
The PRA includes new methods that are unfamiliar to the staff.
When a staff review of the base PRA is necessary, reviewers should begin by evaluating the results and conclusions from available independent peer reviews of the PRA, including those from industry certification or cross-comparison processes. The staff review should also take into account the process used in the peer review (including the review guidelines or standards to which the PRA is compared, the review scope and elements, the qualification and makeup of the review team, etc.). Results from previous staff reviews of the PRA (e.g., from previous applications) could also provide a good starting point. In cases where the PRA is based on the individual plant examination (IPE) or the IPE of externally initiated events (IPEEE) models, reviewers should also be familiar with the request for additional information (RAI) issued by the staff in connection with those examinations, as well as the licensee's responses to those RAls, and the staff evaluation reports regarding the licensee's IPE and IPEEE submittals.
Reviewers could reach a finding that previous industry or staff reviews are sufficient to show that the PRA is of adequate quality in one or more of the review areas for the present application. In such cases, the scope of the eview should be adjusted accordingly. However, reviewers should be aware of potential application-specific differences, and of the currency of the previous review findings with respect to the current plant design and operating procedures.
Quality Assurance Requirements Related to the PRA To the extent that a licensee elects to use PRA as an element to enhance or modify its implementation of activities affecting the safety-related functions of SSCs, appropriate quality requirements will also apply to the PRA. In this context, therefore, a licensee would be expected to control PRA activity in a manner commensurate with its impact on the facility's design and licensing basis. Section 2.5 of RG 1.174 describes the quality elements that apply to the licensee's PRA activities. Reviewers should verify that the quality of analyses and performance programs which affect safety-related equipment and activities will meet the quality guidelines described in RG 1.174.
111.2.2.5 Evaluation of Risk Impact In evaluating the risk impact from an application, reviewers should consider the proposed change in risk with regard to the acceptance guidelines, the cumulative and synergistic effects of the application on the overall plant risk profile, and the licensee's risk management philosophy. Each of these items is discussed in the following subsections.
Acceptance Guidelines for Risk Impact from the Application For many risk-informed applications, the licensee is expected to perform a quantitative estimate of the total impact of a proposed action to demonstrate that Principle 4 (see Section II) has been satisfied.
Rev. 0 - July 1998 SRP 19-16 1
Sectior,2.14 of RG 1.174 discusses the acceptance guidelines for changes to the plant's risk. To i
summarin, regions are established in the two planes generated by a measure of the baseline risk s'
metrics (CDF and LERF) along the x-axis, and the change in those metrics (ACDF and ALERF) along the y-axis (Figures 111-1 and III-2), and acceptance guidelines are established for each region as discussed below. These guidelines are intended for comparison with a full-scope assessment (including internal events, external events, and events that take place under full power, low power and shutdown conditions). However, reviewers should recognize that many PRAs are not full-scope assessments and the use of less than full-scope PRA information may be acceptable as discussed later.
There are two acceptance guidelines, one for CDF and one for LERF, and both should be used. The guidelines for CDF are as follows:
If the application can clearly be shown to decrease CDF, the change is considered to satisfy the relevant principle of risk-informed regulation with respect to CDF. (Because Figure 111-1 is drawn on a log scale, it does not explicitly indicate this region.)
When the calculated increase in CDF is very small (less than 1 x 10~6 per reactor year), the change should be considered regardless of whether there is an assessment of total CDF (Region 111). While there is no requirement for the licensee to quantitatively assess the total CDF, information should be provided to show that there is no indication that the total CDF could considerably exceed 1 x 10' per reactor year. Such an indication could result, for example if the contribution to CDF calculated from a limited-scope analysis (such as that from the IPE or the IPEEE) significantly exceeds 1 x 104 per reactor year, if the licensee has identified a potential vulnerability from a margins-type analysis, or if plant operating m
experience has indicated a potential safety concern.
]
When the calculated mcrease m CDF is in the range of I x 10-6 to 1 x 10-3 per reactor year, applications should be considered only if the licensee can reasonably show that the total CDF is less than 1 x 10d per reactor year (Region II).
Applications which increase CDF by more than 1 x 105 per reactor year (Region I) should not
=
normally be considered.
The CDF-related guidelines listed above are to be applied together with the guidelines for LERF. That is, both sets of guidelines should be satisfied. Specifically, the guidelines for LERF are as follows:
if the application can clearly be shown to decrease LERF, the change is considered to satisfy e
the relevant principle of risk-informed regulation with respect to LERF. (Because Figure III-2 is drawn on a log scale, it does not explicitly indicate this region.)
When the calculated increase in LERF is very small (less than 1 x 104 per reactor year), the e
change should be considered regardless of whether there is an assessment of total LERF (Region III). While there is no requirement for the licensee to quantitatively assess the total LERF, information should be provided to show that there is no indication that the total LERF could considerably exceed I x 105 per reactor year. Such an indication could result, for example, if the contribution to LERF calculated from a limited scope analysis (such as that from the IPE or the IPEEE) significantly exceeds 1 x 10'5 per reactor year, if the licensee has identified a potential vulnerability from a margins-type analysis, or if plant operating j
experience has indicated a potential safety concern.
SRP 19-17 Rev. 0 - July 1998
When the calculated increase in LERF is in the range of 1 x 10" to 1 x 104 per reactor year, applications should be considered only if the licensee can reasonably show that the total LER is less than 1 x 104 per reactor year (Region II).
Applications which increase LERF by more than 1 x 104 per reactor year (Region I) should not normally be considered.
These guidelines are intended to provide assurance that proposed increases in CDF and LERF are sm and are consistent with the intent of the Commission's Safety Goal Policy Statement.
The guidelines discussed above are applicable for full-power, low-power, and shutdown operations.
However, during certain shutdown operations when the containment function is not maintained, the LERF guidelines as defined above are not practical. In such cases, the licensee may use more stringent baseline CDF guidelines (e.g.,104per reactor year) to maintain an equivalent risk profile or may propose an alternative guideline to LERF that meets the intent of Principle 4.
As indicated by the shading in Figures III-I and III-2, the change request should be subjected to technical and management reviews which become more intensive as the calculated results approach the region boundaries. The technical review related to the risk evaluation should address the scope, quality, and robustness of the analysis, including consideration of uncertainties. The scope, level of detail, and quality of analysis is further discussed in Sections III.2.2.2,111.2.2.3, and 111.2.2.4 of this SRP chapter. The robustness of the results can be determined by developing an understanding of the contributors, the sources of uncertainty that impact the results, and their impact on whether the acceptance guidelines are met.
The necessary sophistication of this evaluation depends on both the role the risk assessment plays in the decision and the magnitude of the potential risk impact. For those actions justified primarily by traditional engineering considerations and for which minimal risk impact is anticipated, a bounding estimate may be sufficient. For actions justified primarily by PRA considerations for which a substantial impact is possible or is to be. offset with compensatory measures, an in-depth and comprehensive PRA analysis is generally needed.
Comparison of Results with Acceptance Guidelines in the context of integrated decisioomaking, the acceptance guidelines should not be interpreted as being overly prescriptive. They are intended to provide an indication, in numerical terms, of what is considered acceptable. As such, the numerical values associated with def' ming the regions in Figures 111-1 and 111-2 are approximate values used to indicate the changes that are generally acceptable.
Furthermore, the state of knowledge (or epistemic) uncertainties associated with PRA calculations preclude a definitive decision (based purely on the numerical results) with respect to which region a given application belongs. The intent in making the comparison of the PRA results with the acceptance guidelines is to demonstrate with reasonable assurance that the proposal fulfills Principle 4 (discussed in Section II). The assessment of whether this has been demonstrated must be made on the basis of an understanding of the contributors to the PRA results, and on the impacts of the uncertainties (both those that are explicitly accounted for in the results and those that are not). This is a somewhat subjective process; therefore, in order to complete the assessment, reviewers must carefully document the reasoning behind the decisions.
O Rev. 0 - July 1998 SRP 19-18
i l
l h
As discussed in RG 1.174, PRA values can be affected by particular modeling assumptions that are a V) response to the uncertainty regarding how to correctly model the plant response following an initiating event. Thus, it is important that uncertainties in the PRA results be taken into account in assessing the risk impact and in the risk-informed decisionmaking process to demonstrate the robustness of the l
results. The scope of the required uncertainty analysis is a function of the role that the quantification results play in the decision, and on the significance of the calculated change.
l The general approach to accounting for uncertainty is discussed in Section 2.2.5 of RG 1.174. In tlat discussion, uncertainties are categorized as parameter, model, and completeness uncertainties. In assessing analysis of uncertainties, reviewers should consider the types and souices of uncertainties identified by the licensee, and how those uncertainties have been addressed with reference to the decision guidelines. Specifically, review guidance is as follows.
Parameter uncertainty: Reviewers should determine whether the licensee has accounted for parameter uncertainties in an appropriate manner so that the estimated values for ACDF, ALERF, CDF, and LERF can be regarded as equivalent to mean values. However, this does not imply that a detailed propagation of uncertainties is always necessary; in many cases, it is possible to show that a point estimate is an acceptable approximation of the mean value using qualitative arguments about the risk contributors. For example, if a formal propagation has not been performed, it is necessary for the licensee to demonstrate that the result is not affected by the so-called state of knowledge correlation (specifically, that there are no significant contributing cutsets or scenarios that involve multiple events for which the probabilities are determined using the same parameter, particularly if the parameter value is very uncertain).
m g
it is not uncommon for licensees to use point estimate values without defining probability v/
distributions on the values. In such instances, it is not possible to characterize the point estimate as a mean value. liowever, for the more significant parameters, some characterization of uncertainty is essential to demonstrate that the point estimate is not an optimistic value.
Model uncertainty: Reviewers should determine if the results are strongly impacted by the specific models or assumptions adopted for the assessment of important elements of the PRA, and whether the sensitivity analyses that have been performed (if any) are sufficient to address the most significant uncertainties with respect to these elements.
In some cases, particularly for small changes in risk or for relatively minor changes, there may be relatively few issues related to model uncertainties. In other cases, where the baseline risk values are to be estimated, the modeling issues should include all those that have a significant impact on the evaluation of the baseline risk values. Model uncertainties arise when there are several alternative approaches to the analysis of certain elements of the PRA model. They are typically addressed by adopting a specific model or making a specific assumption. In such cases, the licensee should document why the particular model or assumption used is appropriate both for the base case risk evaluation and for the analysis of the impact of the change. In certain cases, it may be necessary to perform sensitivity analyses using alternative reasonable models or assumptions to demonstrate the robustness of the conclusions. In deciding what are reasonable alternatives, reviewers should consider whether the alternatives have some precedent and whether they have a reasonable engineering basis.
O Reviewers should pay particular attention when the characterization of a model uncertainty is
/
such that the results fall into a bimodal or multi-modal distribution, and one or more of the a
SRP 19-19 Rev. 0 - July 1998 i
i
l modes exceed the acceptance guidelines. The results should then be reviewed on the basis of an evaluation of the significance of the hypotheses associated with those modes that exceed the guidelines.
Comnleteness uncertaintv: Reviewers should determine whether the licensee has adequately l
addressed the limitations in the PRA scope, and other completeness issues either by limiting the scope of the application, or by demonstrating that the impact of the unanalyzed portion of the risk on both the base case risk and on the change in risk is bounded or can be neglected.
Section 111.2.2.2 of this SRP chapter discusses this further.
1 Cumulative and Synergistic Effects from all Applications l
In evaluating the licensee's submittal, reviewers should consider the effects of the proposed changes in light of previously submitted changes implemented by the licensee. Optimally, the PRA used for the l
current application should already model the effects of past applications. However, qualitative and synergistic effects are sometimes difficult to model in the PRA. Therefore, a review of changes in risk l
(both quantifiable and non-quantifiable) from previously submitted changes to the plant's design and operation would provide a means to account for the cumulative and synergistic effects of these changes.
For all previous changes, reviewers should consider the following factors:
l the calculated change in risk for each application (CDF and LERF) and the plant elements
=
(SSCs, procedures, etc.) affected by each change qualitative arguments used to justify the change (if any) and the plant elements affected by
=
those arguments compensatory measures or other commitments used to help justify the change (if any) and the plant elements affected a summary of the results from the moni oring programs (where applicable) and a discussion on how these results have been factored into the PRA or into the current application the plant risk profile to ensure that the accumulation of changes has not created dominant risk contributors if the licensee's submittal includes past changes made to the plant (but not submitted to the NRC) that reduced the plant risk, especially changes related to the current application, reviewers should consider such changes in the integrated decisionmaking process. Benefits from the implementation of the Maintenance Rule can also be credited for the applicable SSCs.
To facilitate future applications, reviewers should summarize the results from the current application using the first three bullets listed above. This summary should be kept in a database maintained by SPSB and available to all staff reviewers, project managers, and regional inspectors. To ensure uniform recordkeeping, the format provided in Table III-2 should be used for this purpose.
Beyond cumulative effects, synergistic effects are also possible, and some of these might not emerge from a quantification of the PRA. For example, if conventional importance ranking approaches are Rev. 0 - July 1998 SRP 19-20
/O employed to determine the importance of SSCs, it would be possible that multiple requirements could Q
be relaxed on certain " low" significant components under multiple applications. If the QA (potentially affecting the failure rate) and the test interval (potentially affecting fault exposure time) were to be relaxed for the same component, the component unavailability could increase more than expected (since failure rate and fault exposure time combine multiplicatively in the calculation of unavailability). If the effects of QA on failure rate could be quantified convincingly, this would be addressed explicitly, but this cannot precently be ensured. As a result, potential exists that different applications might lead to unintended and unquantified synergistic effects on the unavailability of a given component.
Synergistic effects on a given element can be addressed by showing that the basic event model adequately reflects the effects of programmatic activities, and that the calculated unavailability, propagated through the PRA, is consistent with the needed performance with regard to the risk indices and the defense-in-depth concept. However, it is more straightforward simply not io allow for the relaxation of multiple programmatic requirements on a given component, unless demonstrable justification is provided that the risk contribution from the component is negligible for conditions covered by the set of requirements. For example, if IST is relaxed on a given component, it would be preferable not to relax QA as well, unless good arguments are given for allowing both.
Risk Management One of the goals of the review should be to ensure that in the course of the licensee's engineering evaluations, principles of risk management are appropriately applied in the process of evaluating j
changes to current regulatory requirements. For the purposes of this SRP chapter, " risk management" refers to an approach to decisionmaking about safety that seeks to allocate available resources and V) worker dose in such a way as to minimize the risk to public health and safety from plant operations.
The staff should recognize that there is a point of diminishing returns in risk reduction and that some residual risk will be associated with plant operation. Nonetheless, reviewers should expect that licensees will make an effort to identify reasonable and cost-effective measures to control this residual risk as part of the risk-informed regulatory process.
Therefore, as a staff expectation, the process of risk management in risk-informed decisionmaking should not be biased toward eliminating requirements to the exclusion of enhancements that would convey a worthwhile safety benefit. Licensees are expected to apply risk insights in an unbiased way, and licensees who do not satisfy subsidiary safety objectives (as defined in RG 1.174) are expected to seek safety enhancements in conjunction with risk-informed applications.
Therefore, when risk increases are proposed, reviewers should consider plant performance and past changes to the licensing basis to ensure that there is no pattern for a systematic increase in risk.
Insights on the licensee's operational practices, management controls, risk management programs, plant configuration control programs, or performance monitoring programs from previous applications can be obtained from the NRC project managers, the NRC regional offices, or documentation of NRC inspection activities.
111.2.3 Integrated Decisionmaking Process The acceptability of the proposed changes should be reviewed and determined in an integrated fashion.
Staff reviewers should verify that the licensee has used the results of the traditional engineering A
analyses and the risk assessment to ensure that the submittal fulfills the principles listed in Section II of
)
this SRP chapter. Since the roles played by the traditional analyses and the risk analyses in the SRP 19-21 Rev. 0 - July 1998 l
decisionmaking process determine the scope, quality, and robustness required of those analyses, examination of the appropriate inputs and assumptions in the analyses may be necessary for reviewers to conclude with reasonable assurance that the proposal fulfills the stated principles.
When appropriate, the integrated decisionmaking process should include implementation and monitoring strategies that are used to provide confidence in the results of the underlying engineering analyses. In addition, licensees can take compensatory measures which reduce risk to offset incompleteness or uncertainties in the analysis. Compensatory measures can also be used to offset a quantifiable increase in risk with non-quantifiable but expected improvements in safety.
To ensure that the important assumptions used in the engineering analysis to justify the LB change remain valid, the integrated decisionmaking process should ensure that the licensee maintains an appropriate set of programmatic activities (e.g., IST, QA,1S1, maintenance, monitoring) for important elements of the plant response capability. In addition, performance of compensating SSCs should be ensured (through programmatic activities) when these SSCs are used to help justify the relaxation of requirements for other SSCs.
The process used by licensees to integrate traditional and probabilistic engineering evaluations for risk-informed decisionmaking is expected to be well-defined, systematic, and scmtable. Appendix B to this SRP chapter presents review guidance and staff expectations for the licensee's integrated decisionmaking process.
In evaluating the acceptability of a proposed change, reviewers should also address the following factors:
the cumulative impact of previous changes and the trend in CDF and LERF (the licensee's risk management approach) the impact of the proposed change on operational complexity, burden on the operating staff, and overall safety practices plant-specific performance and other factors, including for example, siting factors, inspection findings, performance indicators, operational events, and level 3 PRA information if available the benefit of the change in relation to its CDF/LERF increase, and whether it is practical to accomplish the change.vith a smaller CDF/LERF impact practical actions that could reduce CDF/LERF when there is reason to believe that the baseline a
CDF/LERF are above the guideline values (i.e.,104 and 104 per reactor year)
Review o Combined Change Requests r
in assessing combined change requests, reviewers should evaluate the acceptability of each of the individual changes with respect to the defense-in<!epth and safety margin guidelines discussed in Section 111.2.1 of this SRP chapter. In addition, reviewers should evaluate the overall risk impact of the combined changes using the guidelines discussed in Section 111.2.2 of this SRP chapter.
In evaluating the overall (i.e., combined) risk impact, reviewers should take into account the relationship between the individual changes. For example, in combined change requests for which Rev. 0 - July 1998 SRP 19-22
i l
l g'
individual changes that increase risk are compensated for by other changes that decrease risk, reviewers should evaluate and understand the major contributors to both the risk increase and risk s
b decrease, including the analysis assumptions and uncertainties from each contributor that might affect the decision process. Combining risk impacts from the individual contributors is prudent when the contributors are closely related in terms of analysis assumptions and uncertainty. Contributors could also be related if they impact on the same plant functions, for example. Conversely, for contributors that are not closely related, risk impacts from each change should be evaluated on an individual basis.
Finally, combined changes should not trade many small risk decreases for a large risk increase (i.e.,
create a new significant contributor to risk). It is expected that implementation of combined change requests will improve, or at least maintain, the overall plant risk profile. A desirable risk profile is one in which no contributors are overly dominant. Therefore, proposed changes should not create or exacerbate a risk imbalance either in terms of dominant plant elements (SSCs or operator actions) or accident sequences.
III.3 Element 3: Develo_o Imnlementation and Monitorine Stratecies implementation and monitoring strategies are important in most risk-informed processes since they can provide an early indication of unanticipated degradation of SSCs or other plant performance factors under the proposed changes. In addition, these strategies may be needed to ensure that the plant will effectively maintain the performance of SSCs that are relied upon to justify the proposed change to the LB. Section 2.3 of RG 1.174 provides guidance for the suggested process related to this issue.
O The primary goal of the monitoring program should be to ensure that no adverse degrr.dation occurs
)
because of the changes to the LB. These programs should therefore address the possibility that the aggregate impact of changes which affect a large class of SSCs could lead to an unacceptable increase in the number of failures attributable to unanticipated degradation, including possible increases in common cause failure mechanisms.
Reviewers should evaluate the implementation and monitoring strategies on the basis of findings obtained from the traditional engineering and probabilistic evaluations. When broad implementation is proposed over a short period of time, reviewers should verify that this is consistent with the traditional engineering evaluations, defense-in-depth considerations (including common cause failure), and risk evaluation models and assumptions. When there is a need to gain additional performance insights given a change in requirements, reviewers should verify that the licensee has proposed a phased approach to implementation, if this phased approach involves plan implementation for different SSC groups at different times, reviewers should also assess the basis for the licensee's grouping criteria, keeping in mind the potential common cause failures.
Monitoring should be applied to SSCs in a manner commensurate with their importance to safety as determined by the engineering evaluation that supports the LB change. This monitoring should be contingent on the reliability / availability allocated to SSCs in the risk model (or on performance of operators, where appropriate) used to support the proposed change in regulation. As such, reviewers should ensure that the chosen performance criteria are consistent with the level of performance allocated in the risk analysis, n
When monitoring that is already being performed as part of the Maintenance Rule implementation or as part of other plant programs is also proposed for the current application, reviewers should ensure that SRP 19-23 Rev. 0 - July 1998 1
the monitoring proposed is sufficient for the SSCs affected by the risk-informed application, and the performance criteria chosen are appropriate for the application in question.
As part of the evaluation of the licensee's monitoring program, reviewers should assess the proposed provisions for cause determination, trending of degradation and failures, and corrective actions. The program should be structured such that feedback of infonnation and corrective actions is accomplished in a timely manner, and degradation in SSC performance is detected and corrected before plant safety can be compromised. In cases where monitoring detects degradation, there should be provisions for a trending and corrective action program, or for the SSCs to be refurbished, replaced, or tested / inspected more often (or a combination of these initiatives). The preferred initiative should be selected on the basis of determination regarding the cause of the degradation (whether it is generic, age-related, etc.).
Reviewers should evaluate if the information gathered during monitoring activities is extensive enough to provide a timely indication of component degradation. Since many components are inherently quite reliable, the limited tests on a limited number of similar components may not provide adequate data, especially for newer plants where aging effects may not be detected until the proposed program is fully in place (and the advantages of a phased implementation are lost). One approach to ameliorate this concern would be to include performance data for similar SSCs at other plants with a range of operating times to expand the applicable database over a range of component ages. Such a program would be expected to improve the better chance of early detection of SSC reliability degradation.
Reviewers should evaluate the impact on plant risk and SSC functionality, reliability, and availability given the licensee's proposed implementation and monitoring plan. The benefits from the implementation and monitoring programs should be balanced against any negative impact on risk.
Finally, reviewers should consider the criteria to be applied in deciding what actions are to be taken in cases where performance falls below that predicted by the supporting evaluations. Corrective action procedures should be in place before implementation of the proposed program.
111.4 Element 4: Conduct Staff Evaluation of Submittal in order for the staff to reach a conclusion regarding the acceptability of the proposed LB change on the basis of the review guidance presented in earlier sections, the licensee must submit or make available sufficient engineering and licensing information. In addition, the licensee should request appropriate regulatory action. Requests for proposed changes to the plant's LB typically take the form of requests for license amendments (including changes to or removal of license conditions), technical specification changes, changes to or withdrawal of orders, and changes to programs pursuant to 10 CFR 50.54 (e.g., QA program changes under 10 CFR 50.54(a)). Reviewers should determine if(i) the form of the change request is appropriate for the proposed LB change, (ii) the licensee submitted the information required by the relevant regulation (s) in support of the request, and (iii) the request is in accordance with relevant procedural requirements. For example, license amendments should meet the requirements of 10 CFR 50.90, 50.91, and 50.92, as well as the procedural requirements in 10 CFR 50.4. Where the licensee submits risk information in support of the LB change request, that information should meet the guidance in Section 3 of RG 1.174.
p Licensees have a choice of whether to submit results or insights from risk analyses in support of their LB change request. Where the licensee's proposed change is consistent with the currently approved staff positions, reviewers should reach their determination solely on the basis of traditional engineering analysis without recourse to risk information. (Reviewers may, however, consider any risk information Rev. 0 - July 1998 SRP 19-24 L
i submitted by the licensee.) Where the licensee's proposed change goes beyond currently approved i
staff positions, reviewers should consider both information derived through traditional engineering analysis as well as information derived from risk insights. If the licensee does not submit risk information in support of a LB change which goes beyond currently approved staff positions, reviewers may request that the licensee provide this information. If the licensee chooses not to provide the risk information, reviewers will evaluate the proposed application using traditional engineering analysis and i
determine whether the licensee has provided sufficient information to support the requested change.
In risk-informed change proposals, licensees are expected to identify SSCs with high risk significance l
which are not currently subject to regulatory requirements, or are subject to a le,i er regulation which is not commensurate with their risk significance, or voluntary actions that are key tr ecisionmaking.
In addition, licensees are expected to propose LB changes that will subject such SSCs or voluntary actions to the appropriate level of attention, consistent with their significance. Application-specific regulatory guides set forth specific information on the staff's expectations on this issue. Reviewers should ensure that this application-specific guidance is followed. If there is no guidance, reviewers l
should determine whether any conunitments for enhanced requirements / controls are appropriate for such SSCs or voluntary actions, and ensure that those commitments are reflected in the licensing basis.
Update of the Safety Analysis Report Reviewers should ensure that the proposed changes, when approved by the staff, will be appropriately included in future updates to the licensee's safety analysis report, in addition, the licensee should identify important assumptions (including SSC functional capabilities and performance attributes) which play a key role in supporting the acceptability of the LB change. Since the continued satisfaction of G
these assumptions is necessary to maintain the validity of the safety evaluation, reviewers should verify that such assumptions are reflected by licensee commitments which are incorporated into the safety analysis report. Reviewers should also verify that the licensee has submitted revised FSAR pages, as necessary. This revision should include all the programmatic activities, performance monitoring aspects, and SSC functional performance and availability attributes which form the basis of the request.
This material should also identify those SSCs for which performance should be verified (including nonsafety-related SSCs for which performance and re!iability provide part of the basis for the LB change).
Considerations Related to the National Enviromnental Policy Act In accordance with 10 CFR Part 51, the staff's review process should address environmental protection regulations, such as those from the National Environmental Policy Act (NEPA). Reviewers should use NRR Office Letter 906, Revision 1, and 10 CFR 51.25 to determine how the NEPA requirements are to be addressed. If it is determined necessary, an environmental assessment (EA) should be prepared to assess whether an environmental impact statement (EIS) is required, or whether the staff can reach a finding of no significant impact (FONSI). It is expected that, if all of the guidance and acceptance criteria provided in RG 1.174 are satisfied, the staff should normally be able to reach such a finding for the proposed change.
)
9
\\
SRP 19-25 Rev. 0 - July 1998
Table III-1 (page 1 of 3) 2 Questions to Assist in Establishing the Cause-Effect Relationship LEVEL 1 (INTERNAL EVENTS PRA)
Initiating Events Does the apphcation introduce new iruttatmg events?
Does the application addaess changes that lead to a modification of the initiating event groups?
Does the application necessitate reassessment of the frequencies of the initiating event groups?
Does the application increase the likehhood of a system failure that was bounded by an initiating event group to the extent that it needs to be explicitly considered?
Success Criteria Does the application necessitate modification of the success enteria?
Does the modification of success criteria necessitate changes in other criteria, such as system interdependencies?
Event Trees Does the apphcation address an issue that can be associated with a particular branch, or branches on the event trees, and tf so, is the branching structure adequate?
Does the application necessitate the introduction of new branches or top events to represent concerns not addressed in the event trees?
Does the application necessitate consideration of reordering branch points (i.e., does the application affect the sequence-dependent failure analysis)?
System Reliability Models Does the apphcation impact system design in such a way as to alter system rehability models?
Does the application impact the support functions of the system ir, such a way as to aher the dependencies in the tnodel?
Does the application impact the system performance and, if so, is that impact obscured by conservative modeling techniques?
Parameter Database Can the application be clearly associated with one or more of the basic event definitions, or does it necessitate new basic events?
Does the application necessitate a specialized probability model (e g., time <iependent model, etc.)?
Does the application necessitate modifications to specific parameter values?
Does the application introduce new component failure modes?
Does the application affect the component mission times?
Does the application necessitate that the plant-specific (historical) data be taken into account, and can this be easily achieved by an update of the previous parameters?
Does the application involve a change which may impact parameter values, and do the present estimates reflect the current status of the plant with respect to what is to be changed?
Dependent Failure Analysis Does the application introduce or suggest new common cause failure contributions?
Does the application introduce new asymmetries that might create subgroups within the CCF component groups?
Is the apphcation likely to affect CCF probabilities?
2 Information from Section 3.3 of the EPRI "PSA Applications Guide" provided substantial input to this table.
Rev. 0 - July 1998 SRP 19-26
Table III-1 (page 2 of 3)
Questions to Assist in Establishing the Cause-Effect Relationship Human Reliability Analysis Does the application involve a procedure change?
Does the apphcation involve a new human action?
Does the apphcation charge the available time for human actions?
Does the application affect the human action dependency analysis?
Does the application eliminate or modify an caisting human action?
e Does the application introduce or modify dependencies between plant instrumentation and human acuons?
Is the appbcation concerned with events that have been screened from the model, either in whole or in part
?
Does the apphcation impact a particular performance shaping factor (PSF), or a group of PSFs, and are they explcitly addressed in e
the estimatmo approach (e.g., if the issue is to address training, is training one of the PSFs used in the human reliability analysis)?
Does success in the application hinge on incorporating the impact of changes in PSFs and, if so, do the current estituates reflect the current status of these PSFs?
is it possible that the particular group of human error events that is affected by the chang
- being analyzed has been truncated?
e Does the change address new recovery actions?
Internal Flooding Does the application affect the screening analysis (e.g., does the application result in the location of redundant trams or components into the same flood zone)?
Does the application introduce new flooding sources or increase caisting potential flood inventories?
Does the application affect the status / availability of flood mitigation devices?
e Does the application affect flood propagation pathways?
e e
Does the application affect critical flood heights?
Does the application affect tinung considerations used in the floodmg analysis (e.g., flood flow rates or flood egress rates)?
e Quantification Does the application change any of the basic event probabilities?
Does the apphcation change relative magnitudes of probabihties?
e e
Does the application only make probabilitws smaller?
e la the new result needed in a short-time scale?
Does the application necessitate a change in the truncation limits for the rnodel?
e Does the application affect the
- delete terms' ased during the quanufication process? (i.e., does the application introduce new e
combinanons of maintenance actions or operating modes that are deleted durmg the base case quantification process using the 3
delete function?)
Does the apphcation affect equipment credited for recovery actions (including credit for inter-system or inter-unit crosstics)'
l Analysis of Results l
e Does the application necessitate an assessment rif uncertainty, and is it to be qualitative or quantitative?
l e
Are there unccrtainties in the application that could be clarified by the application of sensitivity studies?
l e
Does the application strategy necessitate an importance analysis to rank contributions?
l Does the application necessitate the performance of an importance, uncertainty, or sensitivity analysis of tim base case PRA?
j Plant Damage State Classification i
Does the application impact the choice of parameters used to defim ; ' ant damage states?
Do the key plant damage states (KPDSs) utilized adequately represent the resuhs of the Level 1 analysis by includmg the plant e
damage states that have a significant frequency of occurrence?
Have those plant damaFe states that have been cluninated in this process been assigned to KPDSs of higher consequence (e.g.,
likehhood of large early release)?
SRP 19-27 Rev. 0 - July 1998 l
Table III-1 (page 3 of 3)
Questions to Assist in Establishing the Cause-Effect Relationship LEVEL 2 (CONTAINMENT ANALYSIS) llave new containment failure modes identified by the apphcanon been addressed in the PRA? Are potential changes accounted e
for?
Are any dependencies among containment tailure modes being changed?
e Does the apphcation involve mechanisms that could lead to containment bypass?
e Does the application involve mechanisms that could cause failure of containment isolation?
e Does the application direcdy affect the occurrence of any severe accident phenomena?
e Does the application necessitate use of risk measures other than large early release?
e Does the application change equipment quahfication to the point where it affects timing of equipment failure relauve to containment e
failure?
Does the application affect core debris path to the sump / suppression pool or to the other portions of the containment?
e Do the selected source term categories adequately represent the revised containment event tree (CET) endpoints? Are CET e
endpoint frequencies changed enough to affect the selecuan of the dominant / representative sequence (s) in the source term binning process?
Does the application affect the timing of release of radonuclides into the environrnent relative to the inication of core melt and e
relauve to the tirne for vessel rupture?
LEVEL 3 (CONSEQUENCE ANALYSIS)
Does the application rrcessitate detailed evacuee doses?
Are individual doses at specific locations needed for this application?
e is evacuation or sheltenng being considered as a mitigation measure?
e Are long-term doses a consideration in this apphcation?
O e
EXTERNAL EVENTS PRA Does the application introduce external hazards not previously evaluated?
e Does the application increase the intensity of existing hazards significandy?
e Are design changes modifying the structural response of the plant being considered?
e Does the change impact the availabihty and performance of necessary mitigation systems for an external hazard?
e Does the apphcation significantly modify the inputs to the plant model conditioned on the external event?
e Are changes being requested for systems designed to mitigate against specific external events?
e Does the application involve availabihty and perfonnance of containment systems under the external hazard?
e LOW POWER and SIIUTDOWN PRA Does the application introduce new irutiating events or change the frequencies of existmg events?
e Does the applicanon affect the scheduling of outage activities?
Does the application affect the ability of the operator to respond to shutdown events?
e Does the apphcation affect the reliability or availabihty of equipment used for shutdt,wn conditions?
e Does the application affect the availability of equipment or instrumentation used for contingency plans?
e O
Rev. 0 - July 1998 SRP 19-28
Table III-2 s
Risk-Informed License Amendment Cumulative Risk Tracking Form (page 1 of 2)
Plant Docket Primary Review Branch Submittal Date TAC Number References (amendment requests, SER, etc., including datesfor each reference)
Type of Amendment Briefly describe amendment request.
O Tech Specs o IST o ISI O Other (specify)
List plant systems, procedures, or other operating characteristics that are afTected by the change.
If a quantitative risk assessment was performed, provide the calculated change in risk (ACDF or ACDIS" and ALERF or ALERIS").
If a quantitative risk assessment was performed, list the base CDF and base LERF (see Note 2).
List qualitative arguments (if any) used to support / Justify the change request.
Ilst compensatory measures or other commitments (if any) used to support / justify the change request and the plant systems, procedures, or other operating characteristics affected.
On the basis of quantitatise arguments, qualitative arguments, and compensatory measures, is application a risk increase, risk neutral, or a risk decrease?
I SRP 19-29 Rev. 0 - July 1998
Table III-2 (continued)
Risk-Informed License Amendment Cumulative Risk Tracking Form (page 2 of 2)
NOTES (1)
Provide core damage probability (CDP) and large early release probabitJy (LERP)for changes that ternporarily affect risk (e.g., technical specification changes). Specify the time periodfor which the condition will exist.
If the application decreases CDF and LERF, a calculation of base CDF and LERFis not (2) a.
required. Ilowever, if the base CDF and/or LERF is included in the submittal, list it here.
b.
If ACDF and AIERF are very small(less than 1 x 10' and i x 10'per reactoryear respectively), there should be an indication that the total CDF does not considerably exceed I x 10'per reactoryear and the total LERF does not considerably exceed 1 x 10' per reactor year.
If ACDFis greater than 10'peryear, or AIERFis greater than 10'peryear, a calculation of c.
total CDF and totalIERF is required.
d-For the listed CDF and LERF, state the operating modes (e.g., low power and shutdown), and initiators (e.g., external events) included.
O O'
Rev. 0 - July 1998 SRP 19-30
t 7 k. r..... I A
s. < i,.... r ~ u i..,,,.i
, 7 44...... si l l 4
,...n.i,......,-
i< 1 :. H Ni i,,,o<,,,,,,,t,.,,i,,,,,,,.
10 3 7 u,....... i n s
p.
s,,,s,...n,i,,,,....
u., i i,,, i,i,,,,,,, i,,,,,.. o 4.'-
i $ i.. i l e...
e 04 i..i <,,,,,,,i..,,, i.......-
104
)
i 10-5 10 CDF "*
d Figu re III-I Acceptance Guidelines
- for Core Damage Frequency (CDF) t u. r..... e a
s..<
i....r,-
u i..,,,,i M
l 64. " i.. i, I l w
.,u <
i,,ni-,,
,,..t.............
,4,,, g, s,
l o. r.... i n s,,, s.,,. ii < i,,,,.,
4 10
""' 8 h " ' '
h"s"
I, "44,
4
' j,!
/,'}
l.. f:.. i li i i mea 7l -
...t.....i..,,i..,.,,,,
R Ms my
+
i M
4 i
10
~p(t :
I-
. ^..,, y $? '.
r 4
-l.
l n E,G %. ; a,
,J %,.eg _gy. ;4, pc:q, i,
,a p.
i l
104 10-5 LERF"*
Figure III-2 Acceptance Guidelines
- The analysis will be subject to increased technical review and management attention as indicated by the darko, Jthe shading of the figure. In the context of the integrated decisionmaking, the boundaries between the regions should not be interpreted as being defini'ive; the numerical values associated with defining the regions in the figure are to be interpreted as indicative values only.
O SRP 19-31 Rev. 0 - July 1998
I IV.
EVALUATION FINDINGS The results of the reviewers' evaluation should reflect a consistent and scrutable integration of the probabilistic considerations and traditional engineering considerations provided by the licensee or applicant and developed independently by the reviewers. To reach a finding of acceptability, reviewers will generally need to show that in light of a small or non-existent increase in risk and a reduced level of conservatism, defense-in-depth and sufficient safety margins are maintained. Findings of acceptability should be supported with logical bases built from an evaluation of the considerations given in Section III of this SRP chapter. Reviewers should also confirm that sufficient information is provided in accordance with the requirements of this SRP chapter, and that the evaluation supports the following conclusions, to be included in the staff's safety evaluation report.
General The proposed change meets the current regulations unless it is explicitly related to a requested exemption or rule change, i.e., a " specific exemption" under 10 CFR 50.12 or a " petition for rulemaking" under 10 CFR 2.802.
The proposed change is consistent with the defense-in-depth philosophy.
The proposed change maintains sufficient safety margins.
When proposed changes result in an increase in CDF or risk, the increases are small and are consistent with the intent of the Commission's Safety Goal Policy Statement.
The impact of the proposed change is monitored using performance-based strategies.
All safety impacts of the proposed change are evaluated in an integrated manner as part of an overall risk management approach in which the licensee is using risk analysis to improve operational and engineering decisions broadly by identifying and taking advantage of opportunities to reduce risk, and not just to eliminate requirements the licensee sees as undesirable. For those cases when risk increases are proposed, the benefits have been described and these benefits are commensurate with the proposed risk increases. The approach used to identify reduced requirements was also used to identify if there are areas where requirements should be increased.
The scope and quality of the engineering analyses (including traditional and probabilistic analyses) conducted to justify the proposed LB change are appropriate for the nature and scope of the change and are derived on the basis of the as-built, as-operated and as-maintained plant, including operating experience at the plant.
The plant-specific PRA supporting the licensee's proposals has been subjected to quality controls such as an independent peer review or certification.
Appropriate consideration of uncertainty has been given to analyses results and interpretation of l
findings, including the use of a program ofinonitoring, feedback, and corrective action to address significant uncertainties, where applicable.
CDF and LERF are used as bases for probabilistic risk assessment guidelines for addressing l
l Rev. 0 - July 1998 SRP 19-32 l
I
l I
l
[
l Principle 4. If the Commission's Safety Goal QHOs have been used in lieu of LERF, the G
implementation of such an approach included justification of the methods and assumptions used j
j in the analysis and treatment of uncertainties.
i Increases m estimated.CDF and LERF resulting from proposed LB changes are limited to small l
l increments, and the cumulative effects of such changes are tracked and considered in the l
ilecision process.
The acceptability of the proposed changes has been evaluated by the licensee in an integrated fashion that ensures that all principles are met.
i Data, methods, and assessment criteria used to support regulatory decisionmaking are clearly documented and available for public review.
Definitian of the Pronomed Chanoe l
Adequate traditional engineering and probabilistic evaluations are available to support the proposed LB change. Plant-specific and relevant industry data and operational experience also support the proposed charge.
Cause-effect relationships have been identified to adequately link the application with the evaluation models, and the proposed models can effectively evaluate or realistically bound the j
effects of the proposed change.
Information from engineering analyses, operational experience, plant-specific performance
=
history have been factored into the decisionmaking process.
d Evahi=*inna of Defense-In-Denth Attributac and Saferv Marains Defense-in-depth is preserved (for example, system redundancy, diversity, and independence
=
are maintained commensurate with the expected frequency and consequence of challenges to l
the system; defenses against potential common cause failures are maintained and the j
introduction of new common cause failure mechanisms is assessed; and defenses against human i
errors are maintained).
Sufficient safety margins are maintained (for example, NRC-approved codes and standards are
]
met or deviations justified; and safety analysis acceptance criteria in the LB are met, or proposed revisions provide sufficient margin to account for analysis and data uncertainty).
l Current regulations have been met, or the proposed exemption is acceptable.
hane of Rick Analysis 1
l l
The licensee's risk analysis satisfactorily addresses all mode / initiator combinations, or The licensee's risk analysis does not analyze all mode / initiator type combinations. However, in t
each instance, the licensee has demonstrated that -
'~%
o suitably redundant and diverse plant response capability is maintained for significant M?
SRP 19-33 Rev. 0 - July 1998
initiators in these modes, and sufficient elements of the plant response capability are subject to programmatic activities o
to ensure suitable performance Irvel of Detail of Risk Analysis The PRA is detailed enough to account for important system and operator dependencies.
Risk insights are consistent with the level of detail modeled in the PRA.
Quality of the PRA There is reasonable assurance of PRA adequacy, as shown by the licensee's process to ensure quality and by a focused-scope application-specific review by the staff.
Results are robust in terms of uncertainties and sensitivities to the key modeling parameters.
Key performance elements for the application have been appropriately classified and
=
performance is backed up by licensee actions.
Evaluation of Risk imnact If the risk-informed application assesses whether it meets Principle 4 by evaluating the change
=
to risk quantitatively, then the following applies:
The application either decreases plant risk, or if an application increases risk, the increase o
is within the guidelines defined in RG 1.174. The cumulative and synergistic effects on risk from the present and previous applications have been addressed. Licensee risk management practices are being followed to minimize the risk from plant operations.
An appropriate consideration of uncenainties is provided in support of the proposed o
application. The licensee showed that even taking into account the uncertainties in the analysis, the evaluation of the change in risk was robust in that there can be confidence in the conclusions drawn with respect to nature of the change compared with the acceptance guidelines. This argument was supported either by explicit propagation or by a qualitative and/or sensitivity analysis showing that no event contributing to the change in risk is subject to significant uncertainty.
If the risk-informed application is based on a qualitative assessment of the change to risk, the application is shown to result in a decrease in plant risk, or is risk neutral, or CDF and LERF increases are shown to be acceptable on the basis of bounding evaluations or sensitivity studies.
Integrated Decisionmaking Process l
Results from traditional engineering analyses and risk analyses have been used to ensure that
=
l the principles for risk-informed decisionmaking have been met.
Potential analysis limitations, uncertainties and conflicts are resolved by use of conservative l
l Rev. 0 - July 1998 SRP 19-34 l
l l
results, or by use of appropriate implementation and monitoring strategies, or by use of
]
appropriate compensatory measures.
~~
The integrated decisionmaking process is well-defined, systematic, repeatable, and scrutable.
Imnlementation and Monitorine Stratecies The implementation process is commensurate with the uncertainty associated with the results of the traditional and probabilistic engineering evaluations.
A monitoring program which could adequately track the performance of equipment covered by the proposed licensing changes was established. It was demonstrated that the procedures and evaluation methods will provide reasonable asst'rance that performance degradation will be detected and that the corrective action plan will ensure that appropriate actions can be taken before SSC functionality and plant safety is compromised. Data from similar plants will be used if needed, In addition to the tracking of performance of SSCs affected by the application, the performance monitoring process also includes tracking the performance of SSCs which support the underlying basis for the decisionmaking.
Licensee Submittal The submittal includes sufficient information to support conclusions regarding the acceptability of the proposed change.
)
The appropriate regulatory action was requested. In addition, pertinent information on the LB v
change will be included in the safety analysis report, technical specifications, or license conditions, as necessary.
The licensee has appropriately committed to the important programmatic and performance assumptions in the PRA and engineering analyses which served as the basis of the LB change.
These include compensatory actions used to justify the change and any new regul.itory requirements for high risk significant SSCs not otherwise subject to existing requirements, commensurate with their risk significance. These commitments are reflected in revisions to the safety analysis report and/or technical specifications, or the staff has imposed appropriate licensee conditions.
l I
V.
IMPI EMENTATION The preceding material is intended to provide guidance to applicants and licensees regarding the NRC l
staff's plans for using this SRP chapter for reviews of applications involving risk-informed changes to l
the plant's design, operations and other activities that require NRC approval.
Except in those cases in which the applicant or licensee proposes an acceptable alternative method for demonstrating that a proposed LB change is acceptable, the method described herein will be used by the staff in its evaluation of such changes.
)
v' SRP 19-35 Rev. 0 - July 1998
VI, REFERENCES 1.
NRC Policy Statement, "Use of Probabilistic Risk Assessment Methods in Nuclear Regulatory Activities," 60 Federal Register (FR) 42622, August 16,1995.
2.
" Framework for Applying Probabilistic Risk Analysis in Reactor Regulation," U.S. Nuclear Regulatory Commission, SECY-95-280, November 27,1995.
3.
" Proposed Agency-Wide Implementation Plan for Probabilistic Risk Assessment,"
U.S. Nuclear Regulatory Commission, SECY-94-219, August 19, 1994.
4.
"An Approach for Using Probabilistic Risk Assessment in Risk-Informed Decisions on Plant-Specific Changes to the Licensing Basis," Regulatory Guide (RG) 1.174, July 1998.
5.
" Quarterly Status Update for the Probabilistic Risk Assessment Implementation Plan,"
U.S. Nuclear Regulatory Commission, SECY-97-234, October 14, 1997.
6.
"PSA Applications Guide," Electric Power Research Institute, EPRI-TR-105396, August 1995.
O Rev. 0 - July 1998 SRP 19-36
e A
APPENDIX A j
GUIDANCE FOR A FOCUSED-SCOPE APPLICATION-SPECIFIC PRA REVIEW t
I
' As stated in Section 111.2.2.4 of this SRP chapter and in Section 2.2.3 of RG 1.174, PRAs that are used in risk-informed submittals to determine risk significance or risk impact should be shown to be of adequate quality. In risk-informed regulation (RIR), licensee submittals are expected to utilize an t
integrated process which combines risk insights from a PRA, together with insights from traditional engineering analyses, supported by performance monitoring and feedback.' The quality of the PRA required to support this process is commensurate with the roles the risk insights play in the fm' al decisionmaking.
Staff evaluation of a licensee's risk-informed application submittal is expected to include a review of the licensee's process for PRA quality assurance. Where necessary, this should be supplemented by a l
general review of the event and fault tree models, data on SSC failures and common cause failures, mission success criteria, initiating event analysis, human reliability analysis, and sequence i
quantification including the analysis of uncertainties. These reviews should be sufficiently detailed to give the staff confidence that the PRA appropriately reflects the plant's design and actual operating l
conditions and practices. Results from previous staff reviews (e.g., from prior applications or from IPE/IPEEE reviews) and from industry reviews (e.g., from independent peer reviews, certification processes, or cross comparisons) should be used, as appropriate.
~
In addition to the general overall review described above, staff reviewers are expected to perform a focused-scope review of the risk analysis on an application-specific basis. This appendix provides s
review guidance for the likely elements of a PRA which may affect or be affected by proposed changes to the LB. Reviewers should choose the relevant parts of this appendix, guided by the application-l specific SRP chapters (where available) and by the cause-effect relationship described in Section 111.2.2.1 of this SRP chapter.
For additional background on the PRA review, the reader is referred to the bibliography provided in Section A.11 of this appendix.
A.1 '
Initiating Event Analvsis a.
Areas of Review Whether or not a PRA includes a particular initiating event depends on the scope of the PRA, the frequency of the given event, the plant systems or other features available to mitigate the event, and the consequences of the event if unmitigated. Proposed plant changes could affect the frequency of initiating events, the probability of mitigating event initiators and, in some cases, event consequences.
In addition, plant changes could potentially introduce new initiating events or increase the importance of events that were previously screened out.
t l
b.
Review Guidance and Procedures For risk-informed applications, reviewers should determine if the licensee followed a systematic approach to determine if initiating events and anticipated plant response are affected by the proposed changes. Reviewers should also determine if the licensee's process includes provisions to evaluate SRP 19-Al Rev. 0 - July 1998
whether the proposed changes can (i) increase the frequency of an initiator already included in the PRA; (ii) increase in the frequency of initiators that were initially screened out in the PRA; (iii) introduce new initiating events; or (iv) affect the grouping of initiating events. These considerations are discussed in more detail in the following paragraphs.
l Applications that change the frequency of an initiator or the ability of the plant to respond to event initiators are relatively easy to model in the risk analysis if the initiators are already included in the base analysis. In such cases, the licensee should have evaluated the impact of the changes directly from the risk model.
In cases where initiators are not included in the original risk analysis based on screening analyses, the licensee should have determined if initiating events previously screened out because of low frequency might now be above the screening threshold as a result of a proposed application. Plant changes could increase the frequency of initiating events that were relatively infrequent to begin with, or these changes could affect SSCs or operator actions that were credited with the satisfactory mitigation of initiating events. If initiating events increased in frequency as a result of an application to the point where it became important (i.e., could no longer be screened out), reviewers should verify that the licensee has modified the scope of the analysis to reflect this change.
Low frequency of an event, by itself, is not usually sufficient as a criterion for screening purposes.
The consequences of non-mitigation of the events also play a big part in this process. For example, interfacing system loss-of-coolant accidents (ISLOCAs) are often assessed as low-frequency events.
However, because of their impact on public health and safety, these ISLOCAs can be important.
Therefore, for potentially high-consequence events, even if the event frequency is below a screening criterion, the features that lead to the frequency being low (for example, surveillance test practices, startup procedures, etc.) should be taken into a: count in reviews of PRA applications.
The licensee should also have evaluated proposed plant changes to determine if the changes could result in initiators that are not previously analyzed in the PRA. For example, changes might enhance the potential for spurious operation of components which might, in turn, cause initiating events, or changes might increase the likelihood for operator errors of commission which could result in plant trips. If the licensee identified mechanisms for producing new initiators, reviewers should ensure that the licensee added those initiators to the risk analysis so that their impacts can be analyzed.
In PRAs, initiating events are usually grouped according to the systems required to respond to the transient. This implies that success criteria for plant systems and operator responses are similar for all events in a group. In addition, events may be screened out when it can be shown that they are bounded in probability and consequence by other similar events. In evaluating risk-informed applications that affect initiating events, reviewers should ensure that grouping criteria used in the base analysis have not been invalidated by the proposed plant changes or, in the case where this is not true, the licensee has made appropriate changes to the event groupings.
Finally, the reader should note that many PRAs model initiating events as single basic events or " black boxes." In RIR, it is preferred that the licensee model initiating events (especially those that result from the loss of support systems) using a fault tree (or equivalent) approach so that system dependencies are fully understood and accounted for. If this is not the case, reviewers should be aware I
of the combination of SSC failures or other events that could lead to the " failure" of the black box.
This would lead to a better understanding of the risk contributors and is especially important in risk categorization applications.
i l
Rev. 0 - July 1998 SRP 19-A2 i
l m-
c.
Evaluation Findings Reviewers should verify that the information provided and review activities conducted support the following conclusions:
The licensee has adequately considered the effects of proposed changes on the frequencies of initiating events analyzed and those previously screened out.
The licensee has demonstrated that the changes do not result in new initiating events or, if new initiators have been identified, these have been added to and analyzed in the risk model.
The licensee has accounted for the proposed changes in the grouping of initiating events.
The decisionmaking process considered the dependencies between the initiating events and the plant's mitigation systems.
A.2 Accident Sequence Analysis mvent Trees) a.
Areas of Review Although the evaluation of risk change from most applications will usually not necessitate changes to the event tree structure or logic, reviewers should be aware that there will be some changes, particularly those involving changes to plant procedures, which might cause a restructuring of the event sequence logic.
l In addition, the application may isolate part of the PRA that is dependent on specific initiating events.
Thus, these initiating events and their associated event trees would have a proportionately greater impact on the evaluation of the change in risk. In this case, these event trees could be candidates for a higher level of scrutiny. For example, if the changes involved the addition or subtraction of a diesel generator, the review would focus on the station blackout event tree and its associated structure and logic. Similarly, if changes involve modification to procedures to cross-tie electrical buses, the review might focus on the loss of offsite power event trees.
b.
Review Guidance and Procedures Event tree sequence models are used to model the responses of plant systems and operations personnel to initiating events. When the LB change request requires the review of event trees, it is important that reviewers become familiar with their structure, and with the assumptions embedded in them. In particular, it is important to identify assumptions or approximations that might impact the application.
Such assumptions and approximations are not always explicitly documented. The guidance provided below discusses approaches that reviewers can adopt to assess the appropriateness of the modeling of the LB change in the event trees.
Reviewers should familiarize themselves with the structure of the event trees and the associated assumptions that are used in the construction of the event trees. Specific issues to consider should include the conditions created by the initiator and the chronological requirements for systems operation and/or operator responses for the different event tree branches. Reviewers should be satisfied that, if simplifications or assumptions were made in the structure and logic of the event trees, these would I
remain justifiable in light of the LB change.
SRP 19-A3 Rev. 0 - July 1998
Reviewers should also study the functional and physical dependencies for each phase of the sequence and, at the same time, the interaction between operators and systems as the sequence unfolds. The timing of the events and time dependencies should also be undetstood. A review of the general structure and philosophy underlying the pertinent plant emergency and abnormal operating procedures will provide valuable insight on the validity of the event tree structure and logic.
Specifically, reviewers should ensure that the following factors are addressed in the evaluation of the LB change:
The event trees reflect changes (if any) to the initiating event groupings.
The models and analyses are consistent with the as-built and as-operated plant, i.e., the functions necessary for safe shutdown are included, relevant systems are credited for each function, and plant emergency operating procedures (EOPs) and abnormal operating procedures (AOPs) are correctly represented. In addition, where the proposed change affects any of these elements, the change is properly modeled.
Changes to the plant's design or operations could affect the dependencies (functional, phenomenological, and operational) among the top events in event trees. Section A.4 of this SRP chapter presents additional detail concerning the review of the dependent failure analysis.
Time-phased evaluation is normally included for sequences with significant time-dependent I
failure modes (e.g., batteries for station blackout sequences) and significant recoveries (e.g.,
AC recovery for SBO sequences). The impact of the change on event timing that could affect the structure or logic of the event trees should be understood.
It is expected that the success criteria used in the event trees will not be affected by many of the changes to the plant's design and operations. In cases where changes could affect the success criteria for front-line or support systems, reviewers should verify that these criteria (hardware requirements, number of trains required, etc.) remain consistent with the required performance criteria (flow, response time, etc.) related to functional requirements. However, even in cases where the change does not affect the success criteria, reviewers should be aware that the success criteria used in the base PRA analysis could affect the conclusions made in the evaluation of the risk change. For example, a component in a three-train system might not be risk-significant if mission success was contingent on the successful operation of one out of the three trains, but this component could become more risk-significant if the success criterion was two-out-of-three or three-out-of-three trains. Section A.5 discusses the review of the success criteria used in accident sequence modeling.
c.
Evaluation Findings Reviewers should verify that information provided and review activities conducted support the following conclusions:
The licensee has adequately considered the effects of proposed changes on the structure and logic of the event trees.
The licensee has addressed the effects of the application on sequence dependent failure analysis, sequence timing, and success criteria.
Rev. 0 - July 1998 SRP 19-A4 t
p
' A.3 Svatem Modeling Annivsis (Fault Trees) a.
Areas of Review Fault trees are used to depict the logical interrelationships of credible plant events (component hardware failures, human errors, or other pertinent events) that can lead to particular failure modes of plant systems in the context of their environment and operation. In RIR, the majority of proposed j
- changes would only be expected to impact the parameten. that are used to quantify the event probabilities modeled in'the fault trees. In such cases, the change will not affect the fault tree logic models themselves. However, in cases where the change relates to a system design change, or where the licensee is proposing temporary changes that require reconfiguration of the system into ones that are not currently modeled, the revised fault trees should be one focus of the staff's review.
Other considerations of which reviewers should be aware in the area of system analysis are whether the application can impact support functions in such a way as to alter the dependencies in the model, and whether the application can impact system performance to an extent that would require changes to the fault tree logic or modeling assumptions.
b.
Review Guidance and Procedures When the review of one or more of the system logic models becomes necessary, this review should include a study of the appropriate system notebooks from the base PRA to understand the modeling
~ haracteristics that may be affected by the change. It should also include an evaluation of the licensee's c
')
process for modeling the system change as well as a spot-check of the revised system models and Q
results. Reviewers should verify that, in modeling the change, the licensee appropriately modified the system logic models to reflect changes in the plant's configuration including changes to the system design, system performance characteristics, system alignments, operational procedures, and operational philosophies < in particular, reviewers should address the following considerations:
a ~
The analysis of the change should account for the effects of the change on the definition of system success. That is, if the proposed application affects component configurations, expected operability conditions, failure modes and their effects, and alternative success and potential failure paths, these should be taken into account. In addition, the licensee should show that the justification used in the original analysis to exclude components, component failure modes, or flow diversion paths, etc remain valid in light of the proposed change. The analysis should also identify and account for changes that could affect environmental conditions that could cause system failure (e.g., room temperature, containment pressure, etc.).
i The analysis should account for interfaces with other systems and dependence on support i
functions; this is particularly important if dependencies on motive power, control power, component cooling, room cooling, or any interlocks have been altered by an application. Other dependencies that licensees should consider include the dependency on automatic system
' initiation ~and the conditions that must exist for automatic start, essential manual actions to initiate or control the system, and the resources required to fulfill mission success (e.g., water sources, air, fuel oil, etc.). When applicable, licensees should factor these dependencies into the analysis of the change.
When proposed changes deal with proceduralized test and maintenance actions or applicable
%/
SRP 19-A5 Rev. 0 - July 1998
technical specification conditions, the modeling of test and maintenance unavailabilities and the modeling of restoration errors for the affected systems / components should be reviewed.
l Changes to the frequency of each test or maintenance activity, its approximate duration, the components repositioned for the action, the verification activities post test and maintenance, and the availability of the system during the test procedure should have been factored into the change analysis.
Operational history (i.e., plant-specific operational experience) should be considered in the i
review of the system models and especially in the review of how the proposed change will j
affect system operation. Considerations like recurring check valve problems (e.g., back-leakages), water hammer events, or flow blockages by sludge or debris should also be considered in the analysis.
The potential for common cause failures including those potentially resulting from the change should have been evaluated and modeled where appropriate. Review guidance for the evaluation of common cause failures is provided in Section A.7 of this SRP chapter.
The function of the modeled system should remain consistent with that required in the event tree models. Success criteria and event sequence conditions should be correctly modeled and consistent with the definition in the event trees.
When fault tree solutions in the form of function cutsets are available, an efficient way to review for the logic in system models is to study the cutsets produced by the solution of the linked fault trees (i.e.,
the fault tree formed by linking the support system fault trees to the system fault tree). In performing this visual inspection, reviewers should compare the results with expectations based on their l
understanding of functional and support system dependencies. The effects of events such as operator l
actions or common cause failures can also be easily verified by an inspection of the function cutsets.
l When expected combinations of failures are not present, reviewers should check to see if these failures have been modeled, or if they have been truncated during model solution, or if the fault tree logic is l
incorrect (e.g., an AND gate in place of an OR gate). In short, a review of cutsets can be one way to l
focus further reviews on other parts of the system modeling analysis.
l c.
Evaluation Findings Reviewers should verify that information provided and review activities conducted support the l
following conclusion:
l The evaluation adequately reflects changes in the plant hardware or procedures, including l
j changes to the system design or alignments, system performance characteristics, support system dependencies, and operational procedures or operational philosophies. Where applicable, these changes are appropriately included in the PRA system models.
A.4 Denendent Failure Analysis a.
Areas of Review Accident progression models and system models should correctly account for dependencies between systems and operator actions needed for accident mitigation. Proposed changes to the plant's design or operations could affect these dependencies; therefore, the evaluation of the risk change should also Rev. 0 - July 1998 SRP 19-A6
consider system-operator dependencies. However, since the modeling of these dependencies requires n
detailed knowledge of the plant systems and procedures, it will not be practical (nor is it intended) for reviewers to verify that all dependencies have been included in the change evaluation. Instead, reviewers should verify that the evaluation utilized a comprehensive and systematic process to look for these dependencies. Reviewers should rely on their experience with similar change analyses (when applicable) or with PRAs of similar plants, but should be aware that dependencies are in many cases plant-specific, and will depend on plant-specific system capabilities and interactions, procedural guidance, and timing of potential accident sequences.
b.
Review Guidance and Procedures Review guidance in this section consists of a discussion of the dependencies that could be important and that could be affected by changes to the plant's design or operations. Although most changes will not alter the original PRA dependent failure analysis, some design or procedure changes could introduce new dependencies or affect existing ones. Therefore, reviewers should be cognizant with regard to the j
following types of dependencies that could exist and could affect the results of the change analysis:
Functional Dependencies: These dependencies occur because the function of one system or component depends on that of another system or component. Functional dependencies include interactions which can occur when the change in the function of a system or component causes a physical change in the environment which results in the failure of another system or component.
Functional dependencies include the following examples:
shared component dependencies (e.g., systems or system trains that depend on a conunon intake or discharge valve) actuation requirement dependencies (e.g., systems that depend on common actuation signals,
=
common actuation circuitry, or common support systems like AC or DC power or instrument air for initiation or actuation) and conditions needed for actuation (e.g., low RPV water level).
isolation requirement dependencies (e.g., conditions that could cause more than one system to isolate, trip, or fail) including environmental conditions (temperature, pressure, and/or humidity), temperature and pressure of fluids being processed, water level status, and radiation levels.
power requirement dependencies (e.g., systems that depend on the same power sources for motive power) cooling requirement dependencies (e.g., systems that depend on the same room cooling
=
subsystem, or the same lube oil cooling subsystem, or systems that depend on the same service water or component cooling water train for cooling) indication requirement dependencies (e.g., systems that depend on the same pressure, temperature, or level instrumentation for operation) phenomenological effect dependencies (e.g., conditions generated during an accident sequence that influence the operability of more than one system), including generation of harsh environments that result in protective trips of systems, loss of pump net positive suction head
}
(NPSH) when containment heat removal is lost, clogging of pump strainers from debris SRP 19-A7 Rev. 0 - July 1998
generated during a LOCA, failure of components outside the containment following containment failure attributable to harsh environment inside the containment, closure of safety relief valves in BWRs on high containment pressure, and coolant pipe breaks or equipment failures following (or resulting from) containment failure operational dependencies (e.g., unavailability of the suppression pool cooling mode for a train of the residual heat removal system when the system is in the low pressure coolant injection mode)
Reviewers should look for evidence that the licensee properly considered the above types of dependencies in the evaluation of the change. In most cases, these dependencies should be explicitly included in the fault tree or event tree logic models; however, in some cases, a qualitative evaluation process may be sufficient.
Human Interaction Dependencies: These dependencies could become important contributors to risk if operator error can result in multiple component failures. Past PRAs show that the following plant conditions could lead to human interaction dependencies that can become important l
1 tests or maintenance that require multiple components to be reconfigured multiple calibrations performed by the same personnel
=
post-accident manual initiation (or backup initiation) of components that require the operator to a
interact with multiple components Reviewers should verify that the licensee's evaluation of risk from proposed changes to plant procedures or changes to operator training included a process to identify these (or similar) activities, and that the licensee evaluated the activities that could be risk contributors.
Component Hardware Failure Dependencies: These dependencies, usually referred to as common cause failures (CCFs), cover the failures of usually identical components which may be caused by i
design, manufacturing, installation, calibration, or operational deficiencies. CCFs are treved quantitatively by common cause failure probabilities or other dependence quantification approaches.
Section A.7 of this SRP chapter presents review guidance related to CCFs.
Spatial Dependencies: Multiple failures could be caused by events that fail all equipment in a defined space or area. These spatially dependent failures include those caused by internal flooding, fires, seismic events, missiles (e.g., turbine missiles), or any cf the other external event initiators. In cases where these events could affect the results of the change evaluation, and where these events are not modeled in the PRA, the dependencies resulting from the unmodeled initiators should be evaluated qualitatively as part of the integrated decisionmaking process. Section 111.2.2 of this SRP chapter discusses the required scope of the PRA in more detail. In addition, the change request should inch de the licensee's consideration of the common influences on component operation such as adverse environment (including excessive temperature, humidity, radiation), inadequate space, inadvertent or spurious sprinkler operation, or routine equipment travel near major components. Reviewers should verify that the change request has used a systematic process to identify potential spatial challenges that could result in multiple failures of SSCs.
O Rev. 0 - July 1998 SRP 19-A8
c.
Evaluation Findings Reviewers should verify that information provided and review activities conducted support the following conclusion:
Dependencies between system and operator interactions have been properly accounted for in the evaluation of the proposed change. Where appropriate, these dependencies have been included in the accident progression models (event trees) and the system analysis models (fault trees).
A.5 Determination of Success CrittIla n.
Areas of Review Guidance in the PRA policy statement and in RG 1.174 stipulates that realistic analysis should be used in PRA implementation. The following discussion is intended to sort out what is meant by " realistic" analysis of success criteria by reference to SAR analysis.
In order to fulfill its intended purpose, SAR analysis is ordinarily based on a set of assumptions containing significant embedded conservatisms. SAR analysis also reflects a postulated single active failure, in addition to whatever event initiated the sequence. When an SAR analysis shows a successful outcome, there is good reason to believe that (apart from beyond-single-failure scenarios) the system will meet or exceed performance requirements for the initiating event considered.
Applying the SAR mission success criterion in a PRA would be conservative, in the sense that the g
probability of failure to meet this performance standard would be greater than probability of failure to meet a more realistic performance standard. However, re-analyzing event sequences with conventional SAR tools would be too burdensome to apply to the large number of scenarios that are defined in the course of a PRA. In addition, the rather specialized computer codes used in SAR analysis may not be appropriate in beyond-single-failure scenarios. Traditionally, development of mission success analyses in PRAs has ranged from the use of faster running models that might not have the same level of quality assurance as the conventional SAR tools, to the extrapolation of results from analyses performed on similar plants.
In order to satisfy the Commission's guideline, then, reviewers should find that the licensee has not distorted the PRA insights by using a systematically conservative bias in mission success criteria, and that mission success criteria used to justify changes to the plant's design or operations have a sound technical basis.
b.
Review Guidance and Procedures When it is determined that the results and conclusions of a risk-informed application are especially sensitive to the choice of mission success criteria, or if the modeling is particularly controversial, reviewers should evaluate the relevant success criteria and the basis for each.
If the basis is analytical, it may be appropriate to evaluate of the code and the input data used. When it is determined that the computer codes used have not received adequate licensee or other industry review, closer examination of the models should also be considered.
I SRP 19-A9 Rev. 0 - July 1998
The models, codes, and inputs used to determine mission success criteria should meet QA standards that are consistent with generally accepted methods. Standards should include configuration control of the analysis inputs and results. The standards do not have to be the same as the standards applicable to SAR analysis, but they should be explicit (i.e., engineering calculations and codes should be verified and quality assured) and they should be formalized as part of the licensee's QA program.
In cases where the basis for the success criteria is lacking, reviewers should either request additional licensee justification or seek independent analysis. Licensee justification could include the use of alternative plausible models to justify the conclusions (thus addressing the model uncertainty), or the redesign of the change such that the change is not affected by the choice of success criteria.
Some mission success criteria can validly be extrapolated between similar plants when a firm basis for the criteria is created at the first plant and when the licensee shows that plant-specific features do not invalidate the comparison.
On an application-specific basis, reviewers should determine whether the definition of the system success criteria will be affected by the application-specific elements or by the elements in the same minimal cutset or accident scenario as the application-specific element. Reviewers should ensure that the success criteria are not so optimistic that they underestimate the required number of components (i.e., overestimate the size of the minimal cutset).
c.
Evaluation Findings In cases where conclusions are sensitive to the mission success criteria, the staff's safety evaluation report should contain findings equivalent to the following:
A technical basis has been established for the mission success criteria used in the analysis.
Analytical elements of the technical basis have received an appropriate level of configuration control and quality assurance. Where comparison with analogous criteria from other plants is possible, this comparison has been justified.
A6 Use of Anpropriate Data a.
Areas of Review In risk-informed applications, it is important that the licensee use appropriate SSC failure data. While plant-specific data is preferred, for plants with little operating history, the only choice might be the use of generic data. Furthermore, when the impact of the change is being modeled as a modification of parameter values, sufficient plant-specific data may not exist to support the modification. The data-related issues are summarized as follows: a) if the impact of the application is to be modeled as a change in parameter values associated with basic events representing modes of unavailability of certain SSCs, these changes should be reasonable and should be supported by technical arguments including plant-specific and generic operational information (when available) and b) the impact of the change should neither be exaggerated nor obscured by the parameter values used for those SSCs unaffected by the change, b.
Review Guidance and Procedures It is to be expected that, for a PRA that has undergone a technical review, parameter values will have Rev. 0 - July 1998 SRP 19-A10
beenjudged to be appropriate, whether they have been evaluated using generic or plant-specific data.
However, since the review was focused on the PRA as a base case model, a different perspective on D
the appropriateness of parameter values may be required for specific applications. Therefore, in evaluating PRA applications, reviewers should focus on those parameter values that have the potential to change the conclusions of the analysis. For example, parameters associated with SSCs that appear in the same cutsets or scenarios as the affected SSCs have the potential to distort the conclusions by decreasing the assessed importance of the change if their values are too low, or by increasing it if their values are too high. Similarly, parameters that contribute to the cutsets or scenarios that do not contain affected SSCs can decrease the importance of the change by being too high, or increase it by being too low.
The failure rates and probabilities used, especially those that directly affect the proposed application, should appropriately consider both plant-specific and generic data. The staff expects that these values will be consistent with generally accepted values from PRAs of similar plants, or the licensee should justify significant deviations on the basis of plant-specific factors. "Significant" in this context can be defined as no greater than a factor of 3 for the mean values of the failure rate or failure probability.
The focus of the review should be on those parameter values that have a significant impact on the results as discussed above, and that deviate significantly from the generally accepted norm.
If the reviewer decides that a more detailed review of the parameter values is appropriate, the following guidance applies. For plant-specific data, reviewers should determine how the licensee used plant records to estimate the number of events or failures, the number of demands, and the operating or standby hours. Reviewers should verify the consistency between the definitions of failure modes and component boundaries used in the risk analysis and the corresponding definitions used in the plant records. When reviewing generic data, it is important to verify that the plant component is typical of the generic industry component. In cases where generic failure rates are used in combination with plant-specific data like test intervals, reviewers should verify that the generic data are applicable for the range of plant data used.
When evaluating the impact of the change, it is important for reviewers to recognize the assumptions that have gone into developing the PRA model. For example, two models are commonly used for events representing the unavailability of a standby component on demand; the standby failure rate model and the constant probability of failure on demand model. The constant probability of failure on demand parameter may be estimated on the basis of an assumed number of demands, implying an average test interval. Use of such a model to investigate the impact of extending test intervals can result in large differences between the unavailabilities of components for which the test intervals differ significantly. Reviewers should be sensitive to this effect, and should ascertain that licensees use appropriate models and parameters for such evaluations.
As another example, in considering plant-specific failure data, poorly performing individual components may have been grouped with other components, allowing their poor performance to be averaged over all components of that type. Poor performance may arise because of inherent characteristics of one member of what would otherwise be considered a uniform population, or may arise because components are operating in a more demanding environment. If these components are grouped together with others for which the operating conditions are more favorable, the failure rates used for the poor performers could be artificially lowered. If requirements are relaxed on the basis of the group failure rate, reduced programmatic attention to these poor performers could lead to a greater-than-expected probability of experiencing an inservice failure of one of these components.
Reviewers should be aware of such effects, and should ensure that the components are grouped SRP 19-A11 Rev. 0 - July 1998 1
appropriately.
When the impact of the change is modeled as a change in the parameter values associated with specific basic events representing modes of unavailability of SSCs, reviewers should focus on whether the change in parameter values is appropriate and reasonable. The licensee is expected to document the rationale behind the change in parameter values, and that rationale should be carefully reviewed.
If generic values are used for the base case parameter values which are candidates for change, reviewers should verify that the conditions under which the generic data apply do not correspond to those which would be more appropriate for a plant with the change incorporated. This should only be a real concern if the plant being changed is somewhat atypical with respect to the issue being addressed by the change. This would not be a concern if plant-specific data were used.
Finally, to validate the data used to justify changes in risk-informed applications, it is important for licensees to monitor the performance of components affected by the application. This monitoring should be performed as the proposed application is phased in. For very reliable SSCs, it may be necessary for the licensee to review available operating experience at other plants for applicability to the licensee's plant to expand the operating experience database. Reviewers should ascertain that the monitoring program is capable of demonstrating that the performance of the com;onents or sys'tems is in accordance with what has been assumed.
- c. Evaluation Findings Reviewers should verify that information provided and review activities conducted support the following conclusions:
O, The failure rates and probabilities used, especially those that directly affect the proposed
=
application, appropriately consider both plant-specific and generic data that are consistent with generally accepted values from PRAs of similar plaats, and deviations (if any) have been justified on the basis of plant-specific factors.
The licensee has systematically considered the possibility that individual components could be performing more poorly than the average associated with their class, and has avoided relaxation for those components to the point where the unavailability of the poor performers would be appreciably worse than that assumed in the risk analysis.
The changes to the parameter values impacted by the application are both justified and reasonable.
Data used to support changes to the plant's design or operations are supported by an
=
appropriate performance monitoring program.
A.7 Modeling of Common Cause Failures a.
Areas of Review Common cause failures (CCFs) represent the failures of components that are caused by common influences such as design, manufacturing, installation, calibration, or operational deficiencies. Since CCFs can fail more than one component at the same time and can occur with greater probability than Rev. 0 - July 1998 SRP 19-Al2 n
would be predicted by the product of the individual component failure probabilities, they can
,m.
1 significantly contribute to plant risk.
x )-
Risk-informed applications that cover SSCs as a group have the potential to affect the CCF probabilities of SSCs within the given group. For the affected components, CCF probabilities could be low or might not even be included in the caseline PRA models based on the operational and engineering evidence driven by current requirements. With proposed changes, there should be assurance that the CCF contribution will not become more significant. In addition, the assessment of the impact of the change can be affected by the CCF probabilities for other components, and can either be exaggerated or obscured depending on the CCF probabilities.
b, Review Guidance and Procedures Reviewers should verify that the PRA addressed potentially significant CCFs and that, where applicable, the CCF modeling has incorporated the effects of the proposed changes. Staff evaluation should include a review of the process used to select common cause component groups.
Specific review guidelines related to risk-informed applications and the assessment of the change are as follows:
Reviewers should verify that industry and especially plant-specific experience involving the failure of two or more components (especially for the application-specific components) from the same cause was analyzed and incorporated into the risk model where appropriate.
For relevant applications, reviewers should check that licensees have appropriately modeled the
^
CCF of groups of equipment that were proposed for the change. In cases where the effects of the application on CCF cannot be easily evaluated or quantified, reviewers should establish that 1
performance monitoring is capable of detecting CCF before multiple failures are likely to occur subsequent to an actual system challenge. In addition, to reduce fault exposure times for potential common cause failures, phased or incremental implementation should be considered as part of the effort to protect against CCF.
Reviewers should ensure that the impact of the change is not inappropriately made insignificant l
by the choice of CCF probabilities for SSCs unaffected by the change. This can occur in two l.
ways. First, the cutsets or scenarios containing events which represent failures of SSCs
]
affected by the change may include CCF contributions from other SSCs which are too small.
Second, the contribution of cutsets or sce iarios which do not contain affected SSCs may be artificially increased by having CCF contributions that are too large so that the impact of the i
l change is obscured. These cases will impact applications involving risk categorization by
~
l lowering the relative contribution (and importances) of the affected SSCs. An understanding of these effects can be obtained from sensitivity analyses performed by removing the pertinent CCFs or by using more realistic values for the CCFs.
A common modeling approximation is to include CCF contr'ibutions only from that combination of SSCs which fails the function of the system. For example, if system success is defined as success of one out of four components, usually only a single term representing a CCF of all four components is included. If the success criterion were two out of four, the corresponding CCF term would represent failure of any three or all four SSCs in the group.
f While probabilistically this usually corresponds to the dominant contributions, care has to be x
SRP 19-A13 Rev. 0 - July 1998 d
w
I taken when the application relies on assessing the impact on risk of having one train unavailable. In this case, the effective success criterion of the remaining part of the system changes, so that in the case of the one-out-of-four system, a CCF of three SSCs becomes a possible contributor. The impact of not modeling the lower-order CCF contributors should be investigated. Note that this can impact applications for which the justification of the change relies on risk categorization, as well as those that require an evaluation of changes to risk.
c.
Evaluation Findings Evaluation findings should include statements to the following effect:
Common cause failure has been suitably addressed, and the licensee has systematically a
identified component groups sharing attributes that correlate with CCF potential and Mt affect the application.
Where applicable, the licensee's performance monitoring program addresses a phased
=
implementation approach to reduce the potential for increased incidence of CCFs attributable to the proposed change.
A.8 Modeling of Human Performance a.
Areas of Review The results of a PRA, and therefore the input it provides to risk-informed decisionmaking, can be very strongly influenced by the modeling of human performance. Plant safety depends significantly on human performance, so it is essential that the PRA treat it carefully. However, the modeling of human performance, typically referred to as human reliability analysis (HRA), is a relatively difficult area; significant variations in approach continue to be encountered, and these can yield significantly different estimates of human error probabilities (HEPs) for what appears to be similar human failure events.
The particular values used for HEPs can significantly influence results of the assessment of the impact of a proposed change.
In addition to the quantification issue, there are questions related to what kind of human actions can appropriately be credited in the context of a particular regulatory finding. As an example, suppose that PRA results appear to support relaxation of requirements for a component based on the argument that even if the component fails, its failure can be recovered with high probability by operator actions outside the control room. The issues of concern here are whether the modeling of the operator action and the evaluation of the failure probability is appropriate, and whether this kind of credit is the sort of compensating measure that is intended by staff guidance to support justification of a relaxation. One further issue involves the impact of human performance which is not explicitly modeled, but is implicit in certain parameter values. An example is the influence of human performance on initiating event frequency. The causes of initiating events are typically not addressed; their impact is included in the frequency in an implicit way, b.
Review Guidance and Procedures l
Reviewers should understand the potentially significant human performance issues that might be affected by the application and how these are reflected in the PRA. This understanding requires a review of the approach used to estimate human error probabilities.
Rev. 0 - July 1998 SRP 19-A14 l
The HRA can impact the assessment of the change in several ways. First, the change may directly G
affect the human failure events (HFEs). Second, the HFEs may represent responses to failures of the SSCs impacted by the change. Finally, HFEs unrelated to the change can obscure or exaggerate the impact of the change (depending on their values) by inappropriately increasing or decreasing the value of the accident sequences unaffected by the change.
When the change directly impacts the HFEs (e.g., as a result of a procedure change or a change in operating practice), reviewers should ensure that the licensee appropriately model the impact; that is reviewers should ensure that the licensee addressed the following questions:
whether new human actions are introduced or whether existing actions are modified or eliminated whether the change affects factors assumed to impact the likelihood of failure (usually called performance shaping factors or PSFs), including: the quality of the procedures; the cues available to the operators; the quality of the information (instrumentation) available to the operators; the quality of the human-machine interface; the location of the interface (s); the complexity of the task; the conditions or context within which the operators are responding, including previous failures, previous actions, etc.; the time available to perform the task; the quality of the training (type and frequency) on the specific ever3:: the crew interactions and the potential for recovery from errors; and the stress on the operators whether the human action dependency analysis is affected whether the application introduces or modifies dependencies between plant instrumentation and human actions whether the screening analysis is affected
=
When HFEs represent responses to failures of the SSCs impacted by the change, reviewers may want to focus their resources on these HFEs in the following ways:
Identify any human actions that compensate for events affected by the proposed application, and ensure that the licensee did not claim inappropriate credit for these events. For human actions that are used to compensate for a basic event probability increasing as a result of proposed changes, licensee actions to ensure operator performance at the level credited in the risk analysis should also be a part of the change request.
Ensure that appropriate justification is provided when the licensee takes credit for post-accident recovery of failed components (repair or other non-proceduralized manual actions, such as manually forcing stuck valves to open). Reviewers should also ascertain whether the identified recovery action is an obvious, feasible (given the time and physical constraints), and supportable by plant programs such as training.
Ensure that the licensee assessed whether the conditions under which the human actions are to be performed have changed significantly so that the HEP should be modified.
Reviewers should also be aware that the impact of the change can be obscured if the accident sequences which do not contain affected SSCs are artificially increased in value by HEPs that are too large.
SRP 19-A15 Rev. 0 - July 1998
)
These cases will impact applications involving risk categorization by lowering the relative contribution of the affected SSCs. An understanding of these effects can be obtained from sensitivity analyses performed by removing the pertinent HEPs or by using more realistic values for the HEPs.
Another consideration associated with the potential masking of important SSCs is that the SSCs might not be included in the model used to perform the evaluation of risk. This can happen in several ways:
Cutsets or scenarios containing the SSCs may be truncated because HEPs in the same cutset or scenario are too low. Such truncation should only be a concern if the logic model was not re-solved to determine the change in risk (for example, in applications that depend on SSC risk ranking using a pre-solved equation). The preferred resolution to this would be a request for re-solution with the appropriate changes made to all affected SSCs. Section A.9 of this SRP chapter discusses this in more detail.
SSCs may not be included in the logic model structure because HEPs are so high that they are assumed to dominate the unavailability of a function, and therefore the associated hardware is not modeled. However, the hardware could still be a contributor to the calculation of risk importance. For example, the hardware (as a group) will have the same risk importance (in terms of Risk Achievement Worth) as the associated HFE. This suggests that the licensees should identify the important operator actions for applications in RIR, as well as the equipment required to perform the specific function associated with the action. The equipment should then be dispositioned in accordance with its importance in achieving that function.
For some complex groups of operator actions (e.g., the response to an ATWS in a BWR, or the choice to go to recirculation rather than RHR in response to a small LOCA in a PWR), the PRA analysts may have chosen to adopt a bounding approach to the accident scenarios which precludes having to address subsequent actions. This could mean that the equipment associated with those actions might be overlooked in the change process.
c.
Evaluation Findings The staff safety evaluation report should include language equivalent in effect to the following:
The modeling of human performance is appropriate.
Post-accident recovery of failed components is modeled in a defensible way. Recovery probabilities are realistically quantified. The formulation of the model shows decisionmakers the degree to which the apparently low risk significance of certain items is dependent on credit for recovery of failed components (restoration of component function, as opposed to actuation of a compensating system).
When human actions are proposed as compensatory measures as part of a proposed change, licensee actions to ensure operator performance at the level credited in the risk analysis (e.g.,
by training, procedures, etc.) are also a part of the change request.
O Rev. 0 - July 1998 SRP 19-A16
A.9 Sequence Ouantificatinn Q
a.
Areas of Review The staff would not generally anticipate the need to perform a detailed review of the quantification of the change in risk; however, some details of the quantification process should be confirmed.
' Specifically, reviewers should be confident that the licensee's evaluation process is sufficient to account for the potential effects of the proposed change on modeling simplifications and assumptions made during the quantification of risk. In addition, the staff should ensure that the chosen sequence truncation limits are appropriate so that important sequences are not discarded and fmal results are not sensitive to the chosen truncation limit.
b.
Review Guidance and Procedures Reviewers should verify that model simplifications and assumptions made during the quantification process are properly accounted for in evaluating of the change in risk, as illustrated by the following examples:
Reviewers should ensure that the licensee accounted for model asymmetries during the application of the PRA models. Asymmetries could result from modeling assumptions (e.g.,
assuming one train to be the operating train, and the second train to be the standby train), from differences in support system alignment, or from actual differences in system design or operating procedures. The licensees should have accounted for these asymmetries when evaluating changes to the affected systems.
O.
Reviewers should ensure that, if cutset/ sequence deletion is performed during quantification,
=
these are correctly addressed in the assessment of risk change. In some quantification processes, cutsets that contain combinations of maintenance actions that are disallowed by the Technical Specifications are deleted from the accident sequence equations after the merging of functional cutset equations. This is done to avoid undue conservatism. If the PRA application deals with Technical Specification allowed outage issues, reviewers should confirm that any impacts on such deletions have been correctly addressed.
Reviewers should ensure that, if operator recovery actions are incorporated after the initial quantification, these actions are still valid in light of the proposed change. Section A 8 of this SRP chapter discusses this in more detail.
. Circular logic in fault trees will cause the quantification process to abort. This is a problem for systems such as the emergency service water system, which provides cooling to the emergency diesel generators, but requires power from those diesel generators when offsite power is lost.
Another example is the mutual dependency between the DC and AC power systems. In situations such as these (i.e., when the physical situation has embedded circular dependencies),
analysts have to break this circularity to allow for model solution. For changes on systems that are affected by circular logic, reviewers should investigate the manner in which the circularity was broken (usually in the sequencing of functions in the event tree) and should verify that the dependency is still being accounted for in the evaluation of the risk change.
SRP 19-A17 Rev. 0 - July 1998
Sequence Truncation The staff prefers that licensees calculate the change in risk from the application by requantifying the base PRA model so that the potential effects of originally truncated events can be accounted for should they become important as a result of an application. If the licensee did not requantify the model, or if the application depended on the risk ranking of SSCs from a pre-solved equation, reviewers should use the guidelines provided below.
Reviewers should be assured (either by documentation provided in the licensee's submittal or by an independent staff analysis) that cutset or scenario truncation did not introduce errors into the application results or the logic of the PRA that affects the application. Staff review could also involve performing (or reviewing) sensitivity studies where the truncation limit is lowered for the dominant sequences and event initiators, and studying the resultant cutsets or scenarios to see if there are any hidden dependencies or unusual / unexpected event combinations (especially if these involve components affected by the proposed application).
Staff review could also include comparing a list of the events affected by the application that is in the final truncated cutset equations to the list of application-specific basic events used in the fault tree and event tree models. This yields a list of events that did not make it pass the truncation process.
Documentation should be available to enable reviewers to determine the reason truncated events are not important to risk.
Finally, in PRA models where common cause failures and human dependencies are incorporated at the sequence level after a truncated set of minimal cutsets has been obtained, reviewers should verify that the truncation criteria used in the PRA do not lead to cutsets involving application-specific components being truncated that could be important if common cause failures or human dependencies are considered.
c.
Evaluation Findings Reviewers should verify that the information provided and review activid conducted support the following conclusions:
The change is appropriately modeled and is properly quamified.
The licensee has satisfactorily established that conclusions are not adversely affected by truncation either because (i) the change in risk from the application was calculated by the requantification of the base model, or (ii) if model requantification was not performed, or if the application depended on the risk ranking of SSCs from a pre-solved cutset equation, the following apply:
The truncation criterion is sufficiently low to ensure stable results, that is, the magnitude o
of the CDF or release frequency will not change as a result of lower truncation limits, and the grouping of SSCs into risk categories will not be affected.
The components affected by the application are, for the most part, not truncated out of the o
model. In cases where they are, a qualitative assessment can demonstrate the reasons why they are unimportant to risk.
O Rev. 0 - July 1998 SRP 19-A18
A.10 Modeling of Containment Response and Changes in Large Early Release Frecuency 9
a.
Areas of Review The purpose of this section is to provide guidance for use in reviewing the licensee's evaluation of changes in LERF stemming from proposed changes to the plant's design or operations.
In general, only a subset of CDF sequences will be affected by a change. Whether or not this subset contributes significantly to LERF depends on several plant-specific characteristics. This section focuses on the characteristics that strongly affect LERF, and identifies review approaches based on these characteristics. It also provides guidance to help reviewers identify the major items related to functional plant capability that directly affect the potential for large early release; to direct reviewers in establishing whether the proposed changes can affect this capability; and to determine whether the licensee has appropriately addressed these items in estimating changes in LERF.
b.
Review Guidance and Procedures There are several ways in which a change to the plant's design or operation can significantly alter LERF, including those that:
Change the frequency of containment bypass sequences (e.g., steam generator tube ruptures and interfacing system LOCAs).
Change the frequency of core damage sequences that pose severe challenges to containment G
(e.g., sequences resulting in elevated reactor coolant system (RCS) pressure during core damage and at vessel failure).
Change the performance of systems involved in containment safety functions (e.g., containment isolation, containment heat removal, containment sprays, hydrogen control, etc.).
Change the performance of systems or operator actions that affect accident management strategies (e.g., depressurization, venting, etc.).
Change the frequency of core damage sequences occurring at shutdown with containment functionality reduced.
The guidance provided below focuses, for each plant type, on particular examples of these general categories.
Based on previous PRAs, draft NUREG/CR-6595 developed some insights on the factors that most strongly affect the estimated likelihood of a large early release. Although plant-specific details may become significant in some cases, it was found that plants of each major containment type tend to be similar in the types of sequences that could lead to a large early release, reflecting strengths and weaknesses of that containment structure and particular features of the core damage sequences that characterize that plant type. Based on these insights, draft NUREG/CR-6595 presents a screening approach to evaluate the frequencies of dominant containment failure modes and bypass events. The purpose of this approach is to provide estimates of LERF, given certain characteristics of core damage sequences as input.
SRP 19-A19 Rev. 0 - July 1998
The review approach presented in this SRP section builds upon the underlying insights from draft NUREG/CR-6595. For each major containment type, particular considerations are suggested for attention in the review process. However, it is not intended to suggest that these considerations exhaust the technical issues that affect the potential for large early release. For example, where plant-specific PRA Level 2 analyses exist, these could provide further insights into LERF considerations for that plant.
For each major containment type, the factors that most strongly affect the potential for large early release (given that a core damage sequence is underway) are as follows:
PWR Large Dry:
Containment bypass Containment isolation RCS depressurization Emergency core cooling (ECC) restoration before vessel failure PWR Ice Condenser:
Containment isolation Containment bypass Hydrogen igniters RCS depressurization ECC restoration before vessel failure BWR Mark I and II:
Containment isolation Containment bypass Venting Containment heat removal: decay heat Containment heat removal: ATWS RCS depressurization ECC restoration before vessel failure BWR Mark III:
All Mark I and Mark II issues Igniters It should be noted that, at some BWRs, many sequences that result in vessel breach have a significant probability of alsa failing the containment. Also, the reader should note that a loss of containment heat removal may significantly contribute to CDF.
In reviewing the calculation of change in LERF for a given plant type, reviewers should consider the following factors:
Containment Bypass:
Whether the proposed change affects systems that are credited in the prevention of, or in response to an initiating event involving a steam generator tube rupture (SGTR) or an ISLOCA.
Whether the proposed change affects the frequency or severity of transients that could result in induced steam generator tube ruptures (ISGTR) (i.e., tube rupture in the course of an accident, caused by elevated temperatures and/or elevated pressure differentials). If the proposed change does not directly affect steam generator tube integrity, and the steam generators in the plant are not experiencing significant degradation, only a qualitative analysis may be needed to ensure Rev. 0 - July 1998 SRP 19-A20
p that the risk of ISGTR is not significantly increased by the proposed change. However, if the plant has suffered a steam generator tube rupture, or has been shut down because of excessive steam generator tube leakage, or has detected tubes which do not meet applicable ASME Code requirements for structural integrity, or has repaired a significant amount of tubes as a result of free span degradation, the application should provide a more thorough analysis of the effects of the proposal on the risk associated with ISGTR.
Containment Isolation:
Whether the proposed change affects systems that perform or support the isolation fenction.
Whether the proposed change affects systems that prevent or mitigate core damage sequences initiated during periods of reduced containment functionality (e.g., shutdown).
Whether the proposed change affects the ability to restore containment function during such periods (e.g., AC power, plant procedures, etc.).
Igniters:
Whether the proposed change affects the igniters or any applicable support systems.
ECC Restoration Before Vessel Failure:
If credit was taken in the estimate of LERF for recovery of cooling before vessel failure,
=
f whether the proposed change affects performance of any system thus credited (including support systems).
Whether the proposed change affects other accident management strategies credited in the PRA (e.g., external vessel flooding).
RCS Pressure at Vessel Failure:
Whether the proposed change affects the capability to depressurize the RCS.
=
Venting:
Whether the proposed change affects the capability to vent the containment.
Containment Heat Removal:
Whether the proposed change affects systems credited in containment heat removal (including front-line and support systems).
Whether the proposed change affects the frequency or severity of ATWS sequences.
For each of the above considerations that apply, reviewers should ascertain that the licensee adequately evaluated the effects and took them into account in calculating the change in LERF.
O SRP 19-A21 Rev. 0 - July 1998
c.
Evaluation Findings The safety evaluation report should contain findings equivalent to the following.
The calculation for the change in LERF resulting from a proposed change has systematically
=
taken into account the dominant causes of containment failure. In particular, the calculation has considered: bypass sequences; sequences posing relatively severe challenges to containment, or sequences occurring during periods of reduced containment functionality (shutdown); performarce of systems involved in containment safety functions, including containment heat removal, sprays, isolation, and restoration of containment functionality (shutdown); and performance of systems involved in accident management strategies.
A.11 Bibliography This section provides a list of documents of that the staff could use as reference or background material during the review process. This bibliography is divided into general categories in the areas of:
desirable PRA attributes, review of the PRA, uncertainty and sensitivity analyses, and use of the PRA in risk ranking. In addition, a bibliography is provided for each of the review categories discussed in Sections A.1 through A.10 of this appendix.
General-Desirable PRA Attributes Electric Power Research Institute, "PSA Applications Guide," EPRI TR-105396, August 1995.
Electric Power Research Institute, " Development of a Quality Pedigree Process and Application to the Duane Arnold Energy Center Probabilistic Safety Assessment," EPRI TR-106575, August 1996 (proprietary document - contact EPRI for availability).
International Atomic Energy Agency, " Procedures for Conducting Probabilistic Safety Assessments of Nuclear Power Plants (Level 1)," IAEA Safety Series No.50-P-4.1992.
USNRC, " Severe Accident Risks: An Assessment for Five U.S. Nuclear Power Plants," NUREG-1150, January 1991.
USNRC, "A Review of NRC Staff Uses of Probabilistic Risk Assessment," NUREG-1489, March 1994.
USNRC, " Individual Plant Examination Program: Perspectives on Reactor Safety and Plant Performance," NUREG-1560, December 1997.
USNRC, "The Use of PRA in Risk-Informed Applications," (Draft for Comment) NUREG-1602, April 1997.
USNRC, "PRA Procedures Guide," NUREG/CR-2300, January 1983.
USNRC, "Probabilistic Safety Analysis Procedures Guide," NUREG/CR-2815 Rev.1, August 1985.
USNRC, " Plan for Implementing Regulatory Review Group Recommendations," SECY-94-003, January 1994.
l Rev. 0 - July 1998 SRP 19-A22
.[
General-Review of the PRA
\\
Boiling Water Reactor Owners' Group, " Report to the Industry on PSA Peer Review Certification Process: Pilot Plant Results," January 1997.
Electric Power Research Institute, " Individual Plant Examination Review Guide," EPRI TR-100369, February 1992.
International Atomic Energy Agency, "IPERS Guidelines for the International Peer Review Service,"
' IAEA-TECDOC-832 Second edition, October 1995.
USNRC, " Individual Plant Examination: Submittal Guidance," NUREG-1335, August 1989.
USNRC,." Procedural and Submittal Guidance for the Individual Plant Examination of External Events (IPEEE) for Severe Accident Vulnerabilities," NUREG-1407, May 1991.
USNRC, "PRA Review Manual," NUREG/CR-3485,1985.
General - PRA Uncedainties and Sensitivity Studies Apostolakis, G.A., " Probability and Risk Assessment: The Subjectivist Viewpoint and Some Suggestions," Nuclear Safety,19(3), pages 305 - 315,1978.
Apostolakis, G.A. and Kaplan, S., " Pitfalls in Risk Calculations," Reliability Engineering, Vol. 2, pages 135 - 145,1981.
Kaplan, S.
and Garrick, B.J., "On the Quantitative Def' ition of Risk," Risk Analysis, Vol.1, pages m
11 - 28, March 1981.
Parry, G.W., and Winter, P.W., " Characterization and Evaluation of Uncertainty in Probabilistic Risk Analysis," Nuclear Safety, 22(1), pages 28 - 42,1981.
Proceedings of Workshop iin Advanced Topics in Risk and Reliability Analysis, Model Uncertainty:Its Characterization and Quantification, held in Annapolis, Maryland, October 20-22,1993, University of Maryland Press,1996.
Special Issue on Treatment of Aleatory and Epistemic Uncertainty, Reliability Engineering and System Safety, Vol. 54, nos 2 and 3, November / December 1996.
USNRC, "A Review of NRC Staff Uses of Probabilistic Risk Assessment," NUREG-1489 Appendix C.6, March 1994.
USNRC, " Sensitivity Analysis Techniques: Self Teaching Curriculum," NUREG/CR-2350, June 1982.
USNRC, " Approaches to Uncertainty Analysis in Probabilistic Risk Assessment," NUREG/CR-4836, January 1988.
A SRP 19-A23 Rev. 0 - July 1998
General - Use of PRA for Risk Ranking USNRC, " Measures of Risk Importance and Their Applications," NUREG/CR-3385, July 1983.
Vesely, W.E., "The Use of Risk Importances for Risk-Based Applications and Risl:-Based Regulation,"
in proceedings of PSA '96, Park City Utah, September 1996.
Initiating Events Electric Power Research Institute, "ATWS-A Reappraisal, Part 3, Frequency of Anticipated Transients," EPRI NP-2330,1982.
Nuclear Safety Analysis Center, " Loss of Offsite Power at U.S. Nuclear Power Plants Through 1991,"
NSAC-182, March 1992.
USNRC, " Evaluation of Station Blackout Accidents at Nuclear Power Plants," NUREG-IC32, June 1988.
USNRC, " Development of Transient initiating Event Frequencies for Use in Probabilistic Risk Assessments," NUREG/CR-3862, May 1985.
USNRC, "Modeling Time to Recovery and Initiating Event Frequency for Loss of Offsite Power Incidents at Nuclear Power Plants," NUREG/CR-5032, January 1988.
USNRC, "ISLOCA Research Program Final Report," NUREG/CR-5928, July 1993.
Accident Sequence Analysis (Event Trees)
USNRC, "PRA Procedures Guide," NUREG/CR-2300 Chapter 3.4, January 1983.
System Modeling Analysis (Fault Trees)
USNRC, " Fault Tree Handbook," NUREG-0492, January 1981.
USNRC, "PRA Procedures Guide," NUREG/CR-2300 Chapter 3.5, January 1983.
Dependent Failure Analysis USNRC, "PRA Procedures Guide," NUREG/CR-2300 Chapter 3.7, January 1983.
Determination of Success Criteria Brookhaven National Laboratory, "MAAP 3.0B Code Evaluation Final Report," FIN L-1499, October 1992.
Electric Power Research Institute, "MAAP Thermal-Hydraulic Quantification Studies," EPRI TR-100741, June 1992.
Rev. 0 - July 1993 SRP 19-A24
l I
t t
t.
p Electric Pcwer Research Institute, "MAAP BWR Application Guidelines," EPRI TR-100742, June
(
1992.
Electric Power Research Institute, "MAAP PWR Application Guidelines for Westinghouse and Combustion Engineering Plants," EPRI TR-100741, June 1992.
Fauske & Associates, Inc., "MAAP 3.0B Users Manual," March 1990.
USNRC, "RELAPS/ MOD 3 Code Manual," NUREG/CR-5535 Volumes 1-5, June 1990.
USNRC, " TRAC-PFl/ MOD 2 Code Manual," NUREG/CR-5673 Volumes 1-4, 1994.
t Westinghouse Electric Corporation, " Reactor Coolant Pemp Seal Performance Following Loss of All AC Power," WCAP-10541, Revision 1.
Use of Appropriate Data -
i Electric Power Research Institute, " Nuclear Plant Reliability: Data Collection and Usage Guides,"
'EPRI TR-100381, April 1992.
Idaho National Engineering Laboratory, " Emergency Diesel Generator Power System Reliability 1987-l' 1993," INEL-95/0035, Febmary 1996.
l Institute of Electrical and Electronics Engineers, " Guide to the Selection and Presentation of Electrical,
,'l(
Electronic and Sensing Component Reliability Data for Nuclear Power Generating Stations," IEEE-(.
STD-500 Rev.1,1984.
International Atomic Energy Agency, " Component Reliability Data for Use in Probabilistic Safety Assessment," IAEA-TECDOC-478, October 1988.
International Atomic Energy Agency, " Evaluation of Reliability Data Sources," IAEA-TECDOC-504, l
April 1989.
l International Atomic Energy Agency, " Survey of Ranges of Component Reliability Data for Use in Probabilistic Safety Assessment," IAEA-TECDOC-508, June 1989.
l l
T-Book, 3rd edition, :" Reliability Data of Components in Nordic Nuclear Power Plants," published by -
ATV Office, Vattenfall AB, Sweden,1992.
I USNRC, " Data Summaries of Licensee Event Reports on Pumps at U.S. Commercial Nuclear Power Plants," NUREG/CR-1025 Rev.1,1982.
l-USNRC, " Data Summaries of Licensee Event Reports of Valves of U.S. Commercial Nuclear Power Plants," NUREG/CR-1363,1982.
l
. USNRC, " Data Summaries of Licensee Event Reports of Selected Instrumentation and Control Components at U.S. Commercial Nuclear Power Plants, January 1,1976 to December 31 1981,"
NUREG/CR-1740,1984.
'(
l i
SRP 19-A25 Rev. 0 - July 1998
USNRC, " Data Sununaries of Licensee Event Reports of Inverters at U.S. Commercial Nuclear Power Plants, January 1,1976 to December 31 1982," NUREG/CR-3867,1984.
USNRC, " Data Summaries of Licensee Event Reports of Protective Relays and Circuit Breakers at U.S. Commercial Nuclear Power Plants, January 1 1976 to December 31 1983," Draft NUREG/CR-4126,1985.
Modeling of Conunon Cause Failures Idaho National Engineering Laboratory, " Common Cause Failure Data Collection and Analysis System," Draft INEL-94/0064, December 1995.
International Atomic Energy Agency, " Guidelines for Conducting Common Cause Failure Analysis in Probabilistic Risk Assessment," IAEA-TEC-DOC 648,1992.
USNRC, " Procedures for Treating Common Cause Failures in Safety and Reliability Studies,"
NUREG/CR-4780 Volumes 1 & 2, January 1988.
Modeling of Iluman Performance Chien, S.H., et. al., "Quantification of Hum a Error Rates Using SLIM-Based Approach," IEEE Fourth Conference on Human Factors and Power Plants,1992.
Electric Power Research Institute, " Systematic Human Action Reliability Procedure," EPRI NP-3583.
June 1984.
Electric Power Research Institute, " Operator Reliability Experiments Using Power Plant Simulators,"
EPRI NP-6937 Volumes 1-3, July 1990 (proprietary document - contact EPRI for availability).
Electric Power Research Institute, " Human Cognitive Reliability Model for PRA analysis," diaft EPRI RP-2170-3, December 1984.
Electric Power Research Institute, "An Approa.h to the Analysis of Operator Actions in Probabilistic Risk Assessment," EPRI TR-100259, June 1991 (proprietary document - contact EPRI for availability).
Electric Power Research Instituw. ' SHARP 1 - A Review of Systematic Human Action Reliability Procedure," EPRI TR-101711, December 1992 (proprietary document - contact EPRI for availability).
USNRC, " Handbook of Human Reliability Analysis with Emphasis on Nuclear Power Plant Applications," NUREG/CR-1278, August 1983.
USNRC, "A Procedure for Conducting a Human Reliability Analysis for Nuclear Power Plants,"
NUREG/CR-2254,1983.
USNRC, "The Use of Performance Shaping Factors and Quantified Expert Judgement in the Evaluation of Human Reliability: An Initial Appraisal," NUREG/CR-2986,1983.
USNRC, " SLIM /MAUD: An Approach to Assessing Human Error Probabilities Using Structured Rev. 0 - July 1998 SRP 19-A26
Expert Judgement," NUREG/CR-3518, Volumes 1 & 2, 1984.
USNRC, " Accident Sequence Evaluation Program - Human Reliability Analysis Procedure,"
NUREG/CR-4772, February 1987.
Sequence Quantification Institute of Electrical and Electronics Engineers, "IEEE Standard for Software Verification and
-Validation Plans," 1EEE Standard 1012-1986.
USNRC, "PRA Procedures Guide," NUREG/CR-2300 Chapter 6, January 1983.
USNRC, " Software Quality Assurance Program and Guidelines," NUREG/BR-0167, February 1993.
Modeling of Contal===nt Response and Changes in Large Early Release Frequency USNRC, " Risk Assessment of Severe Accident-Induced Steam Generator Tube Rupture," (Draft for Comment) NUREG-1570, May 1997.
USNRC, " Evaluation of Severe Accident Risks: Surry Unit 1," NUREG/CR-4551, Vol. 3, Rev.1, Part 1, October 1990.
USNRC, " Evaluation of Severe Accident Risks: Peach Bottom Unit 2," NUREG/CR-4551, Vol. 4, Rev.1, Part 1, December 1990.
[
USNRC, " Evaluation of Severe Accident Risks: Sequoyah Unit 1," NUREG/CR-4551, Vol. 5, Rev.1, Parts 1 and 2, December 1990.
USNRC, " Evaluation of Severe Accident Risks: Grand Gulf Unit 1," NUREG/CR-4551, Vol. 6 Rev.
1, Parts 1 and 2, December 1990.
USNRC,." Integrated Risk Assessment for the LaSalle Unit 2 Nuclear Power Plant: Phenomenology and Risk Uncertainty Evaluation Program (PRUEP)," NUREG/CR-5305, December 1990.
USNRC, " Evaluation of Severe Accident Risks: Zion Unit 1," NUREG/CR-4551, Vol. 7, Rev.1 March 1993.
USNRC, "An Approach for Estimating the Frequencies of Various Containment Failure Modes and Bypass Events," Draft NUREG/CR-6595, November 1997.
SRP 19-A27 Rev. 0 - July 1998
mummmmm---
O l
l I
l be O
O
l APPENDIX B
- (
INTEGRATED DECISIONMAKING Risk-informed applications are expected to require a process to integrate traditional engineering and probabilistic considerations to form the basis for acceptance. In order for this decisionmaking process to be effective in rendering accurate representations of plant safety and risk, the staff anticipates that licensees will have documented guidance to ensure consistent and defensible results. Such guidance would also allow staff reviewers to reconstruct the logic and events involved in the integration process.
This appendix discusses issues that the staff should address during reviews of the licensees' integrated decisionmaking process (sometimes referred to as the " expert panel" process).
a.
Areas of Review Staff reviewers are expected to evaluate proposed changes to the LB by taking into account both traditional and probabilistic engineering considerations. For each change, reviewers should evaluate the licensee's justification for the change and the process by which the results were obtained. In many j
pilot risk-informed applications, licensees have justified changes to the LB through the use of integrated
. decisionmaking panels (or expert panels) especially in cases where there are broad applications of PRA 1
and traditional engineering results over a large number of plant elements (SSCs, operator actions, etc.).
A review of the licensee's integrated decisionmaking process would ensure a better understanding of the reasons, assumptions, approaches, and information used to justify these changes.
[
b.
Review Guidance and Procedures Since the licensee's integrated decisionmakmg process is responsible for justifying the acceptability of the proposed changes to the LB, the staff anticipates that licensees will document the process in a relatively formal fashion. The staff may not routinely audit all of the licensee's findings or recommendations, but the documentation should exist to support such a review, and should be 4
maintained for the life of the plant or until such time at which the recommendations are invalidated by 2 later changes.
5 u
Staff expectations of the integrated decisionmaking process Reviewers should ensure that the licensee's decisionmaking process contains the following attributes:
t The process should be well-defined, systematic, repeatable, and scrutable. This process should be technically defensible and should be sufficiently detailed to allow an independent party to reproduce the major results.
Deliberations should be application-specific. The objectives, proposed for the integrated decisionmaking process for a particular application (particularly, how the results are to be utilized) should be well defined and relevant to the given application.
Membership in the decisionmaking team should include experienced individuals with demonstrated skills and knowledge in relevant engineering disciplines (depending on the (q -
application), plant procedures and operations, plant systems (including operational history),
SRP 19-B1 Rev 0 - July 1998
~
system response and dependencies, operator training and response, details of the plant-specific PRA, and regulatory guidance.
The decisionmaking team should have been advised of the specifics of all proposed changes and the relevant background information associated with the licensing action. In addition, since the judgement will depend, in part, on the results of a risk analysis, it is important that all team members be provided with an interpretation of the results of the risk model and the potential limitations of that model.
The licensee's integrated decisionmaking process should take into account the principles and expectations described in Section II of this SRP chapter.
l In formulating the findings, the licensee should account for both probabilistic and traditional engineering considerations. This should include information from the risk analysis, traditional engineering evaluations and insights, quantitative sensitivity studies, operational experience and historical plant performance, engineering judgment, and current regulatory requirements.
Potential limitations of the risk model should be identified and resolved. In addition, the licensee should individually consider and evaluate all SSCs that are affected by the proposed application but are not modeled in the PRA, on the basis of guidelines similar to those provided later in this appendix or in Section C.2 of Appendix C to this SRP chapter. Finally, the licensee's conclusions should be sufficiently robust with regard to different plausible assumptions and analyses.
When findings or conclusions depend, in part, on the use of compensatory measures, the licensee should justify why the compensatory measures are an appropriate substitute for a proposed relaxation in current requirements. The compensatory measures should also become part of the plant's licensing basis.
Technical information basis for applications involving risk quantification or risk categorization -
The staff expects that the information base supplied to the integrated decisionmaking panel will be capable of supporting the findings that should be made in the context of the specific risk-informed application. For example, in risk quantification and risk categorization applications, the following guidelines should be applicable.
At least the Level 1 portion of the internal events PRA should be formulated in such a way as
=
to support quantification of a change in risk (ACDF and ALERF) and importance measures, and should provide qualitative information (e.g., minimal cutsets) adequate to support defense-in-depth findings.
There should be an inventory of plant response capability for probabilistically significant operating modes and initiating event categories (internal, external, flood, fire, seismic, etc.).
Given a full-scope Level 2 PRA, this requirement could be satisfied by an inventory of event tree success paths, with an indication of the mission success criteria, systems, and SSCs involved in each path. Lacking a full-scope Level 2 PRA, surrogate information should be developed for unanalyzed areas, along the lines described in Section III.2.2.2 of this SRP l
chapter. This requirement is necessary in order to show the safety functions performed by J
SSCs (or other plant elements) affected by the application.
j 1
Rev. 0 - July 1998 SRP 19-B2
' Causal models (determination of cause-and-effect relationships) should be developed to support h
an evaluation (qualitative or quantitative) of the change in risk as a function of the application.
This is necessary in order to relate the application to actual risk indices.
Documentation of inputs to the decisionmaking panel should be part of the process. Reviewers should verify the scope and depth of the information base, especially information supplied regarding modes and/or classes of initiators unanalyzed in the PRA.
Treatment of SSCs not modeled in the PRA PRAs do not model all SSCs involved in performance of safety functions for various reasons.
However, this should not imply that unmodeled SSCs are not important in terms of contributions to plant risk. For example, SSCs are omitted in some cases because the analysts take credit for programmatic activities that ensure a low failure frequency for that item or a short fault exposure time in the event that it does fail. In such cases, even though the PRA results will not reflect the SSC at all, it would be inappropriate to conclude that the programmatic activity is unimportant.
Consequently, one task of the integrated decisionmaking panel is to extrapolate from the PRA and other information r,ources to draw conclusions about SSCs that are not modeled in the PRA. This does not I
mean that the panel is to impute to the PRA high-level results that were not generated in the analysis; however, it does mean that if a success path is modeled in the PRA, the panel is justified in reasoning that unmodeled SSCs in that path are relied upon, if items were screened from the PRA, the panel should be aware of the screening process, in order to avoid violating the basis for the screening.
f For SSCs not modeled in the PRA, reviewers should verify that the decisionmaking panel has performed the following tasks:
Review the PRA assumption base for instances in which initiators were screened out on the basis of credit for SSCs affected by the application.
Review plant operating history for initiating events that might have been prevented by the proposed application.
Review plant operating history for failures of mitigating system trains attributable to events that might have been prevented by the proposed application.
Review accident sequence modeling for instances in which early termination of the analysis obscured challenges to affected SSCs that would normally come into play later than the termination point.
Possible dispositions of the above tasks include the following results:
The item will not affect initiating event frequency or mitigating system performance under reasonably foreseeable circumstances, and the proposed change is warranted.
Although unmodeled, the item already receives and will continue to receive programmatic
- attention commensurate with its significance. In cases where reduced commitments are proposed, adequate justification is provided for this reduction.
_ (_,
".g SRP 19-B3 Rev. 0 - July 1998
The item does not currently receive sufficient programmatic attention, and may be subject to tighter controls.
Reviewers should verify that the safety significance of SSCs not modeled in the PRA (but affected by the proposed application) are appropriately characterized and justified.
Addressing limitations of the risk analysis One objective of the integrated decisionmaking process is to overcome certain limitations of the PRA.
However, this does not include substituting the analyst's judgment for essential PRA results. One reason for developing PRA models is that the complexity of many facilities makes judgment difficult in many contexts.
Generally, if the PRA highlights a plant vulnerability, this should be taken seriously and should not be discounted on the basis ofjudgment. If the analyst can show that the PRA representation of a vulnerability is invalid, then the PRA should be modified, and the licensee should work with the results of the revised PRA.
To address the issue of credit for unmodeled systems that would change a PRA result, the preferred method is to alter the PRA to take the credit. Reviewers should be aware that cases may potentially arise in which credit for an unmodeled system would be seriously complicated by issues of shared support systems, environmental conditions, or other factors such as spatial interaction issues or operator interaction dependencies.
To address the issue of making decisions about SSCs that might influence plant response in unmodeled modes or to unmodeled initiators, the acceptable approach is to proceed on the basis of a structured representation of plant response that shows at least qualitatively the initiating events that may pertain, the systems available to respond to each, the functional dependencies of these systems, and the backups available in the event of failure of any particular SSC. While it is possible to accept program reductions for SSCs that are explicitly shown to play no role in unanalyzed modes, it is more difficult to accept reductions for components that do play a role in unanalyzed (e.g., shutdown) modes. For such instances, conservative methods will be considered prudent.
To address instances in which a PRA model exists but is considered misleading, caution is indicated.
An example would be to down-classify SSCs from a PRA result (i.e., state that a high risk contributor is actually a low contributor), on the basis of paneljudgment. It is not acceptable to place on the record both a PRA and a finding that clearly contradicts it. Although the panel is not expected to take the PRA as absolute truth, the test should be whether the record establishes a clear basis for a finding.
A technical argument that begins with the misleading PRA result and furNshes supplementary information sufficient to justify a relatively minor change to a PRA result, or a qualified interpretation of a PRA result, is satisfactory. A cursory technical argument leading to a conclusion that qualitatively contradicts a major PRA result is an unsatisfactory record.
c.
Evaluation Findings The following language (or language equivalent to this) should appear in the SER, or exceptions should be noted and explained:
The integrated decisionmaking process is appropriate. Appropriate information was available, Rev. 0 - July 1998 SRP 19-B4
suitable issues were raised, the disposition of these issues was systematic and defensible, and
[
the documentation of the findings is traceable and reviewable in principle, so that the basis for conclusions and recommendations is available for scrutiny and review.
The evaluation of risk significance represents appropriate consideration of probabilistic information, traditional engineering evaluations, sensitivity studies, operational experience, engineering judgment, and current regulatory requirements.
The technical information basis was adequate for the scope of the application. In particular, the analysis of success and failure scenarios was adequate to identify the roles played by the SSCs affected by the applicatien, the quantification of the frequency of these scenarios was adequate to establish the safety significance of the SSCs, and the causal models were adequate to establish the effects of the proposed changes in the program.
The safety significance of components affected by the proposed application but not modeled in the PRA was evaluated in a systematic manner. This included a search of components that might contribute to initiating event occurrence, mitigating system components that were not modeled in the PRA because their failure was not expected to dominate system failure in the baseline configuration, and components in systems that do not play a direct role in accident mitigation but do interface with accident mitigating systems.
The process applied by the licensee to evercome limitations of PRA was appropriate. Where decisions were made that do not follow straightforwardly from the PRA, a technical basis was previded that shows how the PRA information and the supplementary information validly combine to support the finding. No findings contradict the PRA in a fundamental way.
Y.
SRP 19-B5 Rev. 0 - July 1998
o APPENDIX C
)
V' CATEGORIZATION OF PLANT ELEMENTS WITII RESPECT TO SAFETY SIGNIFICANCE For several proposed applications in risk-informed regulation, one of the principal activities is the categorization of SSCs and human actions with respect to their safety-significance. This appendix discusses how to review approaches that may be used in this entegorization process.
The first review consideration is the definition of safety-significance as it applies to SSCs and human actions for a specific application. A related, but not identical concept, is that of risk significance. For example, an individual SSC can be identified as being risk-significant if it can be demonstrated that its failure or unavailability contributes significantly to the mea.sures of risk (e.g., CDF and LERF). Safety significance, on the other hand, can be thought of as being related to the role the SSC or human action plays in preventing the occurrence of the undesired end state. Thus, the SSCs and human actions considered when constructing the PRA model have the potential to be safety significant, since they play a role in preventing core damage or large early release. These SSCs and human actions may include those that do not necessarily appear in the final quantified model because they have initially been screened, are assumed to be inherently reliable, or have been truncated from the solution of the model.
In addition, there may be SSCs or human actions not modeled in the PRA that have the potential to be safety significant because they play a role in preventing core damage or large early release.
In reviewing the categorization, it is important to recognize its underlying purpose. Categorization is generally intended to sort the SSCs or human actions into two general groups; those for which some O
change is proposed, and those for which no change is proposed. It is the potential impact of the application on the particular SSCs and human actions and on the measures of risk which ultimately determines which SSCs and human actions should be regarded as safety-significant. Since different applications impact different SSCs and human actions, it is reasonable to expect that the categorization could be different for different applica: ions. Thus, the question being addressed by the application is, for which groups of SSCs and human actions can the change be made such that there will be no more than an insignificant increase in the risk to the health and safety of the public. This impact on overall risk should be related back to the criteria for acceptable changes in the risk measures identified in RG 1.174. It is those groups for which changes can be made that satisfy these criteria that can be regarded as low safety-significant in the context of the specific application. Thus, the most appropriate way to address the categorization is through a requantification of the risk measures. However, the feasibility of performing such risk quantification has been questioned for those applications for which a method for evaluating the impact of the change on SSC unavailability is not obviously available.
In such instances, an acceptable alternative to requantification of risk is to categorize SSCs and human actions using an integrated decisionmaking process (such as the use of an Expert Panel), with PRA importance measures as input. This appendix discusses the issues that reviewers should address for this approach Section C.1 discusses the tecimical issues associated with the use of PRA importance measures, and Section C.2 discusses the use of the importance measures by the decisionmaking panel.
C.1 Use ofImnortance Measures a.
Areas of Review A
)
In the implementation of the Maintenance Rule and in many industry guides for the risk-informed
, %/
/
SRP 19-C1 Rev. 0 - July 1998
applications, the measures most commonly identified for use in the relative risk ranking of SSCs and human actions include the Fussell-Vesely Importance, Risk Reduction Worth, and Risk Achievement Worth. However, in using of these importance measures for risk-informed applications, several issues should be addressed. Most of these issues relate to technical problems that can be resolved through the use of sensitivity studies or appropriate quantification techniques, as discussed in detail later in this section. In addition, there are two issues that reviewers should ensure have been adequately addressed, namely i) that risk rankings apply only to individual contributions and not to combinations or sets of contributors, and ii) that risk rankings are not necessarily related to the risk changes which result from ti.ose contributor changes. When correctly applied and interpreted, component-level importance measures can provide valuable input to the integrated decisionmaking process.
b.
Review Guidance and Procedures Risk ranking results from a PRA can be affected by many factors, the most important being the model assumptions and techniques (e.g., for modeling of human reliability or common cause failures), the data used, or the success criteria chosen. Reviewers should therefore evaluate the licensee's PkA as part of the overall review process. Appendix A to this SRP chapter presents guidance for this review.
In addition to using a PRA of appropriate quality for the application, the licensee should demonstrate the robustness of risk ranking results for conditions and parameters that might not be addressed in the base PRA. Therefore, when importance measures are used to group components or human actions as low safety-significant contributors, the information to be provided to the integrated decisionmaking process should include sensitivity studies and/or other evaluations to demonstrate the sensitivity of the ranking results to the important PRA modeling techniques, assumptions, and data. In assessing this information, reviewers should consider the following issues:
Different risk metrics: Reviewers should ensure that the licensee's ranking process adequately considered risk in terms of both CDF and LERF.
Completeness of risk model: Reviewers should ensure that, when determining safety significance contributions using an internal events PRA, the licensee also considered external events, as well as shutdown and low-power initiators, either by PRA modeling or by the integrated decisionmaking process (as detailed in Section C.2 and Appendix B to this SRP chapter).
Sensitivity analysis for component data uncertainties: The licensee should have addressed the sensitivity of component categorizations to uncertainties in the parameter values. Reviewers should be satisfied that SSC categorization is not affected by data uncertainties.
Sendtivity analysis for common cause failures: CCFs are modeled in PRAs to account for dependent failures of redundant components within a system. As discussed in Appendix A to this SRP chapter, CCF probabilities can impact PRA resuhs by enhancing or obscuring the importance of components.
This should be addressed by the review. A component may be ranked as a high risk contributor mainly because of its contribution to CCFs, or a component may be ranked as a low risk contributor mainly because it has negligible er no contribution to CCFs. In RIR, removing or relaxing requirements may increase the CCF contribution, thereby changing the risk impact of an SSC.
Consideration of multiple failure modes: PRA basic events represent specific failure events and failure modes of SSCs. Reviewers should verify that the licensee performed the categorization by taking into account the combined effects of all associated basic PRA events (such as failure to start and Rev. 0 - July 1998 SRP 19-C2
- / q failure to run), including indirect contributions through associated CCF event probabilities.
iV)
Sensitivity analysis for recovery actions: PRAs typically model recovery actions especially for dominant accident sequences. Quantification of recovery actions typically depends on the time available to diagnose the situation and perform the action, as well as the adequacy of the licensee's training, procedures, and operator knowledge. Estimating the success probability for the recovery actions involves a certain degree of subjectivity. The concerns in this case stem from situations where very high success probabilities are assigned to a sequence, resulting in related components being ranked as low risk contributors. Furthermore, it is not desirable for the categorization of SSCs to be impacted by recovery actions that sometimes are mly modeled for the dominant scenarios. Sensitivity analyses can be used to show how the SSC categorization would change if recovery actions were removed.
Reviewers should ensure that the categorization has not been unduly impacted by the modeling of recovery actions.
Truncation limit: Reviewers should verify that the licensee set the sequence truncation limits low enough so that the truncated set of minimal cutsets or scetarios contains the significant contributors and their logical combinations for the application in question. Depending on the level of PRA detail (module level, component level, or piece-part level), this may translate into a truncation limit from 10 "
to 10-8 per reactor year.
Multiple component considerations: As previously discussed, importance measures are typically evaluated on the basis of individual SSCs or human actions. One potential concern that arises from this practice is that single-event importance measures have the potential to dismiss all elements of a system or group, despite the system or group having a high importance when taken as a whole. (Conversely,
]
there may be grounds for screening out groups of SSCs, owing to the unimportance of the systems of
-[Q which they are elements.) Two potential approaches are used to address the multiple component issue.
The first is to define suitable measures of system or group importance. The second is to choose appropriate criteria for categorization based on component-level importance measures. In both cases, it will be necessary for the licensee to demonstrate that the cumulative impact of the change has been adequately addressed.
While there are no widely accepted definitions of system or group importance measures, it is likely that some licensees will develop new system or group measures. If any are proposed, reviewers should ensure that the measures logically capture the impact of changes to the group. As an example of the issues that arise, consider the following. For front-line systems, one possibility would be to define a Fussell-Vesely type measure of system importance as the sum of the frequencies of sequences involving failure of that system, divided by the sum of all sequence frequencies. Such a measure would need to be carefully interpreted if the numerator included contributions from failures of that system as a result of support systems. Similarly, a Birnbaum-like measure could be defined by quantifying sequences involving the system, conditional on its failure, and summing up those quantities. This would provide a measure of how often the system is critical. However, the support systems again make the situation more complex. To take a two-division plant as an example, front-line failures can occur as a result of failure of support division A in conjunction with failure of front-line division B. Working with a figure of merit determined by the " total failure of support system" would miss contributions of this type, in the absence of appropriately defined group level importance measures, reliance should be made on the integrated decisionmaking process to make the appropriate determination (see Section C.2).
(O)
Relationsidp of hnportance measures to risk changes: Importance measures do not directly relate to SRP 19-C3 Rev. 0 - July 1998 l
changes in risk associated with implementation of a set of changes proposed in an application. Instead, the risk impact is indirectly reflected in the choice of the value of the measure used to determine whether an SSC should be classified as being of high or low safety significance. This is a concern whether importances are evaluated at the component or group level. Therefore, the criteria for categorization into low and high significance should be related to the acceptance guidelines for changes in CDF and LERF. This implies that the criteria should be a function of the base case CDF and LERF, rather than being fixed for all plants. Thus, reviewers should determine how the choice of criteria
' relates to, and conforms with, the acceptance guidelines described in RG 1.174. If component level criteria are used, they should be established taking into account the fact that the allowable risk increase associated with the change should be determined on the basis of simultaneous changes to all members of the category.
c.
Evaluation Findings The SER should incorporate language equivalent to the following, and exceptions (if any) should be noted and explained.
The information provided to the integrated decisionmaking process with regard to determining the risk importance of contributors for a specific application is robust in terms of model inputs and assumptions including issues like the use of the use of both CDF and LERF, completeness of the risk model, and sensitivity of the results to data uncertainties, common cause failure modeling, modeling of human reliability, and truncation limits used.
The categorization addresses the effect of the change on groups of components in a way that is compatible with the risk acceptar.ce guidelines.
C.2 Role of Integrated Decisionmaking in Comnonent Categorization a.
Areas of Review While probabilistic importance analysis can provide valuable information regarding categorization of SSCs or human actions, it should be supported and supplemented by an evaluation based on traditional engineering considerations. This will require using the qualitative insights obtained from the PRA, and considering the maintenance of the defense-in-depth philosophy and sufficient safety margins. One important element of this integrated decisionmaking can be the use of an " expert panel." This section provides guidelines for reviewing the licensee's integrated decisionmaking process in the area of importance categorization, and it supplements the general guidance in Appendix B to this SRP chapter.
b.
Review Guidance and Procedures Identification of functions, systems, and components important to safety: The PRA can provide significant qualitative insights that emerge simply from considering whether and how systems are invoked in particular scenarios. If a front-line system is credited in success paths, it is "important" in some sense, and at least some of its SSCs must also be important in some sense, even if a given single-event importance measure does not reflect this. However, the real importance of a system is a function of whether alternative, diverse systems that could fulfill the same function. Those systems which are the only means of providing the function would be considered more important than those for which there are viable alternatives. A system that supports an important front-line system could also be considered important. This does not mean that all such systems cannot be candidates for relaxing Rev. 0 - July 1998 SRP 19-C4
W
. current requirements; however, it does mean that components in system trains credited in the PRA b)
(
should be explicitly considered during the integrated decisionmaking process.
Either by evaluating the licensee's documentation or by conducting an independent verification, reviewers should complete the following steps:
Identify all systems that are relied upon in plant response to an initiating event, whether explicitly modeled in the PRA or not (e.g., room cooling systems, and instrumentation and control systems associated _with indications rather than control may not be modeled), and identify the function (s) they perform or support.
Determine whether failure of components screened out on the basis that they are elements of
" unimportant" systems could affect a system that is relied upon in the plant's response to an initiating event.
Reviewers should then verify that at least some elements of each h the important systems identified above are considered " safety significant." If this is not the case, reviewers should ascertain what performance is allocated to these items in the PRA, and whether the programmatic activities allocated to these elements are commensurate with the given performance level. If a system is identified as being
' important, but none of its elements is, reviewers should carefully evaluate the licensee's justification.
- As an example, consider the case of a system that contains many redundant flowpaths. Single-event importance analysis will tend to dismiss the flowpaths one at a time, effectively dismissing the group as k
a whole. The focus of the above guidance is that the redundant flowpaths (considered as a subsystem,
/~N and recognizing the function they perform), are important and deserve some attention, even though conventional importance measures would not highlight them. However, in the case of redundant systems, the solution need not always be to assign every redundant path to the high-risk contributor category. In this example, especially if the paths are essentially similar, it is arguably necessary to consider common cause failure. Thus, a program that addresses common cause failure potential by monitoring component perfonnance may provide the necessary protection against loss of the function, w Jie still allowing a decrease in some level of commitment on the individual members of the group.
Verincation oflow safety 4 '.".cer.ce: In evaluating the qualitative risk-informed categorization, reviewers should consider the integrated decisionmaking process and criteria used by the licensee.
- In reviews of the licensee's determination of low safety significance for SSCs or operator actions, reviewers should verify that the licensee appropriately applied risk importance measures and accounted for the results of sensitivity studies. Reviewers should also verify that the licensee considered and compensated for factors such as potential inadequate scope and level of detail of the PRA (see Sections 111.2.2.2 and III.2.2.3 of this SRP chapter). Finally, reviewers should verify that, in categorizing an SSC or operator action as low safety significance, the licensee considered the defense-in-depth philosophy and available safety margins. Section 111.2.1 of this SRP chapter presents review guidance on these topics.
For SSCs not modeled in the PRA, reviewers should verify that the licensee's process determined that the following conditions apply for each SSC that has been proposed as a candidate for relaxation or removal of current requirements:
p The SSC does not perform a safety function, or does not perform a support function to a safety function, or does not complement a safety function.
SRP 19-C5 Rev. 0 - July 1998
The SSC does not support operator actions credited in PRAs for either procedural or recovery
=
I actions.
The failure of the SSC will not result in the eventual occurrence of a PRA initiating event.
The SSC is not a part of a system that acts as a barrier to fission product release during severe accidents.
The failure of the SSC will not result in unintentional releases of radioactive material even in the absence of severe accident conditions.
If any of the above conditions apply, or if SSC performance is difficult to quantify, the licensee should have used a qualitative evaluation process to determine the impact of relaxing requirements on equipment reliability and performance. This evaluation should include identifying those failure modes for which the failure rate may increase, and those for which detection could become more difficult.
Reviewers should then verify that the licensee provided one or more of the following (or similar)
Justifications:
a qualitative discussion on how the change is consistent with the defense-in-depth philosophy and how the change maintains sufficient safety margins a qualitative discussion and historical evidence why these failure modes may be unlikely to occur a qualitative engineering discussion on how such failure modes could be detected in a timely fashion a discussion on what other requirements may be useful to control such failure rate increases a qualitative engineering discussion on why relaxing the requirements may have minimal impact on the failure rate increase l
c.
Evaluation Findings The SER should incorporate language equivalent to the following, and exceptions (if any) should be noted and explained:
The categorization of the SSCs or human actions has adequately captured their significance to safety, and has been performed in such a way that the potential impact of the proposed application results in at most a small increase in the risk to the health and safety of the public.
The input to the integrated decisionmaking process derived from importance measures has been utilized, taking into account the known limitations of importance calculations, and the results have been supplemented by appropriate qualitative considerations.
The integrated decisionmaking process explicitly recognized systems invoked in plant response to initiating events, and ensured that components within these systems are considered for programmatic attention in areas (IST, ISI, etc.) appropriate to their performance characteristics and the level of performance needed from them.
Rev. 0 - July 1998 SRP 19-C6
O O
bb O
e.ee, e.cvc,> o e<co<
lll{1lt lllllt llllll l
NUC WL E
P A
O EN S A A
H R L
I R
TO N
RI L E A
PL
,N AD R B D TS I
TI RA N
E 2YT E
0 E
US 5C S SS
,E 5O 5 M 0
3 M
0 0
I 0
0 S 1
S IO N
W1DIU1 AWONS2 SFCF 0 HNUON5 I-MRR5 NPEMC5 G1NA 0 T TTO6 O1 IC4 N7COI2 ONO1 N
5 T.P R
O OR LE 2 C
DO I
ER S SD B KS 1 D
1 C
M S G
M
?
T O
5 5
5 g
PO S F P
TI E
AR R GS M
N A N D O
S FS G EEM 4
SA 7
I PL AID ll