ML003732128

Attachment 3-PRA Quality for Use in Regulatory Decision-Making Rev 0 7 2000 - S101221-03

ML003732128
Person / Time
Issue date:	07/01/2000
From:	NRC/RES/DRAA/PRAB
To:
References
-pr
Download: ML003732128 (24)

Text

Attachment 3' PRA Quality for Use in Regulatory.

Decision-Making Revision 0 July 2000

'This attachment provides high level acceptance criteria for PRA quality. This document is a draft report which is to be forwarded to the Commission for approval in mid-July.

PRA Quality for Use in Regulatory Decision-Making TABLE OF CONTENTS Section Page

1. IN T R O D U C T IO N..........................................................

1

2. RISK-INFORM ED ACTIVITIES...............................................

1

3. PRA QUALITY IN RISK-INFORMED REGULATION..............................

3

4. PRA TECHNICAL ACCEPTABILITY...........................................

5 4.1 PRA APPLICATION PROCESS...........................................

6 4.2 P RA S C O P E..........................................................

8 4.3 PRA ELEMENTS AND CHARACTERISTICS.................................

9 4 4 P E E R R EV IEW.......................................................

18 4.5 EXPERT PA N EL......................................................

20 LIST OF FIGURES Figure Page 1 Flowchart for Determining Needed PRA Technical Acceptability for a Specific Application

........................................................................6 LIST OF TABLES Table Page 1 Summary of Characteristics and Attributes of an Acceptable Use of a PRA in Risk-Informed A p p licatio n s..............................................................7 2

List of Item s Defining PRA Scope.............................................

8 3

Technical Elements of an Acceptable PRA......................................

9 4

Summary of Characteristics and Attributes of an Acceptable PRA..................

10 5

Summary of Desired Characteristics and Attributes of a Peer Review................

19 6

Summary of Technical Elements for an Expert Panel.............................

20 7

Summary of Desired Characteristics and Attributes of a Expert Panel to Use PRA Results

.......................................................................2 0 ii Revision 0, July 2000

PRA Quality for Use in Regulatory Decision-Making

1. INTRODUCTION Over the past 25 years a number of probabilistic risk assessments (PRAs) have been performed by both the U. S. Nuclear Regulatory Commission (NRC) and the nuclear industry. The scope, depth, and technical content of the PRAs have varied along with their purposes and uses. Results from PRAs have increasingly been used in the regulatory process, starting with generic safety issue prioritization and progressing to regulatory analysis in support of rulemaking and backfits and currently risk-informed regulation, which opens up the possibility of using PRA information in many ways not previously done.

Accordingly, the NRC issued a Policy Statement on the use of PRA in 1995, encouraging its use in all regulatory matters. Since that time, many uses have been implemented or undertaken, including the initiation of work to modify the reactor regulations and inspection program. As a result PRA is becoming a mainstream regulatory tool and, as such, is providing valuable input into the decision-making process regarding the design, operation and maintenance of plants.

Consequently, confidence in the information derived from a PRA is an important issue. That is, the scope of the analysis must be sufficiently broad and the accuracy of the technical content must be of sufficient rigor to justify the specific results and insights from the PRA that are used to support the decision under consideration. Typically PRAs do not address all possible contributions to risk, and therefore, the PRA results are not the sole input to decision-making. Consequently, when judging the acceptability of the PRA, it must be done in the context of the decision-making process, recognizing the role that the PRA results have in that process. In judging the acceptability (or quality) of the PRA, a definition of what is meant by quality is needed. PRA quality is determined by the following elements:

proper scope (e.g., operating modes, initiating events) proper level of detail (e.g., SSC, failure modes and mechanisms) proper analytical tools (e.g., verified and validated codes) proper technology (e.g., thermal-hydraulics, statistics, boolean algebra)

The following issues are addressed in this paper:

Risk-informed activities -

a description of the staff activities that utilize PRA results.

PRA quality in risk-informed regulation -

a description of the role of PRA standards and peer review processes in the NRC review of risk-informed applications.

PRA technical acceptability -

a description of the attributes used to establish the needed confidence of the risk insights used to support a given risk-informed activity.

2. RISK-INFORMED ACTIVITIES Each application may impose somewhat different requirements on the supporting PRA. Therefore, it is important to note what are the different risk-informed activities for which defining PRA technical acceptability is needed. Recent activities include the following:

Risk-Inform 10CFR Part 50: The NRC is evaluating the scope of the special treatment requirements and the technical requirements of 1 OCFR Part 50 and is considering revisions to them, as appropriate, based in part on risk insights obtained from PRAs.

Revision 0, July 2000 I

PRA Quality for Use in Regulatory Decision-Making Reactor Oversight Process: The NRC is increasing the focus of inspection on those activities with the greatest potential impact on safety. Inspection results will routinely be evaluated to determine the risk importance of the findings. Likewise, enforcement sanctions for violations of regulatory requirements will be better linked to the safety significance of inspection findings.

Operating Events Assessment: The NRC is continuing to evaluate the risk significance of operational events and trends in data in conjunction with risk assessments so that safety vulnerabilities can be identified, prioritized, communicated, and resolved on a timely basis.

License Amendments: The NRC has developed Regulatory Guide 1.174 that provides guidance on an acceptable analysis approach to support changes to a plant's licensing basis using plant specific risk information. Application specific regulatory guides have also been developed in the areas of inservice testing, inservice inspection, graded quality assurance and technical specifications. The staff is continuing its reviews of license amendments in these and other areas.

Risk-informed technical specifications: The NRC is continuing to work with industry on several initiatives to further develop risk-informed improvements to the technical specifications.

Examples of these initiatives include the replacement of fixed allowed outage times with a PRA based configuration risk management program, and a definition of preferred end-states for technical specification actions.

Maintenance rule: The NRC has required licensees to monitor the effectiveness of maintenance actions via the maintenance rule (50.65). A new section (a)(4) is being implemented (11-28-00) to help in controlling configuration-specific risks.

For each of the above activities, PRA results are used to determine the risk significance of structures, systems, and components (SSCs), the design and operational features critical to risk, and the events or scenarios important to risk. To make these determinations, the following are needed:

an evaluation of the core damage frequency (CDF), large early release frequency (LERF) and potential for late containment failure of the as-operated and as-built plant an evaluation of the change in CDF and LERF an identification and understanding of the major core damage sequences and their contributors an identification and understanding of the core damage states and phenomena contributing to the large early release of radionuclides and late containment failure an identification of the core damage states, containment system status, and physical phenomena that can contribute to late containment failure the ability to determine the impact of data and specific modeling assumptions on the plant's design and operating features in order to address uncertainties.

Revision 0, July 2000 2

PRA Quality for Use in Regulatory Decision-Making

3. PRA QUALITY IN RISK-INFORMED REGULATION In any risk-informed regulatory decision, the aim is to control the risk so that if any risk increases do occur as a result of a design or operational change, they are small as discussed in Regulatory Guide 1.174. A PRA provides only one part of the information used to make such a decision.

Furthermore, for many applications, the plant-specific PRA, depending on its scope, may provide only a portion of the risk insights. Because the reliance on PRA results will vary from decision to decision, the quality of the PRA must be judged in the context of the decision-making process, and on the way the PRA results are used. The quality of the PRA is what determines the confidence we can have in the results it generates. Whenever confidence in the results is lacking, some compensatory measures should be taken. These include: increased reliance on more traditional analysis; performance monitoring to make sure that any plant changes do not result in unexpected degradation of performance; and limiting the scope of the application so that fewer changes are made, and the potential for risk increase minimized. There will, therefore, generally be a trade-off between the benefit to be obtained from the application and the quality of the risk information.

The first trade-off is made during the initialAnalysis which integrates the risk insights with defense in-depth and safety margins considerations. The degree to which this integration occurs and may be needed is application dependent. Quantitative risk results from PRA calculations are often the most useful and definitive type of analysis performed, but they are generally supplemented by qualitative risk insights and traditional engineering analysis.

Qualitative risk insights include generic results which have been learned from the numerous PRAs that have been performed in the past decades, and from operational experience. For example, if one is deciding which motor operated valves in a plant can be subject to less frequent testing, the plant-specific PRA results can be compared with results from similar plants.

This type of comparison can give support to the licensee's analysis, and reduce the reliance of the staff review on the quality of the licensee PRA.

Traditional engineering analysis provides insight into available margins and defense in depth. In the example of the operational assessment of steam generator tubes discussed below, it is traditional engineering analysis that provides assurance of meeting the structural integrity and leakage criteria. With few exceptions, these assessments are performed without any quantification of risk.

In general, a risk-informed application will require some quantitative risk calculations using PRA methods. In some cases, the use of PRA will be extensive and will be crucial to the success of the application.

There are some proposals for real-time use of the PRA and associated risk management software as a tool to assess plant configuration. The more ambitious proposals involve the use of "risk meters." For example, the NRC and industry are cooperating on the risk informed standard technical specification (RI-STS) project. One element of this project is to replace the traditional limiting conditions for operation (LCO) action statements with a PRA based approach.

When a licensee encounters an LCO, rather than shutting down the plant, they would be authorized to use the plant PRA to determine an appropriate configuration which represents an acceptable level of risk. Applications of this type place a heavy burden on the quality.

There are some applications which, because of the nature of the proposed change, inherently. If the application has such a limitation, and the degree of the limitation, would determine the degree to which confidence is needed in the risk insights used in the decision-making, and subsequently, the level of staff review needed.

Revision 0, July 2000 3

PRA Quality for Use in Regulatory Decision-Making Such a conclusion can lead the staff to conduct a more limited review of the risk estimates, and therefore to place less emphasis on the quality of the PRA than might otherwise have been the case. The staff would also tend to focus its PRA review in specific areas of the PRA.

An example of this was the issue of BWR incremental power uprates. The staff examined the early submittals to determine whether there was reason for concern about undue increases in risk. Five potential areas of impact on risk were postulated and evaluated in the context of two plant applications. The staff concluded that the risk implications of the BWR power uprates was limited by the nature of the application. The staff also concluded that the one area requiring review in some cases is the possibility that increased power levels would result in less time for operator action during an accident. This is an example of how the extent of analysis required to support an application can be circumscribed in advance by examining the inherent risk limitation of the application.

Another example is risk-informed inservice inspection (RI-ISI). In this application, risk significance was used as one criterion for selecting pipe segments to be periodically examined for cracking. The staff spent a great deal of effort reviewing the topical reports and pilot plant submittals. Much of this effort was focused on the quantitative estimates of risk. During the review, however, it became clear that this level of emphasis of PRA results and PRA quality was not necessary. Since ISl involves examination of a sample of pipe segments to identify the onset of cracking problems, the success of the program was very tolerant to errors in the risk significance of the selected segments.

Therefore, the staff review of plant specific submittals referencing the RI-ISI topical will include only a limited scope review of PRA quality.

The implementation of the decision is a function of the confidence we can have in the results of the analysis.

One important factor that can be considered when determining the degree of implementation of the change, is the ability tomonitor the performance to limit the potential risk.

In many applications, the potential risk can be limited by defining specific measures and criteria which must be monitored subsequent to approval. When relying on performance monitoring, the staff must have assurance that the measures truly represent the potential for risk increase, and that the criteria are set at reasonable limits. Moreover, one must be sure that degrading performance can be detected in a timely fashion, long before a significant public health issue results. The impact of the monitoring can be fed back into the analysis to demonstrate how it supports the decision.

An example of this is the management of steam generator tube degradation. The NRC staff has been working with industry to approve licensee use of NEW-97-06, a guidance document for determining what tubes can be left in service and how frequently steam generators need to be inspected. The guidance in NEW-97-06 includes guidance for licensees to perform an operational assessment prior to restart from an outage. Any tubes which exceed certain limits must be repaired or removed from service. The licensee must determine whether the tubes left in service will meet structural strength and leakage criteria at the end of the cycle. If not, the licensee must take compensatory action, such as a mid-cycle inspection. At the end of the cycle, the licensee must perform condition monitoring, in which the actual condition is examined to determine whether the actual performance met the criteria. Any unfavorable deviation of the actual tube behavior from the predicted performance must be accounted for in subsequent operational assessment.

In this example, performance monitoring (condition monitoring) is relied upon to assure that any deviations from acceptance criteria are detected promptly. Moreover, the results are used to improve the analysis techniques to limit potential deviations in future cycles. The NRC staff has decided that the performance monitoring in NEW-97-06 is of sufficient quality and timeliness to Revision 0, July 2000 4

PRA Quality for Use in Regulatory Decision-Making assure acceptable risk from steam generator tube failure. Consequently, the staff concentrates more of its review and inspection effort on the results of the condition monitoring, rather than on the far more complex and time consuming review of pre-cycle predictions.

Finally, when implementing a decision, the licensee may choose to compensate for lack of confidence in the analysis by restricting the degree of implementation. This has been the technique used in several applications involving S.C. categorization into low or high safety significance. In general unless there is compelling evidence that the S.C. is low safety significant it is maintained as high safety significant. This requires a reasonable understanding of the limitations of the PRA.

Another example of risk limitation is the placing of restrictions on the application. For example, risk informed technical specification allowed outage time changes are accompanied by implementation of a configuration risk management program, which requires licensees to examine their plant configuration before voluntarily entering the approved condition.

The NRC review of an application will take all these factors into consideration. The review of PRA quality in particular will focus on those aspects that impact the results used in the decision, and on the degree of confidence required in those results.

4. PRA TECHNICAL ACCEPTABILITY This section discusses, at a relatively high level, the characteristics and attributes of the elements of a process to determine the role and technical acceptability of the PRA used to provide input to the decision. The framework within which the technical acceptability is determined is illustrated in Figure 1. The technical acceptability of risk insights is defined in the context of the specific application and the results used to generate the needed risk insights. The application dictates the scope of risk contributors to be considered when making a decision, and also the level of detail needed in the model.

Associated are the technical elements of the PRA along with their characteristics and attributes needed to ensure that the PRA results are technically correct.

However, not having all the desired scope or level of detail does not invalidate the use of the PRA model, but the results will either have to be supplemented by engineering judgement, or compensated for by including conservatisms, or limitations in the implementation of the decision.

For a given scope of PRA, the technical acceptability of the PRA can be determined by performing a peer review against a defined set of elements and characteristics. However, an expert panel may also be used for this purpose. These items are discussed in the following sub-sections.

Revision 0, July 2000 5

PRA Quality for Use in Regulatory Decision-Making A PLC O

RISK-INFORMED REACTOR APPLICATION j-CTIVITIES:

SRisk-informred 10CFRSO Reactor oversight process

• Operating event-assessment U

License amendments Risk-informed technical specifications ZISION-MAKING PROCESS Maintenance rule iRIK PRA Technical:

Acceptabilityi Scope to yield rskinsghts used in the application Out of PRA scope (compare scope of avalllble PRA to required coverage of risk contributors)

In PRA scope Needed PRA elements and acceptable chersteristica of each element to support PRA scope I

Peer review confirm PRA technical I

acceptabilityi FR Rult$fro s--r" Expert panel I

7ble inputI Expert panel No Docision process to integrates risk determine Importance and ts aceptble resolution I

I Figure 1 Flowchart for Determining Needed PRA Technical Acceptability for a Specific Application.

4.1 PRA APPLICATION PROCESS Applications will differ in the weight given to PRA results in the decision-making process. The weight given will depend on the scope of the PRA as well as its technical quality. For a given scope, the technical quality will determine the degree of confidence the decision-maker can have in the results and their role in the decision-making.

This role of the PRA is determined initially by its ability to produce the results required of the decision, and secondly by the degree of coverage of the risk contributors included in the risk metrics Revision 0, July 2000 6

PRA Quality for Use in Regulatory Decision-Making used in the decision. Given the role has been defined, the next step is to determine the technical acceptability of the PRA to support the results used, identify the differences, determine the importance of the differences, and determine an acceptable resolution forthe important differences.

The characteristics and attributes of this process are described below and summarized in Table 1.

Table 1 Summary of Characteristics and Attributes of an Acceptable Use of a PRA in Risk-Informed Applications The definition of the application identifies the SSCs and plant activities that are the subject of the application. When the application involves a decision on changes to the plant, the cause-effect relationship between the plant change and risk is assessed to identify how the plant change impacts the elements of the PRA model. The results from the PRA to be used in the decision-making process are identified. Therefore, to have confidence in the technical basis of the PRA for a given application, the scope and level of analysis that are needed to produce these results are identified.

In addition, the technical elements for generating these results along with their associated attributes are also identified.

Determination of the adequacy of PRA identifies differences between the existing PRA and the above defined PRA scope, level of detail, elements, characteristics, and attributes and the significance of these differences. It may be determined that the scope of the existing PRA does not provide the required risk information, (for example because it only addresses internal events at full power, and the decision algorithm involves risk from all modes of operation and all initiating events);

or its level of detail does not provide the necessary results (for example because the SSCs or plant operation affected by the decision are not modeled in the PRA); or it does not have the needed Revision 0, July 2000 Element Desired Characteristics and Attributes Definition of Identification of:

the Application SSCs, operator actions and plant operational characteristics affecting the decision for the application cause-effect relationships between the change and the above SSCs, operator actions and plant operational characteristics PRA results that can be used in the decision-making scope of risk contributors needed to support the decision level of analysis needed to support the decision elements of the PRA affected by the application, PRA characteristics and attributes needed to fully support the decision making process Determination determination of whether the existing PRA scope is sufficient to addres of the the risk contributors that impact the decision Adequacy of determination of whether the existing PRA level of detail, including PRA modeled SSCs is sufficient to provide the results necessary to support the decision identification of differences between PRA and the defined needed characteristics and attributes Resolution of Expand PRA to address insufficiencies and differences, or Differences Perform analyses with input from expert panel

• Note: a technical element that is common across the risk assessment application process is documentation S

7

PRA Quality for Use in Regulatory Decision-Making elements, characteristics and attributes (see Section 4.3) for the specific application. For the important differences, a process for resolution is determined (as discussed below).

Resolution of Differences identifies the process for resolution of identified important differences between the standard and the PRA. The resolution process either includes updating the PRA to include the important missing scope, elements and characteristics, as defined by the standard, or performing compensatory measures. These measures involve accounting for deficiencies by an expert panel (see Subsection 4.5).

4.2 PRA SCOPE The scope and level of analysis of a PRA plays an important role in determining the role PRA results can have in the decision-making regulatory activity. The scope of a PRA is defined by the following characteristics:

"* Degree of coverage of plant operating states (POSs) that define the plant's operating mode of concern: from full-power, to low-power, to shutdown modes of operation.

"* Degree of coverage of events, either internal or external to the plant boundary, that cause off normal conditions.

"* Level of characterization of risk:

Level 1 PRA that estimates the core damage frequency (given an event that challenge plant operation occurs).

Level 2 PRA that estimates the containment failure and radionuclide release frequencies (given a core damage state occurs).

Level 3 PRA that estimates the offsite consequences from a release, e.g., early and latent cancer fatalities (given a radionuclide release occurs).

For PRAs used in risk-informed activities, the scope and level of risk analysis are summarized in Table 2.

Table 2 List of Items Defining PRA Scope and Level of Risk Analysis ScopelLevel Desired Elements Definition I

POS full and low power, hot and cold shutdown Initiating internal transients LOCAs floods fires Events external seismic high wind others Risk Level 1: core damage frequency Characterization Level 2: large early release frequency and late containment failure Level 3: not required Revision 0, July 2000 8

PRA Quality for Use in Regulatory Decision-Making Plant operating states (POSs) are used to subdivide the plant operating cycle into unique states to allow modeling of subsequent accident initiating events. Operational characteristics (such as reactor power level; in-vessel temperature, pressure, and coolant level; equipment operability; and changes in decay heat load or plant conditions that allow new success criteria) are examined to identify those important to defining plant operational states. The important characteristics are used to define the states and the fraction of time spent in each state is estimated using plant specific information.

The risk perspective should be based on the total risk connected with the operation of the reactor which includes not only full power operation, but low power and shutdown conditions.

Therefore, to gain the maximum benefit from a PRA, the model should address all modes of operation.

Initiating events identifies the events that have the ability to challenge the condition of the plant.

These events include failure of equipment from either "internal plant causes" such as hardware faults, operator actions, floods or fires, or "external plant causes" such as seismic or high winds.

The risk perspective should be based on the total risk connected with the operation of the reactor which includes events from both internal and external sources. Therefore, to gain the maximum benefit from a PRA, the model should address both internal and external initiating events.

The risk characterization used in risk-informed applications are CDF, LERF (as a surrogate for early fatalities), and the consideration of late containment failure; therefore, to provide the risk perspective for use in decision-making, a Level 1 PRA is required.A Level 2 PRA may be needed (i.e., estimation of the other release beyond a large early release is not needed) if the estimation of LERF for the level 1 PRA is not sufficient to provide insights on application-specific issues, or if late releases can become important for the application.A Level 3 PRA will not be required for the majority of evacuations that do not involve the issue evacuation or other offsite emergency actions.

4.3 PRA ELEMENTS AND CHARACTERISTICS The technical elements of a PRA that provide acceptable results are summarized below in Table 3.

A PRA that is missing one or more of these elements would not be acceptable and, in fact, would not be considered a PRA.

Table 3 Technical Elements of an Acceptable PRA ScopelLevel J Technical Element*

of Analysis Level 1 Initiating event analysis Parameter estimation analysis Success criteria analysis

• Human reliability analysis Accident sequence analysis 0 Quantification analysis Systems analysis

• Interpretation of results Level 2 Plant damage state analysis

° Quantification analysis Accident progression analysis

• Interpretation of results

• Note: a technical element that is common across the scope of the PRA is documentation Each of the elements in Table 3 has associated with it characteristics and attributes needed to ensure that the results are technically correct. These characteristics and attributes are listed in Table 4.

Revision 0, July 2000 9

PRA Quality for Use in Regulatory Decision-Making Table 4 Summary of Characteristics and Attributes of an Acceptable PRA Element Desired Characteristics and Attributes PRA Full Power, Low Power and Shutdown Level I PRA (internal events - transients and LOCAs)

Initiating Event sufficiently detailed identification and characterization of initiators Analysis grouping of individual events according to plant response and mitigating requirements Success Criteria

• based on best-estimate engineering analyses applicable to the actual Analysis plant design and operation codes developed, validated, and verified in sufficient detail analyze the phenomena of interest be applicable in the pressure, temperature, and flow range of interest run by qualified and trained personnel Accident defined in terms of hardware, operator action, and timing requirements Sequence includes necessary and sufficient equipment (safety and non-safety)

Development reasonably expected to be used to mitigate initiators Analysis includes functional, phenomenological, and operational dependencies and interfaces Systems models developed in sufficient detail to:

Analysis reflect the as build as operated plant

• capture impact of dependencies include failure modes that impact the function of the system, including common cause failures, human errors, etc.

Parameter estimation of parameters associated with basic event probability Estimation models that account for plant-specific and generic data Analysis estimation includes a characterization of the uncertainty Human Reliability identification and definition of the human failure events that would Analysis result in initiating events or would impact the mitigation of initiating events quantification of the associated HEPs taking into account scenario (where applicable) and plant-specific factors and including appropriate dependencies Quantification 0 estimation of the CDF for modeled sequences that are not screened Analysis due to truncation, given as a mean value estimation of the accident sequences CDFs for each initiating event group truncation values set relative to the total plant CDF such that the frequency in not significantly impacted Revision 0, July 2000 10

PRA Quality for Use in Regulatory Decision-Making Table 4 Summary of Characteristics and Attributes of an Acceptable PRA Element Desired Characteristics and Attributes Interpretation of identification of the key contributors to CDF: initiating events, accident Results sequences, equipment failures and human errors identification of sources of uncertainty and their impact on the results understanding of the impact of the key assumptions* on the CDF and the identification of the accident sequence and their contributors Level 1 PRA (internal events - flooding)

Flood sufficiently detailed identification and characterization of:

Identification flood areas and SSCs located within each area Analysis flood sources and flood mechanisms the type of water release and capacity the structures functioning as drains and sumps verification of the information through plant walkdowns Flood Evaluation identification and evaluation of Analysis flood propagation paths flood mitigating plant design features and operator actions the susceptibility of SSCs in each flood area to the different types of floods elimination of flood scenarios well defined and justified screening criteria Quantification Identification of flooding induced initiating events on the basis of a Analysis structured and systematic process Estimation of flooding initiating event frequencies that reflect Modification of the Level 1 models to account for flooding effects Level 1 PRA (internal events - fires)

Screening all potentially risk-significant fire areas are identified and addressed Analysis screening criteria are defined and justified necessary walkdowns are performed to confirm the screening decisions

• screening process and results are documented

• unscreened events are subjected to appropriate level of evaluations (including detailed fire PRA evaluations as described below) as needed Fire Initiation all potentially significant fire scenarios in each unscreened area are Analysis addressed fire scenario frequencies reflect plant-specific features fire scenario physical characteristics are defined Fire Damage all potentially significant components are addressed Analysis all potentially significant damage mechanisms are addressed analysis addresses scenario-specific factors affecting fire growth, suppression, and component damage models and data are consistent with experience from actual fire experience as well as experiments Revision 0, July 2000 I I

PRA Quality for Use in Regulatory Decision-Making Table 4 Summary of Characteristics and Attributes of an Acceptable PRA Element Desired Characteristics and Attributes Plant Response all potentially significant fire-induced initiating events are addressed Analysis analysis reflects plant-specific safe shutdown strategy

"* potential circuit interactions which can interfere with safe shutdown are addressed

"* human reliability analysis addresses effect of fire scenario-specific conditions on operator performance Level 1 (external events)

Screening and credible external events (natural and man-made) that may affect the Bounding site are addressed Analysis

• screening and bounding criteria are defined and results are documented necessary walkdowns are performed non-screened events are subjected to appropriate level of evaluations Hazard Analysis the hazard analysis is site and plant-specific the hazard analysis addresses uncertainties Fragility Analysis fragility estimates be plant-specific for important SSCs walkdowns are conducted to identify plant-unique conditions, failure modes, and as-built conditions Level 1 Model important external event caused initiating events that can lead to core Modification damage and large early release are included external event related unique failures and failure modes are incorporated equipment failures from other causes and human errors are included.

When necessary, human error data is modified to reflect unique circumstances related to the external event under consideration 0 unique aspects of common causes, correlations, and dependencies are included the systems model reflects as-built, as-operated plant conditions the integration/quantification accounts for the uncertainties in each of the inputs (i.e., hazard, fragility, system modeling) and final quantitative results such as CDF and LERF the integration/quantification accounts for all dependencies and correlations that affect the results Level 2 PRA Plant Damage identification of the attributes of the core damage scenarios that State Analysis influence severe accident progression, containment performance, and any subsequent radionuclide releases grouping of core damage scenarios with similar attributes into plant damage states Revision 0, July 2000 12

PRA Quality for Use in Regulatory Decision-Making Table 4 Summary of Characteristics and Attributes of an Acceptable PRA Element Desired Characteristics and Attributes Severe Accident use of verified, validated codes by qualified trained users Progression assessment of the credible severe accident phenomena Analysis

• assessment of containment system performance establishment of the capacity of the containment to withstand severe accident environments assessment of accident progression timing, including timing of containment failure use of verified and validated codes run by qualified and trained personnel Quantification estimation of the frequency of different containment failure modes and Analysis resulting radionuclide source terms Interpretation of identification of the contributors to containment failure and resulting Results source terms understanding of the impact of the key assumptions* on Level 2 results Documentation Traceability and

• The documentation is sufficient to facilitate independent peer reviews defensibility

• The documentation describes' all of the important interim and final results, insights, and important sources of uncertainties

• Walkdown process and results are fully described Assumptions include those decisions and judgments that were made in the course of the analysis.

The following provide additional description of the characteristics and attributes in Table 4.

Level 1 PRA (transients and LOCAs)

Initiating event analysis identifies and characterizes those random internal events that both challenge normal plant operation during power or shutdown conditions and require successful mitigation by plant equipment and personnel to prevent core damage from occurring. Events that have occurred at the plant and those that have a reasonable probability of occurring (see Note 1) are identified and characterized. An understanding of the nature of the events is performed such that a grouping of the events into event classes, with the classes defined by similarity of system and plant responses (based on the success criteria), may be performed to manage the large number of potential events that can challenge the plant.

Note 1:

events that result in a bypass of containment may be screened if their frequency of occurrence is less than 10-7, and events that do not result in a bypass of containment may be screened if their frequency of occurrence is less than 10,6 Success criteria analysis determines the minimum requirements for each function (and ultimately the systems used to perform the functions) needed to prevent core damage (or to mitigate a release) given an initiating event occurs. The requirements defining the success criteria are based on acceptable engineering analyses that represent the design and operation of the plant under consideration. The criteria needed for a function to be successful is dependent on the initiator and Revision 0, July 2000 13

PRA Quality for Use in Regulatory Decision-Making the conditions created by the initiator. The code(s) used to perform the analyses for developing the success criteria are validated and verified for both technical integrity and suitability to assess plant conditions for the reactor pressure, temperature and flow range of interest, and accurately analyze the phenomena of interest. Calculations are performed by personnel qualified to perform the types of analyses of interest and are well trained in the use of the code(s).

Accident sequence development analysis models, chronologically, the different possible progression of events (i.e., accident sequences) that can occur from the start of the initiating event to either successful mitigation or to core damage. The accident sequences account for those systems and operator actions that are used (and available) to mitigate the initiator based on the defined success criteria and plant operating procedures (e.g., plant emergency and abnormal operating procedures and as practiced in simulator exercises). The availability of a system includes consideration of the functional, phenomenological and operational dependencies and interfaces between and among the different systems and operator actions during the course of the accident progression.

Systems analysis identifies the different combinations of failures that can preclude the ability of the system to perform its function as defined by the success criteria. The model representing the various failure combinations includes, from an as-built and as-operated perspective, the system hardware and instrumentation (and their associated failure modes) and the human failure events that would prevent the system from performing its defined function. The basic events representing equipment and human failures are developed in sufficient detail in the model to account for dependencies between and among the different systems, and to distinguish the specific equipment or human event (and its failure mechanism) that has a major impact on the system's ability to perform its function.

Parameter estimation analysis quantifies the frequencies of the identified initiators and quantifies the equipment failure probabilities and equipment unavailabilities of the modeled systems. The estimation process includes a mechanism for addressing uncertainties, have the ability to combine different sources of data in a coherent manner, and represents the actual operating history and experience of the plant and applicable generic experience as applicable.

Human reliability analysis identifies and quantifies the human failure events that can negatively impact normal or emergency plant operations.

The human failure events associated with normal plant operation include those events that leave the system (as defined by the success criteria) in an unrevealed, unavailable state. The human failure events associated with emergency plant operation include those events that, if not performed, do not allow the needed system to function.

Quantification of the probabilities of these human failure events are based on plant and accident specific conditions, where applicable, including any dependencies among actions and conditions.

Quantification analysis provides an estimation of the core damage frequency (CDF) given the design, operation and maintenance of the plant. This CDF is based on the summation of the estimated CDF from each initiator class. If truncation of accident sequences and cutsets is applied, truncation limits are set so that the overall model results are not impacted significantly and that important accident sequences are not eliminated. Therefore, the truncation limit can vary for each accident sequence. Consequently, the truncation value is selected so that the accident sequence CDF before and after truncation only differs by less than one significant figure.

Revision 0, July 2000 14

PRA Quality for Use in Regulatory Decision-Making Interpretation of results analysis entails examining and understanding the results of the PRA and identifying the important contributors sorted by initiating events, accident sequences, equipment failures and human errors. Methods such as importance measure calculations (e.g., Fussel-Vesely, risk achievement, risk reduction, and Birnbaum) are used to identify the contributions of various events to the model estimation of core damage frequency for both individual sequences and the model as a total. Sources of uncertainty are identified and their impact o the results analyzed. The sensitivity of the model results to model boundary conditions and other key assumptions is evaluated using sensitivity analyses to look at key assumptions both individually or in logical combinations. The combinations analyzed are chosen to fully account for interactions among the variables.

Level 1 PRA (internal floods)

Flood identification analysis identifies those plant areas where flooding could pose significant risk. Flooding areas are defined on the basis of physical barriers, mitigation features, and propagation pathways. For each flooding area, flood sources due to equipment (e.g., piping, valves, pumps), internal (e;.g., tanks) and external (e.g., rivers) water sources are identified along with the affected SACS. Flooding mechanisms are examined which include failure modes of components, human induced mechanisms, and other water releasing events. Flooding types (e.g., leak, rupture, spray) and flood sizes are determined. Plant walkdowns are performed to verify the accuracy of the information.

Flood evaluation analysis identifies the potential flooding scenarios for each flood source by identifying flood propagation paths of water from the flood source to its accumulation point (e.g.,

pipe and cable penetrations, doors, stairwells, failure of doors or walls). Plant design features or operator actions that have the ability to terminate the flood are identified. Credit given for flood isolation is justified. The susceptibility of each SSC in a flood area to flood-induced mechanisms is examined (e.g., submerge, spray, pipe whip, and jet impingement).

Flood scenarios are developed by examining the potential for propagation and giving credit for flood mitigation. Flood scenarios can be eliminated on the basis of screening criteria. The screening criteria used are well defined and justified.

Quantification analysis provides an estimation of the CDF of the plant due to internal floods.

Flooding induced initiating events that represent the design, operation and experience of the plant are identified and their frequencies quantified. The Level 1 models are modified and the internal flood accident sequences quantified: (1) modify accident sequence models to address flooding phenomena, (2) perform necessary calculations to determine success criteria for flooding mitigation, (3) perform parameter estimation analysis to include flooding as a failure mode, (4) perform human reliability analysis to account for PSFs due to flooding, and (5) quantify internal flood accident sequence CDF.

Modification of the Level 1 models are performed consistent with the characteristics for Level 1 elements for transients and LOCAs.

Level 1 PRA (internal fire)

Screening analysis identifies fire areas where fires could pose a significant risk. Fire areas which are not risk significant can be "screened out" from further consideration in the PRA analysis. Both qualitative and quantitative screening criteria can be used. The former address whether an unsuppressed fire in the area poses a nuclear safety challenge; the latter are compared against a bounding assessment of the fire-induced core damage frequency for the area. The potential for fires involving multiple areas should be addressed. Assumptions used in the screening analysis Revision 0, July 2000 15

PRA Quality for Use in Regulatory Decision-Making should be verified through appropriate plant walkdowns. Key screening analysis assumptions and results, e.g., the area-specific conditional core damage probabilities (assuming fire-induced loss of all equipment in the area), should be documented.

Fire initiation analysis determines the frequency and physical characteristics of the detailed (within-area) fire scenarios analyzed for the unscreened fire areas. The analysis needs to identify a range of scenarios which will be used to represent all possible scenarios in the area. The possibility of seismically-induced fires should be considered. The scenario frequencies should reflect plant-specific experience, and should be quantified in a manner that is consistent with their use in the subsequent fire damage analysis (discussed below). The physical characterization of each scenario should also be in terms that will support the fire damage analysis (especially with respect to fire modeling).

Fire damage analysis determines the conditional probability that sets of potentially risk-significant components (including cables) will be damaged in a particular mode, given a specified fire scenario.

The analysis needs to address components whose failure will cause an initiating event, affect the plant's ability to mitigate an initiating event, or affect potentially risk significant equipment (e.g.,

through suppression system actuation). Damage from heat, smoke, and exposure to suppressants should be considered. If fire models are used to predict fire-induced damage, compartment-specific features (e.g., ventilation, geometry) and target-specific features (e.g., cable location relative to the fire) should be addressed. The fire suppression analysis should account for the scenario-specific time required to detect, respond to, and extinguish the fire. The models and data used to analyze fire growth, fire suppression, and fire-induced component damage should be consistent with experience from actual nuclear power plant fire experience as well as experiments.

Plant response analysis involves the modification of appropriate plant transient and LOCA PRA models to determine the conditional core damage probability, given damage to the set(s) of components defined in the fire damage analysis. All potentially significant fire-induced initiating events, including such "special" events as loss of plant support systems, and interactions between multiple nuclear units during a fire event, should be addressed. The analysis should address the availability of non-fire affected equipment (including control) and any required manual actions. For fire scenarios involving control room abandonment, the analysis should address the circuit interactions raised in NUREG/CR-5088, including the possibility of fire-induced damage prior to transfer to the alternate shutdown panel(s). The human reliability analysis of operator actions should address fire effects on operators (e.g., heat, smoke, loss of lighting, effect on instrumentation) and fire-specific operational issues (e.g., fire response operating procedures, training on these procedures, potential complications in coordinating activities).

Level 1 PRA (external events)

Screening and bounding analysis identifies external events other than earthquake that may challenge plant operations and require successful mitigation by plant equipment and personnel to prevent core damage from occurring. The term "screening out" is used here for the process whereby an external event is excluded from further consideration in the PRA analysis. There are two fundamental screening criteria embedded in the requirements here, as follows: An event can be screened out either (i) if it meets the certain design criteria, or (ii) if it can be shown using a bounding analysis that the mean value of the design-basis hazard used in the plant design is less than 10"5/year, and that the conditional core-damage probability is less than 101, given the occurrence of the design-basis hazard. An external event that cannot be screened out using either of these criteria is subjected to the detailed-analysis.

Revision 0, July 2000 16

PRA Quality for Use in Regulatory Decision-Making Hazard Analysis characterizes non-screened external events and seismic events, generally, as frequencies of occurrence of different sizes of events (e.g., earthquakes with various peak ground accelerations, hurricanes with various maximum wind speeds) at the site. The external events are site specific and include both aleatory and epistemic uncertainties.

Fragility Analysis characterizes conditional probability of failure of important structures, components, and systems whose failure may lead to unacceptable damage to the plant (e.g., core damage) given occurrence of an external event. For important SSCs, the fragility analysis is realistic and plant-specific. The fragility analysis is based on extensive plant-walkdowns reflecting as-built, as-operated conditions.

Level I Model Modification assures that the system models include all important external-event caused initiating events that can lead to core damage or large early release. The system model includes external-event induced SSC failures, non-external-event induced failures (random failures),

and human errors. The system analysis is well coordinated with the fragility analysis and is based on plant walkdowns. The results of the external event hazard analysis, fragility analysis, and system models are assembled to estimate frequencies of core damage and large early release.

Uncertainties in each step are propagated through the process and displayed in the final results.

The quantification process is capable of conducting necessary sensitivity analysis and to identify dominant sequences and contributors.

Level 2 PRA (internal events)

Plant damage state analysis groups similar core damage scenarios together to allow a practical assessment of the severe accident progression and containment response resulting from the full spectrum of core damage accidents identified in the Level 1 analysis. The plant damage state analysis defines the attributes of the core damage scenarios that represent important boundary conditions to the assessment of severe accidents progression and containment response that ultimately affect the resulting source term. The attributes address the dependencies between the containment systems modeled in the Level 2 analysis with the core damage accident sequence models to fully account for mutual dependencies. Core damage scenarios with similar attributes are grouped together to allow for efficient evaluation of the Level 2 response.

Severe accident progression analysis models the different series of events that challenge containment integrity for the core damage scenarios represented in the plant damage states. The accident progressions account for interactions among severe accident phenomena and system and human responses to identify credible containment failure modes including failure to isolate the containment. The timing of major accident events and the subsequent loadings produced on the containment are evaluated against the capacity of the containment to withstand the potential challenges. The containment performance during the severe accident is characterized by the timing (e.g., early versus late), size (e.g., catastrophic versus bypass), and location of any containment failures. The code(s ) used to perform the analysis are validated and verified for both technical integrity and suitability. Calculations are performed by personnel qualified to perform the types of analyses of interest and well trained in the use of the code(s).

Source term analysis characterizes the radiological release to the environment resulting from each severe accident sequence leading to containment failure or bypass. The characterization includes the time, elevation, and energy of the release and the amount, form, and size of the radioactive material that is released to the environment. The source term analysis is sufficient to determine whether a large early release (significant, unmitigated releases from containment in a time frame Revision 0, July 2000 17

PRA Quality for Use in Regulatory Decision-Making prior to effective evacuation of the close-in population such that there is a potential for early health effects) or large late release occurs (significant, unmitigated release from containment in a time frame that allows effective evacuation of the close-in population such that early fatalities are unlikely).

Quantification integrates the accident progression models and source term evaluation to provide estimates of the frequency of radionuclide releases that could be expected following the identified core damage accidents. This quantitative evaluation reflects the different magnitudes and timing of radionuclide releases and specifically allows for identification of the large early release frequency (LERF) and the probability of a large late release.

Interpretation of results analysis entails examining results from importance measure calculations (e.g., Fussel-Vesely, risk achievement, risk reduction, and Birnbaum) to identify the contributions of various events to the model estimation of LERF and large late release probability for both individual sequences and the model as a total. In addition, the sensitivity of the model results to model boundary conditions and other key assumptions is evaluated using sensitivity analyses to look at key assumptions both individually or in logical combinations. The combinations analyzed are chosen to fully account for interactions among the variables.

Documentation Traceability and defensibility provides the necessary information such that the results can easily be reproduced and justified. The sources of information used in the PRA are both referenced and retrievable. The methodology used to perform each aspect of the work is described either through documenting the actual process or through reference to existing methodology documents.

Assumptions 2 made in performing the analyses are identified and documented along with their justification to the extent that the context of the assumption is understood. The results (e.g.,

products and outcomes) from the various analyses are documented.

4.4 PEER REVIEW A peer review process can be used to help establish that the scope, level of detail, and technical acceptability of a PRA are appropriate for selected applications. An acceptable peer review needs to be performed by qualified personnel, needs to be performed according to an established process that compares the PRA against desired characteristics and attributes, and needs to document the results including both strengths and weaknesses of the PRA.

The desired characteristics and attributes for an acceptable peer review of a PRA are described below and summarized in Table 5.

2Assumptions include those decisions and judgments that were made in the course of the analysis.

Revision 0, July 2000 18

PRA Quality for Use in Regulatory Decision-Making Table 5 Summary of Desired Characteristics and Attributes of a Peer Review Element Desired Characteristics and Attributes Team independent with no conflicts of interest Qualifications expertise in all the technical elements of a PRA including integration knowledge of the plant design and operation knowledge of the peer review process Peer Review documented process Process utilize a set of desired PRA characteristics and attributes review PRA methods review application of methods review key assumptions determine if PRA represents as-built and as-operated plant review results of each PRA technical element for reasonableness review PRA maintenance and update process Documentation describe the peer review team qualifications describe the peer review process document where PRA does not meet desired characteristics and attributes assess and document significance of deficiencies The team qualifications determines the credibility and acceptability of the peer reviewers. The peer reviewers can not give any perception of a conflict of interest, therefore, they are independent of the utility whose PRA is being reviewed and have not performed any technical work on the PRA.

The members of the peer review team have technical expertise in the PRA elements they review including experience in the specific methods that are utilized to perform the PRA elements. This technical expertise includes experience in performing (not just reviewing) the work in the element assigned for review. In addition, knowledge of the specific plant design and operation is essential.

Finally, each member of the peer review team is knowledgeable of the peer review process including the desired characteristics and attributes used to assess the acceptability of the PRA.

The peer review process includes a documented procedure to direct the team in evaluating the acceptability of a PRA.

The review process compares the PRA against the desired PRA characteristics and attributes. In addition to reviewing the methods utilized in the PRA, the peer review also determines if the application of those methods were done correctly. The PRA models are compared against the plant design and procedures to validate that they reflect the as-built and as-operated plant. Key assumptions are reviewed to determine if they are appropriate and if they have a significant impact on the PRA results. The PRA results are checked for fidelity with the model structure and also for consistency with the results from PRAs for similar plants. Finally, the peer review process examines the procedures or guidelines in place for updating the PRA to reflect changes in plant design, operation, or experience.

Documentation provides the necessary information such that the peer review process and the findings are both traceable and defensible. A description of the qualifications of the peer review team members and the peer review process are documented. The results of the peer review for each technical element and the PRA update process are described including those areas where the PRA do not meet or exceed the desired characteristics and attributes used in the review Revision 0, July 2000 19

PRA Quality for Use in Regulatory Decision-Making process. This includes an assessment of the importance of any identified deficiencies on the PRA results and potential uses and how these deficiencies were addressed and resolved.

4.5 EXPERT PANEL The PRA results may be integrated into the decision-making process by an expert panel. If an expert panel approach is elected, then there are certain characteristics and attributes that the expert panel needs to meet to be an acceptable alternative. With respect to the PRA, the primary responsibility of the expert panel is to establish the role that PRA results play in the decision, commensurate with the level of confidence in those PRA results. This requires establishing an appreciation of, and compensation for, the limitations of the model, which can be identified by comparison with the desired requirements for technical acceptability. PRA technical acceptability, as discussed above, may be achieved by performing a PRA that meets the desired characteristics and attributes defined for each technical element for the defined scope and level of analysis.

The technical elements needed in an expert panel are summarized below in Table 6.

Table 6 Summary of Technical Elements for an Expert Panel The desired characteristics and attributes to define an acceptable expert panel that are needed to support the identified applications are described below and summarized in Table 7.

Table 7 Summary of Desired Characteristics and Attributes of a Expert Panel to Use PRA Results Revision 0, July 2000 Desired Elements Decision-making process Technical information bases Incorporation of non-PRA Modeled Components Identification of Limitations Panel Member Qualifications Documentation Element Desired Characteristics and Attributes Decision-making

• decision-making process appropriate Process

• appropriate information available 0 evaluation of risk significance represents appropriate consideration of issues Technical adequate for the scope of the analysis Information Bases Incorporation of

• evaluate in a systematic manner the safety significance of items not non-PRA modeled in the PRA but affected by a proposed application (e.g.,

Modeled Items SSCs, modes of operation) 20

PRA Quality for Use in Regulatory Decision-Making Table 7 Summary of Desired Characteristics and Attributes of a Expert Panel to Use PRA Results Element Desired Characteristics and Attributes Identification of process applied by the licensee to overcome limitations of PRA is Limitations appropriate decisions made that do not follow straightforwardly from the PRA need a technical basis that shows how the PRA information and the supplementary information validly combine to support the finding, and no findings contradict the PRA in a fundamental way Panel Member diverse membership including PRA, engineering, operations, etc Qualifications wide knowledge of plant broad understanding of how changes in requirements and issues could affect SSC response training Documentation written procedure of the expert panel process report of the decision concluded by the panel and the basis for the conclusion The decision-making process is based on a written, systematic approach and shown to be appropriate for the decisions the panel is needed to render. The necessary technical information is made available to the panel and is examined to allow the applicable issues to be raised. The issues are disposed of using a systematic and defensible process, and documentation of findings made by the panel are traceable and reviewable. Any evaluation of the risk significance of issues appropriately consider probabilistic information, traditional engineering evaluations, sensitivity studies, operational experience, engineering judgment, and current regulatory requirements.

The technical information bases provides the necessary information for the panel to arrive at a defensible decision. This information is derived from various sources, including, for example, simplified or detailed engineering analyses, specific plant-operational expertise, and expert opinion, and shown to be adequate for the scope of the analysis. Therefore, the used technical information is sufficient to allow analysis (e.g., quantification) of both success and failure scenarios to (1) identify the roles played by the SSCs, and (2) establish the safety significance of the SSCs; and to identify causal models to be used to establish the effects of any proposed changes.

Incorporation of non-PRA modeled items involves evaluating the safety significance items not modeled in the PRA but affected by a proposed application. This systematic evaluation consists of searching for items that might contribute to initiating event occurrence, identifying mitigating system items that were not modeled in the PRA because their failure was not expected to dominate system failure in the baseline configuration, and recognizing items in systems that do not play a direct role in accident mitigation but do interface with accident mitigating systems.

Identification of limitations specifies those aspects in the PRA that decrease the level of confidence in the results, and consequently, to be addressed by the expert panel process. These deficiencies may exist because (1) an item was not modeled in the PRA, (2) an item was inappropriately modeled, or (3) lack of technology to adequately model in the PRA. The process used by the expert panel to resolve the deficiency is based the type of deficiency identified and includes (1) modeling the item in the PRA or accounting for the effects of the item by other means Revision 0, July 2000 21

PRA Quality for Use in Regulatory Decision-Making (e.g., using surrogate components), (2) revising the PRA model to appropriately model the item, or (3) soliciting and using expert opinion to resolve items involving a lack of technology. When a decision made by the panel that does not follow straightforwardly from the PRA, a technical basis is provided that shows how the PRA information and the supplementary information validly combine to support the finding. Further, no findings by the panel can contradict the PRA in a fundamental way.

Panel member qualifications identifies the needed credentials of the panel such that decisions reached by the panel are technically defensible. The panel involves diverse membership including PRA, engineering, Qperations, etc, Plant members have a wide knowledge of plant, and a broad understanding of how changes in requirements and issues could affect SSC response. Training is provided to the members for the activities they are required to perform.

This training is of sufficient depth such that the member can make informed decisions by combining multiple, diverse knowledge sets.

Documentation provides the necessary information such that the expert panel process and its findings are both traceable and defensible. The documentation includes a description of the qualifications of each expert panel member, the written procedures employed by the panel, and a report of any decisions made by the panel including the basis for the conclusions.

Revision 0, July 2000 22