ML071940235

Enclosure 2 Clarification to Regulatory Guide 1.200, Revision 1

ML071940235
Person / Time
Issue date:	07/27/2007
From:	NRC/RES/DRASP
To:
Drouin M
Shared Package
ML071170047	List: ML071940235
References
RG-1.200
Download: ML071940235 (9)

Text

Clarification to Regulatory Guide 1.200, Revision 1 INTRODUCTION Regulatory Guide (RG) 1.200 describes one acceptable approach for determining whether the quality of the probabilistic risk assessment (PRA), in total or the parts that are used to support an application, is sufficient to provide confidence in the results such that the PRA can be used in regulatory decision making for light-water reactors.

Section C of RG 1.200 provides the regulatory position on the technical adequacy of the base PRA. One aspect in defining PRA technical adequacy is the treatment of sources of uncertainty and assumptions (those related to model uncertainty and those related to PRA scope and level of detail). The staffs position on this technical issue is discussed in several places in RG 1.200. While the staffs regulatory position in Revision 1 did not change (from Revision 0 or Draft Guide 1161), additional explanation is being provided to clarify the staffs regulatory position on the treatment of sources of uncertainty and assumptions as stated in Revision 1 to RG 1.200. The staff will incorporate this clarification into the next revision of RG 1.200.

American Society of Mechanical Engineers (ASME) standard RA-S-2002, Standard for Probabilistic Risk Assessment for Nuclear Power Plant Applications, dated April 5, 2002, sets forth requirements for PRAs used to support risk-informed decisions for commercial nuclear power plants. ASME has since issued two addenda (ASME RA-Sa-2003 and ASME RA-Sb-2005). Appendix A to RG 1.200, Revision 1, provides the staffs position on the requirements in this standard, which includes the treatment of assumptions and sources of uncertainty. The staff has No objection with clarification to the definitions in the standard and to certain requirements in the standard on assumptions and sources of uncertainty. (Note that the staffs regulatory position of No objection with clarification is defined as: The staff has no objection to the requirement; however, the requirement, as written, is either unclear or ambiguous; therefore, the staff has provided its understanding of the requirement.) While the staffs regulatory position in Revision 1 in Appendix A to RG 1.200 did not change (from Revision 0 or Draft Guide 1161), additional explanation is being provided to clarify the staffs regulatory position to the ASME standard as documented in Appendix A in Revision 1 to RG 1.200. The staff will incorporate this clarification into the next revision of RG 1.200.

The staffs view with regard to the treatment of sources of uncertainty and assumptions is as follows:

• Parameter uncertainties are addressed through the quantification process.

• Assumptions related to PRA scope and level of detail in the base PRA need to be identified and documented.

• The minimum requirement for the base PRA is that the sources of model uncertainty and related assumptions need to be identified and characterized: it is not necessary to evaluate every source of model uncertainty and every related assumption for the base PRA.

• The impact of the those sources of model uncertainty and related assumptions that can impact the PRA results used to support an application need to be evaluated in the context of that application.

Page 1 of 9

DISCUSSION OF THE ISSUE The staff believes that the analyst must have an understanding of the PRA model before using the PRA results in decision making. This understanding involves (1) the identification of the assumptions related to PRA scope and level of detail, and (2) the identification of the sources of model uncertainty and related assumptions and a characterization of their potential impact on the PRA. This understanding for the sources of model uncertainty and related assumptions is necessary to evaluate their impact on the results of the PRA used to support the decision being considered.

Therefore, for the base PRA (absent an application), the minimum requirement is for the analyst to (1) identify the assumptions related to PRA scope and level of detail, and (2) characterize the sources of model uncertainty and related assumptions, i.e., identify what in the PRA model could be impacted and how. It is in the context of an application that an evaluation (i.e., knowing the extent to which the PRA model is affected by the sources of model uncertainty and related assumptions) is needed.

The staff believes that certain requirements in the ASME standard (1) go beyond what is needed as described above, and (2) as written, are too subjective and therefore difficult to demonstrate compliance.

In the ASME standard, high level requirement (HLR) HLR-QU-E states:

Uncertainties in the PRA results shall be characterized. Key sources of model uncertainty and key assumptions shall be identified, and their potential impact on the results understood.

One of the associated supporting requirements (SR), QU-E4, states:

Cat I -

PROVIDE an assessment of the impact of the key model uncertainties on the results of the PRA.

Cat II -

EVALUATE the sensitivity of the results to key model uncertainties and key assumptions using sensitivity analyses.

Cat III -

EVALUATE the sensitivity of the results to uncertain model boundary conditions and other key assumptions using sensitivity analyses except where such source of uncertainty have been adequately treated in the quantitative uncertainty analysis.

The definitions for key source of uncertainty and key assumption in the ASME standard are:

A key source of uncertainty: a source of uncertainty that is related to an issue for which there is no consensus approach or model and where the choice of approach or model is known to have an impact on the risk profile (e.g., total CDF and total LERF, the set of initiating events and accident sequences that contribute most to CDF and to LERF) or a decision being made using the PRA. Such an impact might occur, for example, by Page 2 of 9

introducing a new functional accident sequence or a change to the overall CDF or LERF estimates significant enough to affect insights gained from the PRA.

Key assumption: an assumption made in response to a key source of uncertainty in the knowledge that a different reasonable alternative assumption would produce different results, or an assumption that results in an approximation made for modeling convenience in the knowledge that a more detailed model would produce different results. For the base PRA, the term different results refers to a change in the plant risk profile (e.g., total CDF and total LERF, the set of initiating events and accident sequences that contribute most to CDF and to LERF) and the associated changes in insights derived from the changes in risk profile. A reasonable alternative assumption is one that has broad acceptance within the technical community and for which the technical basis for consideration is at least as sound as that of the assumption being challenged.

QU-E4 is requiring a quantitative assessment, particularly in Categories II and III, of the key sources of uncertainty and key assumptions. The requirement, QU-E4, is only for those assumptions related to sources of model uncertainty. There are two problems with the requirement as it is written.

(1) An evaluation of the sensitivity of the base PRA results to sources of model uncertainty and related assumptions does not need to be performed. The requirement does attempt to restrict the number of evaluations to only the key ones. However, the definition in the standard for key source of uncertainty and key assumption is too subjective, and does not resolve the problem in an objective and unambiguous manner.

The scope of sources of model uncertainty is not limited with definitions using such words as have an impact on the risk profile ... or a decision being made using the PRA or significant enough to affect insights gained from the PRA. Even a very minor change could be categorized as an impact on the PRA results (e.g., CDF).

Furthermore, it is not known, a priori, which of these sources are relevant to a decision, or even what decisions are to be made. Consequently, every source and related assumption could be considered key.

(2) While an evaluation of any source of model uncertainty or related assumption is not needed for the base PRA, the various sources of model uncertainty and related assumptions do need to be characterized so that they can be addressed in the context of an application. Therefore, the search for candidates needs to be fairly complete (regardless of capability category), because it is not known, a priori, which of the sources of model uncertainty or related assumptions could affect an application.

The goal is to identify and characterize those sources of model uncertainty and related assumptions in the base PRA model that have the potential to impact the PRA results to the extent that a decision being made using the PRA could be affected. This characterization is a qualitative assessment. Characterizing a source of model uncertainty or related assumption means identifying what it affects in the base PRA model. The range of effects can include issues, for example, associated with:

• the determination of success criteria that can influence the event tree structure and can introduce new accident sequences or affect the relative significance of accident sequences Page 3 of 9

• affecting specific basic events This characterization is essential to perform any sort of screening, whether qualitative or quantitative, to limit the scope of sources of model uncertainty and related assumptions that are potentially relevant to an application.

An objective way to identify the sources of model uncertainty and related assumptions relevant to an application is to establish numerical criteria rather than use ambiguous qualitative words as impact or significant enough. It is relatively straightforward to establish such criteria for a specific application. For example, for an application for a licensing base change using the acceptance criteria in RG 1.174, a key source or related assumption could be one that has the potential to change the degree to which the risk acceptance criteria are met, and therefore, could potentially influence the decision; therefore, a source of model uncertainty or related assumption could be considered key if it results in uncertainty regarding whether the result lies in Region II or Region I, or if it results in uncertainty regarding whether the result becomes close to the region boundary or not. It is the numerical acceptance criteria that provide the basis for identifying which sources of model uncertainty and related assumptions can potentially challenge the decision, and therefore, which ones would then be identified as key. In other words, it is in the context of an application that it is important to identify the key sources of model uncertainty and related assumptions.

STAFF CLARIFICATION In Revision 1 to RG 1.200, the staff elaborated on their concern with the ASME standard.

While the staff position has not changed (from Revision 0, Draft Guide 1161, or Revision 1), the staff believes further elaboration is needed. Clarifications are provided below to the text in the main body of RG 1.200, Revision 1, and to Appendix A of RG 1.200, Revision 1.

Main Body of RG 1.200, Revision 1 In several places in the main body of the RG, the staffs regulatory position addresses the treatment of assumptions and sources of uncertainty. The following clarifications are provided to the text:

• Regulatory Position C.1.2.6, Interpretation of Results An important aspect in understanding the base PRA results is knowing what are the sources of uncertainty and assumptions (see Footnote 7 to Section C.3.3.2 for definitions), and understanding their potential impact. Uncertainties can be either parameter or model uncertainties, and assumptions can be related either to PRA scope and level of detail or to model uncertainties. The impact of parameter uncertainties is gained through the actual quantification process. The assumptions related to PRA scope and level of detail are inherent in the structure of the PRA model. The requirements of the applications will determine whether they are acceptable. The impact of model uncertainties and related assumptions can be evaluated qualitatively or quantitatively. The sources of model uncertainty and related assumptions are characterized in terms of how they affect the base PRA model (e.g., introduction of a new basic event, changes to basic event probabilities, change in success criterion, introduction of a new initiating event). Some sources of model uncertainty and related assumptions may be screened from further consideration following an assessment of their potential significance by, for example, qualitative discussion or on the basis of Page 4 of 9

sensitivity analysis. When using sensitivity analyses to screen model uncertainties, model boundary conditions, and other assumptions, the sensitivity analyses address these both individually and in logical combinations. The combinations analyzed are chosen to account for interactions among the variables. The potential conservatism with the successive screening approach used for the analysis of specific scope items such as fire, flooding, or seismic initiating events is assessed.

• Regulatory Position C.1.2.7, Documentation Traceability and defensibility provide the necessary information such that the results can easily be reproduced and justified. The sources of information used in the PRA are both referenced and retrievable. The methodology used to perform each aspect of the work is described either through documenting the actual process or through reference to existing methodology documents. Assumptions related to PRA scope and level of detail are identified and documented. Sources of model uncertainty and related assumptions are (1) identified and their potential impact on the results assessed (see the discussion in Section C.1.2.6) and (2) documented along with their justification to the extent that the context is understood. The results (e.g., products and outcomes) from the various analyses are documented. A source of uncertainty is one that is related to an issue where there is no consensus approach or model (e.g., choice of data source, success criteria, reactor coolant pressure (RCP) seal LOCA model, human reliability model) and where the choice of approach or model is known to have an impact on the PRA results in terms of introducing new accident sequences, change the relative importance of sequences, or significantly affecting the overall CDF or LERF estimates that might have an impact on the use of the PRA in decision making.

• Regulatory Position C.1.3, Table 4, Interpretation of Results Level 1 --

• Identification of sources of model uncertainty and their potential impact on the base PRA results

• Understanding of the impact of the assumptions related to model uncertainties on the CDF and the identification of the accident sequence and their contributors Level 2 --

• Identification of sources of model uncertainty and their potential impact on the base PRA results

• Understanding of the impact of the assumptions related to model uncertainties on Level 2 results

• Regulatory Position C.3.3.2, Assessment of Assumptions and Approximations, Footnote 7 An assumption is a decision or judgment that is made in the development of the PRA model. An assumption is either related to a source of model uncertainty or is related to scope or level of detail.

An assumption related to a model uncertainty is made with the knowledge that a different reasonable alternative assumption exists. A reasonable alternative assumption is one that has broad acceptance within the technical community and for which the technical basis for consideration is at least as sound as that of the assumption being made.

Page 5 of 9

An assumption related to scope or level of detail is one that is made for modeling convenience.

An assumption is labeled key when it may influence (i.e., have the potential to change) the decision being made. Therefore, a key assumption is identified in the context of an application.

A source of model uncertainty is one that is related to an issue in which there is no consensus approach or model and where the choice of approach or model is known to have an effect on the PRA model (e.g., introduction of a new basic event, changes to basic event probabilities, change in success criterion, introduction of a new initiating event).

A source of model uncertainty is labeled key when it could impact the PRA results that are being used in a decision, and consequently, may influence the decision being made.

Therefore, a key source of model uncertainty is identified in the context of an application. This impact would need to be significant enough that it changes the degree to which the risk acceptance criteria are met, and therefore, could potentially influence the decision. For example, for an application for a licensing base change using the acceptance criteria in RA 1.174, a source of model uncertainty or related assumption could be considered key if it results in uncertainty regarding whether the result lies in Region II or Region I, or if it results in uncertainty regarding whether the result becomes close to the region boundary or not.

Appendix A to RG 1.200, Revision 1 Appendix A to RG 1.200, Revision 1 provides the staffs position on the requirements in ASME including Addenda A and B. In several places in Appendix A, the staffs regulatory position addresses the treatment of assumptions and sources of uncertainty within the requirements of the ASME standard. The following clarifications are provided to the staffs objections (as noted in Appendix A to RG 1.200, Revision 1, recommended additions to the ASME requirement are shown in bolded text (i.e., bold) and recommended deletions in strikeout text (i.e., strikeout)):

• Table A-1, Chapter 2, Key Assumption, Staff Issue

. . . . See the discussion on staff regulatory positions C.1.2.6, C.1.2.7, and C.3.3.2.

• Table A-1, Chapter 2, Key Source of Uncertainty, Staff Issue

. . . . See the discussion on staff regulatory positions C.1.2.6, C.1.2.7, and C.3.3.2.

• Table A-1, Chapter 2, Key Assumption and Key Source of Uncertainty, Staff Position and Resolution key assumption: an assumption made in response to a key source of uncertainty in the knowledge that a different reasonable alternative assumption would produce different results, or an assumption that results in an approximation made for modeling convenience in the knowledge that a more detailed model would produce different results. For the base PRA, the term different results refers to a change in the plant risk profile (e.g., total CDF and total LERF, the set of initiating events and accident sequences that contribute most to CDF and to LERF) and the associated changes in Page 6 of 9

insights derived from the changes in risk profile. A reasonable alternative assumption is one that has broad acceptance within the technical community and for which the technical basis for consideration is at least as sound as that of the assumption being challenged.

key source of uncertainty: a source of uncertainty that is related to an issue for which there is no consensus approach or model and where the choice of approach or model is known to have an impact on the risk profile (e.g., total CDF and total LERF, the set of initiating events and accident sequences that contribute most to CDF and to LERF) or a decision being made using the PRA. Such an impact might occur, for example, by introducing a new functional accident sequence or a change to the overall CDF or LERF estimates significant enough to affect insights gained from the PRA.

An assumption is a decision or judgment that is made in the development of the PRA model. An assumption is either related to a source of model uncertainty or is related to scope or level of detail.

An assumption related to a model uncertainty is made with the knowledge that a different reasonable alternative assumption exists. A reasonable alternative assumption is one that has broad acceptance within the technical community and for which the technical basis for consideration is at least as sound as that of the assumption being made.

An assumption related to scope or level of detail is one that is made for modeling convenience.

An assumption is labeled key when it may influence (i.e., have the potential to change) the decision being made. Therefore, a key assumption is identified in the context of an application.

A source of model uncertainty is one that is related to an issue in which there is no consensus approach or model and where the choice of approach or model is known to have an effect on the PRA model (e.g., introduction of a new basic event, changes to basic event probabilities, change in success criterion, introduction of a new initiating event).

A source of model uncertainty is labeled key when it could impact the PRA results that are being used in a decision, and consequently, may influence the decision being made. Therefore, a key source of model uncertainty is identified in the context of an application. This impact would need to be significant enough that it changes the degree to which the risk acceptance criteria are met, and therefore, could potentially influence the decision. For example, for an application for a licensing base change using the acceptance criteria in RA 1.1.74, a source of model uncertainty or related assumption could be considered key if it results in uncertainty regarding whether the result lies in Region II or Region I, or if it results in uncertainty regarding whether the result becomes close to the region boundary or not.

Page 7 of 9

• Table A-1, Table 4.5.8-2(e), HLR-QU-E, Staff Position and Resolution Uncertainties in the PRA Key Sources of model uncertainty and key related assumptions.

• Table A-1, QU-E1, Staff Position and Resolution IDENTIFY key sources of model uncertainty. (Note: In identifying the sources of model uncertainty, an acceptable process and generic list is provided in EPRI 1009652 as endorsed in NUREG-1855 (to be issued for public review and comment in August 2007)).

• Table A-1, QU-E2, Staff Position and Resolution IDENTIFY key assumptions made in response to model uncertainties the development of the PRA model (Note: In identifying the related assumptions, an acceptable process and generic list is provided in EPRI 1009652 as endorsed in NUREG-1855 (to be issued for public review and comment in August 2007)).

• Table A-1, QU-E4, Staff Issue Understanding of the model uncertainties and related assumptions is an essential aspect of the uncertainty analysis. In addition, . . . . definition of source of uncertainty.

It is only in the context of an application that the need exists to quantitatively assess the impact of a source of model uncertainty or related assumption.

• Table A-1, QU-E4, Staff Position and Resolution Cat I: PROVIDE an assessment of the impact . . . . Cat II: EVALUATE the sensitivity of the results . . . . Cat III: EVALUATE the sensitivity of the results . . . .treated in the quantitative uncertainty analysis.

Cat I, II, III For each source of model uncertainty and related assumption identified in QU-E1 and QU-E2, respectively, IDENTIFY how the PRA model is affected (e.g.,

introduction of a new basic event, changes to basic event probabilities, change in success criterion, introduction of a new initiating event) [Note (1)].

• Table A-1, QU-F4, Staff Position and Resolution DOCUMENT the characterization of the key assumptions and sources of model uncertainty and related assumptions (as identified in QU-E4)., such as: ..... spacial dependencies, etc.

• Table A-1, IE-D3, AS-C3, SC-C3, SY-C3, HR-I3, DA-E3, IF-F3, LE-G4, Staff Position and Resolution DOCUMENT the key assumptions and key sources of model uncertainty and related assumptions (as identified in QU-E1 and QU-E2) associated with . . .

Page 8 of 9

• Table A-1, 6.1, Peer Review, Purpose, Staff Issue

. . . A key objective of the peer review . . . this goal is to be clearly understood by the peer review team. Another key objective is for the peer review to assess the appropriateness of the assumptions related to sources of model uncertainty.

Page 9 of 9