ML20217D487
| ML20217D487 | |
| Person / Time | |
|---|---|
| Issue date: | 12/12/1997 |
| From: | Callan L NRC OFFICE OF THE EXECUTIVE DIRECTOR FOR OPERATIONS (EDO) |
| To: | |
| References | |
| SECY-97-287, SECY-97-287-01, SECY-97-287-1, SECY-97-287-R, NUDOCS 9803300035 | |
| Download: ML20217D487 (11) | |
Text
_ _ _ _ _ - _ _ - - - - - - _ - _
......................oo RELEASED TOTHE PDR o
$ =w%
3lo?5l%9 f/D f
,i date Wash
%...../
POLICY ISSUE (Notation Vote)
December 12.1997 SECY-97-287 F_QB:
The Commissioners FROM-L. Joseph Callan Executive Director for Operations
SUBJECT:
FINAL REGULATORY GUIDANCE ON RISK-INFORMED REGULATION:
POLICY ISSUES l
PURPOSE:
To request Commission approval of staff recommendations on policy issues related to the final version of risk-informed regulatory guidance.
SUMMARY
The staff has identified four policy issues during the preparation of the final version of Regulatory Guide DG-1061. The fourissues are: probabilistic acceptance guidelines relative i
to very small increases in core damage frequency and large early release frequency, treatment of uncertainties, acceptance gu.'delines for shutdown operations, and acceptance guidelines for temporary plant changes. This paper discusses and provides recommendations regarding each of these issues.
BACKGROUND:
For the past two years, the staff has had under development and review a set of regulatory guides (RG) and standard review plan (SRP) sections which describe how probabilistic risk Contacts:
Mark A. Cunningham, RES 415-6189 SECY NOTE:
To be made publicly available when the final SRM is Gareth Parry, NRR made available.
415-1464 gj
~
9803300035 971212 5
y # g [;ne 9
eor 8ecv g
97-287 R eDR i
f;g
The Commissioners -
2 assessment should be used in a risk'-informed process to change those portions of the current licensing basis (CLB) for nuclear reactors which require NRC review and approval. SECY - 077, dated April 8,1997, provided drafts of the staff's initial set of regulatory guidance documents. This set included a general regulatory guide providing basic principles and -
expectations (DG-1061) and associated general SRP (Chapter 19) and application-specific
- RG/SRPs for inservice testing, technical specifications, and graded quality assurance (RG only).. In a June 5,1997, staff requirement's memorandum (SRM), the Commission approved publication of the documents for public comment. A public workshop was held on August 11-13,1997, to provide an overview of the documents, answer questions, and obtain comments.
In addition to the comments received at the workshop, the staff has received approximately 40 sets of written comments. The public comment period for these documents expired on September 30,1997.
The staff has been participating with specific licensees in pilot programs in parallel with
' development and review of these regulatory guidance documents. These programs, initiated to help develop and assess the feasibility of risk-informed guidance, include review of a graded quality assurance program at the South Texas Project, an inservice testing program at the Comanche Peak Station, and technical specifications with the Combustion Engineering Owner's Group.
DISCUSSION:
The public comments received on the draft guidance documents, as well as interactions with industry in the pilot programs, have resulted in the identification of several policy issues requiring Commission decisions. Specifically, three policy issues have been identified for resolution as part of finalizing DG-1061. These issues are:
Probabilistic acceptance guidelines relative to very small increases in core damage p
frequency (CDF) and large early release frequency (LERF)
Treatment of uncertainties in comparison of PRA results with the acceptance guidelines Acceptance guidelines for shutdown operations in addition, a potential policy issue regarding acceptance guidelines for temporary plant changes has been identified. However, as discussed below, the staff recommends that consideration of this issue not be addressed as part of finalizing DG-1061, but be deferred for a possible future revision of this guide.
Each of these issues is discussed below, including a description of the issue, attemative approaches for resolution, and a staff recommendation. In order to meet the scheduled completion dates for the regulatory guidance documents, the staff intends to proceed with
. finalization of the documents presuming Commission approval of these recommendations. The Commission will be provided the final DG-1061 and its SRP for approval in December.1997.
. The other documents will be provided in March 1998.
j i
I i
a 3
Probabilistic Acceotance Guidelines Permittina Very Small CDF/LERF Increases Principle 4 in DG-1061 states that " proposed increases in risk and their cumulative effect, are small, and do not cause the NRC Safety Goals to be exceeded." The document then defines probabilistic acceptance guidelines intended to assure that this principle is met.
The probabilistic acceptance guidelines contained in DG-1061 include a baseline core damage frequency (CDF) guideline of 10" per reactor year and a baseline large early releases frequency (LERF) guideline of 104 per reactor year. These guidelines are used to establish a baseline value of CDF/LERF, above which further increases in these measures would not normally be permitted.
Plants with baseline CDF/LERF above these guidelines would be permitted to propose only changes which result in no change (i.e., are " risk-neutral") or a reduction in CDF/LERF. This guideline is intended to be compared with mean CDF/LERF estimates for all initiating events (i.e., internal events and external events during power and shutdown operations). The current CDF acceptance guideline provides a quantitative statement of the Commission's interest in preventing core damaga accidents and maintaining a balance between accident prevention and mitigation, as discussed in the Safety Goal Policy Statement, and as practiced in Commission-approved staff activities, including the application of the staff's regulatory analysis guidelines and the advanced reactor review process.2 As described in SECY-97-077, dated April 8,1997, the LERF guideline is derived from the j
Commission's Safety Goal Quantitative Health Objectives (QHOs) and provides a measure of accident mitigation and assurance that the OHOs are not exceeded. In effect, the guideline value for LERF is a surrogate for the Commission's OHO on early fatality risk. The Commission had also proposed in the Safety Goal Policy Statement a general performance guideline of 104 per reactor year for a large release of radioactive material to the environment. This guideline was proposed for staff evaluation and the results of the staff evaluation were reported in SECY-93-138, dated May 19,1993. Although work on defining a large release to be used with a frequency of 104per reactor year was stopped (as reported in SECY-93-138), increased Using the process described in DG-1061, proposed changes with CDF and/or LERF increases would not normally be considered by the staff if the baseline CDF or LERF are exceeded, whether or not they may be acceptable with respect to the other parts of the DG-1061 integrated decision making process (i.e., with respect to defense in depth and safety margins).
2 The CDF guideline was developed from the Commission's August 1986 Safety Goal Policy Statement, SECY-89-102, and the Commission's June 6,1990 SRM on SECY-89-102.
The policy statement discussed the Commission's intentions in preventing core damaging accidents (stated in qualitative terms), indicated the scope of the policy statement encompassed essentially all initiating events in reactors (except sabotage), and specified that mean values would be used for comparisons with the goals, with "due consideration of uncertainties." SECY-89-102 proposed a subsidiary, quantitative, core damage frequency objective, with a value of 1 x 104 per reactor year. The Commission's June 6,1990, SRM approved use of this subsidiary objective.
4 NRC rnanagement attention will be given to proposed changes that cause LERF to increase by more than.one percent of the guideline value of 10-5 per reactor year. This will help ensure that-the intent of the Commission's general performance guideline is considered in the review of
. proposed risk-informed changes requiring NRC approval.
The staffs primary concem relative to implementation of the current draft guidelines is that they result in the staff and licensees expending significant resources (analysis and review) on unimportant issues (i.e., CDF and LERF changes so small that they cannot be meaningfully estimated). This is in direct conflict with the goal of risk-informed regulation to focus licensee and staff resources on the most risk-significant issues. In addition, industry comment on DG-1061 has questioned the appropriateness of the baseline CDF guideline since it is significantly more restrictive than the Safety Goal Quantitative Health Objectives, since it would significantly limit the potential for many plants to propose changes (i.e., many plents would not meet the
. guideline when considering all initiating events), and since it would be a disincentive to licensees to quantify the risk from all initiating events (e.g., from shutdown operations).
The staff believes that the use of the baseline probabilistic acceptance guidelines for CDF and LERF is consistent with Commission policy and practice, and should be maintained.
However, in response to lessons leamed from pilot projects and to public comment, which suggested that it may be appropriate also to permit very small CDF/LERF increases independent of the baseline CDF/LERF, the staff has considered the following two attemative approaches to address this issue:
Reolace the current " risk-neutral" nolicy for olants with CDFs above 104oer reactor vear and/or LERFs above 10-5 oer reactor vaar with one that oermits very small calculated increases in these measures. This attemative would permit very smal! CDF/LERF increases without a detailed assessment of the baseline CDF/LERF. The principal benefits from this is increased opportunities for licensees to propose changes which have very little significance to CDF/LERF but could reduce regulatory burdens, making this more consistently with the philosophy of risk-informed regulation. However, it has the detriment of allowing plants above the CDF/LERF acceptance guidelines to move further away from the guidelines, albeit in very small increments.
in quantitative terms, the staff proposes that "very small" in this context mean an 4
increase of less than 104 per reactor year in core damage frequency or 10 per reactor year in large early release frequency. These values represent one percent of the baseline CDF/LERF guidelines, and are considered by the staff to be reasonable guideline values given typical calculated frequencies of core damage and LERF, typical calculated frequencies of important accident sequences, the guidance contained in the Commission's Regulatory Analysis Guidelines, and the margin between the CDF and LERF mean values and the QHOs, in addition, values of 104 per reactor year for l
changes of CDF, and corresponding changes of 104 for LERF, are considered to_ be representative of the limit of resolution for PRA models in the sense that minor changes in the quantification process (e.g., truncation limits, addition of recovery factors) can easily produce changes in the calculated risk metrics in excess of these values.
I i
5
- Retain the current " risk neutral
- coliev This alternative to retain the current risk neutral guideline is equivalent to permitting no increase in CDF/LERF for plants with baseline CDF/LERF above the draft guidelines of 104 per reactor year (for CDF) or 104 per reactor year (for LERF). This attemative implies that plants above the guidelines will get 4
no further away from the guidelines. However, this is, in the context of probabilistic risk assessments, an impractical definition, since licensees and the staff could not eliminate the possibility that many changes that appear " neutral" are actually extremeiy small increases.
The staff recommends that calculated CDF increases of less than 104 per reactor year, and calculated LERF increases of less than 10 per reactor year, be considered "very small" and be permitted for proposed changes independent of the baseline CDF/LERF. However, where there are indications (quantitative or qualitative) that the baseline CDF and/or LERF are above
{
I the guideline values, the staff will request licensees to consider and document why the proposed change cannot be accomplished with or without other changes in a fashion that is practical 8 and results in a risk reduction. In addition, the cumulative effect of such changes would need to be tracked to look at the long term trend in plant risk as a consideration in the decision process. Accordingly, the staff intends to monitor the cumulative effect of approved changes to prevent overall increases in CDF/LERF which would violate the staffs Principle 4.
Treatment of Uncertainties in Comoarison of PRA Results with the Acceptance Guidelines In the context of the risk-informed framework described in DG-1061, in which the PRA calculation is but one of several inputs to the decision-making, the purpose of performing a PRA and comparing the results with the acceptance guidelines is to help assess whether Principle 4, namely, that " proposed increases in risk, and their cumulative effect, are small and do not cause the NRC Safety Goals to be exceeded," is satisfied. This puts a different perspective on the role of the PRA, and the role of uncertainty analysis than would be the case for an approach in which the risk assessment would play a more dominant role, and there would be a need for greater confidence in the assessment of the risk. However, even for the less rigorous determination of risk required for the risk-informed process, in many cases, the decision on whether the principle has been met will not be straightforward, since uncertainties in calculated values and from unquantified factors can affect the staffs confidence as to whether the guidelines have been met. The degree to which one can have confidence that the principle has been satisfied should directly affect how much weight is given to this element of the integrated decision making.
SECY-97-221, dated September 30,1997, provides a description of the issues associated with the characterization of uncertainty in PRAs, and the comparison of these results with numerical acceptance guidelines, and discusses alternative approaches for making the comparisons. The formal approach that has been used in PRAs to characterize uncertainty is to develop a i
8 One approach for assessing the practicality of such changes is contained in the
. industry document NUMARC 91-04.
h
. probability distribution on the value of a risk metric, where the probability associated with a particular value represents a measure of the analysts' degree of belief that value is a bound on the true value. In SECY-97-221, three possible approaches were identified for comparison of such distributions with acceptance guidelines.
Use the mean values of the metrics and their increments in the comoarison with the acceptance guidelines The use of mean values is conceptually simple, and is consistent with classical decision-making. Furthermore, evaluation of the mean value incorporates consideration of those uncertainties explicitly captured in the model.
However, the mean, as with any other single estimate derived from a distribution, is a summary measure that does not fully take into account the information content of the probability distribution.
Use the oercentile of the distribution corresoondina to the value of the acceptance cuideline as the dearea of confidence that the coal has been met. This is an intuitively appealing approach to the use of the probability distribution. Mcwever, the form of the
- tails of the distributions is to some extent arbitrary, so use of high percentile values might give a false sense of assurance. Adoption of this attemative would require a policy decision on what level of assurance was acceptable. Historically, while no particular assurance level has been proven to be more acceptable than others, it is typical to see assurance levels of.95 as being characteristic of acceptability. In addition, since the guidelines proposed are based on the Commission's Safety Goals and subsidiary objectives, which were meant to be compared with mean values, this option would require a reevaluation of the gu;delines themselves.
A third alternative is to adoot the idea that there is some tolerance associated with meetina the coal. A oerformance obiective would be defined with which the mean value is to be comoared. and a hiaher acal. with which the 95th nercentile is comoared. This has the direct recognition that the guideline is not a simple go/no-go limit, and that there is a tolerance band rather than a simple limit. This approach would require a change in policy to determine the form of the acceptance goals, and in particular to establish the upper (percentile) goal. The approach suffers from the same concem as the other two options in that the shape of the distribution is still not being used, and the mean and 95th percentiles are sensitive to fluctuations in the tails of the distributions.
From a theoretical standpoint, there is no clear advantage to choosing any one of these approaches over the others. They are'all subject to the criticisms that the complete distribution is not being fully used, and that the form of the distributions for characterizing the ir.put uncertainties (particularly parameter uncertainties) is to some extent arbitrary, and that both the mean and the higher percentiles are sensitive to changes in the tails of the distributions.
However, as discussed in SECY-97-221, establishing a probability distribution that represents
' the uncertainty in PRA results is not straightforward. Even if it were possible for an analyst to i
- a 7-develop a characterization of the total uncertainty in PRA results using statistical measures.
such a characterization would, because of the variability in PRA modeling assumptions and approximations, necessarily be subjective.3 Because of the lack of standardization in the performance of PRAs, different analysts adopt different model assumptions and approximations, which make PRA results potentially more variable than they would be on the basis of design and operational differences. Furthermore, incompleteness uncertainties in particular are impossible to deal with statistically because they address unanalyzed contributors to risk.
it was suggested in SECY-97-221 that, in the context of understanding how the uncertainty impacts the decision-making process, it is often more valuable to the reviewer to separate the
]
facts (i.e., the evidence used) and the analysis of that evidence from the value judgements of the particular analyst. Thus an approach to the analysis of model uncertainty, for example, would be a) to identify the sources of uncertainty, b) to identify a reasonable set of attematives, and c) to understand how the risk changes under those different alternatives or assumptions.
Two approaches for characterizing the uncertainty in the results of a PRA for comparison with the numerical acceptance guidelines that represent the extremes in terms of formal statistical approaches are:
Construct a orobability distribution that encomoasses all sources of uncertainty for comoarison with one or more auidelines. This approach is perhaps the most attractive from a formal theoretical point of view. However, it would be very resource intensive for any analysis that involves model uncertainty. Furthermore, there is no way to address issues of incompleteness without performing some sort of analysis and, without some sort of standard approach, the input probability distributions will be analyst dependent, and therefore subjective, and thus any degree of confidence generated will also be subjective.
Address carametric uncertainty and any exolicit model uncertainties in the assessment of mean values. Perform sensitivity studiet to evaluate the imoact of usina altamate models for the orincioal imolicit model uncertainties. and use cuantitative analvses or qualitative analyses as necessary to address incompleteness as appropnate to the decision ar'd the acceptance guidelines. This is essentially the approach suggested in DG-1061, and has the major advantage that it is consistent with the state of the art. The method avoids the implicit effect of the value judgements of the analysts, and makes the evidence used in making the decision more visible in that it focuses attention on the assumptions and approximations made by the analysts. However, the approach is not -
formal, and might be considered as too subjective. In reality, it is not more so than the first approach, where the subjectivity is harder to see.
Given the perspective that the role of the uncertainty analysis is to help verify that Principle 4 is met, the second approach described above is recommended. The mean value (or other
7 3-8 appropriate point estimate if it can be argued to be close enough to the mean value)is appropriate for comparing with the acceptance guidelines. In recommending that the mean -
value should be used, the staff's overriding consideration is that the DG-1061 acceptance guidelines were established with the Commission's Safety Goals and subsidiary objectives in mind, and that those goals were meant to be compared with mean values. For the distributions generated in typical PRAs, the mean values typically corresponded to the region of the 70th to 80th percentiles, and coupled with a sensitivity analysis focused on the most important contributors to uncertainty, can be used for effective decision making. The sources of uncertainty related to modeling or incompleteness should be identified along with whether there are any reasonable attemate assumptions or missing contributions that could change the results significantly enough to change the assessment. In this approach, the role of the uncertainty analysis is essentially as a tool for analyzing the results of the PRA, to determine their robustness and to highlight possible areas of concern.- These possible areas of concem could be associated with dubious approximations or unfounded assumptions, or could reflect a genuine lack of knowledge where there is no clearly preferred modeling approach. However, it
. is in this role as an aid to communication that uncertainty analysis can be most valuable.
l 1
Decision making in light of these uncertainties thus requires weighing the different issues that can impact the decision when comparing one or two calculated numbers to a set of guidelines.
This implies that it is not just the numerical values of the various measures of risk and their changes that are important, but it is also important to understand what contributes to the results.
SECY-97-221 also stres' sed the need to consider the role of the uncertainty analysis in the i
context of the form of the acceptance guidelines and of the particular decision being made. It was suggested that for some decisions the uncertainty could be designed out of the decision-making processi For example, if the acceptance guidelines are modified as recommended, so that, for very small calculated increases in CDF/LERF, a detailed assessment of the baseline CDF and LERF is not required, the consideration of uncertainty, including incompleteness, can be restricted to that associated with the analysis of the changes in CDF and LERF. As another example, it might be possible to design the proposed change so that certain model uncertainties are of no concem.
Accordingly, the staff recommends that the basic approach for treating uncertainties contained in DG-1061 be retained in the final version of the regulatory guide, but be clarified to provide a better description of what the licensee should consider and address in his submittal to identify and account for the important sources of uncertainty. For "very small" risk increases as defined above, this would limit uncertainty analysis to that associated with the changes in CDF and LERF and the use of sensitivity analysis to test the changes in CDF and LERF against the acceptance guidelines.' For larger risk increases uncertainty and sensitivity analysis would also apply to the baseline CDF and LERF.
l 9
Acceptance Guidelines for Risks from Shutdown Operations Public comment on DG-1061 noted that conditions relating to the definition of large early release frequency can be quite different for shutdown conditions versus power operations.
Thus, the LERF definition developed using perspectives of full power accidents may be inapplicable for shutdown accidents.
This comment is consistent with the staff's current understanding of shutdown risk. For example, the reactor core's radionuclide inventory is quite different, such that the potential for early fatality consequences (which is a key factor in the current definition of LERF) is reduced relative to that in accidents initiated during power operations. In addition, containment mitigative features available during power operations may not be available during portions of shutdown operations. That is, containment engineered safety features (e.g., suppression.
- pools) or the containment itself may not be functional. In such circumstances, CDF and LERF are essentially equivalent.
The staff recommends that additional consideration of possible additional shutdown risk acceptance guidelines be undertaken as part of the research effort (beginning in FY 1999) to better understand risks from accidents in these plant conditions. The results of this work will be discussed with the ACRS, and, if appropriate, public comment and Commission approval sought for the use of any such guidelines. If developed and approved, these guidelines would be documented in a future revision of DG-1061.
In the interim, the staff recommends that the current CDF and LERF guidelines be maintained for both power and shutdown operations. However, if the proposed change involves equipment used in shutdown operations when containment functions are not available, licensees will have the flexibility to propose a reasonable definition for LERF considering the reduced radionuclide inventory or to rely solely on an assessment of core damage (i.e., CDFs below the 10-5 per reactor year LERF guideline). It is recommended that DG-1061 be revised to be consistent with this interim approach.
Acceotance Guidelines for Temoorary Plant Confiourations The probabilistic acceptance guidelines described in DG-1061 are measures of time-averaged CDFs and LERFs. That is, for example, the CDF guideline of 10"per reactor year refers to the average CDF over the year, considering all plant operations and configurations during that time period, in some situations (e.g., certain temporary plant configurations with equipment out of service), the conditional CDF (for the period of time that the plant is in such situations) may be considerably higher than the average. Comments received on the draft guidance suggest that j
1
- l0 m
g O
10 an additional set of guidelines may be appropriate to limit the conditional CDF and LERF in such circumstances. The staff has considered these comments and believes that they merit additional assessment. As such, the staff proposes to undertake a more detailed assessment of such guidelines.* This assessment will be discussed with the ACRS, and, if appropriate, public comment and Commission approval sought for the use of any such guidelines. If developed and approved, these guidelines would be documented in a future revision of DG-1061. In the interim, the staff recommends that only time-averaged guidelines be used, except as supplemented in DG-1065 for technical specification allowed outage times.
COORDINATION:
This paper has been coordinated with OGC, which has no legal objection.
RECOMMENDATIONS:
in summary, the staff recommends that the Commission approve the following with respect to each of the policy issues:
Permitting Very Small Risk Increases The staff recommends that CDF increases of 4
less than 104 per reactor year, and LERF increases of less than 10 per reactor year, be considered "very small' and be permitted for proposed changes independent of the baseline calculated CDF/LERF, provided a cumulative acceptance guideline is established and the cumulative effect is tracked.
Treatment of Uncertainties in Companson of PRA Results With the Acceptance Guidehnes The staff recommends that the basic approach for treating uncertainties contained in DG-1061 be retained in the final version of the regulatory guide, but be clarified to provide better guidance to licensees.
Acceptance Guidehnes for Shutdown Conditions: The staff recommends that additional consideration of possible additional shutdown risk acceptance guidelines be undertaken as part of the RES-led effort (beginning in FY 1999) to better understand risks from accidents in these plant conditions. In the interim, the staff recommends that the CDF and LERF guidelines be maintained for both power and shutdown operations.
)
Licensees will have the flexibility to propose a reasonable definition for LERF considering the reduced radionuclide inventory or to rely solely on an assessment of core damage (i.e., CDFs below the 104 LERF guideline). it is recommended DG-1061 be revised to be consistent with this interim approach.
u Anema*=nce Ge" >es for Temocrary Plant Confiauratians: The staff recommends that an assessment of acceptance guidelines for temporary plant changes be undertaken.
However, the staff also recommends that this development be deferred for future Thia assessment can be accomplished 'within the planned resources estimated for risk-4 informed regulatory initiative activities in RES and NRR.
11 revisions of DG-1061, to permit adequate staff analysis, ACRS review, public comment, and Commission approval. In the interim, the staff recommends that only time-average guidelines be used.
In order to meet the scheduled completion dates for the regulatory guidance documents, the staff intends to proceed with finalization of the documents presuming Commission approval of these recommendations. The Commission will be provided the final DG-1061 and its SRP for
- approval by the end of December 1997. These documents will assume Commission approval of the recommendations and will need to be modified as appropriate to reflect other Commission guidance.
I L. J ph Callan Executive Director for Operations SECY NOTE:
Commissioners' completed vote sheets / comments should be provided directly to the Office of the Secretary by cob Wednesday, December 31, 1997.
Commission staff office comments, if any, should be submitted to the Commissioners NLT Monday, December 22, 1997, with an information copy to the Office of the Secretary.
If the paper is of such a nature that it requires additional review and comment, the Commissioners and the Secretariat should be apprised of when comments may be expected.
DISTRIBUTION:
OPA 1
' I