ML20197F193
| ML20197F193 | |
| Person / Time | |
|---|---|
| Issue date: | 12/16/1997 |
| From: | Seale R Advisory Committee on Reactor Safeguards |
| To: | Shirley Ann Jackson, The Chairman NRC COMMISSION (OCM) |
| References | |
| ACRS-R-1740, FACA, NUDOCS 9712300153 | |
| Download: ML20197F193 (8) | |
Text
ACRSR-1740
,#oa %,%,,
UNITED $ TAT t t PDR 8'
NUCLEAR REGULATORY COMMISSION o
,{
I AoVISORY COMHlf TEE ON RE ACTOR SAFEouARDS
/
W A EHINGTON, D. C. 20$f>S y s.*
December 16. 1997
'ihe Honorable Shirley Ann Jackson Chairman U.S. Nuclear Regulatory Comission Washington. D.C. 20555 0001
Dear Chairman Jackson:
SUBJECT:
TREATMENT OF UNCERTAINTIES VERSUS POINT VALUES IN THE PRA RELATED DECISIONMAKING PP.0 CESS During the 443rd. 444th. 446th, and 447th meetings of the Advisory Committee on Reactor Safeguards. July 911. September 3 5. November 6 7. and December 3 6.
1997. respectively, we met with representatives of the NRC staff to discuss issues included in the Staff Requirements Memorandum dated May 27. 1997, regarding the use of uncertainty versus point values in the PRA related decisionmaking process (Reference 1).
Our Subcommittee on Reliability and Probabilistic Risk Assessment (kvRA) met with the staff and industry representatives to discuss these matters on July 7. August 28. October 2122. and November 12 13, 1997.
Background
Uncertainty has always been of concern to nuclear power regulators. As early as 1956. Willard F. Libby. Acting Chairman of the Atomic Energy Commission (AEC),
wrote to the Congressional Joint Committee on Atomic Energy that *it is incumbent upon the new industry and the Government to make every effort to recognize every possible event or series of events which could result in the release of unsafe amounts of radioactive material to the surroundings and to take all steps necessary to reduce to a reasonable minimum the probability that such events will occur in a manner causing serious overexposure to the public."
(Reference 2)
Even though Dr. Libby used the word " probability." about 20 years would pass before systematic calculations of probabilities would be produced for the
'possible event or series of events" to which he referred.
The " reasonable minimum" of the unquantified probability that was achieved at that time was attained through the development and application of the concepts of defense in t
depth and safety margins.
D DI B
f g23 3 971216 R-1740 PDR y.pjH 5OM
2-l Defense in depth is advocated in numerous documents as the principal means of i
' controlling the (still unquantified) probability of accidents.
For example, during the 1971 hearings on emergency core cooling, the AEC staff stated: The i
safety goal, therefore, is the prevention of exposure of people to this radioactivity, This goal can be achieved with a high degree of assurance, j
dithough not perfectly (emphasia added), by use of the concept of defense in depth.,.The three separate lines of the defense in depth provided for power i
reactors are considered appropriate to reduce to an acceptable value the probability and potential consequences of radf oactive releases," - (Reference 3)
- Although the approaches of defense in depth and safety margins have served the industry well ftom the safety perspective, they were intended to be conservatiw l
and, as implemented today, thev impose a heav/ regulatory burden. The level of safety was not quantified.
The first call for a more rational approach to regulation based on improved understanding of risk came in 1967 from r. Reginald Farmer (Reference 4) of the linited Kingdom Atomic Energy Authority. The React e t
Safety Study (WASH 1400)(Reference 5) soon followed in 1975.
Not surprisingly, the WASH 1400 study itself proved to be conservative in some areas. e.g.. the-analysis of the containment, and nonconservative in others, e.g., the analysis of earthquake; and ftres.
There has been tremendous progress in our understanding of the risks from nuclear ponr plants since that study (a history of PRA developments since WASH 1400 is given in Reference 6),
Realizing that the availability of risk numbers made it possible to reexamine the question of how safe is Nfe enough, the Commission issued the safety goal policy in 1986 (R?ference 7). The recognition that uncertainties had to be dealt with is reflected in the following three statements frec the policy statement:
3 Statement 1:
'It is the Commission's intent that the risks from all the various initiating mechanisms be taken into account to the best of the capability of current evaluation techniques,"
i Statement 11:
"To the extent practicable, the Commission intends to ensure that the quantitative techniques used for regulatory decisionmaking take into account the potential uncertainties that exist so that an estimate can be made on the confidence level to be ascribed to the quantitative results."
Statement Ill:
"The Commission has adopted the use of mean estimates for the purposes of implementing theiquantitative objecti,es of this safety goal
~
policy...."
The Commission's safety goals were derived from societel considerations 1.e.,
independent of the PRA state of the art.
Even though they were expressed both j
.w--
+- -..
..m.wm
-,.,.,e-.
._,_4..,---.,w.,,n,,-
..,..r-
. - -,.,,m.m,.-%..--m,,,.-,._..<_.,----m,-w u,-
..em-_m,-.-,w.,-.ar-,-.
3
, qualitatively and quantitatively, it was clear that the Commission did not intend to simply compare a PRA
- point estimate" (however it was defined) with the numerical goals.
The Issue As noted above, the numerical estimates that PRAs produce have been scrutinized to an extraordinary degree since the early days of WASH-1400.
Sometimes the debate regarding the accuracy of these numbers detracts from the intended use of PRA.
It is not the intent to regulate on the basis of risk estimates alone (thus.
- risk informed" regulation). The objective is to gain enough confidence in the numerical probabilities of a set of accident scenarios so that the traditional approaches (defense in depth and safety margins) that have already been applied to this set can be better managed.
This means either relaxing some existing requirements, if proven burdensome and non contributing to risk reduction, or adding new requirements. if the traditional approaches have not covered some detrimental events.
The preceding discussion suggests that the question regarding the quality of PRA results ought not to be an absolute one, but, rather, a comparative one.
Therefore, we offer the following observation:
Observation 1:
6cn PRA results and insights are proposed to be used in the regulatory process, the question to be asked should be:
To what degree is there confidence that the use of PRA results and insights will improve on the existing regulatory system for the problem of interest?
The words "PRA results and insights' include the set of dominant scenarios to risk (or core damage, as the case may be) as well as an assessment of the uncertainties regarding the frequencies of these scenarios. The utilization of PRA results and insights depends on our confidence that their use will improve the regulations in accordance with the Commission's vision. It is definitely not a case of PRA versus the traditional approach.
In Observation 1. the key words are *will improve." There is improvement when the regulations contribute to the safe and efficient use of nuclear materials, as per
-the recently articulated vision of the Commissior.:
"In implementation of w,
mission. Nuclear Regulatory Comission actions enable the Nation to safely anc efficiently use nuclear materials."
(Reference 8)
~.
4 e
t
. Uncertainties As our brief historical review has demonstrated. the uncertainties regarding off-normal events and incidents in nuclear power plants have been of concern since the early days of reactor regulation.
In the early seventies, quantifying the uncertainties was synonymous with developing probability distributions for the i
failure rates and the frequencies of accident initiators.
This explicit quantification of uncertainties posed a new problem to safety analysts.
They soon discovered that the interpretation of the concept of probability was controversial among mathematicians. Several schools of thought were available, of which the frequentist and the Bayesian schools were dominant.
When the nuclear debate was heating up in the mid seventies, the analysts were reluctant to get involved in an additional controversy.
This attitude, although understandable in the context of the times, was unfortunate, because it led to confusion and the.crception that uncertainty analysis was controversial and to be avoided.
it also led to some circumlocutions. For example, the WASH 1400 treatment of failure rates is purely Bayesian, yet that voluminous report does not acknowledge this fact explicitly.
Similarly, the NUREG 1150 studies (Reference 9) claimed to elicit " weighting factors" from the experts, rather than admit that they were eliciting probabilities. Although " officially." both frequentist and Bayesian viewpoints were equally valid, no PRA had been done using frequentist methods because it cannot be done.
Industry sponsored PRAs however, have readily acknowledged using Bayesian methods in an explicit way (Reference 10).
It is now known that uncertainties in failure rates and other parameters appearing in PRA can be quantified via probability distributions using available generic and plant-specific data and appropriate Bayesian methods.
The propagation of these (*istributions through the PRA logic diagrams is straightforward using standard computer packages.
We believe that there is no excuse for failing to do an uncertainty analysis on the parameters of the PRA models. Therefore, we offer the following observation:
Observation 2:
The Bayesian interpretation of probability provides the appropriate framework for PRA.
Probability distributions for the parameters of PRA models, e.g., failure rates, should be developed using all available evidence and propagated to produce the probability distribution of the quantity of interest e.g.
core damage frequency (CDF) and large, early release frequency (LERF).
~
i Since regulators must confront uncertainties, it is evident that, if PRA is to be used as in our Observation 1 the probability distributions of Observation 2 must be derived. Anything less does rot represent what is actually known about these failure rates.
This brings up the issue of " point estimates." for which we offer the following observation:
Observation 3:
The oaly
- point estimates" that are unambiguously defined are those that are sumary measures of a probability distrioution: e.g., the mean value, the median value, a.id various percentile values.
Ill defined " point estimates." such as "best estimates." have limited utility.
Point estimates are valuable for screening purposes after a convin:ing case has been made that the uncertainties have been handled apprcpriately, e.g.. they are either negligible or have been bounded in fact the use of such point values is an important tool in screening the thousands of minimal cut sets that a PRA produces.
Such use however, should be followed by a rigorous uncertainty analysis of the dominant sequences.
The uncertainties of interest in reactor regulation have been terrred " state of-knowledge" uncertainties (Reference 11) or, more recently.
'epistemic" uncertainties (References 12.13). The parameter uncertainties that are referred to in Observation 2 are only a part of the total epistemic uncertainties.
Uncertainties resulting from model assumptions and approximations are also epistemic and more difficult to quantify. Examples would include models used for evaluating severe accident phepomena in Level 11 PRAs.
Model uncertainty is the key to any use of PRA results. When events or processes are modeled poorly or not at all, there is uncertainty that has not been quantified in the sense that it is not part of the probability distributions proouced by propagating parameter uncertainties. The fact that uncertainty is not quantified does not mean, however, that nothing is known about it.
The PRA structure provides a good framework within which these uncertainties can be assessed qualitatively through sensitivity analyses or other means (see, for example. Reference 14),
These uncertainties exist independently of whether or not they are quantified in PRAs, Recalling Observation 1. use of PRA insights must include a qualitative description of unquantified uncertainties, in addition to those that have been quantified.
Any PRA based argument for easing the regulatory requirements of the traditional approach is weakened when the unquantified uncertainties are very large and pertinent to the application.
Therefore, we offer the following observation:
L.
i 6-i l
o-f Observation 4:
Regulatory decisions must be made in the light of all the relevant i
uncertainties.
These include the uncertainties quantified in PRAs. as
[
well as significant unquantified uncertainties. Although
- point" values, i
defined as in Observation 3, can be useful for screening purposes, they are sunmary measures of the probability distributions and should not be the sole basis for decisionmaking.
}
a The deliberation on uncertainties that we are recommending is best accomplished l-
,Jnsidering the scenarios tMt dominate the event of interest.
The set of t
sominant scenar'os is one of the most important results of PRA and has been proven to be very useful in risk management (Reference 16). A discussion of the overall uncertainties without a discussion of the sources of uncertainties is of f
_ limited value. Thus, we offer the following observation:
i i
Observation 5:
The dominant scenarios should be an integral part of the deliberation on uncertainties.
The regulatory decisions of immediate interest are those related to requests for changes in the current licensing basis (CLB).
In discussing uncertainties. it is important to consider nossible benefits of the proposed change. For example, a change that reduces the regulatory burden in certain areas could allow the reallocation of resources to more risk significant issues and activities.
Therefore, we offer the following observation:
[
Observation 6:
The unquantified uncertainties associated with a proposed change in the CLB should include the possible beneficial impact of the proposed change on plant safety.
Finally, we note that the decisionmaking process described in Regulatory Guide
.1.174 treats uncertainties and point values in a manner consistent with our y
e v
4 c
._,,.a_.,_,
,..,._.,1
t 9
7 recommendation! as discussed in our report dated December 11, 1997.
(Reference 16)
Sincerely.
R.
L,
'"le Chairman Referene,gs:
1.
Staff Requirements Memorandum dated May 27,:1997, from John C.
- Hoyle, Secretary of the Commission, to_ John _T.
Larkins, ACRS
Subject:
'teeting with Advisory Committee on Reactor Safeguards on Friday, May 2, 1997,
' i 2.
Letter dated March 14,1956, from Willard F. Libby, Acting Chairmaq of the Atomic Energy Comiss_1on to Senator B, Hickenlooper, Joint Committee on Atomic Energy, reproduced in: 0, Okrent, Nuclear Reactor Safety The l
t University of Wisconsin Press Hadison, 1981.
3.
Testimony of the AEC Regulatory Staff at a Public Rulemaking Hearing on Interim Acceptance Criteria for Emergency Core Cooling Systems for Light-Water Reactors on January 27, 1972, Issued December 28, 1971 V,
S.
Atomic Energy Commission.
4.
F.
Reginald Farmer,
" Reactor Safety and Siting:
A Proposed Risk Criterion " Nuclear Safety, Vol. 8, No. 6, pp. 539 548, Nov Dec. 1967.
- 5.
U. S.
Nuclear Regulatory Conmission, NUREG-75/014, " Reactor Safety Study.
An Assessment of Accident Risks in U.S. Nuclear Power Plants WASH 1400,"
October 1975, 6.
Eric S 3cckjord, Mark C. Cunningham, and Joseph A. Murphy, *Probabilistic Safety Assessment Development in the United States 1972 1990, Ref f abf if ty Engineering and System Safety, Vol, 39, pp,159170,1993.
7, U, S.
Nuclear Regulatory Commission, " Safety Goals for the Operation of Nuclear Power Plants: Policy Statement," Tederal Register, Vol, 51, p.
30028. August 21, 1986, 8.
U.S. Nuclear Regulatory Comission, "Strategfc Plan: ffscal Year 1997 -
-Ffscal Year 2000 " September 1997.
-t a
.ce, r,-
,~.,y
,-mm 3
-,yw
--m,
,.,--,-4-e,
-g--e-
,e--
...e ec.-w w.,-
,e
,--n
-',w1
l i
i
.g..
1 9.
,U. S.
Nuclear Regulatory Commission. NUREG 1150. " Severe Accident Risks:
i An Assessment for Five US Nuclear Powr Plants." December 1990, i
10.
Pickard Lowe, and Garrick. Inc.. Westinghouse Electric Corporation, and i
Fauske & Asse.tates. Inc.. "Ifon Probabfifstfc Safety Study." Report prepared for Commonwealth Edison Company Chicago.1981.
=.
i
. 11.
Stanley Kaplan and B. John Garrick, *0n the Quantitative Definition of
[
Risk." Risk Analysis. Vol. 1. No.
- 1. pp. 11 28. March 1981.
[
12.-
George E. Apostolakis.
"A Coi..entary on Model Uncertainty."
in U S..
Nuclear Regulate y Commission. NUREG/CP 0138. Proceedings of Workshop 1 In Advanced Topics in Risk and Reliability Analysis: Model Uncertainty.
l Its Characterization and Quantifft.stion, October 20 22, 1993.
I 13.
Garcth W. Parry, "The Characterization of Uncertainty in Probabilistic l
R1sk Assessments oi Compiex Systems." Re11ab111ty Engineering and System i
Safety Vol. 54 pp.119126,1996.
l 1
14.
Dennis Bley, Stanley Kaplan, and David Johnson.
"The Strengths and i
Limitations of PSA: Where We Stand." Reliability Engineerfng and System i
Sa /ety, Vol. 38, pp. 3 26,1992.
15.
Hardyros Kazarians. Nathan Siu, and George Apostolakis. " Risk Management App 11 cation of Fire Risk Ana1ysis.* Proceedings of the First international Syrtposium on Fire Safety Science, pp. 1029 1038. C.E. Grant and P.J.
Pagni. Editors. Hemisphere Publishing Corporation. New York, 1986.
16.
Report dated December 11, 1997 from R.
L.
Seale. Chairman, ACRS. to Shirley Ann Jackson, Chairman. NRC.
Subject:
" Proposed Final Regulatory Guide 1.174 and Standard Review Plan Chapter 19 for Risk-Informed i
Performance Based Regulation" l
l t
_. _.. - - -.,. - _.., _.. - _ ~. - - _.
,_,..._..._.,.,.m~,-
. -. ~....,. ~ - -
.