ML22145A315

The 95/95 Acceptance Criterion in Risk-Informed Regulation

ML22145A315
Person / Time
Issue date:	06/01/2019
From:	Yuri Orechwa NRC/NRR/DSS/SFNB
To:
Yuri Orechwa NRR/DSS 301-415-1057
Shared Package
ML22145A298	List: ML22145A315 ML22145A316 ML22145A317
References
Download: ML22145A315 (18)

Text

The 95/95 Acceptance Criterion in Risk-Informed Regulation The 95/95 Delusion Yuri Orechwa NRR/DSS/SNPB June xx, 2019

2 I Curr-ent_ Working_ Draft_Report.nb I. Introduction A previous White Paper (Ref. l) discussed a systematic process for identifying and understanding the deterministic computation of the uncertainties associated with the formulation of an analysis framework, that enhances the traditional PRA analyses at L 1, by exploiting the results obtained routinely today in licensing basis analyses of reactor transients.

The results of such an analysis set the initial conditions for the subsequent L2 and L3 a_nalyses, with which the safety to the public is assessed. This sequence of analyses estimates the amount of radioactive material released from the fuel elements throughout the core to the public under accident conditions. ln the continuing spirit "that understanding the formulation of Risk-Based Analysis is the 'sine qua non' for making Risk-Jnformed Decisions", we further elaborate the analysis of Ref. I. In particular, the previous analysis used a statistical hypothesis testing approach with respect to a fixed failure margin at rhe 95/95 acceptance level of a single fuel element. We intend to expand this analysis to the mean core damage, which is more appropriate for decision making through the application of statistical decision theoretic principles. (Ref. 2) Moreover, we shall address the appropriateness of 95/95 tolerance limit statistic in safety analyses in the context of nuclear reactor safety.

Our methodology is analogous to studies of the behavior of stock market returns. There the motivation and theory are based on that "Evidence on the form of the distributions of returns on securities and portfolios is important for several reasons. For the investor, the form of the distribution is a major factor in determining the risk of investment." (Ref.3, p. 17) The analogy to" the distributions of returns on securities" will be in our study of the probability density function of the figures of merit, in particular the peak cladding temperature, that is the independent variable of the loss ( core damage) function. The expected core damage will be the measure of risk. The key observation is that probability alone is not risk!

Thus, at the heart of the discussion is the expression for the mean core damage (CD) estimated from a best-estimate plus uncertainty (BEPU) result of an analysis of a reactor transient:

Eq. 1.1

Current_ Working_Draft_ Report.nb I 3 The objective is the deconstruction of this expression, and, thereby, gain insight to it's place in reactor safety analyses. It is recognized that the core damage is a function of a number of correlated figures of merit. For ease of illustration and discussion, we focus on one, the peak clad temperature Tµ. The more realistic multivariate case is numerically messy, but a straightfonvard extension of the argument presented here. This allows us, without loss of generality, to focus on and illuminate the central issues in our argument of risk.

II. Formulation of the Mean Damage for L2 and L3 PRA Analysis based on L 1 LBA Results The analysis in Ref. l is based on a statistical hypothesis testing framework. In that context, the estimate of the level of damage ofa single fuel element was quantified as a*M, where M is the damage at a threshold (the limiting peak clad temperature Tµ, for example) and a the probability that the peak clad temperature Tµ has exceeded the clad temperature at the damage threshold Ly. We then have the following relationship where the damage is modified by (1 - /3), the sensitivity of the measurement instrument (i.e.

computer code in this application) that the damage margin has been exceeded. This formulation is instructive, but we feel unsatisfactory for risk analysis.

The fuel element damage formulation from Ref. l, however, is suggestive of a phenomenologically descriptive formulation of an estimate of core damage for use as an initial condition in L2 through L3 PRA.

To this end, we derive, to first order, the following decision theoretic formulation of the expected core damage <CD> at L 1, based on the phenomenological analysis of a given peak clad temperature probability density function associated with a single fuel element, and a loss function in the form of a peak clad temperature core-damage function. This, in principle, allows an estimate, to first order, of a risk-informed assessment of the relative mean source of the health effects at L3 of new reactor designs and due to design changes of operating reactors.

4 I Current_ Working_Dra~_ Report.nb Let us define the risk of core damage as the mean estimate of core damage based on a.

probability density function f(Tpl µP, erµ), constructed on the basis of a best-estimate plus uncertainty (BEPU) transient analysis of a reactor accident sequence. The additional required information for the risk estimate is a "loss" function CD(Tp), that in our case we shall refer to as a "damage" function. This then results in the following expression (Eq. I. l) for the mean core damage:

Eq.Ll Let us examine the integrand of this expression by first considering separately the functions in this risk formulation; The two component functions, CD(Tµ) the core damage as a function of the peak clad temperature, and f(TµI µ P, o p ), the probability density function of the peak cladding temperature. The interaction between these two functions is critical in determining the implied mean core damage base on BEPU analyses results. At issue is the phenomenological behavior of each function, and the interpretation of it's contribution to the mean core damage for risk-informed decisions.

A. A Phenomenological Loss Function CD(Tp) that reflects Core Damage.

We develop our phenomenological core damage function CD(Tp) by considering, without loss of generality, a minimalist core model consisting of cylindrical geometry and a homogeneous composition. Let us assume a hemispherical flux distribution in the core. This results in a hemispherical power distribution, and for uniform flow as a hemispherical clad temperature distribution of Tp as shown in Fig. I. We associate each planar ring with a ring of fuel pins with a the same Tp.

Current_ Working_Dra~_Report.nb I S

)' p Fig. 1 Peak Cladding Temperature Tp as a Function of Core Radius The relation between Tµ and the core radius r is shown in Fig. 2, in an axial planar perspective, for no pin failures, partial core failure and full core failure for a fuel pin at radius

r. The phenomenological effect this illustrates is that as the fuel performance code computes Tµ>Ly for the lead pin at the center, as the calculated Tµ of the center pin increases more and more adjacent pins fail.

6 I Current_ Working_ Draft_Report.nb "Tp(a) > Ly Full Core Failure" "Tp(r) > Ly Partial Core Failure" "Tp(r) ~ Ly No Pin Failures" Fig. 2 Peak Cladding Temperature TJJ as a Function of Core Radius Let the core consist of N identical fuel elements, that are each associated with a unit cell area (UC). Thus, the planar active core region has an area Ac= N*UC. Furthermore, we consider that each fuel pin that fails (by Tp > Ly) releases a fixed amount of 'damage' (PD) per fuel pin, say in the form radio active isotopes, upon exceeding the acceptance limit Ly.

Current_ Working_Dra~_Report.nb I 7 Fig. 3 Variation of the Area of Failed Fuel Pins for Peak Cladding Temperature TP as a Function of Core Radius Therefore, as the peak clad temperature of the peak pin ( core center pin) increases (Fig. 3 ), the concentric area containing all the failed pins increases. We, therefore, can express the number of failed fuel pins for an area associated encompassed by T v(r) (Fig. 2), as no. failed pins = nr2/UC, for Tp(r).

We, therefore, have CD(Tp(r)) = PD*(nr2/UC ).

Eq. 11.2 Given our assumption of a hemispheric core power distribution, we can scale the radius to T v*

and let Ca = [(Tp(a) - Ly) I a]. We have the necessary relationship

8 I Current_ Working_ Draft_Report.nb Substituting this relation into Eq. (11.2) we obtain the necessary the core damage function Eq. I 1.3 Thal is, CD is quadratic in Tp(r).

Jt is, of course, recognized that in a realistic case the analytic relationship will not be as

'clean' as in Eq. 11.3. However, it will be a power law dependent on the peak to average power distribution of the core under consideration.

III. Analysis of Core Damage Risk Based on a Perfect Information Probability Density Function of the Peak Cladding Temperature Tp and the Acceptance Limit Lr As a generic representative for our argument, we shall consider the normal probability density function for ease and familiarity. Any continuous pdf will have similar phenomenological implications. For example, such as the lognormal pdf that is more appropriate in many cases.

As perfect information, we define two normal pdfs for which we know the true mean and standard deviation, say NB( 10, 1) and Ne( 1, 6.45). (We shall not associate dimensions with these, since only their relations are of interest.) In practice, we have estimates of these at best.

So any shortcomings in the methodology under perfect information will likely be more pronounced in practice. The parameters were chosen specifically so that both have the same 0.95 quantile. Thus, for both pdf's y = 0.95, and, therefore, L,, = 11.65. This is graphically presented in Fig. 4.

Current_ Working_Draft_Report.nb I 9 0.4 0.3 0.2 0.1

-20 0

10 20 Fig. 4. N8(10, 1) - Black, Nc(l, 6.45) - Green and L0_95 - Red.

These two normal distributions are not unique with respect to having the same quantile value 11.65. We can, in principle, construct an infinite number of normal distributions that have the same quantile by adjusting the mean and variance appropriately.

In the previous section above, we developed a phenomenological relationship between the peak clad temperature of the peak pin and core damage. In BEPU analysis of reactor systems, this relationship is not considered. The acceptance criteria are stated in terms of the probabilities of two nonintersecting events: {Tp ::5 Ly} and {Tp > Ly} as Prob{Tp ::5 Ly} = 0.95 and Prob{Tp > Ly} = 0.05. Both distributions in Fig. 4, for example, meet this BEPU criterion.

If we let the two distributions be associated with two different reactors, they are both acceptable. However, we might ask is one "riskier" than the other? Looking at the distributions in Fig. 4, an "interocular" test might conclude that the reactor associated with the black distribution is riskier. Since the frequency of high T P close to Ly is much greater than that for the reactor associated with the distribution in green. This conclusion could be based, for example, on the intuitive analogy with standing at the edge of a cliff as opposed to further back.

This BEPU decision framework is not only not unique, but also only assigns a same probability to infinite possible designs and not a unique expected damage to each. Let us consider Fig. 4 further, in the context of Eq. I.I of the expected core damage associated with

• any BEPU result. Our interest is the interaction between CD(Tµ(r)) and f(Tp I µ p, (T p)- For this we can set CD(Tp(r)) = [Tp(r) - Ly]2 and capture the phenomenological behavior of

10 I Current_ Working_Draft_ Report.nb interest as in Fig. 5..

70 60 50 40 30 20 10 12 14 16 18 20 Fig. 5. Core Damage Function CD(Tp(r)) = [Tp(r) - Ly]2 for event {Tp > Ly}.

The expected damage for both systems is dependent only on the event {Tp > Ly} for a single fuel pin, while the two pdfs are dependent on the specific designs of the two reactor systems and are shown in Fig. 6.

0.4 0.3 0.2 0.1

-20 20 Fig. 6. Superposition of expected core damage components CD(Tµ) and f(Tp I µv, <rp)*

Since the expected core damage for both systems is dependent only on the event { T P > Ly},

Current_ Working_Draft_ Report.nb I 11 we need only examine the asymptotic relationship with respect to the abscissa, as shown in Fig. 7. Mathematically the asymptotic behaviors of the functions are: CD ~ x 2 and the pdfs e-x 2

• Therefore, as x oo, the pdfs 0 (black and green curves) faster than x 2 oo (blue curve).

0.10 12.0 12.5 13.0 13.5 14.0 14.5 15.0 Fig. 7. Expected core damage components CD(Tp) and f(Tp I µ1n er p) for event {Tp >

Ly}.

The integrand of Eq. I. I for the two pdfs is shown in Fig. 8, and illustrates the importance of the contribution to < CD > of the "tails" of the pdfs for event { Tµ > Ly}. The thicker tail of the green pdf contributes more to CD than the more rapidly decaying black pdf.

12 14 16 18 20

12 I Current_ Working_ Draft_ Report. nb Fig. 8. Integrand of <CD> for event {Tv > Ly}*

The mean core damage for the two systems given by Eq.1.1 is then:

< CD >Black = 0.015

< CD >Green = 0.467.

Thus, based on the traditional BEPU acceptance criterion Prob{Tµ ::s: Ly} = 0.95, both systems are acceptable under perfect information. Our interocular test suggested that the system represented by the black pdf is 'riskier'. Phenomenological analysis, that takes core damage into account, shows that both ~re incorrect, and that the system represented by the green curve will have a much greater risk of core damage. Thus, in order to make a risk-informed decision based on BEPU results, it is necessary to introduce a physically relevant loss function, that gives the relationship between the peak clad temperature and the damage incurred as a result of exceeding an experimrnt ;:illy clr1rrminecl limiting trmprniturr ch1r to uncrrtainty.

IV. Analysis Based on an Imperfect (i.e. Sample) Information Probability Density Function of the Peak Cladding Temperature Tp and the Acceptance Limit Lr Jn practice, we only have a random sample of size N from the true pdf. Thereby, BEPU analyses give only imperfect information for making a risk assessment with regard to core damage. ( It is important to note that we assume that the uncertainty is due only to the random effects of the model's parameters. i.e. it is phenomenologically correct) ln order to see how "imperfect" the results are we study two basic estimation methods: parametric and non-parametric. In the case of the former, we shall use a representative of the class of response surface methods. For the latter, in light of the current in-vogue approach at NRC, a tolerance interval method; in particular using the so called 95/95 methodology of Wilks' Theorem (Ref.

4) to quantify the level of safety. We assess these approaches by applying them to the previous analysis above of perfect information via the two specified pdfs shown in Fig. 4. The core damage function is the same and not an issue; we focus on the implication of the effect of the limited information content in the BEPU sample for making risk informed decisions with

Current_ Working_Draft_ Report.nb I 13 regard to reactor safety.

A. Base Case Sample Data Our objective is to study the loss of information when we have only a random sample drawn from the "true" pdf of the figure of merit. to the end, let us first consider a base case sample from each of the two distribution functions N 8 ( I 0, I) and Na( I, 6.45) shown in Fig.4. The key objective is to demonstrate the weakness of the so caIJed 95/95 methodology in light of the estimation of the mean core damage (Eq. I. J) for assessing nuclear reactor safety.

We draw random samples of size N = 59, so as to be consistent with the 95/95 methodology.

ForNs( IO, I):

10.2107, 9.38212, 10.1792, 10.1587, 12.1323, 10.6931, 10.3642, 8.52441, 10.4009, 10.1661, 10.5556, 10.4479, 9.92655, 9.21098, 10.4392, 9.7536, 10.9363, 9.60991, 10. 9228, 9.15423, 10.1566, 10.7537, 9.60641, 11.1431, 9.28669, 11.4952, 8.81195, 10.4316, 10.9711, 9.844, 10.113, 10.0296, 9.15593, 9.92208, 10.7389, 8. 27512, 9.93952, 10.8391, 8. 56407, 9.66609, 11. 2341, 7.58957, 10.4807, 7.23384, 11.1549, 8.25832, 11.2367, 9.15294, 10.001, 10.9748, 9.46498, 10.7563, 10.5194, 11.8907, 10.7448, 9.39109, 11.1513, 12.2136, 8.97911 )

and for Ne(!, 6.45)

{12.5798, 9. 66408, 0.740544, -0.9746, - 3.46813, - 4.83713, - 3.14985, 2.52295, 4.69621, - 0.944088,

- 8.02446, 1.19877, 5.76318, 2.14461, - 1.08628, - 2.93579, - 12.5708, - 1.97193, 1.42524, 2.69249, 3.8633, -3. 74272, - 15.3513, 1.3456, 3. 828, 1.05397, 5.98373, 7.16034, 3.80897, - 1.21239, 12.0271, 0.0894731, 6.94144, 4.11539, -2.09345, 3.52815, -7.7629, 0.16986, 5.61613, 7.95565,

4.47253, 1.66266, 3.63721, 13.1487, - 7.31508, 0.973678, - 4.08423, 17.2706, 13.3061, 10.9996, 7.18058, -0.454851, 6.32628, - 1.06634, - 9. 64521, - 5.5967, - 3.36068, 2. 94812, - 0.664466 }

Table I. Basic Distribution Moments for Perfect vs. Sample Information in the Tl,'O Base Cases Perf. Inf.BLK Perf. Inf.GRE Sampel Inf. BLK Sampel Inf.GRE mean 10 1

10.09

1. 53 st.dev.

1 6.45 1.03 6. 37 skew 0

0

- 0.45

- 0. 049 kurt.

3 3

3.26 3.31 T ( 59)

NA NA 12.21 17.27

14 I Current_ Working_ Draft_ Report.nb B. Non-Parametric Estimation ( a la Wilks)

Recall from Ref. I that our basic task is to distinguish between the probabilities of the two events: { T/J ~ L y } No damage, and { Tp > Ly } damage. Thus, the following identity holds:

Eq.Jll.4 In addition, the cumulative probability density function in the case of perfect information is given by the expression:

Eq. 111.5 Where y is the fixed acceptable portion of the pdf f (T Iµ, er). Thus, with perfect information in the form of the analytic form of the pdf f(TI µ, er), we can compute the quantile qy exactly with the relationship given in Eq.lll.5. However, when we do not know the analytic form of f(TI µ, er), we must estimate Ly with a random sample of Ts via a BEPU analysis with a computer code.

It is instructive to derive the Wilks' relationship for estimating the quantile qy associated with y from imperfect (i.e. sample) information. To this end, we consider a set of N runs of the BEPU code for a transient analysis to obtain N different values of T µ (Since TP is a random variable in a BEPU methodology.)

Let us order the N calculated values as an ordered set !T(i)}f~1, where T( I) < T(2) < T(3) <

...... < T(N). From identity Eq.111.4, it follows that Prob{ Tµ > Ly } = 1 - Prob{ Tµ ~ Ly }.

Eq.111.6

Current_ Working_ Draft_Report. nb I 15 Since T(i)'s are assumed to be independent continuous random variables with common, but unknown, pdf,s we can express, Prob{T(N)::; Ly }=

Prob{T(l)::; Ly }xProb{T(2)::; Ly }x... xProb{T(N)::; Ly}.

Therefore we get with the application of Eq.JII.5 that Prob {T(N) ::; Ly } = yN.

Substituting this expression into Eq.111.6, and defining f3 = Prob{T(N) > Ly}, we have Wilk 's expression for one random variate as Eq. 111.7 There are two basic interpretations of this expression (Ref. ):

A. f3 is the probability that the event {T(N) > Ly} is greater than that of they-quantile of the true but unknown pdf f(T[ µ, a") that generated the sample { T(i) }% 1*

B. f3 is the probability that the interval ( -oo, T(N) ] covers a larger portion than y of the of the true but unknown pdf f(T[ µ, o-).

Recall in the case of perfect information in Sec.JI.B, Fig. 4, we saw that an infinite number of unique pdfs are consistent with the event {T" > Ly} for the same quantile Ly. Similarly, for an outcome of a maximum value T(N) in our imperfect (sample) information analysis of N code runs, there is an infinite set of y/(3 combinations that satisfy Wilks' formula for fixed N as shown by the Blue curve in Fig.9. Are the other points y/{J on the Blue curve equally "risky" to 95/95? Wilks' formula is silent with regard to the implications on the risk states associated with them. This is clear, since the basic premise is that the pdf is unknown, and, by extension, the shape of the tail for event {Tp > Ly}, which is critical for the estimation of the risk.

16 I Current_ Working_Dra~_ Report.nb p Confidence Level 1.. ~==============~

0.8 0.6 0.4 0.2 0.90 0.92 0.94 0.96 0.98 y Probability Con 1.00 Fig. 9. Wilks' formula Eq.111.7 with N=59 (Blue) and 0.95/0.95 (Red).

Similarity Relations between Perfect Information and Wilks' yl/3 Methodology Acceptance Criteria Formulations for Peak Clad Tem-perature.

Perfect Information Sample Information Prob { Tp > Ly } = /3 Prob { T(N) > Ly } = /3

/3

=

/3 Ly -

quantile (0.95)

Current_ Working_ Draft_Report.nb I 17 T (N) -

quantile nonparametric estimate f (T I µ, er) - known analytic form T (N) -

random variable

( - oo, Ly ) -

specified interval

( - oo, T (N)) -

random interval

µ I er -

defines quantile y I /3 - defines N References

1) Y. Orechwa, "Some Formal Considerations in the Application of Best-Estimate Plus Uncertainty Results in Licensing Basis Analysis to Risk "1nformed" Decision Making for Reactor Safety," Internal NRC/NRR/DSS White Paper, June 14, 2018.

2) M. H. De Groot, Optimal Statistical Decisions, McGraw-Hill, New York, 1970.

3) Eugene F. Fama, Foundations of Finance, Basic Books, Inc. New York, 1976. (p. 17).

4) Y. Orechwa, "Best-Estimate Analysis and Decision Making Under Uncertainty," Proc. Int.

Mtg. Updates inBest-Estimate Methods in Nuclear Installations Safety analysis (BE-2004),

November 14-1 8, Washington D.C., American Nuclear Society (2004).

5) I. Guttman, Statistical Tolerance Regions: Classical and Bayesian, Griffin's Statistical Monographs & Courses, Hafner, Darien Conn., 1970.

18 I Current_ Working_Draft_ Report. nb

6) U.S. Nuclear Regulatory Commission, " PRA Procedures Guide," NUREG/CR-2300, Vol. I (January 1983).

7) L. Pal and M. Makai, "Remarks on the statistical aspects of the safety analysis."

arXiv:physics/05 l 1140v I, November 2005.

8) A. Guba, Makai and L. Pal, "Statistical Aspects of Best-Estimate Methods - I," Reliab.

Eng. Syst. Saf., 80, 2 I 7 (2003 ).

9) M. Makai, L. Pal, "Best estimate method and safety analysis II." Reliab. Eng. Syst. Saf.,

91, 222 (2006).