ML19199A246
| ML19199A246 | |
| Person / Time | |
|---|---|
| Site: | Crane  |
| Issue date: | 03/16/1979 |
| From: | Abramson L, Moore R Metropolitan Edison Co |
| To: | Atomic Safety and Licensing Board Panel |
| Shared Package | |
| ML19199A245 | List: |
| References | |
| NUDOCS 7904100051 | |
| Download: ML19199A246 (14) | |
Text
.
UNITED STATES OF AMERICA NUCLEAR REGULATORY COMMIbSION BEFORE THE ATOMIC SAFETY AND LICENFIiG APPEAL 30APD In the Matter of
)
)
METROPOLITAN EDISON CCMPANY, ET AL.
)
Docket No. 50-320
)
(Three Mile Island Tiuclear Staticn,
)
Unit 2)
)
SUPPLEMENTAL TESTIMONY OF R. MOORE AND L. sLRAMSON IN RESPCNSE TO ALAB-525 APPLIED STATISTI_CS ERANCH OFFICE OF MANAGEMENT AND PROGRAM ANALYSIS U.S. NUCLEAR REGULATORY COMMISSION 1T332 March 16, 1979 7 9 0410 0 OSI1
In ALAB-525 (February 1,1979), the Board made several ccmments on the methodology of estimating the areal crash density at TMI-2 as presented.n our testimony submitted at the December 12 hearing session in this proceeding (prefiled testimony following Tr. 378).
The purpose of this addition is to respond to the Board comments.
Before responding to the specific ccccents, however, we feel that a shorc discussion of the rationale for our methodology would be helpful.
Le choice of apprcach was motivated by the requirement to analyze the uncertainty associated with the point estimates.
There are two sources of uncertainty -
model uncertainty and statistical uncertainty.
Model uncertainty stems from the particular choice of assumptions made about the underlying relations among the problem parameters and varichies and the possibility that the assumptions might be in error.
Statistical uncertainty stems from the randon nature of the observations and the possibility that the cbservations might not be representative of the assumed model.
One way to handle model uncertainty is to chcose the model assumptions sucn that any plausible departure from them would be in a corservative direction.
This is the approach we adopted.
Before adopting our particular model assumptions, we reviewed the applicant's approach involving the choice of a specific functional form for the conditional crash densities.
This can be a useful approach, provided that the assumed functional form is correct, since it makes use of all the data to estimate the unkncwn parameters.
However, if the assumed functional form is incorrect, then using it can lead to significant estimation errors.
Instead of trying to use data distant frca TMI-2 to estimate the probability of a hit at TMI-2, we based 12~333 our estimates only on data in the vicinity of TMI-2.
(By the assumpticn of model independence between r and e, justifica on pages 3-4 of our testimony, we treat r and 9 separately.)
Our estir'ates are then based on the mild assumption that the conditional crash densities for r and e are approximately constant in the vicinity of TMI-2.
If the cui,oitional crash densities are not approximately constant, then it is plausible to assume that they would be concave decreasing (see Tables 9/. and 98, revised 12/S/78), so that a chord joining any two point: on the curve would lie wholly above the curve.
(The exporential form assumed by the applicant has this property.)
In such a case, an estimate made with our methodolcgy would be conservatively biased, i.e., it would tend to overestimate the density.
In addition to minimi::ing the model uncertainty, our methodology was also designed to estimate the statistical uncertainty in a straightforward manner.
This we did by the method of confidence intervals, which require no extra assumptions in addition to thcse already made for the point estimate.
In contrast, the applicant assumed a prior distribution and a likelihcod function in order to apply a Bayesian analysis to estimate the uncertainty.
Thus, our methcdclogy requires fewer and weaker (i.e., assuming less) assumptions than does the applicant's metFodology, both for the point estimates and the uncertainty analysis.
On page 12 of AL AB-525, the Board referred to the "very irregular angular probability distribution" produced by the staff's methodology and claimed that it fails to decrease regularly as the angle e (measured from the runway center-line) increases.
In addition, the Board noted that the staff mcdel " appears to yield a zero probability for a crash within large segments of angle within the 0-5 mile range."
Our testimuny was focused cn the preselected location of TMI-2.
For this use we felt that our methodology represented a conservative yet reasonable treatment of available data.
Although our methodology was not specifically designed to apply to an arbitrary point in the (r, e) plane it is certainly possible to do so,with modifications for " edge effects" and zero-valued estimates, as follcn.
To be consistent with the data ava;lable, we confine our attention to that portion of the (r, 0) plane defined by 0 5r1 5 and 0 5 e < 90 We omit consideration of angles greater than 90' because the data indicates that the density appears to be increasing in this regicn, and our methodology is designed for a decreasing density.
Because of the apparent special situation for angles greater than 90, it is our judgment that this case deserves special study before an appropriate estimator can be devised.
The approach descriLed in our testimony '.ns to l'ase our estimates of the crash densities for r and e at a point (r, e ) cn the obscrved number g
g of hits in an interval of width 1.5 niles roughly centered at r and an g
interval of width 15' roughly centered at e.
This approach can be applied g
throughout the region of interest except where r or e is near the edge g
g of the region.
Accordingly, the edge effect modification to our methodology is to base our estimates on the number of hits ir, the half-nile wide intervals
[0, 0.5] for 0 < r < 0.5 and [4.5, 5.0] for 4.5 1rg1 5.0, respectively, g
and on the 5 wide interval [0, 5'] for 0 < e
< 5*.
(Note th t the 15' wide g
interval [80, 95 ] is used if 85* 1 eg1 90.
Even though we do not estimate the density for e
> 90, using the cbserved hits for e
> 90 still g
g leads to a conservative estimate of density for 85' s eg1 90.)
13 N
_4-The second mcdificaticn to cur methodology is designed to obtain a non-zero estimate for the crash density at all points.
There are several ways to do this.
The one m use is to assume one additional hit at r or e g
g for those points for which the modified tethodology described above would yield an estimate of zero for the crash density of either r or 0.
The additicn of this pseudo-hit raises both the numerator and dencninator by one and yields a ccnservative non-zero estimate o# the crash density.
The results of applying this redified methed: logy to the data in Tables 9A and 95 are shcwn in Figs.1 and 2.
Of the four estinated crash densitim, three are essentially monotonically decreasing and t:'e fourtii (the crash density of e for takeoffs) is somewhat irreguicr.E For TM:-2, 2.7 miles and e
= 34.
Frca Figs.1 and 2, gT(r ) =.133, gL(r ) =.133, r =
g g
g g
n.T(O ) =.764, h (So) =.143.
Since the mcdifications discussed ahnve were o
L not needed at the TMI-2 location, these estimates are identical to those on page 7 of our testimony.
It is worth noting that the areas under the four estimated densities in 4igs. 1 and 2 are a!l slightly greater than one.
This is a reflection of the conservatisc.s intrcduced by the modifications for edge effects and zero-valued estimates.
This phencmenon is not a matter for concern, since the purpose of our modified methodology is to obtain point estimates rather than estimates of the ca si ties as a whole.
If the latter were required, then an adjustment could be made so that the areas would be equal to one.
1/
- Duc mainly to the absence of cbserved takeoff hits between 5' and 20.
This does not imply that the "true" density is zero between 5' anc 20 Because of the small total numoer of hits (15), a considerable degree of irregularity is to be expected in the estimated density.
9 n.T $.
- 0. 5 - - -
t 1
I
- L__,
TAKEOFFS 0.4
- - - - - - LANDI NGS
- 0. 3 -
I g( r) i
- 0. 2 -
1 I
L __,
t L---
- u__,
0.1 _
i
- L__,
L__,
t_ _ _
0 i
r i
4 i
1 2
3 4
3 r(miles)
FIGURE 1.
ESTIMATED CRASH DENSITY FOR r (PER MILE) 1.T M"P
- 3. 5 -
- 3. 0 -
- 2. 5 -
TAKEOFFS LANDINGS 2.0 -
h(e)
- 1. 5 -
q 1.0 -
0.5 -
r- ;
~~
t
'- L r-y i
L___________..___.___J 0
10 20 30 40 50 60 70 80 90 e(degrees)
FIGURE 2 ESTIMATED CRASH DENSITY FOR 9 (FER RADIAid 1.'? 338 P
ALAB-525 also discussedU our calculation of confidence limits for the areal crash densities and adduced reasons why our confidence limits might be overly conservative.
While the observations made by ALAB-525 are well-taken,
the approach we used appeared to us as the only feasible one at the time we developed our tmtimony.
While, in principle, exact confidence limits can be determined from the model assumptions and the observed data, this would involve very extensive computations for tr' case at hand.
As an alternative, we used the Bonferroni method of calculating bounds for the exact confidence limits.
Since this method is a very general one, it yields bounds which might be overly conservative fer any particular case.3/
Fucthermore, the Bonferroni method yields only upper bounds on the exact confidence limits, so that no estimcte of the degree of conservatism is possible.
Cespite these drawbacks, we were unawre of any other feasible approach, and so we used the conservative Bonferroni bounds as presented in Table IV of our testimony.
Upon reviewing our testimony, both written and oral, we have subsequently discovered that it is possible to calculate less conservative upper bounds on the exact confidence limits.
Furthermore, it is possible to also calculate a lower bcund for the exact 9C% confidence limits.
Since the upper and icwer bounds for the 90% confidence limits c,enerally differ by a factor of two, it 2_/The description of confidence level given by footnote 9 on page 12 of ALAB-525 is misleading.
Ar. upper confidence limit L is the endpoint of a random interval
'0,L).
The confidence level is the probability that the interval (0, L) will cover the unknown parameter.
Since the parameter is fixed, no probability is associated with it.
E0ur bounds were cbtained by multiplying tnree ccnfidence limit;, each with confi5nce (1 - g), and calling the product a bound on a confidence limit with confidence level (1 - 39).
As was pointed out by ALAB-5ES, there is an intrinsic conservatism in this calculatica, regardless of tne degree of independence or dependence among the three factors.
1'd 3 3 9
. is our judgment that our revised procedure for calculating bounds on the exact confidence limits yields values wr not overly c'.nservative and therefore no further refinements are felt to be worthwhile.
The revised bounds on the exact confidence limits are presented in Table IV (revised) for 80,5 and 90 confidence levels, together with the corresponding values frcm Table IV of our testimony.
Because the Benferroni method is applied to only two factors in our revised approach, ile it was applied to three factors in our original testimony, the 803 and 90!; con-fidence level bounds in Table IV (revised) correspond to the 705 and 855 confidence level bounds in Table IV, respectively.
(There are no revised bounds in Table IV (revised) corresponding to the 97% bounds in Table IV because there were no corresponding tables in the source paper we used for our. revised approach.)
A: compared with the bounds in Table IV, the revised bounds in Table IV (revised) show a double improvement.
First, the confidence levels are increased and second, the upper bounds on the exact confidence limits ara decreased.
Furthermore, except for the relatively unimportant case cf S
nonscheduled takeoffs !, the upper and los bounds for the 90% level differ by about a.' actor of two.
i 4/
- Frcm pace 15 of the tes timony of Darrell G. Eisenhut, nonscheduled takeoffs contribute less than 10% to Ptotal, the probability of a " heavy" aircraf t impacting iMI-2.
IT 310
_9_
BCUt!DS ON EXACT C0iiFICE? ICE LIMITS ( x 10~9) 70%
1 80%
11 853 1
90%
l Bound Upper Upper f
Upper Lower Upper ESTIMATED Bound Bound Boand i
Sound SCHEDULED RATE (Table IV)
(revised)
(TableIV)]
(revi ed) s
~9 TAKEOFFS 4.9 x 10 36 23 53 15.4 32.3
-9 LANDIIiGS 2.0 x 10 10 7
13 5.3 9.0 fiONSCHEDULED
~9 TAKEOFFS 32 x 10 420 273 670 103
'409 LANDIliGS I
~9 39 x 10 210 148 290 101 196 I
i i
Table I'/ (revised).
Estimated values and bounds on exact confidence limits for areal crash densities at TMI-2 for a U. S. carriet aircraft engaged in a relevant cperation (prcbability per square mile)
The source of this revision was the discovery that the estimates of the crash densities g(r) and h(s) (see page 7 of our testimony) are statistically independent for both takeoffs and landings.
This fact allcws us to calculate approximateE 90% and 951 confidence 1imits for the conditicnal crash densities D (fo' 0 ) and T
0 D (#o' 0 ).
(See " Confidence Intervals for the Product of Two Binomial Parameters",
L 0
by Rober+
2hler, Journal of the American Statistical Asscciation, Cecember 1957, 482-49..)
The appgoximate confidence limits are then multiplie: by the
,1 3/
" Based cn the Poisson acproximation to the bincaial.
As discussed en page 5 of the apperdix to our testimony, this approximation yields conservative confidence 1imits.
1.T N41
. approximate confidence limits for the off-runway crash rates to get conservative confidence limits for the areal crash densities using the Boaferroni method discussed on page 5 of the appendix.
The lower bound for the 90% confidence limit is obtained by multiplying the approximate 90%
cor.fidence limits for the conditional cra_, densities by the es timated off-runway crash probabilities rrom Table I on page 2 of our testimony.
If the values from Table I were equal to the true crash rates, this procedure would yield approximate 905 confidence limits for the areal crash densities, but since the values in Table I are only estimates of the true crash rates, this procedure yields lower bounds.
It should be noted that it is not all obvious that the estiraated crash densities for r and e are statistically independent.
From Table 9A for takeoffs, the three hits in 2 < a < 3.5 and the three hits in 25 < 0 < 40 have one hit in common and we believed that this ccomon hit would induce a positive correlation between the estimates of the crash densities as calculated on page 7 of our testimony.
The relevant data is summarized in the following table of takeoff hits.
Radial Distancc Ancular Distribution (miles)
[0, ?5 )
[25, 40 )
[40, 100 ]
I
[0, 2.0 )
5 1
3 9
[2.0,3.5) 0 1
2 3
[3.5,5.0]
2 1
0 3
7 3
5 15 1T 302
. Our model assumes that each of the 15 takeoff hits impacts in one of the nine boxes of the table according to the following joint distribution, where p; and q are probabilities such that p) + p2 + P3 *91+92+93
- l' Radial Distance Angular Distribution (miles)
[0, 25 )
[25, 40 )
[40', 100 ]
[0, 2.0 )
p)q) p)q2 P913 Pl
[2.0, 3.5 )
p2 1 9
P922 P923 P 2
[3.5,5.0]
p9 P932 P933 P
31 3
9j 92 9
I 3
The assumption that the probability of a hit in any box is the product of the marginal probabilities is equivalent to the assumption on page 4 of our testimony that "r and c for off-runway hits are distributed independently".
For the care at hand, the problen is to estimate p2 2, the probability 9
that an off-runway crash will impact in the box including TMI-2k There are two ways to do the estimation.
The first is to use the the ratio of the observed number of hits in the box including TMI-2 to the total number of off-runway hits.
For this case, the estimate would be 1/15 =.057.
The 5/ e conditional crash density 0(r, s ) estimated in our testimony differs Th o
o frcm p292 by a normalization factor wnich yields the probability of impact per square mile.
This normalization factor is an exact quantity and it is coitted in this discussion for convenience and to allcw us to focus most directly on the statistical issues.
Its caission does not affect any of these statistical issues.
1T 343
. second - thod is to estimate p2 nd separately and then multiply.
For 2
3 3
this case. the estimate would be 3 g =.04.
We use the secon.re thod because it has smaller variance than the first.
To consider
..e issue of independence, denote the estimates of p2 and by h and h, respectively.
Even though we have model independence as q2 2
2 exoressed in the joint distribution table above, this does not necessarily implythath nd h are statistically independent.
For this case 2
2
_0+1+2
^
P2 15
_1+1+1
^
92 15 where we have deccmposed h and q ccording to the cbserved data.
In 2
2
- general,
_ "21 + "22 + "23 P2 N
_ "12 + "22 + "32
^
92 N
where n.. is the observed number of hits in row i and column j of the 1J 3 x 3 data table and N is the total number of hits.
It is the presence of n in both p2 and h that led us to believe that h and h
"#8 "Ut 22 2
2 2
statistically independent and, in fact, are positively correlated.
- However, because the total number of hits is fixed, n is negatively correlated with 22 all of the other n ) and, in particular, n22 j
is negatively correlated with g3-345
.. (n21 + "23) nd with (n12 + "32),
It turns out that this negative correlation exactly balances the positive correlation induced by the presence of n so that 22 and h are, in fact, statistically independent.
p2 2
Since the estimated conditional crash density D(r, e ) is the product of g
g g( r) two statistically independent quantities nd h(9), we can use Buehler's r
tables to calculate approximate confidence limits for the conditional crash densities for takeoffs and landings.
However, the estimated conditional crash densities are not independent of the estimated accident rate, since both depend on the same set of accidents and there is no mechanism to cancel out this dependence.
(Nuterical calculation indicates that the confidence limits for the accident rate and the conditional crash density are negatively correlated.)
It is for this reason that we use the Bonferroni method to calculate bounds on the exact ccnfidence limits for the areal crash densities.
12~345