ML19206B337
| ML19206B337 | |
| Person / Time | |
|---|---|
| Site: | Crane  |
| Issue date: | 03/16/1979 |
| From: | Moore R NRC OFFICE OF MANAGEMENT AND PROGRAM ANALYSIS (MPA) |
| To: | |
| Shared Package | |
| ML19206B328 | List: |
| References | |
| NUDOCS 7905090224 | |
| Download: ML19206B337 (14) | |
Text
UNITED STATES OF AMERICA NUCLEAR REGULATORY COMMISSION BEFORE THE ATOMIC SAFETY AND LICENSING APPEAL BOARL In the Matter of
)
)
METROPOLITAN EDIS0ti COMPANY, ET AL.
)
Docket No. 50-320
)
(Three Mile Island Nuclear Station,
)
Unit 2)
)
SUPPLEMENTAL TESTIMONY OF R. MOORE AND L. ABRAMSON IN RESPONSE TO ALAB-525 APPLIED STATISTICS BRANCH OFFICE OF MANAGEMENT AND PROGRAM ANALYSIS U.S. NUCLEAR REGULATORY COMMISSION March 16, 1979 24 101 TD05090 n 4<
0
In ALAS-525 (February 1,1979), the Board made several comments on the methodology of estimating the areal crash density at TMI-2 as presented in our testimony submitted at the December 12 hearing session in this proceeding (prefiled testimony following Tr. 378).
The purpose of this addition is to respond to the Board comments.
Before responding to the specific comments, however, we feel that a short discussion of the rationale for our methodology would be helpful.
Our choice of approach was motivated by the requirement to analyze the uncertainty associated with the point estimates. There are two sources of uncertainty -
model uncertainty and statistical uncertainty.
Model uncertainty stems from the particular choice of assumptions made about the underlying relations among the problem parameters and variables and the possibility that the assumptions might be in error.
Statistical uncertainty stems from the random nature of the observations and the possibility that the observations might not be representative of the assumed model.
One way to handle model uncertainty is to choose the model assumptions such that any plausible departure from them would be in a conservative direction.
This is the approach we adopted.
Before adopting our particular model assumptions, we reviewed the applicant's approach involving the choice of a specific functional form for the conditional crash densities.
This can be a useful approach, provided that the assumed functional form is correct, Lince it makes use of all the data to estimate the unknown parameters.
However, if the assumed functional form is incorrect, then using it can lead to significant estimation errors.
Instead of trying to use data distant from TMI-2 to estimate the probability of a hit at TMI-2, we based 24 102 our estimates only on data in the vicinity of TMI-2.
(By the assumption of model independence between r and e, justified on pages 3-4 of our testimony, we treat r and e separately.) Our estimates are then based on the mild assumption that the conditional crash densities for r and e are approximately constant in the vicinity of TMI-2.
If the conditional crash densities are not approximately constant, then it is plausible to assume that they would be concave decreasing (see Tables 9A and 98, revised 12/8/78), so that a chord joining any two points on the curve would lie wholly above the curve.
(The exponential form assumed by the applicent has this property.)
In such a case, an estimate made with our methodology would be conservatively biased, i.e., it would tend to overestimate the density.
In addition to minimizing the model uncertainty, our methodology was also designed to estimate the statistical uncertainty in a straightforward This we did by the method of confidence intervals, which require nc manner.
extra assumptions in addition to those already made for the point estimate.
In contrast, the applicant assumed a p.-ior distribution and a likelihood function in order to apply a Bayesian analysis to estimate the uncertainty.
Thus, our methodology requires fewer and weaker (i.e., assuming less) assumptions than does the applicant's methodology, both for the point estimates and the uncertainty analysis.
On page 12 of ALAS-525, the Board referred to the "very irregular angular probability distribution" produced by the staff's methodology and claimed that it fails to decrease regularly as the angle e (measured from the runway center-line) increases.
In addition, the Board noted that the staff model " appears to yield a zero probability for a crash within large segments of angle withi-the 0-5 mile range."
24 103 Our testimony was focused on the preselected location of TMI-2.
For this use we felt that our methodology represented a conservative yet reasonable treatment of available data.
Although our methodology was not specifically designed to apply to an arbitrary point in the (r, e) plane it is certainly possible to do so,with modifications for " edge effects" and zero-valued estimates, as follows.
To be consistent with the data available, we confine our attention to that portion of the (r, e) plane defined by 0 1r1 5 and 0 $e1 90. We omit consideration of angles greater than 90 because the data indicates that the density appears to be increasing in this region, and our methodology is designed for a decreasing density.
Because of the apparent special situation for angles greater than 90, it is our judgment that this case deserves special study before an appropriate estimator can be devised.
The approach described in our testimony was to base our estimates of the crash denaities for r and e at a point (r, e ) on the observed number g
g of hits in an interval of width 1.5 miles roughly centered at r and an g
interval of width 15 roughly centered at e. This approach can be applied g
throughout the region of interest except where r or e is near the edge g
g of the region.
Accordingly, the edge effect modification to our methodology is to base our estimates on the number of hits in the half-mile wide intervals
[0, 0.5] for 0 c r < 0.5 and [4.5, 5.0] for 4.5 1rg1 5.0, respectively, g
and on the 5 wide interval [0, 5 ] for 0 < e
< 5.
(Note that the 15 wide g
interval [80, 95 ] is used if 85 1eg1 90.
Even though we do not estimate the density for e > 90, using the observed hits for o > 90 still g
g leads to a conservative estimate of density for 85 seg1 90.)
o.^,
1,04 c
The second modification to our methodology is designed to obtain a non-zero estimate for the crash density at all points.
There are several ways to do this.
The one we use is to assume one additional hit at r or e g
g for those points for which the modified methodology described above would yield an estimate of zero for the crash dens.ty of either r or e.
The addition of tnis pseudo-hit raises both the numerator and denominator by one and yields a conservative non-zero estimate of the crash density.
The results of applying this modified methodology to the data in Tables 9A and 93 are shown in Figs.1 and 2.
Of the four estimated crash densities, three are essentially monotonically decreasing and the fourth (the crash density of e for takeoffs) is somewhat irregular.E or TMI-2, F
r = 2.7 miles and e = 34.
From Figs.1 and 2, gT(r ) =.133, gL(r ) =.133, g
g g
g h (9 ) =.764, h (9 ) =.143.
Since the modifications discussed above were T 0 L 0 not needed at the TMI-2 location, these estimates are identical to those on page 7 of our testimony.
It is worth noting that the areas under the four estimated densities in Figs.1 and 2 are all slightly greater tnan one. This is a reflection of the conservatisms introduced by the modifications for edge effects and zero-valued estimates.
This phenomenon is not a matter for concern, since the purpose of our modified methodology is to obtain point estimates rather than estimates of the densities as a whole.
If the latter were required, then an adjustment could be made so that the areas would be equal to one.
E Due mainly to the absence of observed takeoff hits between 5 and 20.
This does not imply that the "true" density is zero between 5 and 20 Because of the small total number of hits (15), a considerable degree of irregularity is to be expected in the estimated density.
u.,
10 5 g
i
- 0. 5 -- -
t I
I
- L__,
TAKEOFFS 0.4 r
- - - - - - LAND I f1G S
- 0. 3 -
-- m i
g(r)
,__q a
1 0.2 I
L._ _,
i L._--
u-_,
0.1 _
i
- L__,
L__
0 i
r 6
i e
a i
1 2
3 4
5
-(miles)
FIGURE 1.
ESTIMATED CRASH DENSITY FOR r (PER MILE) 24 106
- 3. 5 ~
- 3. 0 - --
- 2. 5 -
TAKEOFFS IDINGS 2.0 -
h(e)
- 1. 5 -
._q 1.0 -
~~
0.5 -
_ _F 1t t_ _
' -t r- -
L___.
______s 0
10 20 30 40 50*
60 70 80 90 e(degrees)
FIGURE 2 ESTIMATED CRASH DENSITV FOR e (PER RADIAN) 24 107
. ALAB-525 also discussed 2/ our calculation of confidence limits for the areal crash densities and adduced reasons why our confidence limits might be overly conservative. While the observations made by ALAB-525 are well-taken,
the approach we used appeared to us as the only feasible one at the time we developed our testimony. While, in principle, exact confidence limits can be determined from the model assumptions and the observed data, this would involve very extensive computations for thc case at hand.
As an alternative, we used the Bonferroni method of calculating bounds for the exact confidence limits.
Since this method is a very general one, it yields bounds which might be overly conservative for any particular case.El Furthermore, the Bonferroni method yields only upper bounds on the exact confidence limits, so that no estimate of the degree of conservatism is possible.
Despi te these drawbacks, we were unawart of any other feasible approach, and so we used the conservative Bonferroni bounds as presented in Table IV of our testimony.
Upon reviewing our testimony, both written ar.d oral, we have subsequently discovered that it is possible to calculate less conservative upper bounds on the exact confidence limits.
Furthermore, it is possible to also calculate a lower bound for the exact 90% confidence limits.
Since the upper and lower bounds for the 90% confidence limits generally differ by a factor of two, it 2/ ne description of confidence level given by footnote 9 on page 12 of ALAB-525 T
is misleading. An upper confidence limit L is the endpoint of a random interval (0,L).
The confidence level is the probability that the interval (0, L) will cover the unknown parameter.
Since the parameter is fixed, no probability is associated with it.
5/ ur bounds were obtained by multiplying three confidence limits, each with 0
confidence (1 - g), and calling the product a bound on a confidence limit with confidence level (1 - 3 ).
As was pointed out by ALAB-525, there is an intrinsic 9
conservatism in this calculation, regardless of the degree of independence or dependence among the three factors.
9 !s, 108 c
i is our judgment tnat our revised procedure for calculating bounds on the exact confidence limits yields values which are not overly conservative and therefore no further refinements are felt to be worthwhile.
The revised bounds on the exact confidence limits are presented in Table IV (revised) for 80% and 90% confidence leve ls, together with the a
corresponding values from Table IV of our testimony.
Because the Bonferroni method is applied to only two factors in cue revised approach while it was applied to three factors in our original testimony, the 80% ar.d 90% con-fidence level bounds in Table IV (revised) correspond to the 70% and 85%
confidence level bounds in Table I'!, respectively.
(There are no revised bounds in Table IV (revised) corresponding to the 97% bounds in Table IV because there were no corresponding tables in the source paper we used for our revised approach.)
As compared with the bounds in Table IV, the revised bounds in Table IV (revised) show a double improvement.
First, the confidence levels are increased and second, the ucper bounds on the exact confidence limits are decreased.
Furthermore, except for the relatively unimportant case of nonscheduled takeoffs /S the upper and lower bounds for the 90% level differ by about a factor of two.
$/ rom page 15 of the testimony of Darrell G. Eisenhut, nonscheduled takeoffs F
contribute less than 10% to P.otal, the probability of a " heavy" aircraft impacting TMI-2.
7A 1, @s.9 w
. BOUNDS ON EXACT C0ilFIDEf4CE LIMITS ( x 10-9) 1 70%
i 80%
1 85%
90%
Upper Upper Upper Lower Upper ESTIMATED Bound Bound Bound Bound Bound SCHEDULED RATE (Table IV)
(revised)
(Table IV)
(revised)
-9 TAKEOFFS 4.9 x 10 36 23 53 15.4 32.3
-9 LANDIliGS 2.0 x 10 10 7
13 5.3 9.0 tiONSCHEDULED
-9 TAKEOFFS 32 x 10 420 273 670 103 409
-9 LANDINGS 39 x 10 210 148 290 101 196 Table IV (revised).
Estimated values and bounds on exact confidence limits for areal crash densities at TMI-2 for a U. S. carrier aircraft engaged in a relevant operation (probability per square mile)
The source of this revision was the discovery that the estimates of the crash densities g(r) and h(s) (see page 7 of our testimony) are statistically independent for both takeoffs and landings.
This fact allows us to calculate approximate _/
5 90% and 95% confidence limits for the conditional crash densities D-(
, e ) and g
g D (r, e ).
(See " Confidence Intervals for the Product of Two Binomial Parameters",
g g
by Robert J. Buehler, Journal of the American Statistical Association, December 1957,482-493.) The approximate confidance limits are then multiplied by the 5/ ased on the Poisson approximation to tl.e binomial.
B As discussed on page 5 of the appendix to our testimony, this approximation yields conservative confidence limits.
24 110
, approximate confidence limits for the off-runway crash rates to get conservative confidence limits for the areal crash densities using the Bonferroni method discussed on page 5 of the appendix.
The lower bound for the 905 confidence limit is obtained by multiplying the approximate 90*;
confidence limits for the conditional crash densities by the estinated off-runway crash probabilities from Table I on page 2 of our testimony.
If the values from Table I were equal to the true crash rates, this procedure would yield approximate 90% confidence limits for the areal crash densities, but since the values in Table I are only estimates of the true crash rates, this procedure yields lower bounds.
It should be noted that it is not all obvious that the estimated crash densities for r and e are statistically independent.
From Table 9A for takeoffs, tne three hits in 2 < e < 3.5 and the three hits in 25 < e < 40 have one hit in common and we believed that this common hit would induce a positive correlation between the estimates of the crash densities as calculated on page 7 of our testimony.
The relevant data is summarized in the following table of takeoff hits.
Radial Distance Anaular Distribution (miles)
[0, 25 )
[25, 40 )
[40, 100 ]
[0, 2.0) 5 1
3 9
[2.0,3.5) 0 1
2 3
[3.5,5.0]
2 1
0 3
7 3
5 15
}.0, \\\\
, Gud modG1 assumes that each of the 15 takeoff hits impacts in one of the nine boxes of the table according to the following joint distribution,
- I" where p; and q are probabilities such that p) + p2 + P3 *91+92+93 Radial Distance Ancular Distribution (miles)
[0, 25 )
[25, 40 )
[40, 100 ]
[0, 2.0 )
p)q) p)q2 P1 P913
[2.0, 3.5 )
p2 1 P2 P923 9
P922
[3.5,5.0]
p3 1 P
P933 3
9 P932 91 q2 9
I 3
The assumption that the probability of a hit in any box is the product of the marginal probabilities is equivalent to the assumption on page 4 of our testimony that "r and e for off-runway hits are distributed independently".
For the case at hand, the problem is to estimate p2 2, the probability 9
that an off-runway crash will impact in the box including TMI-2_{ There are 6
two w?ys to do the estimation.
The first is to use the the ratio of the observed number of hits in the box including TMI-2 to the total number of off-runway hits.
For this case, the estimate would be 1/15 =.067.
The S e conditional crash density D(r, 9 ) estimated in our testimony differs Th o
0 from p242 by a normalization factor which yields tne probability of impact per square mile.
This normalization factor is an exact quantity and it is omitted in this discussion for convenience and to allow us to focus most directly on the statistical issues.
Its omission
- es not affect any of these statistical issues.
24 112
. second method is to estimate p2 and q2 separately and then multiply.
For h=.04.
this case, the estimate would be We use the second method because it has smaller variance than the first.
To consider the issue of independence, denote the estimates of p2 and by h and k, respectively.
Even though we have model independence as q2 2
2 expressed in the joint distribution table above, this does not necessarily imply that h and k are statistically independent.
For this case 2
2
_0+1+2 P2 15
_1+1+1 2
15 where we have decomposed h and h according to the observed data.
In 2
2
- general,
- _ "21 + "22 + "23 p2 N
^, "12 + "22 + "32 92 N
where n;) is the observed number of hits in row i and column j of the 3 x 3 data table and N is the total number of hits.
It is the presence of n in both h and h that led us to believe that h and h are not 22 2
2 2
2 statistically independent and, in fact, are positively correlated.
- However, because the total number of hits is fixed, n is negatively correlated with 22 all of the other n and, in particular, n is negatively correlated with jj 22 24 113
. (n21 * "23) and with (n12 * "32).
It turns out that this negative correlation exactly balances the positive correlation induced by the presence of n so that 22 p2 and q2 are, in fact, statistically independent.
Since the estimated conditional crash density D(r, e ) is the product of g
g g(r) two statistically independent quantities nd h(e), we can use Buehler's r
tables to calculate approximate conficence limits for the conditional crash densities for takeoffs and landings. However, the estimated conditional crash densities are not independent of the estimated accident rate, since both depend on the same set of accidents and there is no mechanism to cancel out this dependence.
(Numerical calculation indicatas that the confidence limits for the accident rate and the conditional crash density are negatively correlated.)
It is for this reason that we use the Bonferroni method to calculate bounds on the exact confidence limits for the areal crash densities.
} !,
\\.