ML19322D374

Joint Affidavit Providing Analysis of Effect of Updated Data on Previously Submitted Testimony & Supplemental Testimony Re Aircraft Accidents

ML19322D374
Person / Time
Site:	Crane
Issue date:	02/01/1980
From:	Abramson L, Moore R NRC OFFICE OF MANAGEMENT AND PROGRAM ANALYSIS (MPA)
To:
Shared Package
ML19322D370	List: ML19322D369 ML19322D371 ML19322D372 ML19322D373 ML19322D374 ML19322D375
References
NUDOCS 8002120336
Download: ML19322D374 (7)

Text

'

O UNITED STATES OF AMERICA NUCLEAR REGULATORY COMMISSION BEFORE THE ATOMIC SAFETY AND LICENSING APPEAL BOARD In the Matter of

)

)

METROPOLITAN EDISON COMPANY, ET AL.

)

Docket No. 50-320

)

(Three Mile Island Nuclear Station,

)

Unit 2)

)

JOINT AFFIDAVIT OF ROGER H. MOORE AND LEE R. ABRAMSON STATE OF MARYLAND

)

COUNTY OF MONTGOMERY)

I, Roger H. Moore, being duly sworn, depose and state:

1.

I am the Chief, Applied Statistics Branch, Office of Management and Program Analysis, U.S. Nuclear Regulatory Commission, Washington, D.C.

20555.

2.

I have previously testified in this proceeding and my Statement of 1

Professional Qualifications is incorporated in the transer.ipt (following Tr. 373).

I, Lee R. Abramson, being duly sworn, depose and state:

3.

I am a statistical adviser, Applied Statistics Branch, Office of Management and Program Analysis, U.S. Nuclear Regulatory Commission, Washing ~ ton, D.C.

20555.

4.

I have previously testified in this proceeding and my Statement of Professional Qualifications is incorporated in the transcript (following Tr. 374).

800232n334

, We, the affiants Roger H. Moore and Lee R. Abramson, being duly sworn, depose and state:

5.

We are jointly responsible for and participated in the preparation of the attached document entitled " Analysis of the Effects of Updated Data on the Previously Submitted Testimony and Sapplemental Testimony of

't. Moore and L. Abramson", and, if called upon, we would testify as set forth therein.

Q r,

f\\ n. J k hOmr Roger H. Moore

'e s,'

,Af l'~ (!(j

_ iv, vA u Lee R. Abramson Subscribed and sworn to before me this / E day of February, 1980 4

J_kk Notary Public

! /YNe My Commission expires:

41

/

V

ANALYSIS OF THE EFFECTS OF UPDATED DATA ON THE PREVIOUSLY SUBMITTED TESTIMONY AND SUPPLEMENTAL TESTIMONY OF R. MOORE AND L. ABRAMSON Since the hearing on December 11-12, 1978, an additional year's worth of data (1978) has become available. We were provided with this updated data and requested to examine the effects of this adoitional data on our testimony as submitted November 30, 1978 and revised December 8,1978 (to be denoted by "T") and on our supplemental testimony submitted March 16, 1979 (to be denoted by "ST").

With the addition of the 1978 data, Table I in "T" is updated as follows:

SCHEDULED NONSCHEDULED OPERATIONS HITS RATE OPERATIONS HITS RATE 6

6 (X10)

(X10-6)

(X10 )

(X10-6)

TAKE0FFS 90.9 11 0,12 2.47 2

0.81 LANDINGS 90.9 26 0.29 2.47 14 5.67 Table I (updated). Off-runway destruct accidents for all U.S. carrier aircraf t, 1956-1978.

Since there were no off-runway destruct takeoff accidents in 1978, our estimate of the conditional crash density for takeoffs as given in "T" does not change.

From Table 4 Addition, the 1978 data increases the total number of takeoff accidents from 40 to 42 and the number of hits in the interval 2.0 < r < 3.5 surrounding TMI from 8 to 10. The updated values for gt(r )

g and hL (e,) are

. gt(r ) = h h =,.159 per mile g

h (e ) = h

• f =.136 per radian.

L g As on page 8 of "T", the updated estimate of the conditional crash density for landings is D (r, e ; = (.159}I2.f I =.00801 per square mile.

g g g

The estimate areal crash densities based on the 23-year period 1956-1978 is given by Table III (updated).

SCHELULED NONSCHEDULED

-9

-9 TAKE0FFS 4.6 x 10 30 x 10

-9

-9 LANDINGS 2.3 x 10 45 x 10 Table III (updated). Estimated areal crash densities at TMI-2.for a U.S. carrier aircraft engaged in a relevant operation (probability per square mile).

The inclusion of the 1978 data results in a decrease of 6 percentE in the estimated areal crash densities for takeoffs and an increase of 15 percentUin the areal crash densities for landings.

The magnitudes of these changes are not at all surprising, since they are well within the un-certainty bands on the estimates given in "T"..

E cheduled takeoffs dec5 ease from 4 9 x 10-9 to 4.6 x 10-9 and nonscheduled S

takeoffs from 32 x 10 ' to 30 x 10-9 E cheduled landings incbease from 2.g x 10

-9 to 2.3 x 10-9 and nonscheduled S

landings from 39 x 10- to 45 x 10

. Since the uncertainty results in "T" were replaced by the 'evised uncertainty analysis of "ST," we see no point to updating the confidence limits given in "T".

The inclusion of the 1978 data has very little effect on our supple-mental testimony of March 16, 1979. Since there were no off-runwav destruct takeoff accidents in 1978, the estimated crash densities for takeoffs from figures 1 and 2 remain unchanged. The updated estimated crash densities for landings look very much like their counterparts in "ST" and are given in Figures 1 and 2 (updated). No change in the discussion is necessary.

In the uncertainty analysis, the bounds on the exact confidence limits change by a maximum of 16 percent] and are presented in Table IV (revised 3

and updated).

In thiL table, the two columns headed " Upper Bound (Table IV)"

have been updated with the 1978 data.

B0UNDS ON EXACT CONFIDENCE LIMITS ( x 10-9) 70%

80%

85%

90%

Upper Upper Upper Lower Upper ESTIMATED Bound Bound Bound Bound Bound SCHEDULED RATE (Table IV)

(revised)

(Table IV)

(revised)

-9 TAKEOFFS 4.6 x 10 34.2 22.1 50.3 14.6 30.6

~9 LANDINGS 2.3 x 10 10.2 7.7 14.0 5.9 9.7 NONSCHEDULED

~9 TAKEOFFS 30 x 10 401 261 640 98 390 LANDINGS 45 x 10-9 224 168 31 2 117 21 5 TableIV(revisedandupdated). Estimated values and bounds on exact confidence limits for areal crash densities at TMI-2 for a U.S. carrier aircraft engaged in a relevant operation (probability per square mile)

E e bounds for takeoffs decrease by about 5 percent and the bounds for Th landings increase from 8 to 16 percent.

. l 0.5 -

i 1

_q

)

I l

l TAKEOFFS

'-~~

0.4

~~~---

LANDINGS 0.3 O

a g(r) 1 0.2 -

i 1

lL ___ _,

L__,

O.1

'--]

I Il-3

.n

'~

~'

0 l

1 2

3 4

5 r(miles)

FIGURE 1 (UPDATED).

ESTIMATED CRASH DENSITY FOR r (PER MILE) l l

l

l

_ 3.5 "

3.0 -...

2.5 "

TAKEOFFS

~~~~~~~

LANDINGS 2.0 -

h(e)

1. 5 -

~~I

1. 0 -

0. 5 -

...T m 1

L3 LT r --

w J

0 10*

20*

30*

40*

50*

60*

70*

80*

90*

e(degrees)

FIGURE 2 (UPDATED).

ESTIMATED CRASH DENSITY FOR e (PER RADIAN)

.-2 4.-,.,.

- - - - - *,A_.msa-J 4

v-4

-a6_.~

,_e,*-.s.a.2.m-m-.

e.e z.2-s

.f

.\\ -

e i

s i

)

4 ATTACHMENT 2 i

SUPPLEMENTAL TESTIMONY OF R. MOORE AND i

L. ABRAMSON IN RESPONSE TO ALAB-525 k

1 4

i 5

l 1

4 I.

i t

i t i

I i

i s

a i

(

l l

e L__

q l

e a -

A A[

OAAA

, n, py 0

4 i

r i

w-f

'y 9-g-

.-w+

,w-wa--

g_g3,

+

w

+

-,--,---'-p w

e, e n--

r--p---g

.e,-ge g,--,y--

w-wy g

g-'-

O i

1 UNITCD STATES OF AMERICA NUCLEAR REGULATORY COMMISSION BEFORE THE ATOMIC SAFETY AND LICENSING APPEAL BOARD In the Matter of METROPOLITAN EDISON COMPANY, ET AL.

Docket No. 50-320 (Three Mile Island Nuclear Station, Unit 2)

SUPPLEMENTAL TESTIMONY OF R. MOORE AND L. ABRAMSON IN RESPONSE TO ALAB-525 APPLIED STATISTICS BRANCH OFFICE OF MANAGEMENT AND PROGRAM ANALYSIS U.S. NUCLEAR REGULATORY COMMISSION March 16, 1979 h u*'

qc)o5b9 0 9 4

i i

In ALAB-525 (February 1,1979), the Board made several coments on the methodology of estimating the areal crash density at TMI-2 as presented in our testimony submitted at the December 12 hearing session in this proceeding

~

l (prefiled testimony following Tr. 378).

The purpose of this addition is to respond to the Board coments.

.\\

l Before responding to the specific comments, however, we feel that a short discussion of the rationale for our methodology would be helpful. Our choice of approach was motivated by the requirement to analyze the uncertainty associated with the point estimates. There are two sources of uncertainty -

model uncertainty and statistical uncertainty.

Model uncertainty stems from the particular choice of assumptions made about the underlying relations among the problem parameters and variables and the possibility that the assumptions might be in error.

Statistical uncertainty stems from the random nature of the observations and the possibility that the observations might not be representative of the assumed model.

One way to handle model uncertainty is to choose the model assumptions such that any plausible departure from them would be in a conservative direction. This is the approach we adopted.

Before adopting our particular model assumptions, we reviewed the applicant's approach involving the choice of a specific functional fom for the ccnditiensi crash densities. This can be a useful approach, provided that the assumed functional form is correct, since it makes use of all the data to estimate the unknown parameters.

s However, if the assumed functional form is incorrect, then using it can lead to significant estimation errors.

Instead of trying to use data distant from TMI-2 to estimate the probability of a hit at TMI-2, we based s.

a e

e-

u.

our estimates only on data in the vicinity of TMI-2.

(By the assumption of model independence between r und e, justified on pages 3-4 of.our testimony, we treat r and e separately.) Our estimates are then based on the mild assumption that the conditional crash densities for r and e are approximately constant in the vicinity of TMI-2.

If the conditional crash densities are not approximately constant, then it is plausible to assume that they would be concave decreasing (see Tables 9A and 98, revised 12/8/78), so that a chord ioining any two points on the curve would lie wholly above the curve.

(The exponential form assumed by the applicant has this property.)

In such a case, an estimate made with our methodology would be conservatively biased, i.e., it would tend to overestimate the density.

In addition to minimizing the model uncertainty, our methodology was also designed to estimate the statistical uncertainty in a straightforward manner.

This we did by the method of confidence intervals, which require no extra assumptions in addition to those already made for the point estimate.

In contrast, the. applicant assumed a prior distribution and a likelihood function in order to apply a Bayesian analysis to estimate the uncertainty.

Thus, our methodology requires fewer and weaker (i.e., assuming less) assumptions than does the applicant's methodology, both for the point l

estimates and the uncertainty analysis.

On page 12 of ALAB-525, the Board referred to the "very irregular angular probability distribution" produced by the staff's methodology and claimed that it fails to decrease regularly as the angle e (measured from the runway center-line) increases.

In addition, the Board noted that the staff model " appears to yield a zero probability for a crash within large segments of angle within the 0-5 mile range."

s

~

s

. i Dur testimony was focused on the preselected location of TMI-2.

For this use we felt that our methodology represented a conservative yet reasonable treatment of available data. Although our methodology was not specifically designed to apply to an arbitrary point in the (r, e) plane it is certainly possible to do so.with modifications for " edge effects" and zero-valued estimates, as follows.

To be consistent with the data available, we confine our attention to that portion of the (r, e) plane defined by 0 < r < 5 and 0 < e < 90*.

We omit consideration of angles greater than 90* because the data indicates that the density appears to be increasing in this region, and our methodology is designed for a decreasing density.

Because of the apparent special situation for angles greater than 90*, it is our judgment that this case deserves special study before an appropriate estimator can be devised.

i The approach described in our testimony was to 'm our estimates of the crash densities for r and e at a point (r, e ) cn the observed number g

g of hits in an interval of width 1.5 miles roughly centered at r, and an interval of width 15* roughly centered at e. This approach can be applied g

throughout the region of interest except where r or e, is near the edge o

of the region.

Accordingly, the edge effect modification to our methodology is to base our estimates on the number of hits in the half-mile wide intervals-

[0, 0.5] for 0 < r 2 0.5 and [4.5, 5.0] for 4.5 1ro1 5.0, respectively, o

and on the 5* wide interval [0, 5*] for 0 < eg < 5*.

(Note that the 15* wide interval [80*, 95'] is used if 85* 10,1 90*.

Even though we do not estimate the density for e, > 90*, using the observed hits for e, > 90* still 1eads to a conservative estimate of density for 85* 1 e 1 90*.)

n

o

-4 The second modification to our methodology is designed to obtain a non-zero estimate for the crash density at all points. There are several ways to do this. The one we use is to assume one additional hit at r or e g

g for those points for which the modified methodology described above would yield an estimate of zero for the crash density of either r or o.

The addition of this pseudo-hit raises both the numerator and denominator by one and yields a conservative non-zero estimate of the crash density.

The results of applying this modified methodology to the data in Tables 9A and 98 are shown in Figs.1 and 2.

Of the four estimated crash densities, three are essentially monotonically decreasir.g and the fourth (the crash density of e for takeoffs) is somewhat irregular.E or TMI-2, F

= 2.7 miles and e, = 34*.

From Figs.1 and 2, gT(r ) =.133, gL(r ) =.133, rn g

g h (e ) =.764, h (e ) =.143.

Since the modifications discussed above were T g L n not needed at the TMI-2 location, these estimates are identical to those on page 7 of our testimony.

It is worth noting that the areas under the four estimated densities in Figs.1 and 2 are all slightly greater than one. This is a reflection of the conservatisms introduced by the modifications for edge effects and zero-valued estimates. This plenomenon is not a matter for concern, since the purpose of our modified methodolqv is to obtain point estimates rather than estimates of the densities as a whole.

If the latter were required, then an adjustment could be made so that the areas would be equal to one.

l E e mainly to the absence of observed takeoff hits between 5* and 20.

This Du does not imply that the "true" density is zero between 5* and 20*.

Because of the small total number of hits (15), a considerable degree of irregularity is to be expected in the estimated density.

O e

f

0. 5 --- -]

I 1

IL-,

TAKEOFFS 0.4

- - - - - - LAN D I N G S 1

0. 3 -.

8' 9(r)

__q s

0. 2 ~

l i

L __,

I l__

8---1 0.1 -

i t--_,

L.. 3 8- - - -

0 i

r i

1 a

i 1

2 3

4 5

r(miles)

FIGURE 1.

ESTIMATED CRASH DENSITY FOR r (PER MILE) i I

~

~

-~-

~

t

3. 5 -

'I b

?

1 l

3.0 - --

i

2. 5 -

- TAKEOFFS

- - - - - - LAND I N GS 2.0 -

h(e)

1. 5 -

. 3

1. 0 -

~~

0. 5 -

_.T 1 1

u 1 -1 r- -

L - _ _ _ __.- -

._ _ J i

=

O 10*

20*

30*

40*

50*

60*

70*

80 90*

i e(degrees) l.I l3 FIGURE 2.

ESTIMATED CRASH DENSITY FOR e (PER RADIAN)

I l

J i

I f

i

. ALAB-525 also discussed] our calculation of confidence limits for the 2

areal crash densities and adduced reasons why our confidence limits might be overly conservative. While the observations made by ALAB-525 are well-taken, the approach we used appeared to us as the only feasible one at the time we developed our testimony. While, in principle, exact confidence limits can be determined from the model assumptions and the observed data, this would involve very extensive computations for the case at hand.

As an alternative, we used the Bonferroni method of calculating bounds for the exact confidence limits. Since this method is a very general one, it yields bounds which might be overly conservative for any particular case.E Furthermore, the l

Bonferront method yields only upper bounds on the exact confidence limits, so that no estimate of the degree of conservatism is possible.

Despite these drawbacks, we were unaware of any other feasible approach, and so we used the conservative Bonferroni bounds as presented in Table IV of our testimony.

Upon reviewing our testimony, both written and oral, we have subsequently discovered that it is possible to calculate less conservative upper bounds on the exact confidence limits.

Furthennore, it is possible to also calculate a 4

lower bound for the exact 90% confidener limits.

Since the upper and lower tounds for the 90% confidence limits generally differ by a factor of two, it U e description of confidence level given by footnote 9 on page 12 of ALAB-525 Th is misleading. An upper confidence limit L is the endpoint of r. random interval (0,L). The confidence level is the probability that the interval (0, L) will cover the unknown parameter. Since the parameter is fixed, no probability is associated with it.

E ur bounds were obtained by multiplying three confidence limits, each with 0

confidence (1 - g), and calling the product a bound on a confidence limit with confidence level (1 - 3g). As was pointed out by ALAB-525, there is an intrinsic conservatism in this calculation, regardless of the degree of independence or dependence among the three factors.

_.0

r

. is our judgment that our revised procedure fer calculating bounds on the exact confidence limits yields values which are not overly conservative and therefore no further refinements are felt to be worthwhile.

The revised bounds on the exact confidence limits are presented in Table IV (revised) for 80% and 90% confidence levels, together with the corresponding values from Table IV of our testimony.

Because the Bonferroni method is applied to only two factors in our revised approach while it was applied to three factors in our original testimony, the 80% and 90% con-fidence level bounds in Table IV (revised) correspond to the 70% and 85%

confidence level bounds in Table IV, respectively.

(There are no revised bounds in Table IV (revised) corresponding to the 97% bounds in Table IV because there were no corresponding tables in the source paper we used for ourrevisedapproach.)

As compared with the bounds in Table IV, the revised bounds in Table IV (revised) show a double improvement.

First, the confidence levels are increased and second, the upper bounds on the exact confidence limits are decreased.

Furthennore, except for the relatively unimportant case of nonscheduled takeoffsM, the upper and lower bounds for the 90% level differ by about a factor of two.

U rom page 15 of the testimony of Darrell G. Eisenhut, nonscheduled takeoffs F

contribute less than 10% to Ptotal, the probability of a "hea'vy" aircraft impacting TMI-2.

i

. /

I BOUNDS ON EXACT CONFIDENCE 1.IMITS( x 10-9)

)

70%

80%

85%

90%

Upper Upper Upper Lower i Upper ESTIMATED Bound Bound Bound Bound Bound SCHEDULED PATE (Table IV)

(revised)

(Table IV)

(revised)

TAY,E0FFS 4.9 x 10-9 36 23 53 15.4 32.3

~'

LANDINGS 2.0 x 10-9 10 7

13 5.3 9.0 NONSCHEDULED TAKE0FFS 32 x 10-9 420 273 670 103 409 LANDINGS 39 x 10-9 210 148 290 101 196 Table IV (revised).

Estimated values and bounds on exact confident.c limits for areal crash densities at TMI-2 for a U. S. carrier aircraft engaged in a relevant operation (probability per square mile)

The source of this revision was the discovery that the estimates of the crash densities g(r) and h(e) (see p. age 7 of our testimony) are statistically independent for both takeoffs and landings. This fact ailows us to calculate approximateE 90% and 95% confidence limits for the conditional crash densities D (r, e ) and y o g

D (r, e ).

(See " Confidence Intervals for the Product of Two Binomial Parameters",

g g g

by Robert J. Buehler, Journal of the American Statistical Association, December 1957,482-493.) The approximate confidence limits are then multiplied by the 5_/ ased on the Poisson approximation to the binomial. As discussed on page 5 B

of the appendix to our testimony, this approximation yields conservative confidence limits, 9

, l approximate confidence limits for the off-runway crash rates to get conservative confidence limits for the areal crash dens { ties using the Bonferrdni method discussed on page 5 of the appendix. The lower bound for the 90% confidence limit is obtained by multiplytng the approximate 90%

confidence limits for the conditional crash densities by the estimated 4

off-runway crash probabilities from Table I on page 2 of our testimony.

If the values from Table I were equal to the true crash rates, this procedure would yield approximate 90% confidence limits for the areal crash densities, but since :he values in Table I are only estimates of the t' rue crash rates, this procedure yields lower bounds.

It should be noted that it is not all obvious that the estimated crash densities for r and e are s(atistically independent., From Table 9A for takeoffs, the three hits in 2 < e < 3.5 and the three hits in 25* < e < 40*

have one hit in common and we believed that this common hit would induce a positive correlation between the estimates of the crash densities as calculated on page 7 of our testimony.

The relevant data is sunmarized in the following table of takeoff hits.

Radial Distance Angular Distribution (miles)

[0',25*)

[25*,40')

[40',100*]

[0,2.0) 5 1

3 9

[2.0,3.5) 0 1

2 3

[3.5,5.0]

2 1

0 3

7 3

5 15

b t Our model assumes that each of the 15 takeoff hits impacts in one of the nine boxes of the table according to the following joint distribution, where p and q) are probabilities such that pj + p2 + P3*91+92+93 " I*

g Radial Distance Angular Distribution (miles)

[0*, 25 )

[25*, 40* )

[40*,100*]

l

[0, 2.0 )

P913 P1 pqjj p9j2

[2,0, 3.5 )

p2 1 P923 P2 9

P922

["$.5,5.0]

P931 P932 p3 3 p3 9

I 91 92 93 The assumption that the probability of a hit in any box is the product of the marginal probabilities is equivalent to the assumption on page 4 of our testimony that "r and e for off-runway hits are distributed independently".

For the case at hand, the problem is to estimate p2 2, the probability 9

that an off-runway crash will impact in the box including TMI-2k There are two ways to do the estimation. The first is to use the the ratio of the observed number of hits in the box including TMI-2 to the total number of off-runway hits.

For this case, the estimate would be 1/15 =.067.

The E e conditional crash density D(r, e ) estimated in our testimony differs Th o

o from p292 by a nonnalization factor which yields the probability of impact per square mile. This normalization factor is an exact quantity and it is omitted in this discussion for convenience and to allow us to focus most directly on the statistical issues.

Its omission does not affect any of these statistical issues.

9

second method is to estimate p2 and q2 separately and then multiply.

For this case, the estimate would be h h =.04.

We use the second method because it has smaller variance than the first.

To consider the issue of independence, denote the estimates of p2 and q2 by p2 and q2, respectively.

Even though we have model independence as expressed in the joint distribution table above, this does not necessarily imply that h and q2 are statistically independent.

For this case 2

,0+1+2 p2 15 2,1+1+1

^

9 15 where we have decomposed p2 and q2 according to the observed data.

In

general, 21 + "22 + "23 n

p2

• N "12 * "22 * "32 92*

N where n ) is the observed number of hits in row i and column j of the g

3 x 3 data table and N is the total number of hits.

It is the presence in both h and q2 that led us to believe that h and h are not of n 2

2 2

22 statistically independent and, in fact, are positively correlated. However, because the total number of hits is fixed, n is negatively correlated with 22 all of the other n );and, in particular, n22 9

is negatively correlated with l

, t (n21 + "23) and with (n12 + "32).

It turns out that this negative correlation exactly balances the positive correlation induced by the presence of n s that 22 p2 and q2 are, in fact, statistically independent.

Since the estimated conditional crash density D(r, e ) is the product of o

o 9(

two statistically independent quantities and h(e), we can use Buehler's r

tables to calculate approximate confidence limits for the conditional crash densities for takeoffs and landings. However, the estimated conditional crash densities are not independent of the estimated accident rate, since both depend on the same set of accidents and there is no mechanism to cancel out this dependence.

(Numerical calculation indicates that the confidence limits for the accident rate and the conditional crash density are negatively correlated.)

It is for this reason that we use the Bonferroni method to calculate bounds on the exact confidence limits for the areal crash densities.

I

\\

.