ML20237G654
| ML20237G654 | |
| Person / Time | |
|---|---|
| Site: | Shoreham File:Long Island Lighting Company icon.png |
| Issue date: | 05/26/1987 |
| From: | SUFFOLK COUNTY, NY |
| To: | Atomic Safety and Licensing Board Panel |
| References | |
| OL-5-I-SC-099, OL-5-I-SC-99, NUDOCS 8709020355 | |
| Download: ML20237G654 (67) | |
Text
322-8& 5 g/z&/87 6
l 1_ se en my i
UNITED STATES OF AMERICA
'g7 /G 25 A9 :rg NUCLEAR REGULATORY COMMISSION Before the Atomic Safety and Licensino Board f i'.
I uuv Ay,
}
l l
l 1
)
In the Matter of
)
)
LONG ISLAND LIGHTING COMPANY
)
Docket No. 50-322-OL-5
)
(EP Exercise)
(Shoreham Nuclear Power Station,
)
Unit 1)
)
)
l i
TESTIMONY OF GARY A. SIMON -AliD sir.Ptir N COLDE ON BEHALP OF SUFFOLK COUNTY CONCERNING i
CONTENTION EX 21
)
1 I
April 6, 1987 I
B709020355 870526
)
PDR ADOCK 05000322 G
PDR l
1 1
L -- -
4
{
\\
UNITED STATES OF AMERICA NUCL 5:AR REGULATORY COMMISSION l
Before the Atomic Safety and Licensitic Board l
l
)
In the Matter of
)
I
)
LONG ISLAND LIGHTING COMPANY
)
Docket No. 50-322-OL-5
)
(EP Exercise)
(Shoreham Nuclear Power Station,
)
Unit 1)
)
)
l l
TESTIMONY OF GARY A. SIMON AND-STEPHE N C ON BEHALF OF SUFFOLK COUNTY l
CONCERNING
~
CONTENTION EX 21 9
i April 6, 1987 4
l l
i i
i i
I 1
i i
l l
i j NUCLEAR SEGULATORY COMM155105 g R9 osetet sedn-21s.oe.r e,n iei e,b, no.
1 la the mettet eO_= A% #se[a 8,4 WM o,,.u[j e
/
bteft-IDE NTIFIED_
Applieset RECEIVID laterveeer REJECTED CCet's Off'r j
CootreWor Daft M~-M-97 1
Other Witeest geporter_
MMu 8Au i
1 I
UNITED STATES OF AMERICA NUCLEAR REGULATORY COMMISSION Before the Atomic Safety and Licensino Board.
)
In the Matter of
)
)
LONG ISLAND LIGHTING COMPANY
)
Docket No. 50-322-OL-5
)
(EP Exercise)
(Shcreham Nuclear Power Station,
)
Unit 1)
)
)
TESTIMONY OF GARY A.
SIMON AND STEPHEN COLE 0N BEHALF OF SUFFOLE, COUNTY CONCERNING CONTENTION EX 21 I.
IDENTIFICATION OF WITNESSES Q.
Please state your names and positions.
A.
(Simon)
My name is Gary A. Simon.
I am an Associate Professor, in the Department of Statistics and Operatlons Research, in the Graduate School of Business Administration of New York University.
N le)
My name is Stephen Cole.
I of Sociology at the Sta iversit iew York at Stony Brook.
I am also President of Data 1 sts, Inc., a consulting; firm enga n conducting applied sociologi surveys and p
1s.
Q.
Briefly summarize your experience and professional qualifications.
A.
(Simon)
I received a B.S. degree in Mathematics from Carnegie Mellon University, and a Ph.D. in Statistics from Stanford University.
Since receiving my Ph.D. in 1971, I have been teaching and working in the field of statistics.
My experience has included research, teaching, and applications.
Over the years, I have taught many different subjects, at various levels, including, for example, sampling, regression analysis, analysis of categorical data, multivariate analysis, analysis of experiments, and time series.
For several years, I have also been engaged in statistical consulting.
The subject' matter har varied widely, and the range of statistical issues has been quite large.
This work has included such things as sample surveyn, product testing, and time series, and other applications of statistical and sampling principles.
~
I am the author of numerous articles and publications on various subjects in the field of statistics.
For five years (1979-1983) I was an Associate Editor of the Journal of the.
American Statistical Association.
My professional qualifica-tions, publications, and experience are described in detail in my recume which is Attachment 1 hereto.
le)
My professional qualifications were dec abed N
in my Testimony 6 ntentions Ex 38, Ex 39, Ex 22.
Ex'44, Ex 40.C, and Ex 49.C, date arch 13, 1987, my Vita was
's thereto.
I would like d, however, that my experience on matters pertinent this test ny has involved the practical application data gathering technigt This has included the develop t of samples for more than 150 app
- d and basic research udies.
I am also the author of The Sociolocic x
- Method, 0,
3rd Edition, which is a basic text on the logic and tpc niques of data gathering.
II.
PURPOSE OF TESTIMONY 0.
What is the purpose of this testimony?
A.
The purpose is to address matters raised in Contention Ex 21 arising out of the exercise of LILCO's-offsite emergency plan (the " Plan") for the Shoreham Nuclear Power Station which was held on February 13, 1986 (the " Exercise").
Unless otherwise stated, the testimony which follows is sponsored by both of us.
Q.
What do you understand to have been the purpose of the Exercise?
l A.
We understand that under the NRC's regulations, there must be a FEMA-graded exercise of the offsite emergency plan for a nuclear power plant before a license authorizing full power operation of such a plant is issued.
The purpose of such an exercise is to allow FEMA, and then the NRC, to determine whether that plan can and will be implemented in a real emergency to provide adequate protection to the public health and safety.
We understand that LILCO contends that the Exercise was a " full
)
i participation" exercise, which, according to the NRC's regula-1 tions, means that it was intended to demonstrate an ability to l
implement an integrated and coordinated overall response to a l
l radiological emergency.
Finally, we understand that the NRC's regulations state that full participation exercises are to
)
include " testing the major observable portions" of the offsite plan as well as mobilization of " personnel and other resources in
)
sufficient numbers to verify the capability to respond to the
)
accident scenario."
10 CFR Part 50, Appendix E, S IV.F.1, n.
4.
Q.
In your opinion, is it possible for an exercise of an emergency response plan to yield results that could permit conclusions to be drawn about the ability to implement such a plan in the event r an actual radiological emergency?
A.
We believe that it may be possible, but the evaluation of an exercise would have to be considerably different from that which was undertaken by FEMA with respect to the LILCO Exercise.
Q.
Why?
A.-
We will discuss our conclusions on this in detail' below, but, in general, our opinion is based on two facts:
the Exercise included no quantitatively stated performance objec-tives; and, the evaluation process was done without reasonable
. thought about sample sizes or random selection mechanisms.
The result of these two facts, in our opinion, is that FEMA's con-clusions about whether Exercise objectives were satisfied, and other generalizations about the ability to implement the LILCO Plan drawn from Exercise events, are not valid.
l Q.
Please describe your understanding of how FEMA reviewed and evaluated the performance of LILCO and other' response per-sonnel during the. Exercise.
A.
Again, we will discuss this in more detail below, but in general, the procedure was the following.
Prior to the Exer-cise, FEMA and LILCO agreed on Exercise " objectives," which described general types of activities to be " demonstrated" by.the Exercise players in response to the hypothetical accident postu-lated in the Exercise scenario.
These objectives are listed in the FEMA Post-Exercise Assessment Report
(" FEMA Report") at pages 9-15.
On the day of the Exercise, FEMA.had a total of 38 evalua-tors, placed at various locations,.who observed and evaluated the performance of particular activities, or the performances of 1 - _ _ _ _ _ _ _ - _ _ _ _ _ _ _ _ _ _ _ - _ _
i particular players, and based on those observations, reached conclusions about whether the pre-identified Exercise objectives had been met, partly met, or not met.
Their observations and conclusions formed the basis for the findings contained in the FEMA Report.
)
Q.
Please state the basic allegation of Contention Ex 21.
A.
The basic contention is as follows:
FEMA had no basis or insufficient data to support its conclusion that certain exercise objectives identified in subparts A-F below were met or were partly met during the exer-cise in that even with respect to the events and activities which FEMA did observe, the j
samples which FEMA reviewed were much too i
small to permit valid generalizations or to J
support FEMA's conclusions concerning these objectives.
Therefore, FEMA's conclusions on such objectives cannot support a finding of reasonable assurance that the LILCO Plan can and will be implemented as recaired by 10 CFR S 50.47(a)(2).
Thus, the exereir>e precludes a finding of reasonable assurance that adequate protective measures can and will be taken in a Shoreham accident as required by 10 CFR S 50.47(a)(1).
The LILCO Plan is, therefore, fundamentally flawed.
The Contention then goes on to provide specific examples in subparts A through F.
Q.
Do you agree with that contention?
a _ _ _ - _ _ _ _ _ _ _ _ _ _ - _ _ -
H A.
Yes, we do.
1 III.
OVERVIEW A.
Samolino i
Q.
What is meant by the term " sample?"
l A.
The term is used to refer to a subset of a population.
A well-chosen sample should tell us useful information about the whole population.
The technique of sampling is'used not just for purely scientific research, but also in many other areas of data gathering.
For example, surveys of human populations depend critically on the techniques of scientific saepling.
Q.
What is the purpose of sampling when gathering data?
I A.
Sampling is used when one-is interested in finding out something about a population, but it is difficult or impossible to examine the entire population.
Investigators in such in-stances may instead select samples from the population and examine them, rather than the whole population.
If the inves-tigator intends to use the sample to reach conclusions.about the whole population, then the selection of the.cample must be done very carefully. _ _ - _ _ _ _ _ _ _ _ - _ - - _ _
I J
Sampling has many applications, and it is used for many reasons.
First, it is enormously efficient.
Thus, if one' wanted to determine how frequently a particular characteristic (or opinion, or fact) occurred in a pooulation of 30,000 persons, a I
sample of 200 chosen from that population might provide an excellent estimate.
For example, politica] polls using fewer than 2000 voters give excellent results which can predict-the i
outcomes of elections involving millions of voters.
Similarly, if one wished to learn about the attitudes of students at a particular high school regarding smoking and drinking, it would be unnecessarily time-consuming and expensive actually to query every student individually about these issues.
Instead, one could learn nearly everything about the student attitudes by surveying a statistical sample o-f the student population.
l
)
Second, the sampling process forces the data gatherer to l
think very carefully about what he or she is trying to do.
In l
order to select properly a representative sample, the investiga-tor must first identify the relevant questions about the popula-tion which are necessary to obtain the data he or she seeks, and must also decide the goal of the investigation:
that is, how accurate a prediction, with respect to the whole population, does one wish to obtain from the sample.
l Q.
Is there more than one type of sampling?
A.
Yes.
The major distinctions among the methods depend on which facts about the population are known in advance and how these facts are used.
In the simplest instance, nothing is known about the population (except perhaps names), and the only method which should be seriously considered is simple random sampling.
If other facts are known, such as age or sex, then these facts might possibly be used to improve the precision of estimated l
quantitles.
These other techniques go under names like strati-i fled sampling, ratio estimation, quota sampling, and others.
The techniques still involve random sampling, but in a more compli-cated way.
Random sampling has been entrenched in the scientific methodology for at least 50 years.
The technique is usually advocated on the ground that it avoids chance biases which might result from haphazard or purposeful selection.
Another benefit, which is not so widely appreciated, is that random sampling enables us to make straightforward probability statements about the quantitative results which arise from the work.
Q.
In general, how does one determine whac is an appro-priate sample size when one wishes to gather data? ________________ _
A.
The answer depends upon what the user wishes to do with the sample data.
Some investigations are exploratory and others are decision-making.
l Q.
Please explain the difference between these two types of investigations.
A.
In an exploratory investigation, the goal is simply to learn some particular fact about, oi to explore some particular facet of, the population.
Political polling, product-use surveys (have you ever used such and such a product?), and opinion re-search fit into this category of research.
On the other hand, a decision-making investigation is designed to gather data in order to make a decision about the population.
A common example would be " acceptance sampling":
based on the results of a sample from a large shipment of merchandise, a yes/no decision must be made on whether to purchase the shipment.
Or, based on the results of a survey about a new product, company managers must make a yes/no decision about whether to market the product.
Q.
If the investigation is purely exploratory, how is sample size determined?.
]
1 I
i A,
In an exploratory investigation, two questions j
determine the appropriate sample size.
They are:
(1) what is the tolerable error or how accurate do you want l
l to be (i.e., the difference between the proportion of people in the sample with the investigated characteristic and'the propor-tion of people in the total population with that characteris-tic)?; and, (2) what is the probability of achieving that error or less (i.e., how likely is it that we can obtain as small e difference, or smaller, between the sample and the population proportions)?
The end result of the investigation can be expressed as a confidence interval statement.
An example of such a statement is:
"I am 95% confident that the true proportion in the popula-tion of voters who intend to vote for candidate X is 62%, plus or j
minus 5%."
The 5% figure in the plus-or-minus statement represents the tolerable error, or the answer-to the first question above.
The confidence level, here 95%, represents the l
probability of achieving the tolerable' error, or the answer to the second question above.
Most statisticians use the 95% level most of the' time, but obviously other confidence level values may be used. l
1 The sample size for an exploratory investigation can be determined once the objective is identified in terms of the confidence level and the tolerable error as just illustrated.
In generci, the smaller the tolerable error, and the higher the confidence level, the larger the sample must be.
Q.
Was FEMA's evaluation of the Shoreham Exercise an exploratory investigation?
4ko A.
No.
Based upon our review ofqFEMA Report, the objective of the FEMA investigation apparently was to determine whether the overall task assigned to each group of individuals in i
a particular LERO job category can be performed.
That goal calls l
for the making of a decision -- whether Exercise objectives were or were not satisfied -- based on the data gathered from the sample.
In fact, according to the FEMA Report, that is what FEMA did.
Q.
Please explain how one determines the necessary sample size for a decision-making investigation.
A.
In a decision-making investigation, there are three items which must be identified to determine sample size:
(1) the target value; (2) the definitely unacceptable (or " bad") values;. _ _ _ _ _ _ _ _ _ _ _ _ - _
and (3) the probability with which one wishes to be able to distinguish the target from any of the unacceptable or " bad" values.
(1)
The target value is a nonstatistical statement about the population.
It represents, in numerical terms, a statement of our hopes about the population.
For instance, in the context of acceptance sampling of watermelons, we might set the target value at 80% -- meaning that we would regard the lot of melons as a reasonable purchase if 80% of them are salable.
In the context of the evaluation of the LILCO Exercise, the target value is the minimum proportion of players who must be able to perform a defined task correctly in order to conclude that an Exercise objective has been met.
This figure might, for example, he 90%
for some of the components of the Exercise.
(2)
The definitely unacceptable or " bad" value, basically defines how much deviation from the target value would be acceptable, even though~not right on target.
It, in essence, defines what constitutes unambiguous failure which you want to be protected against.
For the acceptance sampling of watermelons, again as an example, the target value was 80%.
The range of unacceptable values might be 0% to 60%.
For a component of the LILCO Exercise with a target value of 90%, the range of unacceptable values might be 0% to 70%. __ -
l (3)
The probability of distinguishing the target from the 1
unacceptable or " bad" values is called the " statistical power."
l Using the acceptance sampling example, one might wish to
~
distinguish between the target value of 80% and the nearest unacceptable percentage of 70%.
Question (3) requests the probability with which we want to be able to make this distinction.
In the context of the Exercise, using the values suggested above, one would have to specify the probability with which one wants to distinguish 90% from 70%.
In general, the more accurately you want to be able to predict from the sample to the entire population -- that is, the larger the probability chosen for making the distinction between the target value and the unacceptable proportions -- the larger the sample must be.
An example helps to illustrate.
Suppose that a company decides to conduct a marketing survey to determine whether to market a new product.
Management decides that if 70% of the l
people surveyed prefer the company's new product to its j
l competitor's, it will market the new one.
(70% is the target l
l value.)
Now, suppose the results of the survey are that 62% of the people sampled prefer the new product, with an error of 12%
-- that is, the confidence interval is 62%, plus-or-minus 12%. _ _ _ - - _
e..
Because the target value -- 70% -- is within the confidence interval, according to the decision made before the survey was conducted, management should go ahead and market the new product.
Clearly, however, it should have been explained to manage-ment prior to conducting the survey, that because' sampling does not permit absolutely precise predictions, it will have to.make decisions even though its precise target values may not be achieved.
In this case, if the true value were 68%, rather than the targeted 70%, the decision to market would not be a bad one.
l If the true value were 45%, it could be that the decision to market would be a disaster.
l l
This illustrates the need to define ahead of time the " bad" J
values and the desired probability of distinguishing the target from the " bad" values.
In the case of the company we have been discussing, the " bad" values may be 0% to 45%, or even 0% to 55%.
The smaller the difference between the target and the " bad" values, the larger the sample size must be.
If the actual 1
population proportion preferring the new product is a " bad" value, then we want the confidence interval to exclude the target with specified large probability.
l >
i 1
As a technical aside, we note that the sample size issue can u
be handled with slightly more precision if both the target value and " bad" values are separately specified.
However, identifying l
just the difference between the target value and the nearest j
" bad" value also provides good results.
I B.
FEMA's Conclusions Are Not Valid 1
Q.
How does all this relate to FEMA's evaluation of the Exercise?
A.
During the Exercise, FEMA observed and evaluated only a portion of the LILCO players in particular job categories (e.o.,
Bus Drivero, Route Alert Drivers, Traffic Guides, etc.), rather than the entire population of these workers.1 As noted above, the FEMA evaluation was a decision-making investigation, designed to determine whether Exercise objectives, involving the performance of particular emergency functions, were met.
In order to determine the appropriate size of the samples I
it reviewed in the Exercise, FEMA should have, in advance, specified:
its target value (what proportion of adequate player performances constitutes meeting the Exercise objective); its unacceptable or " bad" value (what proportions of adeguate
^
l There were certain job categories for which FEMA did evaluate the entire population, at least of the shift which participated in the Exercise, for example, the LERO Spokesperson 0;
at the ENC.
We discuss in this testimony, however, those job categories referenced in Contention Ex 21, as to which only samples were reviewed by FEMA. A
4
,3 s
s x
3 1
perforniance would constitute unambiguously or definitely not
, meeting the Exercise ob,jective); snd the probability with which it wished to be able to m4ke the distinction.
We see no evidence that this was done, even in an informal way.
O.
If you do not sort out all these issues in advance, but simply choose a sample size based on other grounds, will the sample be worthicss?
A.
tiot necessarily.
A large sample, if properly selected, will almost always be somewhat informative-That is, sometimes, o
g{
e large sample selected without regard to the issues discussed jk above will nonetheless succeed in revealing blatant aspects of the population.
On the other hand, small samples, chosen without regard to the matters just discussed, will produce results with such large error bounds that they are virtually meaningless.
O.
Do you know how FEMA determined the size of the samples (i.e., the number of " demonstrations" observed) which it evalu-fj ated dur!*g the Shoreham Exercise?
A.
We understand that in deposition testimony, the FEMA witnesses stated that the number of demonstrations FEMA decided to observe during the Shoreham Exercise was based upon a combina-tion of two factorr,:
the number of demonstrations of similar types FEMA had observed at other exercises; and the number of F
1 evaluators FEMA had available to observe demonstrations at Shoreham, and their locations during the Exercise.
- Egg, e.q,,
Transcript of Deposition of Roger B. Kowieski, Thomas Baldwin, and Joseph B. Keller (January 29, 1987) (hereafter, " FEMA Panel Depo."), Et 253, 260, 268-70.
In other words, we understand that the samples sizes were not selected based on an analysis of the I
difficulty of the task to be demonstrated, the number of persons expected to perform the task, the number of tasks necessary to be performed in order to implement the LILCO Plan, or how accurately FEMA wished to predict the ability of the entire population.
Thus, from what we have been able to review, we conclude that FEMA essentially made no reasoned sample-size decisions j
based on what it was trying to determine or how accurately it was l
trying to do so.
Furthermore, we have no evidence that randomi-zation, as opposed to a casual, haphazard selection process, l
played any role in FEMA's design of its evaluation, i
I Q.
In your opinion, does the way the samples were chosen for FEMA's review of the Shoreham Exercise affect the validity of i
the conclusions reached by FEMA as a result of that review?
l l
A.
Yes.
We believe that FEMA's failure to consider the l
l sampling process, and to use, at least in a modified format, the principles of random sampling in its evaluation, greatly diminishes the validity of FEMA's conclusions.
The standards of 1
)
1 I
serious scientific work demand random sampling, and we ought to I
expect no less here.
It is the public's health and safety which j
are at issue in this endeavor; accordingly, in our opinion, every effort should be made to assure that conclusions about whether the public could be protected in the event of a nuclear accident are accurate, reasonably arrived at, and valid.
I l
O.
Is there a way that random sampling principles could have been applied to the FEMA review?
A.
Yes.
For each category of workers observed during the l
Exercise (gtgz, bus drivers, route alert drivers, etc.), FEMA should have stated a target value, and a difference from that target which it desired to detect with large probability (i.e.,
the unambiguously " bad" value it wanted to be protected against).
For example, the target may be that 80% of the workers in each category be capable of performing as required.
But FEMA would want to know with high probability, say 90%, if the true propor-
]
tion of adequate performar.ces is only 50%.
Thus, it would want its 80% target value to be, with high probability, outside a confidence interval if the true value were as bad as 50%, so that it would have a high probability of not deciding its target had been met when, in fact, it had not.
Based on these decisions concerning target values, bad values, and the desired probability of distinguishing between the two, an appropriate sample size could have been determined.
0 Why would the use of random sampling principles be desirable in the circumstances presented by the Shoreham Exercise?
A.
The object of the FEMA evaluation is to enable the NRC to make a prediction about what would happen in a real radiolog-ical emergency at Shoreham.
Thus, the data collected during the Exercise must permit an inference about the behavior of the LILCO players in a real emergency.
As we stated earlier, since it is the public's health and safety which are at stake here, it is essential that the best, most accurate, and most credible tools be used in evaluating the capability of LILCO to protect the public.
S LILCO's witness Dr. Mileti has stated th e should be concernedswith how well an organization per rms during an N
emergency, rather than with how well ind 1 duals perform during an emergency, and that therefore, pling doesn't matter.
Do x
you agree?
N N
.' hat in this case a A.
We certa ly agree with Dr. Mileti t
N an organizati "LERO" -- is responsible for perf' arming the tasks an anaging the emergency response to an accident a Shore am.
But, insofar as Dr. Mileti tries to use that point to l
\\ _ _ _ - _ _ _ _ _ _ _ _ _ _ _ _
rgue that, therefore, the performance of individuals in LERO du ng the Exercise is not important, we totally disagree w th him.
The proh{em here is essentially one of measureme t.
How can FEMA, the NRC, a d the public determine if LILCO's lan can be implemented adequa ely to-deal with a Shoreham a ident and
" verify the capabilit to respond to the accid t scenario"?
Since we cannot perform n experiment to fin the answer to this question (e.a., actually h a real acci snt and see how LILCO manages the response to it),
w must lo for other ways of measuring and testing our hypoth ses.
The idea of having a drill or exercise of an emergency plan i theoretically a good one.
But in conducting such an exerci 6 an attempting to draw conclusions from it, how is F to det mine if the organization, in this case LCO, has dem strated an ability to implement the Plan proper)y?
FEMA must look at how samples of individuals -- the mob ized personnel referre to in Appendix E to 10 CFR Part 50 -
performed the required tasks during the Exercise and then se data derived from these indiv' dual performances to make an assessment of the capability f the organization as a whole to deal with a real emergency.
I Dr. Mileti's position were correct, there would be no j
\\
nee to look at any individual performances or indeed at any N
i lamentation capabilities; rather it would be sufficient simply _ _ _ _ _ _ _ _ _ _ _ - _ _ _.
1l i
I J
to look at the organizational structure and determine if there ex< ts an organizational mechanism which, in theory, could arry out t required tasks.
This results in a situation in ich q
there wou d be no way of knowing how well that organiz tional
)
1 structure a (ually worked.
It would be contrary to he Appendix E requirement at an exercise include "mobilizat*on of j
personnel and othe resources in sufficient nu ers to verify the capability to respond o the accident scenar'o."
(Emphasis 1
l added.)
It would also bescontrary to the nO's licensing requirement that there be urances t t adequate protective N
measures can and will be taken.
1 As sociologists studying o ganiz lons know well, there is frequently a substantial dif'erence betw en the formal structure of an organization and th day to day opera on and capabilities of that organization.
Suffolk County Cor 4 tion Ex 50 Testimony, March 20, 987, particularly pages 200 18 relating to 1
1 LILCO's organizat nal structure.
It also may be po sible for a N
utility such a LILCO to have an " adequate" emergency p an from a structural p t of view, but an organization that is com etely incapable f operationalizing this plan.
Dr. Mileti is say
.g we should ook only at the plan (does a part ' olar mechanism exis to e nduct a certain task?), not at the ability or success of th o
anization to implement that plan. -The latter can only be measured by looking at how adequate random samples of indiv' duals in specified organizational positions perform the necess ry tasks i
the exercise.
Gi.en that the only way to realistically de ermine an organ-ization's bility to respond to an emergency 's to have some kind of drill to e mine the performance of ind iduals carrying out the organizatiottal tasks, it is desirabl in evaluating the N
organization's imp ementation capabil' y, or " grading" it, to have the individuals \\ ho perform th se tasks randomly selected, to have samples which a e large e ough to make at least approxi-mately accurate projectio to he population, and to have some quantitative criteria for e uating what is a good or a bad performance.
Thus, while A se most general terms, we may be interested in the capac' y of t organization -- LERO -- to N
achieve it's objective the only wa to determine this is to see if an adequate samp e of the people ab igned to fulfill certain roles in that or nization can adequatel perform the particular tasks assigned o them in the organization structure.
Thus, we are not only concerned with whether the organizational structure has a mech nism designed to " address" issues, bu also with whether hat structure likely can work in an emerg cy situation and wh ther the organization can successfully implem t and i
perf rm the tasks necessary to appropriately and effect vely deal wi h those issues during a real emergency.
It is this la ter
.atter of implementation capability which we understand the 1 ___- _____ -____
i I
J l
)
Exercise, and FEMA's evaluation of it, were intended to add ss.
You cannot address that issue without a close focus on in lvidual drformance,particularlysinceanorganizationlikeLE is no mor than the sum
.. the individual capabilities.
i l
1 Q.
Dr. Mileti also claims that the relevan social unit in evaluating a organization's ability to handle ergencies is the organization i elf.
Do you agree with this?
I I
A.
No.
The p'oper unit of analysi in research depends upon what the problem If the probl m were defined in the q
)
following way:
"Can util ties in the United States with nuclear power plants adequately resp nd to mergencies which might N
develop at these plants?" (a qu e ion in which the NRC or Congress might be interested f r licy-making purposes on a 1
nationwide basis), then the elevant unit of analysis would be the various operators of clear power lants.
LILCO, for 1
example, would be one it in a study of m ny different utilities
)
operating nuclear p1 nts.
This study would h ve to be carried out by selecting random sample of operators and conducting l
drills for eac member of the sample.
(For each d 11 conducted it would be ecessary to randomly select the individu ls in the particul organization to perform designated tasks.)
concly ion then might be drawn based on data from the samp e 1
i ab t the ability of all utilities to handle emergencies at nuclear plants.
24 -
_ _ - _ - _ _ _ _ _ _ - - _ _ _ _ ~
I If, however, the question is "Can a particular operator l
Inadequately respond to an emergency which might develop at th op rator's nuclear plant?," then a different type of study ould hav to be conducted.
From a scientific point of view, he best I
type study would require the particular utility to espond
)
without otice to 100 different hypothetical emerg cies at its j
plant.
Fo each of the 100 exercises, there wou be a quantita-tive way of g ding the performance of the uti ity or other j
l organization in harge of responding to the mergency.
The 1
organization woul be required to mobiliz personnel from within each organizational osition at random nd without prior notice.
l If the organization co ld then demon trate that it could perform
)
I well in 90 or more of th emergen es, we could then have a fair l
degree of confidence -- at he 0% level -- that that particular l
organization was capable of quately responding to most, although certainly not all emer encies at the plant.
l Such an experime clearly is im ossible.
Given that only I
one exercise is ini ially required to d termine if an operator 1
can and will be le to implement a plan protect the public in an emergency, t is even more important to ke sure that we have systematic ays of evaluating the performance capabilities as demonstr ed during the Exercise.
This can only be achieved by havin adequate random. samples of individuals in p rticular K
4 org nizational positions perform tasks, and having t p rformances graded by some objective standards.,
d
l l
Q.
Dr. Mileti has claimed in deposition testimony that i i
l 1s i possible to get representative samples on a small popul ion 1
and th t a sample of 30 from a population ot 60 is no bett than l
l a sample f 1 frem that same population.
Do you agree?
I A.
We completely disagree with Dr. Mileti's c nclusion.
j The testimony bp which you referred is the follow g:
i It woul be inappropriate, for examp e, to 1
take --
at me think hypothetical If there j
l were a poQ of 60 ambulette poten al drivers I
and you selpeted 30, randomly, s artistically, l
i where each h d a known non-zero chance of being select 1, the basic rul for probability sampling, I d
't think you uld have a 1
stronger basis or generali ing to the pool, j
the population 60, than if you looked to one because you n't ge a representative i
sample on that sm 1a pulation.
l l
Transcript of Deposition of D is Mileti (January 8, 1987), at l
l 144.
It is certainly p ssible to draw random samples from small populations, and th larger these sampl are, the better will be the ability to ge eralize from the sample to the population.
A sample of 30 f m a population of 60 is far superior to a sample l
l of one.
Or Mileti made a similar error when he asse ted in his depo ition (p. 139) that if we had a sample of 19 o t of 20, it w
ld not enable us to generalize to the one on which we had no f ata.
To the contrary, if the entire population consis d of i _ _ _ - - _ _ _ _ _ - - _
'\\xqnly 20, and if we knew that 19 could perform the'.
tasks adegh ely, we would know for certain that least 95% of the population could perform adequately.
is would be very useful information.
1 It is for these re ons, which flect fundamental princi-l l
ples of sampling, at we agree with Conte on Ex 21 that the l
l sample sizes ey luated by FEMA during the Exerci were too small i
to justif 'or validate conclusions going to the entire opulation l
l or th implementability of the major aspects of the LILCO P as l
which FEMA observed only selected individual performances.
l l
IV.
DISCUSSION OF CONTENTION EX 21 SUBPARTS l
A.
Suboart A:
Route Alertina j
Q.
Please state subpart A of Contention Ex 21.
A.
Subpart A of Contention Ex 21 states:
Under the LILCO Plan, 60 Route Alert Drivers are expected to be available to drive through the EPZ to notify the public of an emergency in the event of siren failure.
Egg Plan, Fig.
2.1.1; OPIP 3.3.4, Att. 1.
Specifically, a total of 20 Route Alert Drivers are to be available at each of the Port Jefferson, Patchogue and Riverhead Staging Areas to drive l
33, 26 and 30 routes, respectively.
S_gg Plan, 1
Fig. 2.1.1; OPIP 3.3.4, Att. 3.
During the exercise, however, only three Route Alert Drivers (one from each Staging Area) were dispatched by LILCO and observed by FEMA.
FEMA Report at 57, 64 and 74; geg g.1.sq FEMA Report at 20.
Based solely on these observa-tions, FEMA concluded that objectives FIELD 5, SA 9, and EOC 15 were met or partly met.
FEMA l
c !
l l
Report at 34, 56, 57, 62, 64, 72 and 74.
The sample upon which FEMA based these conclusions was much too small to justify FEMA's generali-zations with respect to all Route Alert Driv-ers, or LILCO's ability to notify the public in the entire EPZ.
Furthermore, even with respect to the small l
sample observed by FEMA, FEMA found that the l
time necessary to brief, deploy, and drive even one route (i.e., assuming only one failed siren for each Staging Area) was excessive.
FEMA Report at 57, 64 and 74.
In light of the small sample reviewed by FEMA and its conclu-sions even with respect to the three Route Alert Drivers it did observe, FEMA's conclu-sions with respect to objectives FIELD 5, SA 9, and EOC 15 are without basis and invalid.
The exercise results concerning those objectives preclude a finding of reasonable assurance that adequate protective measures can and will be taken in the event of a Shoreham accident.
Q.
Do you agree with the allegations contained in that subpart?
A.
Yes, we do.
Q.
Would you briefly summarize the basis for your agree-ment?
A.
The number of route alert drivs.s evaluated by FEMA is insufficient to reach any particularly useful or valid conclu-sions about the entire population of route alert drivers.
A sample size of three, out of a total of 60, is simply not large enough.
We understand that FEMA evaluated three route alert drivers, one from each staging area.
FEMA Panel Depo. Tr. 254- _ - __ ___- _ _ _ _ _ _
55.
A sample. size of one driver out of a total of 20 drivers in each staging area is insufficient to justify conclusions about the entire population of drivers in each staging area.
Simi-larly, a review of the driving of one route out of a total of 26, 30, or 33 routes per staging area, is also insufficient.
As we note below, FEMA found with reLpect to each staging area the pertinent exercise objective -- FIELD 5 -- to have been " partly met."
Because the sample sizes upon which FEMA's conclusions were based were so small, however, the Exercise in fact afforded no basis to make even those limited conclusions.
Q.
Please state the objectives referenced in subpart A.
A.
Those objectives are:
SA 9:
Demonstrate the ability to dispatch and direct emergency workers in the field.
FIELD 5:
Demonstrate the ability to provide backup public alerting, if necessary, in the event of partial siren system failure.
EOC 15:
Demonstrate the capabllity for providing both an alert signal and an informational or instructional message to the population on an area-wide basis throughout the 10-mile EPZ, within 15 minutes (to be simulated).
Q.
What were FEMA's conclusions, based on the Exercise results, with respect to each of these objectives? -
A.
FEMA Report at 34.
It found SA9metattheRiverheadffa (see Exercise Evaluation Critique Form ("EECF"), SA 9, (S. McIntosh]) and partly met at the Patchogue Staging Area (see FEMA Report at 62) and the Port Jefferson Staging Area (sgg EECF, SA 9, (P. Weberg]).
With respect to FIELD 5, which goes most directly to the activities of the Route Alert Drivers, it found the objective partly met at Patchogue, Riverhead, and Port Jefferson.
EECFs, FIELD 5.
More specifically, the FEMA conclusions relating to route alert 1
drivers, based upon its limited evaluation, were the following:
I For Riverhead:
Route alert drivers.
should be expediti-ously dispatched.
The objective to demonstrate the ability to l
provide backup public alerting, if necessary, in the event of a partial siren system failure was partly met (Field 5).
The Route Alert Driver was thoroughly briefed and provided with a kit which included a map of the area covered by siren # 89.
However, the map had i
l no mileage or distance scale, making it diffi-cult to determine where to begin and end route alerting on roads in the area covered by the siren.
Mobile public address equipment was I
promptly mounted on the assigned vehicles and I
was demonstrated to be operable.
A written message was provided but was not broadcast.
All streets were traversed at an appropriate speed, and the driver completed the entire route in thirty-five (35) minutes.
- However, seventy-eight (78) minutes elapsed from the time the LERO EOC informed the staging area of l
the simulated siren failure until the public alerting was completed.
A portion of the elapsed time included fifteen (15) minutes for holding a briefing and distributing the packet.
It is recommended that the Plan for 30 -
backup route alerting should be reviewed and revised as necessary to reduce the time needed for route alerting.
FEMA Report at 74.
For Patchogue:
The objective of demonstrating the ability to provide backup public alerting, if necessary, in the event of partial siren system failure was partly met (Pield 5).
Mounting and opera-tion of the mobile public address units assigned to the Route Alert Drivers was demon-strated at the staging area.
The driver who was evaluated had good knowledge of the route planinthey@ peed.
ffected area and drove at an appropriate s He knew how to give verbal instructions if anyone should approach the vehicle with questions regarding the prescripted message.
Total time from dispatch to the beginning of the route was nineteen (19 $'"with another fifty-one (51) minutes needed to complete the route itself because of the length of the route.
It is recommended that the plans for backup route alerting should be reviewed and revised as necessary to reduce the time needed for public alerting.
FEMA Report at 64.
For Port Jefferson:
The numerous field activities deployed from Port Jefferson Staging Area were generally 1
well organized and implemented according to I
the Plan.
The first activity demonstrated was route alerting (Field 5).
A message was re-ceived at the staging area at 0822 indicating siren # 26 had not sounded (simulated) and that backup route alerting would be required.
l A route alerting crew was promptly briefed, given a packet containing a nap of the failed siren's coverage area, and oispatched.
The driver obtained a mobile public ad6ress system from the emergency equipment van located in the staging area parking lot and at tached it to his car.
The Route Alert Driver followed procedures correctly, driving at a realis-j tically slow speed (approximately five (5) miles per hour) and covering the area in a systematic fashion.
About half of the total assigned area was covered.
It should be i
! 0 Q
J
noted, however, that alerting half of the siren coverage area took the driver ninety (90) minutes; presumably it would take three (3) hours for one (1) driver to cover the entire area.
It is recommended that each siren coverage area should be assessed to determine an appropriate number of drivers that would be required to drive the area.
Where necessary, multiple drivers should be deployed to reduce alerting time.
FEMA Report at 57.
I Q.
How does the sample size chosen by FEMA for its evalua-tion during the Exercise relate to the conclusions you have just cited?
A.
First of all, as explained earlier, before making any decision about how large a sample one needs to have to reach valid conclusions about the entire population, it is necessary first to define the performance target which will be the basis of the evaluation, and also to state the probability with which one wishes to be able to detect a specified " bad" level of perform-ance.
None of this was done by FEMA.
Therefore, there is absolutely no scientific or other basis for determining that data from the three route alert drivers observed by FEMA are adequate to permit a valid generalization to the entire population.
We cannot conceive of a realistic set of objectives for this problem for which a sample of size three would be adequate.
(This would be true when randomization is used correctly; without __________a
proper randomization, things are even worse.) Thus, while it appears significant that every one of the route alert drivers observed by FEMA took too long to drive the route, from a sample theory point of view, that evidence, standing alone, would not support a conclusion that it would take all 60 route alert drivers too long to drive their routes.
Of course, it does support the conclusion that not all routes could be driven in a timely manner.
Similarly, the fact that FEMA found all three of the route alert drivers that it evaluated to be capable of performing other aspects of their jobs (e.a., being briefed, mounting public address systems on their cars, finding their routes, and reading maps), while interesting, does not allow a conclusion that all or even a substantial portion of the 60 route alert drivers would be capable of such performances.
Q.
Please explain.
A.
The sample size used by FEMA can only result in con-fidence intervals which are.so wide that the data are meaning-less.
The table below indicates how pointless it is to use a sample of three values to try to make a judgment about a popula-tion of 60.
The number of successful drivers in a sample of three can be one of the values 0, 1,
2, or 3.
Here are the confidence interval statements that result from this:
33 -
95% confidence interval Number of successful number of successful Previous, in drivers in samole in cooulation percent terms 2 0 of 3 0 to 37 (out of 60) 0% to 62%
1 of 3 1 to 51 (out of 60) 2% to 85%
2 of 3 9 to 59 (out of 60) 15% to 98%
3 of 3 23 to 60 (out of 60) 38% to 100%
Clearly, the confidence intervals are terribly wide, ranging from 62 percentaae coints to 83 oercentaae coints.
In terms of whether the route alerting functions could be implemented in a real emergency, -- and whether the Exercise permitted FEMA "to verify the capability to respond to the accident scenario" -- it is essentially meaningless to say "I am 95% certain that anywhere from one to fiftv-one (out of 60) of the route alert drivers would perform properly," or "zero to thirty-seven," "nine to fifty-nine," or " twenty-three to sixty."
Furthermore, observe that all possible intervals contain the value 60%.
Thus, if the target value were 60%, then o procedure using a sample size of 3 would be useless in providing data on how likely it is that the general population would achieve or not achieve that target, because all possible intervals contain the value 60%.
l 2
The " percent terms" column refers to the previous informa-tion.
For instance, 0 out of 60 is 0% and 37 out of 60 is 62%. '
Thus, our overall conclusion concerning the matters raised in subpart A is that the data gathered by FEMA concerning three route alert drivers are insufficient to support a conclusion that all or a substantial majority of route alert drivers could or would perform as demonstrated by the three who were observed.
As a result, there is no valid basis for a conclusion that exercise objective FIELD 5 was met, or even partly met.
The only conclu-sions possible from the data gathered by FEMA are that in some instances it would take too long to drive the routes, and that some drivers would know how to perform their jobs.
There is insufficient evidence from the sample of three observed by FEMA to conclude that all or even a majority of the drivers could perform as required or that all or a majority of the routes could be driven in a timely manner.
Q.
How large would the sample have to be to permit valid generalizations about the 60 Route Alert Drivers?
A.
We cannot answer that without having someone define the target values, " bad" values, and desired probabilities.
We can say, however, that using a sample of size 10 would let us distinguish (with probability 70%) values about 40 percentage points apart.
This could be useful if our target value were 75%
and our range of " bad" values were 0% to 35%.
The sample of size 10 is capable of making distinctions of this size.
Finer distinctions would, of course, require larger sample sizes.. _ _ _ _ _ _ _ _ _
l l
B.
Subpart B:
Bus Drivers Q.
Please state subpart B of Contention Ex 21.
l A.
That subpart states as follows.
l l
According to the LILCO Plan, evacuation of the i
transit-dependent population requires 333 LILCO bus drivers to retrieve 333 buses from i
20 bus company locations, and to make 377 bus trips (using 236 buses) over 43 separate routes within the EPZ, plus 97 trips to the Nassau Coliseum.
See OPIP 3.6.4; App. A at i
IV-74a thru -165d.
According to the Plan, 108 bus drivers are required to make 169 bus trips from the two transfer points under the juris-4 diction of the Port Jefferson Staging Area, l
100 bus drivers are required to make 139 bus trips from the five transfer points under the j
jurisdiction of the Riverhead Staging Area, and 125 bus drivers are required to make 166 bus trips from the four transfer points under the jurisdiction of the Patchogue Staging I
Area.
See OPIP 3.6.4, Att. 11; gf. Plan, Fig.
2.1.1.
During the exercise, FEMA observed only two l
bus drivers from each of the Riverhead and l
Port Jefferson Staging Areas.
FEMA Report at 58, 74.
After retrieving a bus from the assigned bus company yard, they each drove one l
EPZ route, then returned to their assigned transfer point and proceeded to the Nassau i
Coliseum.
Based solely on its observation of two bus drivers making one run each, FEMA concluded that objective FIELD 9 was met by the Port Jefferson and Riverhead Staging Areas.
FEMA Report at 58 and 74.
In addi-I tion, FEMA observed four bus drivers from the
(
Patchogue Staging Area attempt to retrieve l
buses and drive bus routes.
FEMA Report at i
64-65.
FEMA concluded, on the basis of those drivers' inability to pick up buses or to drive routes properly or in a timely manner, that objective FIELD 9 was not met at the Patchogue Staging Area.
Id.
Nonetheless, i
based upon its observations of eight drivers making a total of eight bus runs, at least
(
1
three of which were judged unsatisfactory by FEMA, FEMA concluded that objective EOC 16 was j
met.
FEMA Report at 34.
i 1
In light of the small sampling of bus drivers reviewed by FEMA, and FEMA's conclusions regarding the entire sample of bus drivers reviewed at the Patchogue Staging Area, FEMA's conclusions as to objective FIELD 9 for Port i
Jefferson and Riverhead, and as to EOC 16 for the entire EPZ, are without basis and invalid, and fail to justify FEMA's generalizations about all LILCO bus drivers under the LILCO Plan.
The exercise results concerning those l
objectives thus preclude a finding of reason-able assurance that adequate protective measures can and will be taken in the event of a Shoreham accident.
Q.
Do you agree with the allegations contained in j
l subpart B?
)
1 A.
Yes we do.
Q.
Please state the exercise objectives referenced in I
subpart B.
A.
Those exercise objectives are l
i FIELD 9:
Demonstrate a sample of resources necessary to implement an orderly evacuation of all or part of the 10-mile EPZ.
EOC 16:
Demonstrate the organizational ability to manage an orderly evacuation of all or part of the 10-mile EPZ including the water portion.
Q.
What were FEMA's conclusions concerning those I
t objectives?
! i i
I 1
A.
FEMA concluded as follcws:
With respect to Patchogue:
s 1
The objective of demonstrating a sample of evacuation of all or,part of the;10-mile EPZ resources necessary to implement an orderly:
was not met.(Field 9)..
Four (4). Bus Drivers dispatched from the Patchogue Staging Area were evaluated.
Two (2) of the drivers were able to pick up buses at designated yards, proceed to assigned' transfer points and drive I
their assigned route, although one (1)cof them I
missed part of his assigned evacuation route.-
More confusion was evident on the part of the L
other drivers.
One (1) driver proceeded'to j
the wrong points and completed his route only i
after being prompted by the Federal evaluator.
j The other driver took over two (2)-hours ~to i
get to his transfer point from the staging l
area because he initially went to the wrong bus garage.
l FEMA Report at 64-65.- With respect'to Riverhead:
l i
A sample of resources necessary to implement an orderly evacuation of all or part of the 10-mile EPZ was successfully demonstrated at the Riverhead Staging Area and on.the two (2) general population evacuation bus routes l
(Field 9).
FEMA Report at 74.
And, with respect to Port Jefferson:
Two (2) randomly selected bus routes were I
evaluated for the exercise.
Both routen were l
implemented according to plan (Field 9).
At the staging area, the drivers were briefed, assigned to a bus company, and issued route maps to the assigned bus company lot and from the lot to the transfer point..Each driver was issued route-specific maps and directions j
at the transfer point, drove the route cor-
~
l rectly, and arrived at the Reception Center.
l FEMA Report at 58.
.38 -
l l
l l
i l
I As far as objective EOC 16 is concerned, the FEMA Report states:
The LERO EOC met the exercise objective of demonstrating the organizational ability to manage an orderly evacuation of all or part of the 10-mile EPZ including the water portion (EOC 16).
l l
FEMA Report at 34.
Q.
Why do you agree with the allegations contained in subpart B?
1 i
A.
Our reasons are similar to those we have set out above with respect to subpart A and with respect to the FEMA evaluation 1
as a whole.
First, there is no evidence that FEMA had any j
l scientific or other basis related to the overall objective of the evaluation, for determining the size of the samples it would review to reach conclusions about LILCO's ability to implement its bussing plan for evacuees without transportation.
At the very least, some effort should have been made to identify what level of performance it desired and/or is necessary to constitu-a successful implementation of that plan.
In the absence of such an analysis, and the use of such analysis in arriving at a sample size, there can be no validity attached to the FEMA conclusions, particularly since the gross sample size was so small.
l
, l
Second, even if it were assumed that the strict rules of scientific random sampling were not to be applied in this instance, the data gathered by FEMA still do not permit valid generalizations to the entire population of LERO bus drivers.
Regardless of what the target objective is --
i.e.,
performance of individual bus drivers, performance of bus drivers from particular staging areas, or overall completion of bus routes --
the samples evaluated by FEMA were much too small to permit valid I
inferences to the remainder of the population.
Thus, looking at l
eight out of 333 bus drivers, or a sample of 2.4% of those l
drivers, is inadequate to determine if there are significant 1
departures from the desired performance targets or to determine the actual probability of good performance from the population as a whole, particularly since the sample was itself drawn ~from only
)
l a little over one-third of the population.
Indeed, in this case, l
vhen at least three of the eight drivers evaluated performed inadequately in FEMA's judgment, a positive conclusion concerning the performance capabilities of the entire population would be particularly improper.
The table below summarizes the results which can be obtained l
by using a sample of size 8, taken from a population of 333, 1
l
____-y l
95% confidenco interval Number of successful number of successful Previous, in drivers in samole in population percent terms 0 of 8 0 to 121 (out of 333) 0% to 36%
1 of 8 3 to 166 (out of 333) 1% to 50%
l l
2 of 8 16 to 211 (out of 333) 5% to 63%
3 of 8 38 to 235 (out of 333) 11% to 71%
4 of 8 66 to 267 (out of 333) 20% to 80%
l 5 of 8 98 to 295 (out of 333) 29% to 89%
6 of 8 122 to 217 (out of 333) 37% to 95%
7 of 8 167 to 330 (out of 333) 50% to 99%
8 of 8 212 to 333 (out of 333) 64% to 1004 l
This table graphically illustrates why the sample size used by FEMA cannot yield meaningful results.
The confidence inter-vals are all much too wide to permit valid or meaningful conclu-sions about performance capabilities of the population as a whole.3 Of course, the specific results of FEMA's actual review during the Exercise are the relevant data here.
It found that only five of the eight drivers evaluated could perform ade-q'ta t ely.
That corresponds to a confidence interval of 29% to 89%.
In other words, the result of FEMA's evaluation of the 3
For example, given a sample of size 8, even if every sampled driver had performed successfully, the confidence interval would still run from 64% to 100%.
This means that one could be 95%
confident that anywhere from 64% to 100% of the total number of necessary drivers would be able to perform properly in a real emergency.
That much uncertainty attached to a conclusion makes it of little, if any, value. (
L ___ _
-LILCO Exercise is that we can say with 95% confidence that between 29% and 89% of the LILCO bus drivers required to implement the bussing plan for the transit-dependent population could adequately perform their jobs in a Shoreham accident.
Clearly, such a conclusion is essentially meaningless, and certainly cannot be used to support a conclusion that there is reasonable assurance that the bussing plan can and will be implemented effectively.
Our overall conclusion, therefore, is like that discussed above with respect to route alert drivers:
that is, because of the haphazardly selected sample of bus drivers and the small size of the sample, one cannot validly or with any significant degree of confidence conclude from the Exercise data that all or even a substantial portion of the LILCO bus drivers could properly perform in the event of a real emergency, or that all or a substantial po: Tion of the bus routes necessary to be driven in a real emergency could be so driven properly.
Q.
Why wouldn't a haphazard selection of bus drivers by FEMA evaluators be an adequate way to select the individuals to be evaluated?
l l
A.
It may well be that such a selection process would be a i
good substitute for a scientifically random process.
However, we l
have no way of knowing what kind of biases were injected into the
process by the logistic-related criteria which FEMA used in part in making its selections.
Thus, the selections based upon ease of FEMA evaluators to observe particular bus routes may have resulted in selection of routes that were particularly easy, particularly difficult, etc.
We have no way of knowing that.
Sample size and randomization are both important aspects of any investigation.
Our primary criticism of the FEMA process is not with respect to its failure to use random selection so much as with respect to the size of the samples it reviewed.
- Here, the sample sizes are so small that they preclude reasonable findings (even if randomization were properly done).
According-ly, most of our discussion focuses on the primary problem of the FEMA evaluat M - sample sizes.
Q.
What size sample should FEMA have used to reach valid conclusions about LILCO's ability to implement its busing plan for the transit dependent?
A.
As we said in response to the earlier similar question, we cannot answer w3.thout being given the necessary data about target values, " bad" values, and desired probabilities of detecting the differences.
However, a sample of size 10 would let us distinguish (with probability 70%) values about 40 percentage points apart.
Again, this could be useful if our,
i l
l
,\\
target value were 75% and our range of " bad" values were 0% to l
1 35%.
Again, finer distinctions would require larger sample l
sizes.
C.
Suboart C:
School Testino l
Q.
Please state subpart C of Contention Ex 21.
l A.
It states as follows:
l Eleven school districts (with 33 schools), one l
BOCES supervisory district, two parochial f
schools, and 13 nursery schools have school buildings located within the EPZ.
In addi-i tion, there are seven school districts with no school buildings within the EPZ but whose i
I district boundaries include residences within the EP2 (i.e., with children who live in the l
EPZ); and, there are three parochial schools j
and one BOCES supervisory district with i
buildings located just beyond the EPZ boundary.
Sag OPIP 3.6.5, Att. 3; gfz App. A at IV-i.69 thru -171.
During the exercise, FEMA observed two buses report to one school l
(Shoreham-Wading River High School), where two bus drS. vers allegedly received maps and instru:tions.
FEMA Report at 43.
FEMA was not able to observe the organizational ability to effect an early dismissal of schools, however.
FEMA Report at 38.
FEMA also l
observed one bus, driven by a LILCO employee, l
arrive at the Ridge Elementary School (almost three hours after it had been " requested" in a free play message).
FEMA Report at 38.
Neither of the two Shoreham-Wading River buses observed by FEMA drove any early dismissal or evacuation routes or transported any children, nor had either of the bus drivers received any l
training or information concerning any plans for protecting or evacuating school children in the event of a Shoreham emergency.
Egg FEMA Report at 44-45.
The free play message
" requesting" a bus to Ridge Elementary School was injected by FEMA at 10:30; however a bus driver was not dispatched to pick up a bus
1 s
until 40 minutes after the r'equest was j
received by the Patchogue Staciag Area.
FEMA Report at 66.
The bus did..ct arrive at the school until 1:23 (FEMA Report at 38) and, according to LILCO, even as late as 4:23, it had not arrived at the Reception Center.
The bus did not transport any children, and neither the driver, the FEMA evaluator, nor any other exercise player, had any contact
)
with any officials from the Ridge School or the Longwood School District.
The FEMA evaluator merely observed the bus arrive ai i
and leave the school.
Based solely upon these observations, however, i
and despite FEMA's acknowledged failure to l
observe LILCO's ability.to effect an early dismissal, and the irrelevance of the situation postulated in the Ridge School free play message to the evacuation procedures in the LILCO Plan (see Contention Ex 2d.G).
FEMA nonetheless concluded that objectives FIELD 15 and 16 and EOC 20 were met.
FEMA Report at 38, 43.
1 The limited observations and irrelevant sample l
l upon which FEMA based these coitclusions do not justify FEMA's generalization about the abil-ity to evacuate all schools, school districts, and school children in the EPZ and the ability to accomplish an early dismissal of all such schools, districts and children in the event of a Shoreham emergency.
In light of FEMA's l
limited observation, the irrelevance of the Ridge School free play message, and the size of the sample reviewed by FEMA, FEMA's conclu-
~
sions as to objectives FIELD 15 and 16 and EOC 20 are without basis and invalid.
The exer-l cise results concerning those objectives pre-clade a finding of reasonable assurance that adequate protective measures can and will be taken in the event of a Shoreham accident. _ _ _ _ _ _ _ _ _ _ -
0 Do you agree with the allegations in that subpart?
)
i I
A.
Yes, we do.
Q.
Please' state the Exercise objectives identified in the I
subpart.
i A.
They are:
F:: ELD 15: Demonstrate a sample of resources necessary to effect an early dismissal of schools within the 10-mile EPZ (to be simulated out of sequence, if appropriate).
FIELD 16: Demonstrate a sample of recources necessary to effect an orderly evacuation of schools within the 10-mile EPZ.
1 i
EOC 20:
Demonstrate the organizational ability necessary to effect an orderly evacuation of schools within the 10-mile EPZ.
If this protective action is not recommended by the decision-makers, e.g.,
schools were dismissed early, a free play controllers I
message may be inserted to demonstrate this activity.
A related objective, EOC 18, is also referenced in the rvan Awry
{
faubpart, as FEMA stated it could not observe "(a) demonstration i
of the organizational ability necessary to effect an early dismissal of schools within the 10-mile EPZ FEMA Report at 38.
Q.
What were FEMA's conclusions concerning those objec-l tives based on the Exercise results?
4,
.________._.-._._m._m_
A.
Frankly, FEMA's conclusions relating to these objec-tives are somewhat unclear and confusing.
As best we can tell, however, the following is what happened.
FEMA did not observe early dismissal-related activities at t;he EOC.
In the FEMA Report, FEMA states that this could not be observed because, under the scenario as exercised, schools were simulated to have
(
l already been closed.
FEMA Report at 38.
FEMA also did not i
l observe activities relating to objective EOC 20 (evacuation) at the EOC.
FEMA Report at 38.
The evaluator who was supposed to observe this objective (which would be " demonstrated" by the interjection of a free play message ~equesting "LERO" to send a bus to Ridge School) noted in the comment section on his Exercise j
Evaluation Critique Form that "(o]bserver not informed that message had been inserted."
EECF, EOC 20 (A. Smith)
FEMA concluded, however, that objective EOC 20 was met, apparently because the free play message did elicit a response:
[EOC 20 (evacuation)] was met as demonstrated in response to a free play message inserted to
. demonstrate this activity.
A free play mes-sage requesting school bus resources to assist in transporting forty (40) children from Ridge Elementary School was given to the Evacuation Coordinator by the Exercise Controller at the LERO EOC at approximately 1030.
In a 1041 message the pertinent information was communi-cated to the Special Pooulation Bus Dispatcher at the Patchogue Staging Area, requesting that a bus be picked up at the United Bus Company.
Coordination was effected with the superin-tendent of the Longwood Central School Dis-trict.in which Ridge Elementary School is located, to confirm arrival of the bus.
Ar-rival was confirmed at about 1323 but it was noted that the bus had not yet arrived at the Reception Center.
The Reception Center was -
contacted and requested to inform the Public School Coordinator at the LERO EOC when the bus arrived.
i FEMA Report at 38.
FEMA's handling of objectives FIELD 15 and FIELD 16 is particularly confusing.
The evaluator who went to 1
the Shoreham-Wading River High School and learned from personnel there that tney had requested buses (17 simulated; 2 real) to come and pick up children who were at the school at the time the l
school received notification from the EOC to close concluded l
that, on the basis of what he learned, both objective FIELD 15 l
and objective FIELD 16 had been met.
See EECFs, FIELD 15, FIELD 16 (W. Gasper).
l On the other hand, the evaluator who observed the Ridge l
School bus demonstration apparently concluded that objective i
FIELD 16 was only partly met.
There was apparently some mix-up in the planning phase of the Exercise, as the Ridge Schcol bus i
evaluator was listed on, and recorded his observations on, an l
Exercise Evaluation Critique Form for objective FIELD 9.
The FEMA Report concludes that objective FIELD 15 was met, and that objective FIELD 16 was both met, and partly met.
l The objective of demonstrating a sample of resources necessary to effect an orderly evac-uation of schools within the 10-mile EPZ was partly met at the Patchogue Staging Area (Field 16).
The Bus Dispatcher at the staging area arranged for one (1) bus to simulate the evacuation of forty (40) children to the Reception Center, based on a LERO EOC request.
The driver was familiar with his function and I
l followed his directions very well.
- However, i
the staging area took forty (40) minutes to i
l dispatch the driver after the request was received.
j l
l FEMA Report at 66.
And:
I A sample of resources necessary to effect an j
early dismis3a3 of schools or an orderly' evac-l uation of schools was demonstrated through the simulated dispatch of seventeen (17) buses to the Shoreham-Wading River High School and the release of students for transportation back to their homes (Field 15 and 16).
The dismissal actions were implemented by the Superintendent of the school district.
The bus company, which is under contract to the school district and available at any time, was notified and l
dispatched two (2) buses to the high school.
FEMA Report at 43.
Q.
What can you conclude, from these FEMA statements, about FEMA's conclusions concerning capabilities relating to the protection of school children?
I l
A.
Insofar as FEMA acknowledges that the Exercise provided l
it with no data upon which to make a judgment concerning the satisfaction of any of the schools-related exercise objectives, we agree with FEMA.
Insofar, however, as FEMA does purport to draw conclusions based upon its limited observations during the 1
l Exercise, about the ability to evacuate, or to conduct an early dismissal, cf school children during a Shoreham. emergency, we l
believe such conclusions have no validity because there are j
insufficient data upon which to base them.
l s lw
l i
I Q.
Please explain.
I A.
The primary basis for this opinion is discussed in detail in the testimony of the Governments' witnesses on Contention Ex 15/16 and will not be repeated here.
We note only that in failing to observe any activities at all by any of the school districts or personnel other than a very few by personnel affiliated with the Shoreham-Wading River Central School District or bus company under centract to it, there is, literally, no basis for drawing conclusions about their capabilities from the Exercise results, particularly in the absence of any evidence that their needs, resources, and capabilities are the same as those of the Shoreham-Wading River Central School District, which
)
was the only district " evaluated" by FEMA.
In addition, however, even with respect to the Shoreham-Wading River Central School District, which did participate in the Exercise, the sample evalut.ted by FEMA was insufficient to permit generalizations about the capability of that district to cor. duct an evacuation of its pupils.
The FEMA evaluator who evaluated the Shoreham-Wading River demonstration was supposed l
to:
- 1) evaluate the contact between the High School and the EOC; 1
1
- 2) board one of the buses sent to the school and evaluate the driver's knowledge of the route, emergency procedures, and dosimetry; 3) ride the bus to the Nassau Coliseum Reception Center (or to any other reception location designated by the l l
" School District"); and, 4) remain on the bus as it goes to the EWDF.
Egg Shoreham Exercise -- Evaluator Assignment / Location Instructions, at 26.
In fact, the evaluator did little of this.
When he arrived at the Shoreham-Wading River High School, the Superintendent had i
i already received the call from the EOC and contacted the bus company that provides buses to the school.
The two buses sent by the bus company left the school while the FEMA evaluator was having his initial conversation with the Superintendent.
Deposition of William Gasper (December 1, 1986), Tr. 26-28; 35-36.
The evaluator apparently had some discussion about early dismissal with the Superintendent and the principal of the High School,4 then sat around and waited for something else to happen.
Gasper Depo., 27-28.
Eventually, the evaluator went to the Seaman Bus Company and spoke to the dlspatcher.
The dispatcher said that, before 8:00 a.m., he had received a call from the School District Superintendent requesting 17 buses.
Twa were actually dispatched.
The evaluator also interviewed the bus dispatcher about the bus drivers' knowledge of dosimetry, KI, and radiation 4
LILCO denies that the principal participated in the Exer-cise.
Apparently, the other person Mr. Gasper spoke with at the High School was the Director of Business Affairs for the Shoreham-Wading River Central School District.
Egg LILCO Admission No. 12.. - __-_-________ _ ___
4 i
exposure control.
Gasper Depo., Tr. 30; 39-41.5 After his interview with the bus dispatcher, the FEMA evaluator returned to his motel.
Gasper Depo., Tr. 36.
Thus, the FEMA evaluator vent to one school (the High School) in the Shoreham-Wading River Central School District.
He l
I spoke only to the Superintendent and one other official.
He i
spoke to no teachers, no school children, no bus drivers.
He did not observe any of the bus drivers even simulating the picking up
]
of children and driving their routes.
He did not see or evaluate a reception location for the children from the school.
In short, the evaluator had no basis for concluding even that the Shoreham-Wading River High School -- let alone all the other schools in the Shoreham-Wading River Central School District --
could successfully complete an early dismissal of children in the event of a Shorehan emergency.
Yet, the evaluator concluded that both the capability to implement an early dismissal and the ween capability to imrlement an svahat-imahad been met.
The demon-strations upon which these conclusions rest are clearly both a quantitatively and qualitatively inadequate basis for concluding that the protective actions can and will be taken at the 33 public schools located within the EPZ.
1 1
5 According to the dispatcher, the bus company had no docime-try equipment available; employees of the bus company had no training in dosimetry or exposure control.
KI was not available at the bus company, nor hs.d employees been trained in its use.
Gasper Depo. Tr., 39-41.
52 -
i l
The only other demonstration of LILCO's ability to accom-plish or assist in an evacuation of schools was the dispatch by LILCO, in response to a simulated request, of a bus to the Ridge School in the Longwood School District.
The facts concerning this demonstration are set out in the Governments' testimony on l
l Contentions Ex 15/16 (Section II.C).
Although player documents j
note telephone calls between LERO and the Superintendent of the Longwood School District in connection with the Ridge school free play message, it is apparent that these phone calls were merely j
simulated.
The entire simulation was done without any contacts between LILCO, school district personnel, officia.ls or teachers at the school involved, or school children.
Because none of j
l these persons participated, the demonstration was a complete irrelevancy in terms of actual school preparedness.
Even if actual contacts had been made between "LERO" and the school to arrange for the bus, a demonstration of the ability to get one l
1 bus to one school (even if done perfectly) would be an inadequate display of LILCO's ability to assist in the evactation of all the school children -- at both public and private schools -- in the EPZ.
j It is debatable whether any useful information at all was collected.
If we take the position that we do indeed have relevant randomly-sampled instances of trial runs for these parts of the Exercise, then the conclusions about the ability to evacu-i _ - _ _ _ _ _ -
l 1
l l
l l
ate the schools cannot be reached because of the very small l
sample sizes.
Samples of size one or size two are simply inadequate for reaching conclusions about a population.
l l
i D.
Sub_part D:
Soecial Facilities O.
Please state subpart D of Contention Ex 21.
l I
A.
Subpart D states as follows:
I According to the LILCO Plan, evacuation of the institutionalized, mobility-impaired popula-i tion in the EPZ (excluding residents at I
facilities which LILCO assumes would not re-quire transportation assistance and those in the three hospitals and the Suffolk Infirmary (cf. App. A at IV-175 and OPIP 3.6.5, Att. 2))
{
requires 23 ambulances and 178 ambulettes from 11 ambulance companies to make runs from the approximately 10 special facilities in the EPZ l
requiring transportation assistance from LILCO (there are, altogether, 19 health care facili-ties, excluding hospitals, at 23 locations throughout the EPZ) to unidentified receptio'n centers outside the EPZ.
Ege OPIP 3.6.5; App.
A at IV-166a thru -168, IV-173 thru -178.
i During the exercise, FEMA observed only one ambulance driver and gng ambulette driver each l
drive one route.
Based solely on its obser-i vation of these two activities, FEMA concluded that objective FIELD 13 was met.
FEMA Report I
at 43.
The sample upon which FEMA based this conclusion was much too small to justify FEMA's generalization about all ambulance and l
ambulette drivers and LILCO's ability to evac-uate institutionalized, mobility-impaired individuals from all the special facilities located in the EPZ.
In addition, based upon its observation of two drivers, FEMA concluded that objective EOC 16 was met with respect to all the institutionalized, mobility-impaired residents of the EPZ.
FEMA Report at 34.
In light of the size of the sample reviewed by FEMA, FEMA's conclusions as to objectives FIELD 13 and EOC 16 are without basis and J
invalid.
The exercise results concerning those objectives preclude a finding of reasonable assurance that adequate protective measures can and will be taken in the event of a Shoreham accident.
l l
l l
Q.
Do you agree with the allegations in subpart E.
l l
l A.
Yes we do.
I l
I l
Q.
Please state the objectives referenced in that subpart.
i A.
Those objectives are:
l l
l FIELD 13:
Demonstrate a sample of resources necessary to effect an orderly evacuation of the institu-tionalized mobility-impaired individuals within the 10-mile EPZ.
EOC 16:
Demonstrate the organizational ability to manage an orderly evacuation of all or part of the 10-mile EP7 including the water portion.
Q.
What were FEMA's conclusions concerning those objec-i tives, based on the Exercise results?
j i
l A.
The only pertinent FEMA conclusion concerning the ability to effect an evacuation of the institutionalized i
mobility-impaired individuals in the EPZ was that objective FIELD 13 "was met with respect to the ambulance and ambulette demonstrations."
FEMA Report at 43.
While noting that the LERO Ambulance Coordinator had 6 ambulances and 6 ambulettes report to the EOC, FEMA observed the driving of only 1 ambulance route (to l o-----------------------
the Our Lady of Perpetual Help Convent) and only 1 ambulette route (to the United Cerebral Palsy Residence).
FEMA Report at 43.
Those two vehicles were dispatched within 25 minutes and 44 minutes, respectively, from the time they were " requested."
FEMA Report at 36.
FEMA found objective EOC 16 met, without any J
particular reference to the mobility-impaired institutionalized' population in the EPZ.
FEMA Report at 34.
t Q.
In your opinion, are FEMA't-conclusions valid?
A.
No, for the same basic reasons we have discussed with i
l respect to the other subparcs of Contention Ex 21, and with
\\
respect to FEMA's overall approach to its evaluation of the l
I Exercise.
Specifically, a sample of one, which was the basis of l
FEMA's conclusions about ambulance evacuations, and also about 1
ambulette evacuations, is of no use, when the goal is to draw I
conclusions about a whole population of size 23 (ambulances) or l
i l
size 178 (ambulettes).
Similarly, for the reasons we have already discussed with respect to other aspects of the FEMA eval-uation, even if we were to combine the ambulance and ambulette I
populations, a sample size of two would not be sufficient to l
permit generalizations about a total population of 201.
of course, it is difficult to speculate about what an l
l adequate sample of ambulances and ambulettes would be, as it is i
l 1
l unclear from the LILCO Plan how many runs each vehicle would have l
56 -
to make, and where the vehicles would have to go.
LILCO has not, for the most part, designated reception locations for special facilities evacuees, or for the home bound mobility impaired.
Presumably, the geographic locations of these reception areas would have an impact on how many runs particular vehicles could make, on how many vehicles and personnel would be needed, and on what sample of vehicles, routes, and personnel would yield meaningful data concerning LILCO's ability to implement its Plan.
Furthermore, even though FEMA may have been able to observe that during the Exercise, LILCO had prearrar.ged to have 6
]ggg ambulance drivers and 6 ambulette drivers report to the EOC, that observation provides no data or basis for,E conclusions about the ability of any of those drivers, aside from the two evaluated by FEMA, to perform the activities necessary to effect an evacuation of the residents of 10 separate special facilities to unidenti-fled reception centers outside the EPZ, which is the point of Exercise objective FIELD 13.
Thus, using the same analysis described above, the confidence intervals for a sample of size two from a population of 201 are:
95% confidence interval Number of successful number of successful Previous, in outcomes in sample outcomes in occulation cercent terms 0 of 2 0 to 155 (out of 201) 0% to 77%
1 of 2 6 to 195 (out of 201) 3% to 97%
2 of 2 46 to 101 (out of 201) 23% to 100% - - _ _ _
I l
l This table combines the 23 ambulances and the 178 ambulettes for i
the purposes of making the calculation.
If we separately
)
considered a sample of one from 23 and a sample of one from 178, l
)
then the resulting intervals would be even less precise.
l l
While, for the reasons already stated above, we cannot say-with precision what sample size should have been used to evaluate l
ambulance and ambulette drivers, if FEMA had increased it to l
I include the 12 drivers who reported to the EOC during the l
Exercise, then (assuming this constitutes a random sample) the I
sample would have been able (with probability 95%) to distinguish values about 40 percentage points apart.
This is still not l
impressive power, but it is dramatically better than what one gets with a sample size of two.
E.
Suboart E:
Traffic Control l
Q.
Please state subpart E of Contention Ex 21.
j l
A.
Subpart E states as follows:
Under the LILCO Plan, 165 Traffic Guides are I
expected to man 130 traffic control points j
(TCPs) in order to implement an evacuation of the EPZ.
OPIP 3.6.3, Att.
4.
More speci-fically, 72 Traffic Guides from the Port j
Jefferson Staging Area are expected to man 58 l
TCPs; 41 Guides from the Patchogue Staging l
Area are expected to man 28 TCPs; and 52 i
Guides from the Riverhead Staging Area are i
expected to man 44 TCPs.
LILCO Plan, Fig.
l 2.1.1; OPIP 3.6.3, Att. 4.
During the exer-1 I
i l
! l 1
l y
cise, however, FEMA apparently only observed a total of 27 TCPs manned by 32 Guides (10 TCPs with 10 Guides from Port Jefferson; nine TCPs with 14 Guides from Patchogue; and eight TCPs with eight Guides from Riverhead).
See FEMA 1
Report at 57, 64 and 74.
Based solely on these observations, FEMA concluded that ob-jectives FIELD 6 and 11, SA 9, and EOC 16 and l
19 were met or partly met.
FEMA Report at 34, l
38, 56, 57, 62, 64, 65, 72, 74 and 75.
The sample upon which FEMA based these conclusions was much too small, however, to justify FEMA's i
generalizations with respect to all Traffic l
Guides, all TCPs, and evacuation and access control of the entire EPZ.
l Furthermore, even with respect to the small l
sample observed by FEMA, FEMA found (i) that the time between deployment from the staging area and arrival at TCPs was excessive --
l 1.e.,
two hours after declaration of a general cac.gsacy urra 7213 mTr se het'r sf kar the first evacuation recommendation -- for the eight TCPs from the Riverhead Staging Area l
(FEMA Report at 74); (ii) that only one out of l
the 14 Guides interviewed from the Patchogue Staging Area knew the location of the Reception Center (FEMA Report at 64); and, (iii) that communications between the Port Jefferson Staging Area and Traffic Guides was sometimes difficult due to poor radio recep-tion (FEMA Report at 56).
In addition, FEMA acknowledged that it was unable to render any evaluation regarding the timeliness of TCP I
set-up with respect to any of the 10 TCPs within the Port Jefferson Staging Area's l
jurisdiction.
FEMA Report at 57.
In light of l
the small sample reviewed by FEMA and its l
conclusions even with respect to the few TCPs i
it did observe, FEMA's conclusions with re-spect to objectives FIELD 6 and 11, SA 9, and I
EOC 16 and 19 are without basis and invalid.
The exercise results concerning those objec-tives preclude a finding of reasonable assur-ance that adequate protective measures can and i
will be taken in the event of a Shoreham j
accident.
i i
i j I
Q.
Do you agree with the allegations in subpart E?
A.
We agree in part.
l i
i 1
Q.
Please state the objectives referenced in that subpart.
i l
A.
Those objectives are:
l FIELD 6:
Demonstrate that access control points can be
)
established and staffed by Traffic Control Guides l
l in a timely manner.
I FIELD ll:
Demonstrate a sample of resources necessary to control access to an evacuated area (Traffic Guides).
l SA 9:
Demonstrate the ability to dispatch to and direct emergency workers in the field.
EOC 16:
Demonstrate the organizational ability to manage an orderly evacuation of all or part of the 10-mile EPZ including the water portion.
EOC 19:
Demonstrate the organizational ability necessary j
to control access to an evacuated area.
l l
In our opinion, the most pertinent objectives to the main point of subpart E are FIELD 6, EOC 16, and EOC 19.
Q.
What were FEMA's conclusions concerning those objectives?
A.
FEMA found objective EOC 16 met, but without specific reference to the performance or capabilities of Traffic Guides.
FEMA Report at 34.
It also found EOC 19 met, in the following words:
)
)
An exercise objective was met by demonstrating the organizational ability.necessary to con-trol access to an evacuated area (EOC 19).
The Traffic Control Point Coordinator de'cer-1 mined which TCPs needed to be staffed.
This j
information was communicated to the staging l
areas according to the Plan.
The Evacuation Coordinator initially notified the Coast Guard at about 0755.
FEMA Report at 38.
FEMA's conclusions about FIELD 6 vere the
]
following with respect to the 10 Traffic Guides supposedly evaluated from the Port Jefferson staging area:
l
'l Traffic Guide teams were briefed and dis-patched in priority order, as planned.
Time-liness of TCP setup could not be evaluated I
because the Federal evaluator was delayed at j
- ncther location.
However, all tne TCPs nere established at the correct loca-i tions, and each team demonstrated thorough l
knowledge of their assignment (Field 6).
Each I
I team understood its role in providing. guidance to motorists and was aware of LERO policy re-garding interaction with county / local police.
1 FEMA Report at 57.
With respect to the 14 Traffic Guides l
l evaluated from the Patchogue staging area:
The objective of' demonstrating that access control points can be established and staffed by Traffic Guides in a timely manner was l
partly met (Field 6).
All nine (9) TCPs evaluated were fully staffed in a timely l
manner.
All personnel. understood the concept of operations at their respective locations, including timely radio communication and check-1n, as well as proper placement, as appropriate of barricades and cones.
- However, l
only one (1) Traffic Guide out of the fourteen (14) who were interviewed at nine (9) TCPs
~
knew the location of the Reception Center, and I
one (1) Traffic Guide thought the general public was to be directed to the EWDF.
1
! d
FEMA Report at 64.
Finally, with respect to the eight Traffic Guides evaluated from the Riverhead staging area:
The objective to demonstrate that TCPs can be established and staffed by Traffic Guides in a timely manner was partly met (Field 6).
Eight (8) TCPs were observed in the Riverhead Staging Area's jurisdiction.
The time between deployment of Traffic Guides from the staging area and their arrival at TCPs was excessive, taking between fifty (50) and seventy (70) minutes.
Each Traffic Guide communi-cated successfully by radio with the staging area.
The Traffic Guides were all at the proper locations and were familiar with where to guide traffic, how to interact with the county police, and the limitations of their authority.
FEMA Report at 74.
Q.
You mentioned that ye agree in part with the alle-gations in subpart E.
Please-explain.
A.
The size of the samples of Traffic Guides evaluated by FEMA during the Exercise are not nearly as insufficient as was the case for the other job categories we have discussed in this testimony.
Thus, in the absence of information about the target values, bad values, and desired probabilities of detecting the difference, we cannot conclude, as we did for the other cate-gories, that under all circumstances, the Traffic Guide samples were too small.
In fact, the sample size chosen by FEMA (32 out of 165 for the sites combined) is able (with probability 70%) to distinguish values about 22 percentage points apart.
This could be useful if, for instance, the target value were 75% while the rangs tf unacceptable values were 0% to 53%.
We suspect that finer statements would be useful here, and we would have recommended a larger cample size.
However, the sample of 32 is not without value.
F.
Subpart F:
Impediments to Evacuation Q.
Please state subpart F of Contention Ex 21.
A.
Subpart F states as follows:
During the exercise, FEMA chose only two loca-tions to observe LILCO's ability to deal with impediments to evacuation (objective EOC 17).
These two locations -- in the vicinity of TCPs 41 and 124 -- both involved simulated traffic obstructions; they were the result of a simu-lated accident involving a gravel truck and three cars (TCP 124) and a simulated accident 1
involving a fuel truck (TCP 41).
FEMA Report at 30, 36-38.
i FEMA was unable to observe LILCO's response to the fuel truck impediment (agg FEMA Report at 37, 57-58), found LILCO's response to the gravel truck impediment inadequate (FEMA Report at 36-38), and concluded that objective EOC 7 was only partly met and objective EOC 17 was not met (FEMA Report at 29-30, 36-38).
FEMA nonetheless concluded that objectives EOC 16 and FIELD 9 for the Port Jefferson Staging Area were met and that objective FIELD 10 for the Patchogue Staging Area was partly met (FEMA Report at 34, 58, 65).
Those conclu-sions are without basis and invalid, in light of the following facts:
(i)
FEMA never even observed LILCO's response to the fuel truck impediment (which occurred within the Port Jefferson Staging l
Area's jurisdiction (agg App. A, Fig.
8, and FEMA Report at 57-58);
(ii) LILCO did not respond to the fuel i
truck impediment until more than three hours after the free play message had been injected at 11:00 (FEMA Report at 36-37, 57);
(iii) There were substantial delays in LILCO's response to the gravel truck irapedi-ment (which occurred within the Patchogue Staging Area's jurisdiction, see App. A, Fig.
8, and FEMA Report at 65);
(iv) The response was inadequate for removal of the gravel truck impediment (only one tow truck was dispatched even though the simulated accident involved three cars and the gravel truck, and no scraper was sent to remove spilled gravel from the road) (FEMA Report at 37, 65);
(v)
At least two hours elapsed before LILCO's Transportation Support Coordinator was informed that a bus evacuation route was potentially blocked by the gravel truck impediment (FEMA Report at 36).
In addition, the sample upon which FEMA based its conclusions regarding objectives EOC 16 and FIELD 9 for the Port Jefferson Staging Area and FIELD 10 for the Patchogue Staging Area was much too small to justify FEMA's generalizations with respect to the entire EPZ and LILCO's ability to implement an evacuation during a Shoreham accident.
Under the LILCO Plan, 165 Traffic Guides are expected to man 130 traffic control points in order to imple-ment an evacuation of the EPZ.
App. A, Fig.
8; OPIP 3.6.3, Att.
4.
These locatione are manned by LILCO because traffic must be kept moving through them in order to permit evacuees to follow the routes prescribed in the LILCO Plan.
The evacuation time estimates used by the LILCO players during the exercise were premised on the assumption that such prescribed routes were in fact available for full capacity use throughout the evacuation process.
Thus, there are a substantial number of places where impediments are likely to occur because of anticipated heavy traffic, and where their occurrence would have signifi-cant impact upon the ability to implement an evacuation, but which were ignored in FEMA's evaluation of LILCO's ability to remove im-pediments.
Furthermore, the two locations a
I
)
actually chosen for the impediment demonstra-
)
tions during the. exercise are virtually the i
only locations on the evacuation routes chosen for the. impediment messages which would allow easy re-routing of evacuation traffic around the traffic impediments, assuming arcuendo, LILCO had been able to respond to the loca-tions of the impediments.in a timely manner.
In light of the small sample of traffic impediment locations observed by FEMA, FEMA's i
conclusions with respect to the single attempted impediment response it did observe, and the unique and unrepresentative impediment I
locations chosen by. FEMA, FEMA's conclusions I
with respect to objectives EOC 16, FIELD 9 for the Port Jefferson Staging Area and FIELD 10 for the Patchogue Staging Area are without basis and invalid, thereby precluding a finding of reasonable assurance that adequate i
protective measures can and will be taken in the event of a Shoreham accident.
l Q.
Do you agree with the allegations in subpart F?
I l
l A.
While we generally agree, we also understand that the matters raised in that subpart have already been addressed in testimony sponsored by other witnesses concerning Contention j
i Ex 41.
Therefore, we do not discuss them here.
I 1
\\
l Q.
Does that complete your testimony?
A.
Yes.
l l _______-_-_______ - -
7-1 I
j i
I
)
1 ATTACHMENT 1 a
l l
i U'
Gary Simon Personal Resume
)
November 1986 I
I Education:
BS Mathematics, Carnegie-Mellon University, Pittsburgh, Pennsylvania, 1966.
l PhD Statistics, Stanford University, Palo Alto, j
l California, 1972.
l Positions:
United States Peace Corps, morked with Department i
of Applied Mathematics, University of the 1
Philippines,- Los Banos, September 1966 to August 1968.
Assistant Professor, Department of Statistics, Princeton University, Princeton, New Jersey, September 1971 to August 1975.
Associate Professor, Department of Statistics, State University of New York, Stony Brook, New York, September 1975 to August 1981.
Associate Professor, " Department of Statistics and Operations Research, Graduate-School of Business Administration, New York University, New York, New York, September 1981 to date.
Miscellaneous Proofreader for the Wiley Series in Si,4istics through Beatrice Shube, Wiley-Interscience, New York.
Associate
- Editor, Journal of the American Statistical Association, January 1979 %o December 1981 Consultant to Sallup Organization through paul I
Perry, Gallup Organization, Princeton, New Jersey, 1977-1978.
Consultant-to Committen on Science and Public Policy (COSPUP),
National Academy of Sciences, 1979-1980.
Recipient of Chance 11or's Award for Excellence in Teaching (in the SUNY system), May 1979.
Lecturer in the Chautauqua short-course program, 1979-1980 and 1980-1981.
Consultant to Research Testing Laboratories, Little Neck, Queens, New York, 1981 to date.
L
Consultant to Corporation Counsel, City of New York, March 1983 to April 1984.
Consultant to marketing sectica,
- CITICORP, 575 Lexington Avenue, New York, November 1983 to date.
Consultant to Consumers Union, Mt.
- Vernon, New York, 1985 to date.
Consultant to Neissner, Tisch, and Kleinberg, 275 Madison Avenue, New York, 1986.
j Personal:
Born 24 APRIL 1945, Wilkes-Barre, Pennsylvania.
Married, four children.
BOOK REVIEWS Basic Concept s in Information Theory and Statistics, by A. M.
Mathat and P.M.
Rathie, reviewed in Technometrdes, vol 18, 1976, p 502.
R. A.
- Fisher, the Life of a Scient'ist, by Joan Fisher Box, l
reviewed in SIAM News, vol 11, 85, October 1978, pp 5-6.
Statistics, by D. Freedman, R. Pisani, and R. Purves, reviewed in Journal of the American Statistical Association,.vol 74, 1979, pp 927-928.
(This review was subsequently incorporated into the publisher's advertising. )
The Analysis of Cross-Tabulated Data, by Graham J. B.
- Upton, reviewed in Technometrics, vol 21, 1979, pp 581-582.
Measures of Association for Cross-Classifications, by Leo A.
Goodman and William H.
- Kruskal, reviewed in Journal of the American Statistical Association, vol 75, 1980, p 1036.
Beginning Statistics with Data Analysis, by Mostel}er, Rienberg, and Rourke; Modern Elementary Statistics, 6th edition, John E.
- Freund, reviewed in Journal of the Hmerican Statistical Association, vol 80, 1985, p 240.
Statistical Sampling for Accountants, by T.M.F.
- Smith, and Practical Statistical Sampling for Auditors, by Arthur J.
- Wilburn, Qoint review of two books) reviewed in Journal of the American Statistical Association, vol 80, 1985, p 1078.
1
I
\\
%g-c.
PUBLICATIONS Additivity of Information in Exponential Family Probability Laws, Journal of the American Statistical Association, vol 68, 1973, pp 478-482.
I Bainful Employment Following Head Injury, Prognostic Fact ors, i
Archives of Neurology, y 29, 1973, pp 111-116, with co-authors A.
l Dresser and four others.
Alternative Analyses for the Singly-Ordered Contingency ' Table, Journal of the American Statistical Association, vol 69, 1974, pp l
971-976.
A Nonparametric Test of Total Independence Based On Kendall's Tau, Biometrika, vol 74, 1977, pp 277-282.
1 Computer Simulation Swindles, with Application to Estimates of Location and Dispersion, Applied Statistics, vol 25,
- 1976, pp 266-274.
l Multivariate Generalizations of Kendall's Tau, with Application to Data Reduction, Journal of the American Statistical l
Association, vol 72, 1977, pp 367-376.
l Comments on paper by Schruben and Margolin, Journal of the American Statistical Association, vol 73, 1978, pp 520-521.
Efficacies of Measures of Association for Ordinal Contingency
- Tables, Journal of the American Statistical Association, vol 73, 1978, pp 545-551.
The Sign Test when Ties are Present:
THe Problem of Confidence Intervals, American Statistical, vol 33, 1979, pp 140-142, with co-author John Emerson.
The Probability of primary Tooth Exfoliation: A Reference Source for Programs that Treat Children during their Mixed Dentition, Journal of Public Health Dentistry, vol 41,
- 4, Fall 1981, pp 236-243, with co-authors L. Ripa and three others.
Chance and Consensus in Peer Review, Science, vol 214, 20 NOV 81, pp 881-886, with co-authors J. Cole and S. Cole.
A General Expression for the Variance-Covariance Matrix of Estimates of Gene Frequency:
the Effects of Departures from Hardy-Weinberg Equilibrium, Annals of Human Genetics, vol 48, 1984, pp 283-286, with co-author Nancy R. Mende11.
Diagnostics for Missing Data in Least Squares Regression, Journal f
of the American Statistical Association, vol 81,
- 1986, pp 501-509, with co-author Jeffrey Simonoff.