ML20214A043
| ML20214A043 | |
| Person / Time | |
|---|---|
| Site: | Braidwood  |
| Issue date: | 11/13/1986 |
| From: | Atomic Safety and Licensing Board Panel |
| To: | |
| References | |
| CON-#486-1670, CON-#486-1970 OL, NUDOCS 8611190174 | |
| Download: ML20214A043 (136) | |
Text
.
ORGNII_
n UNITED STATES kj NUCLEAR REGULATORY COMMISSION IN THE N1ATTER OF:
DOCKET NO: 50-456 OL 50-457 OL COMMONWEALTH EDISON COMPANY (Braidwood Station, Units 1 and 2)
LOCATION:
CHICAGO, ILLINOIS PAGES: 17067 - 17201 DATE:
THURSDAY, NOVEMBER 13, 1986
,0\\
0\\
ACE-FEDERAL REPORTERS, INC.
n is'J O?icialReporters 444 North Capitol Street 1i190174 861113 Washington, D.C. 20001 hR
/* DOC K 0500 6
(202) 34 -3 00 NATIONWIDE COVERAGE
l 17067 i
1 UNITED STATES OF AMERICA 2
NUCLEAR REGULATORY COMMISSION 3
BEFORE THE ATOMIC SAFETY AND LICENSING BOARD 4
12;;;_;;;__;;;;;;;
In the Matter of:
6
(Braidwood Station, Units 1 8
and 2)
_ ; ; ; ; ; _ ; ; ; ; ; _ ; ; _ ; ;x 9
10 Pages 17067 - 17201 11 United States District Courthouse Courtroom 1743 O
12 219 South Dearborn Street Chicago, Illinois 60604 13 Thursday, November 13, 1986.
14 l
15 The hearing in the above-entitled matter reconvened 16 at 9:00 A.
M.
17 i
BEFORE:
18 i
JUDGE HERBERT GROSSMAN, Chairman 19 Atomic Safety and Licensing Board 4
I U.
S. Nuclear Regulatory Commission 20 Washington, D.
C.
1 21 JUDGE RICH ARD F. COLE, Member, Atomic Safety and Licensing Board 22 U.
S. Nuclear Regulatory Commission Washington, D.
C.
23 JUDGE A. DIXON CALLIH AN, Member, 24 Atomic Safety and Licensing Board U.
S. Nuclear Regulatory Commission
["'N
( )
25 Washington, D.
C.
Sonntag Reporting Service, Ltd.
Geneva, Illinois 60134 (312) 232-0262
17068 O
1 APPEARANCES:
2 On behalf of the Applicant:
3 MICHAEL I.
MILLER, ESO.
Isham, Lincoln & Beale 4
Three First National Plaza Chicago, Illinois 60602 5
On behalf of the Nuclear Regulatory 6
Commission Staff:
7 GREGORY ALAN BERRY, ESQ.
ELAINE I. CHAN, ESO.
8 U.
S. Nuclear Regulatory Commission 7335 Old Georgetown Road 9
Bethesda, Maryland 20014 10 On behalf of the Intervenors:
11 ROBERT GUILD, ESQ.
13 14 15 16 17 18 19 20 21 22 23 24 25 Sonntag Reporting Service, Ltd.
Geneva, Illinois 60134 (312) 232-0262
i 4
17069 I
1 EXHIBIT INDEX Marked Received
(
2 Intervenors' Exhibit No.192 17104 I
3 i
l 4
l I
5 i
i 6
i 7
i 8
~
9 t
l 10 i
11
.l 12 j
13 1
14 i
j 15 I
16 l
17 18 i
19 l
20 l
21 t
22 23 i
24 i
25 r
i
{
Sonntag Reporting Service, Ltd.
I t
Geneva, Illinois 60134 i
(312) 232-0262
I i
17070 1
TESTIMONY OF MARTIN R.
FRANKEL 2
DIRECT EXAMINATION BY MR. MILLER:
17076 3
Prefiled testimony of 4
Martin R.
Frankel 17082 5
DIRECT EXAMINATION (Continued)
BY MR. MILLER:
17083 6
CROSS EXAMINATION 7
BY MR. GUILD:
17098 8
CROSS EXAMINATION BY MS. CH AN :
17128 9
BOARD EXAMINATION 10 BY JUDGE GROSSMAN:
17134 11 BOARD EXAMINATION BY JUDGE COLE:
17144 t
s 12 BOARD EXAMINATION 13 BY JUDGE CALLIH AN:
17152 14 BOARD EXAMINATION BY JUDGE GROSSMAN:
17153 15 BOARD EXAMINATION 16 BY JUDGE COLE:
17154 17 REDIRECT EXAMINATION BY MR. MILLER:
17155 18 RECROSS EXAMINATION 19 BY MR. GUILD:
17164 1
20 REDIRECT EXAMINATION (Continued)
BY MR. MILLER:
17188 21 BOARD EXAMINATION I
22 BY JUDGE GROSSMAN:
17191 23 RECROSS EXAMINATION (Continued)
BY MR. GUILD:
17192 24 25 x
Sonntag Reporting Service, Ltd.
Geneva, Illinois 60134 (312) 232-0262 t
1
l 17071
/~
(
1 JUDGE GROSSMAN:
The hearing is reconvened.
2 This is the 90th day of hearing.
3 Before we begin with our next witness, we've 4
thought about completing the testimony in the case, and 5
we still want to shoot for the end of next week, whether 6
or not we can get Mr. McGregor on, and I think we ought 7
to plan on having Mr. Gardner next week.
8 We will give Staff certainly the opportunity to 9
recall Mr. Gardner after Mr. McGregor if some issues 10 arise in Mr. McGregor's testimony for which Mr. Gardner 11 is needed.
12 But I think that we really don't want to hold up 13 completing the remainder of the case until Mr. McGregor 14 is able to appear.
15 So if Staff can get Mr. Gardner's prefiled 16 testimony out hopefully by the end of tomorrow -- is 17 that a possibility, Mr. Berry?
18 MR. BERRY:
Very, very remote, Mr. Chairman, 19 although we will endeavor on that.
We'll try.
20 JUDGE GROSSMAN :
Okay.
If not --
21 MR. BERRY:
I just don't think so, Mr.
22 Chairman; but we will make it available to the parties 23 at the beginning of next week.
24 JUDGE GROSSMAN:
Okay.
Monday, of course, 25 we're not meeting, so perhaps you can do that sometime Sonntag Reporting Service, Ltd.
Geneva, Illinois 60134
.f (312) 232-0262
17072
("~)
1 during the day on Monday.
2 MR. BERRY:
We certainly will try.
Certainly 3
no later than lunchtime on Tuesday.
4 But again, Mr. Chairman, Staff counsel is in the 5
hearing and the witnesses are --
6 JUDGE GROSSMAN:
Well, we have tomorrow and 7
we have Monday, so maybe you can work on those days.
8 I know that is a tough schedule because there are 9
other witnesses to prepare cross examination for.
10 MR. BERRY:
We certainly will try, Mr.
Il Chairman.
O}
12 The other point that I would just make -- or the a
13 question I would just raise is whether the Intervenors 14 have reached a determination as to whether they're going 15 to -- they have any witnesses to present in this 16 connection.
17 JUDGE GROSSMAN:
Well, I believe, from what 18 I've heard, that aside from rebuttal of witnesses not 19 yet heard, Mr. Guild doesn't have any rebuttal 20 testimony, but is reserving the right to rebut Dr.
l 21 Frankel, if necessary, and the others.
22 Is that correct, Mr. Guild?
23 MR. GUILD:
That's a correct understanding, 24 Judge.
25 JUDGE GROSSMAN:
Okay.
Sonntag Reporting Service, Ltd.
Geneva, Illinois 60134 (312) 232-0262
17073
,m v
1 MR. BERRY:
Are the others Mr. Laney and Mr.
2 Hulin?
3 MR. GUILD:
Yes, certainly.
4 MR. BERRY:
Very well, Mr. Chairman.
5 Staff will certainly use its best efforts to 6
accommodate all the parties and the Board.
7 We are just as anxious to conclude this hearing as 8
any party.
9 JUDGE GROSSMAN:
Okay, fine.
10 Mr. Guild, I hope that you'll be able to prepare 11 for all those experts next week over the extra couple of
[
\\
12 days that you have, tomorrow and Monday, plus the b
13 weekend.
14 MR. GUILD:
Sir, I think that's pressing it 15 quite a bit.
16 I wanted the Board to understand as well that we've 17 identified this mass of documentation that was produced 18 by Mr. Hii.
19 I reviewed that during a full day this week and am 20 waiting for the copies of that material to be supplied 21 for a detailed review.
22 But I think that -- I appreciate the Board's desire 23 to complete the hearing expeditiously, but we've got 24 what then will be three principal experts that you' re 25 scheduling next week, including Mr. Gardner.
Sonntag Reporting Service, Ltd.
Geneva, Illinoi s 60134 (312) 232-0262
17074 1
I have not seen Mr. Gardner'E testimony,.'so even 2
with the Staff's assurances that'I won't be surprised, I 3
think I should -- I have to reserve until I see his 4
testimony to know what the substance of it will be.
5 Hulin, Laney, both principal experts for Applicant, 6
as well as trying te review and evaluate,a mass of new 7
documentation.
8 So it's quite an ambitious undertaking, even with 9
the two days off scheduled.
4 10 JUDGE GROSSMAN:
Okay.
11 I consider Mr. Hii's documents and the surprise 12 elements that might be present in Mr. Gardner's 13 testimony as being loose ends that we can't expect you 14 to know about for next week.
i 15 But we want to get done with everything that is 16 already cognizable, and that means, to the best we can, 17 those prepared experts.
l 18 MR. GUILD:
I'm ready for them, Judge.
I 19 expect to be ready next week for Messrs. Hulin and j
20 Laney.
It's the other wild cards that I can't account 21 for particularly.
22 JUDGE GROSSMAN :
Okay.
Maybe the loose ends 23 then can be taken care of the week follokine, or,if
~
24 that's not adequate, maybe we won't even ineet the week N
25 following and take care of the loose ends.
l l
\\
j Sonntag Reporting Servi ~e_,
Ltd.
l Geneva, Illinois 60134 (312) 232-0262
4 17075 i
,,~
1 I don't know what the loose ends are, and it may l
2 require a pause of a week or so, but we'll just have to 3
play that by ear.
4 Whatever, though, is contemplated now in the 5
testimony I would hope would be completed by the end of 6
next week, then; but I don't want to spend any further 7
time on this.
8 Mr. Miller, did you wish to say anything?
9 There's no need to if --
10 MR. MILLER:
No, your Honor, except that we 11 oppose any further extension and support the Board's 12 desire to complete next week so that we can all get 13 started on the findings.
14 JUDGE GROSSMAN:
Okay, fine.
15 Are there any other preliminary matters?
16 If not, then, Mr. Miller, please call your next 17 witness.
18 MR. MILLER:
Thank you, your Honor.
19 We call Dr. Martin Frankel to the stand.
20 JUDGE GROSSMAN:
Okay.
Dr. Frankel, will you 21 stand, please, and raise your right hand.
22 (The witness was thereupon duly sworn.)
23 JUDGE GROSSMAN:
Please be seated.
24 Mr. Miller.
25 MR. MILLER:
Thank you, Judge Grossman.
Sonntag Reporting Service, Ltd.
Geneva, Illinois 60134 (312) 232-0262
17076 4
iV 1
MARTIN R.
FRANKEL 2
called as a witness by the Applicant herein, having been 3
first duly sworn, was examined and testified as follows:
4 DIRECT EXAMINATION 5
BY MR. MILLER:
6 Q
Dr. Frankel, would you state your name for the record, 7
please?
8 A
Martin R.
Frankel.
9 0
And what is your current business address?
10 A
My current business address is 17 Lexington Avenue, New 11 York, New York, 10010, and that's Baruch College,
[
12 B-A-R-U-C-H College, CUNY.
b) 13 Q
By whom are you employed, Dr. Frankel?
14 A
I'm employed by the City University of New York.
15 Q
What is your profession or occupation?
16 A
I'm Professor of Statistics and Computer Information 17 Systems at Baruch College of New York.
18 Q
Dr. Frankel, do you have before you a 27-page document 19 entitled " Rebuttal Testimony of Martin R.
Frankel On i
20 Rorem QA Subcontention 2, Harassment and Intimidation"?
21 A
Yes.
22 0
There are two attachments to that document, are there 1
23 not?
i 24 A
Yes.
)
25 Q
Dr. Frankel, we have previously circulated to the Board
(
l Sonntag Reporting Service, Ltd.
l Geneva, Illinois 60134 l
(312) 232-0262
17077 mU 1
and parties certain changes to the prefiled testimony.
2 These are changes in numbers that appear at Pages 3
13,14,18,19 and 20, and on Page 2 of Attachment 1 to 4
the document that's before you.
5 A
Yes.
6 MR. MILLER:
I'd simply inquire of the Board 7
and the parties as to whether or not they have those 8
changes in the prefiled testimony of Dr. Frankel?
9 MR. GUILD:
I don't have the -- I've received 10 them in the mail.
11 I would appreciate it if the witness would simply
[J
\\
12 identify them f rom the stand, Judge.
13 MR. MILLER:
Let me just go through them very 14 quickly, then.
15 BY MR. MILLER:
16 Q
Dr. Frankel, would you turn first to Page 13 of the 17 document.
18 Would you state for the record the change that 19 appears on that page?
20 A
On Page 13, the last line, the number 98.909 percent 21 should be changed to read 98.822 percent.
22 O
Sir, on the next page, Page 14?
23 A
On the following page on the second line, the first 24 number, 98.893, should be changed to read 98.719, and 25 the second number, 98.925, should be changed to read Sonntag Reporting Service, Ltd.
Geneva, Illinois 60134 (312) 232-0262 4
17078 v
1 98.940.
2 Q
All right, sir.
3 Now turn to Page 18.
Would you tell us what 4
changes you made there?
5 A
In the fifth line of Answer 21, 98.893 should be changed 6
to read 98.719 and 98.925 should be changed to read 7
98.940.
8 The line immediately following that, 0.24314 9
percent should be changed to read 0.29620 percent.
10 In the formula that is on the -- effectively the 11 ninth line in Answer 21, the numbers should be changed h
12 from -- the first number should be -- instead of 98.893, J
13 it should be 98.719; the second number, from 98.925 to 14 98.940; the division is by -- instead of 0.24314, it 15 should be 0.29620; and the final number should be --
16 instead of 0.1316, should be minus 0.746.
17 The third line f rom the bottom of Page 18, the 18 number minus 0.1316 should be changed to read minus 19 0.746.
20 0
All right, sir.
21 On the next page Page 19, would you describe the 22 changes?
23 A
On the eighth line down on Page 19, the number minus i
24 0.1316 should be changed to read minus 0.746.
25 Q
Turning to Page 20, what changes are made there?
Sonntag Reporting Service, Ltd.
J Geneva, Illinois 60134 (312) 232-0262
17079 pb 1
A The 14th line down, 0.7 percent should be changed to 0.8 2
percent, and on the next line, 0.5 percent should be 3
changed to read 0.6 percent.
4 0
Dr. Frankel, turning to Page 2 of Attachment 1 to your 5
prefiled testimony, can you describe for us what 6
changes, if any, appear there?
7 A
I wonder if counsel could give me a copy of that.
8 0
(Indicating.)
9 A
Thank you.
10 On Page 2, the first formula that reads R equals, 11 in both the numerator -- in the numerator of that 12 formula there is a Y equal, the first thing, and that 13 should be eliminated.
There's an X equal, and that 14 should be eliminated as well.
15 MR. GUILD:
Eliminate what, Dr. Frankel; the 16 X and the Y?
17 THE WITNESS:
Where it says Y equals, that 18 should be deleted, and where it says X equals, that 19 should be deleted.
20 So what you see in that formula in the numerator, 21 it begins with a summation sign.
22 MR. GUILD:
Yes.
23 THE WITNESS:
In the denominator, it begins 24 with a summation sign.
r 25 BY MR. MILLER:
Sonntag Reporting Service, Ltd.
Geneva, Illinois 60134 (312) 232-0262
17080 O
t t
V 1
Q Dr. Frankel, could you describe for the record why the 2
numbers that appear in the text of your testimony which 3
you've just testified to have changed?
4 A
Yes.
Mr. Miller, there's also a change on Page 4.
5 0
Oh.
I'm sorry.
6 This is Page 4 of Attachment 27 7
A Yes.
8 It's rather complicated to describe in words.
I 9
wonder if there's an alternate way of getting that into 10 the record correctly.
11 JUDGE GROSSMAN:
Well, it's going to be in O
12 the record correctly anyway, so we don't have to bother.
(~-]\\
13 THE WITNESS:
There was -- a typo in the 14 formula is what occurred.
15 MR. MILLER:
Unless Mr. Guild is going to 16 immediately go to that formula for his cross 17 examination, perhaps Dr. Frankel can supply it to Mr.
18 Guild at the break.
19 MR. GUILD:
I'll rearrange my cross 20 examination to defer that question until later.
21 (Laughter.)
22 BY MR. MILLER:
23 Q
Dr. Frankel, I'd ask what the reason was for the changes 24 in the numbers in the text of your testimony.
25 A
It was my understanding that the data base changed, and Sonntag Reporting Service, Ltd.
Geneva, Illinois 60134 (312) 232-0262
17081 O
/b 1
Mr. Orlov provided me with revised figures.
I went 2
through my calculations again with those revised 3
figures.
4 Q
All right, sir.
5 I'd like to call your attention to Page 21 of your 6
prepared testimony.
7 Are there any changes on that page?
8 A
Yes.
In the answer to Question 25, the fourth line 9
down, the word " item" should be deleted and the words 10
" inspection point" should be inserted.
11 0
Dr. Frankel, can you tell us what the reason for that 12 change is?
13 A
Well, quite frankly, I know that when I wrote it, I 14 wrote it as inspection point, and I just never picked it 15 up when I proofread the drafts.
16 0
With the changes that you've just described, is the 17 testimony true and correct to the best of your knowledge 18 and belief?
19 A
Yes, to the best of my knowledge.
20 MR. MILLER:
Your Honor, at this point in 21 time, I would ask that Dr. Frankel's prepared testimony 22 and the attachments be bound into the record as if read.
23 JUDGE GROSSMAN:
Any objection?
24 MR. GUILD:
Mr. Chairman, subject to cross 25 examination, we have no objection.
Sonntag Reporting Service, Ltd.
Geneva, Illinois 60134 f
(312) 232-0262
4 17082 I
T 1
JUDGE GROSSMAN:
Miss Chan?
i 2
MS. CH AN :
No objection from the Staff, your 3
Honor.
4 JUDGE GROSSMAN:
Okay.
The testimony is 5
admitted and will be bound in the record as though 6
stated here at hearing.
i 7
8 9
10 11 12 13 14 15 16 17 i
l 18 4
1 19 i
j 20 21 22 l
23 I
24
^5 i
l Sonntag Reporting Service, Ltd.
I Geneva, Illinois 60134 (312) 232-0262
a
.O' UNITED STATES OF AMERICA NUCLEAR REGULATORY COMMISSION BEFORE THE ATOMIC SAFETY AND LICENSING BOARD In the Matter of
)
)
COMMONWEALTH EDISON COMPANY
)
Docket Nos. 50-456
)
50-457 (Braidwood Station Units 1 and 2)
)
REBUTTAL TESTIMONY OF MARTIN R. FRANKEL (ON ROREM Q. A. SUBCONTENTION 2)
(Harassment and Intimidation)
O
%tngniE, 1986
\\
(1)
\\
REBUTTAL TESTIMONY OF MARTIN R. FRANKEL ON THE CONSTRUCTION SAMPLE REINSPECTION ELEMENT OF THE BRAIDWOOD CONSTRUCTION 4
ASSESSMENT PROGRAM Q.1.
Please state your full name for the record.
^
A.1.
Martin R. Frankel Q.2.
Please describe your present positions and your job responsibilities.
A.2.
At the present time I am Professor of Statistics, Bernard Baruch College, City University of New York, 17 Lexington Avenue, New York, New York 10010.
I am responsible for the teaching of all graduate and
()
undergraduate courses in survey sampling.
In addi-tion, I teach courses in general statistics and in computer languages.
I have been at Baruch College since 1971 with the exception of a two year period when I was an Assistant Professor of Statistics in the l
Graduate School of Business of the University of Chicago.
I also serve as Technical Director of the National Cpinion Research Center, University of Chicago.
In
~
this position I am responsible for the statistical and technical quality of all contract survey research conducted by the Center.
i k
i l
i-
(
t I
g Q.3.
Please describe your educational and professional background.
1 A.3.
I hold an AB degree in Mathematics from the University 1
of North Carolina.
I hold an M.A. degree in Mathe-matical Statistics and a Ph.D. degree in Mathematical Sociology from the University of Michigan.
My doc-toral dissertation was in the area of inference from complex probability samples.
This dissertation, which was published by the Institute of Social Research of the University of Michigan under the title Inference From Complex Samples, is currently in its fifth printing.
I have been actively involved in the use of probabil-ity sampling techniques for a period of 21 years.
Over this time period I have been involved in the design, selection and implementation of more than 100 different large scale samples.
This work has been carried out for federal agencies, universities, an international organization and business firms.
The major professional organization for applied statisticians in the United States is the American Statistical Association.
I was elected a Fellow of the Association in 1979 for my work in the area of
()
probability survey sampling.
I have served as i
1 i...
.(
Chairman of the Association's Section on Survey Research Methods and its Advisory committee to the U.
S. Bureau of the Census.
I also served as an associate Editor of the Journal of the American Statistical Association for a period of 8 years.
In addition to the title mentioned above, I am co-author of 2 books in the area of survey sampling.
I am coauthor and author respectively of the chapters on probability sampling in The Handbook of Marketing Research (McGraw Hill, 1974) and the Handbook of Survey Research (Academic Press).
I have published articles on survey sampling in various scientific
()
journals.
I am one of the four members of the Editorial Board of the 8 volume Encyclopedia of Statistical Sciences (John Wiley and Sons).
I was elected to membership in the International statistical Institute in 1983. At the present time I serve as a member of the National Academy of Sciences Panel on Occupational Safety and Health Statistics.
Q.4.
Are you familiar with the Construction Sample Rein-spection (CSR) element of the Braidwood Construction Assessment Program (BCAP)?
A.4.
Yes.
I have had an active involvement with BCAP and specifically with the CSR element of BCAP since August of 1984.
t t..
.O
(
Q.5.
Can you define tne terms probability sample, non-probability sample an6 random sample?
A.S.
A probability sample is a sample tnat is selected by a procedure that gives eacn element in a defined popula-tion a Known, calculable, non-zero probability of being incluceo in tne sample.
A non-probability sample is any sample tnat coes not fall unoer tne definition of a probability sample.
2ne term random sample is often useo tnree different ways.
O In tne formal tneory of probability sampling it is useo to descrice a type of probability sample in wnien all combinations of elements of a given size in the population ano all subsets of tnis size nave an equal enance of being selected into tne sample.
In tnis context, ranoon, samples of elements may be oetinec as "selecteo witnout replacement" or "witn replacement."
In general statistical tneory, tne term rancom sample is used to describe a sample from"a population tnat may be treateo matnematically as tne product of 1
indepenoent, identically distributed random variables.
As I will alscuss later, tnere are
.O numerous instances where samples which do not satisfy the probability sampling definition of random s'amples are treated as random samples in various analytical and inferential procedures.
The term random is also used by the general population and the media that serve this population. In this context the term does not seem to have any clearly defined meaning.
Q.6.
As a general matter, in cases where it is possible to collect data about each item in a given population, is there any reason to sample the population instead?
()
A.6.
There are various situations when it may be preferable to examine a sample of items from a population rather than the entire population.
This preference for sampling may come about for three distinct reasons.
First, time constraints may make it impossible to undertake a complete population examination but it may
)
be possible to undertake and complete a sample study within the allotted time.
Second, it may be difficult to justify the cost of a complete population examination but it may be cost effective to examine a sample.
Third, and sometimes most importantly, the j
use of a sample may produce study results that are more precise than the results that would be produced O.
by a complete population enumeration or study.
This g
i
may occur because the imprecision and uncertainty that is introduced because a sample is used may be less than the imprecision and uncertainty associated with undertaking a large scale study. For example, data collectors may not do as good a job counting a large number of items as they would if they had fewer items to count.
In other words, on a per item or case basis, the errors associated with undertaking a large scale study may be significantly greater than those that occur in a smaller study.
This difference in imprecision may more than offset the errors that occur because only a sample if examined.
A somewhat extreme example of the superiority of a sample over a complete population enumeration or study is provided by the decennial US census.
Given the number of temporary employees that must be hired and trained in order to conduct the decennial population census, it is generally recognized that a more accurate projection of the number of inhabitants in the entire United States, as a whole, would be 1
obtained if the count were carried out on a sample basis.
In fact, it is a sample study, known as the PES (Post Enumeration Survey), that is used in order to evaluate the degree to which a census over or under count may have occurred.
l
-s-
Q.7.
Can you describe the role of probability samples in k
drawing inferences from a sample to a larger population?
A.7.
The use of probability sampling methods generally assure that objective statistical inferences may be drawn about the large population from which the sample was selected.
More specifically, support for one of the assumptions that must be made in order to apply various theories of mathematical statistics may be directly' linked to the sample selection process.
In less technical terms, the use of probability sampling methods maker questions about sample adequacy amenable to answer via objective statistical methods.
(
The methods and techniques of probability sampling were first introduced in the late 1930s and early 1940s.
Since that time it has been recognized that probability sampling generally simplifies the drawing of inferences to the larger population.
This simpli-fication is directly linked to the built in "objecti-vity" of the methods used in the selection of probability samples.
f At the present time probability samples are used in order to produce a number of national level O{
statistical series that inform Federal Government
(.
policy decisiens.
Examples of these statistics include the size of the labor force, the unemployment rate, the size of the money supply and various measures of health status.
Q.8.
Can you describe the role of non-probability samples in drawing inferences from the sample to a larger population?
A.8.
While it is true that probability sampling is generally preferred to non-probability sampling, this does not preclude the use of non-probability samples in making inferences about larger populations.
There are many instances which involve public welfare and
()
safety in which policy decisions are made on the basis 1
i of non-probability samples.
i Examples of the use of non-probability samples in this context include the approval of drugs for general distribution and testing of products for the sStis-faction of safety standards.
Non-probability samples are somewhat more difficult to use than probability samples, for the purpose of making inferences to a large population.
This is because the adequacy of a non-probability sample is not assured by the sample selection method, but C) instead, must be evaluated by an individual or
{
(
i l
individuals who have substantive knowledge and g
experience related to the quantities under study.
Q.9.
Can you please define the terms reliability and con-fidence level?
A.9.
Both of these terms are used in a number of different contexts.
In the present context the term reliability is used to denote the complement of the error or failure percentage rate.
More specifically the re-liability percentage is 100% minus the failure per-centage.
For example if the manufacturer of a certain product guaranteed that parts would pass certain tests
()
with a reliability rate of 99% this means that the manufacturer was guaranteeing-that not more than 1% of the parts would not pass these tests.
The term confidence level is used in its standard statistical context.
Specifically the term con-fidence level is a measure of the probability associated with the truth of a statistical statement.
If a statement has a 95% confidence level this means that the probability that the statement is true is 95%.
l Q.lO.
What kind of sampling program is the CSR?
A.10.
The CSR made use of both probability samples and non-
)
probability samples in various construction category c
.g.
()
populations.
The nature of these populations is des-cribed in Dr. Kaushal's testimony.
For each construction category population a probability sample was selected via the use of random selection without replacement.
Under this type of sampling plan, each element in a population is given the same chance of selection as every other element in the population.
Furthermore, the probabilities of selection among various elements are statistically independent.
As initially designed, the CSR sample sizes were i
specified in sequential form, which allowed for possible sample expansion.
In this context the feasible sample sizes were chosen so as to assure a minimum statement of 95% reliability with 95%
confidence.
The non-probability evomponent of the CSR involved the selection of additional population elements based on general engineering judgment.
Criteria for selection within this context are discussed in the testimony of Dr. Kaushal.
O t - _ _ - _ _
()
Q.ll.
Is the use of both a probability and non-probability
\\
sample an unusual practice?
A.ll.
No.
This " dual" approach is often used in quality assurance and auditing examinations in order to provide objective and subjective information.
Each type of sample serves as a complement to the other.
Q.12.
What inferential statements about the quality of the electrical work at Braidwood can be made based on the results of the CSR?
A.12.
Given that no design significant discrepancies were found in any of the CSR construction categories, objective statistical statements concerning the
(}
absence of design significant discrepancies may be t
made at the 95% level of reliability with 95%
confidence for each of the electrical construction categories for the period covered by the CSR program.
With respect to objective statistical inferences, it should be noted that for certain populations the sample sizes exceeded levels required for objective statements of 95% reliability at 95% confidence.
For these populations the levels of reliability and confidence are correspondingly higher.
In addition, higher levels of reliability and confidence will result when the results are combined across all
()
electrical populations.
(. -. _ _ _ _ _ _ _
O'
(
As a statistician I am not qualified to discuss the subjective inferences that may be made from the'non-probability portion of tne sample.
Q.13.
Dr. Frankel, for purposes of the following questions, I define tne term " agreement rate" as the number of inspection points determined by CSR inspections to be acceptable (i.e., non-discrepant) for any hardware item, divided by the total number of inspection points reinspectea for tne same item.
Is it possible to use objective statistical methods to evaluate wnetner tne error rates or agreement rates for L. K. Comstock QC inspections differed when we compare the period prior to July 1, 1982 and the period on or after July 1, 1982?
A.13 Yes, I nave performed such an evaluation.
O f
Knat were tne conclusions of tnis evaluation?
Q.14.
A.14.
My evaluation snoweo tnat tnere was not a statisti-cally significant difference between the agreement rate for L. K. Comstocn QC inspections prior to July 1, 1982 ano the agreement rate for L. K. Comstock inspections on or after July 1, 1982.
Q.15.
How did you go about doing that evaluation?
A.15.
As is described in Dr. Kausnal's testimony, random sampling was one of the methods used to select sampics of items in eacn of six electrical populations covered by the CSR.
i
O(
rj For eacn item in an electrical population tnat was selected tnrough random sampling, I was provided by Edison witn counts of tne number of inspection points and the number of discrepancy points, and the number of agreement points (inspection points-discrepancy Points = agreements points).
Tnese counts, which were provided on an item by item basis, were combineo for all six construction categories and divided into two parts: counts associatec witn Comstocn anspections r
that were performed prior to July 1, 1982 and counts associateo witn Constocn inspections tnat were performed on or after July 1, 1982.
()
{
Since tnis cata was basec on a probability sample, it was possicle, using ob]ective statistical methods, to ca'rry out a projection of tnese agreement rates to tne population of all Consteen inspections covered by tne CSA defined electrical populations.
Furtnermore, it was possible to carry out this projection separately for tne time perico prior to July 1, 1982 and tne time,
perioo on or after July 1, 1982.
O.16.
Knat did tnese projections snow?
A.16.
These projections showed that the overall agreement rate for Comsteen QC inspection prior to June 30, 1964 OI was Juker7Fif The agreement rates for the periods
(
--.m.
()
('4" prior to July 1,1982 and on or after July 1,1982
,L were 98.893% and 98.925% respectively.
35 0 %
In order to obtain$$.i o
an obj ctive confirmation that this apparently small difference in sample projections was indeed insignificant and trivial, I conducted a statistical hypothesis test.
Q.17.
What is a statistical hypothesis test?
A.17.
A statistical hypothesis test is a paradigm for inference which involves a formal set of steps.
These steps ultimately result in a test conclusion of
()
" acceptance" or " rejection" of a stated hypcthesis or
(
assertion.
Q.18.
Please describe how you carried out the statistical hypothesis test.
A.18.
In the present context the statistical test was carried out in four steps:
STEP 1, Statement of assumptions and hypotheses; STEF 2.
Specification of significance level and formulation of test statistic and critical region; STEP 3.
Computation of the test statistic; and STEP 4.
Statement of results and conclusions.
O,
-t '.
d._ _
Revi
O
(
Q.19.
Please describe the first step of your statistical hypothesis test.
A.19 In the first step of a statistical test, the population and sample are defined and two mutually exclusive and exhaustive hypothesis are formulated.
In the present situation, there are two populations:
~
Comstock inspections of work covered by the CSR that occurred prior to July 1, 1982 and Comstock inspec-tions of work covered by the CSR that occurred on or after July 1, 1982.
()
The sample of these two populations consisted of a stratified random sample of elements from 6 strata (the six construction catagories) with total sample size equal to 467.
This represents the total number of hardware items in the six construction categories which were selected using random sampling methods and for which at least one inspection point was included in the data that I used in my projections.
The two hypotheses are as follows:
NULL HYPOTHESIS:
The agreement rate for pre-July 1, 1982 Comstock inspections covered by CSR is the same j
as the agreement rate for post-July 1, 1982 Comstock O) inspections covered by CSR.
(_
a e
()
ALTERNATIVE HYPOTHESIS:
The agreement rate for
(
pre-July 1, 1982 Comstock inspections covered by CSR is different from the agreement rate for post-July 1, 1982 Comstock inspections covered by CSR.
Q.20.
Please describe Step 2 of your statistical hypothesis
- test, i.e.,
your specification of significance level and formulation of test statistic and critical region.
A.20.
The next step in conducing a statistical hypothesis test is the specification of the " level" of significance or alpha level.
Level of significance specifies the probability that the statistical test will REJECT the NULL hypothesis if the NULL hypothesis is indeed TRUE.
In general the lower the level of significance, the more conservative is the test.
The most commonly used value for an alpha level in statistical hypothe. sis tests is 5%, but depending upon the application, alpha levels sometimes range between 20% and 0.1%.
As an initial starting point the test was carried out using a 5% alpha level.
As shall be seen later, the test results for this CSR data are invariant over a broad range of alpha levels.
The test statistic is the numerical quantity that will be derived from the data obtained from the two samples
[}
(Comstock inspections pre and post July 1, 1982).
f i
(
1 __-
O
{
The test statistic (TS) for the hypotheses stated in step 1 consists of the difference between the pre and post July 1, 1982 agreement rates divided by the standard error of this difference.
AGREEMENT RATE PRE 7/1/82 - AGREEMENT RATE POST 7/1/82 TS=
STANDARD ERROR OF DIFFERENCE The formula for the standard error of the difference may be found in most statistical sampling texts.
It is shown in Attachment 2C (Frankel-1).
4 The critical region for the test is defined as the values of the test statistic (TS) which will result in i
the rejection of the NULL hypothesis.
If the test statistic does not fall within the critical region the NULL hypothesis will be ACCEPTED.
The formula for the critical region can also be found in most statistical sampling texts.
Note that the critical region is a function of the the alpha level selected.
For the test statistic in this context, the critical region is 1.96 or greater or -1.96 or less.
In other words if the test statistic is greater than or equal to 1.96 or less than or equal to -1.96 the NULL hypothesis of no difference will be REJECTED.
If the test statistic falls inside the range -1.96 to s
f - -.
F e
/
+1.96, the test statistic can not be REJECTED.
It is therefore ACCEPTED.
9 21.
,Please describe Step 3 in your statistical hypothesis test.
A.21.
The third step in conduct of a statistical test Anvolves the cc.putation of the test statistic based on the actual data.
In the present situation Jhe two error rates are 8 (U E ,[o l. 70 E 7. - T D 8 4 0 M
g and _" 02".0;graspectivel The standard error ofthedifferenceisC.24%4;;.%y.
0296*AD7e A
A *-JA i;'
, hus the test statistic TS is calculated as O(
(
AGREEMENT RATE PRE 7/1/E2 - AGREEMENT RATE POST 7/1/82 75=
S!;JOARD ERROR OF D2FFERENCE h M d, x.0 7= - W. 7 3 w yg,ip 80 1
i
.n M r;T A i n i i O e li (p 2.0 W f n^
-onf Q.22.
Please explain the results of the statistical hypothesis test.
j A.22.
The final step in a statistical test involves a statement of the test conclusions based upon the test statistic and the critical region.
In the present
-of
.-a situation, the test statistic (TS = WMh) fal;,s well inside the range -1.96 to 1.96.
Thus, the O(
comc1usion of the test is acct,TxscE of the uutt
- i Revised Frankel Testimony
O,, i h'ypothe si s.
In other words, the test does NOT find a statistically significant difference between the agreement rates before and after July 1, 1982.
It should be noted that had the alpha level been different, the conclusion would have been the same.
For example if the test had been carried out at the 20% alpha level, the critical region would range from 0M
-0 d"l%
-1.28 to +1.28.
The test statistic remains -0 : ;;s, which falls well within this region..Had the test been carried out at a 0.1% alpha level, the critical region would range from -3.29 to +3.29.
Again, the test statistic falls well within this region, c
Q 23.
Are there ar.y add:::enal features of stat:stical tests that should be noted?
A.23.
Yes.
In addition to considering the ' level of signi-ficance associated with a test, it is also sour..i' practice to examine a quantity known as the r
statistical " power of the test" or simply the power.
Statistical power refers to the probability that the i
test process will REJECT the NULL hypothesis when it is, in fact, FALSE.
Examination of power is somewhat more complex since there are a number of different ways that tht NULL C:)'
s k,
19-Revised Frankel Testimony
O hypothesis might be false.
For example in the present I.
context, the hypothesized difference in agreement rates before and after July 1, 1982 might be one percent (1.0%, absolute), or nine tenths of one percent (0.9%, absolute) or some other value.
Since the possible ways in which the NULL hypothesis might be false can vary, the statistical power of the test is determined for various alternative scenarios i
i under which the NULL hypothesis might be false.
I For the test described above, the power is in excess
(
of 99.0% against a situation in which the agreement rates differ by 1% (absolute) or more.
This power level in excess of 99% holds for differences as small
/
as (absolute).
The power level is in excess of O *We V
95% for differences as small as Q,6% (absolute).
l Q.24' Dr. Frankel, what would be the conclusion of a statistical test if you were to compare the agreement rates for the period prior to August 1,1983 and on or after August 1, 1983?
A.24.
Application of the same test procedure shows the same conclusion of no statistically significant difference at any of the levels which are typically used for significance testing.
This includes the 0.1% alpha
( )(
l :
. O
\\
level, tne 1% alpna level, tne 5% alpha level, the 10%
alpha level, and the 20% alpha level.
Q.25.
Dr. Frankel, could you please describe in more detail now you combined items in different construction categories for purposes of tne analyses cescribec in your previous answers?
A.25.
On tne casis or oiscussions witn Mr. Del George and Dr. Kausnal it was determined that the parameter of choice for tne assessment of tne effectiveness of issW /Wf Comstock QC inspections was tne $42m specific all reinspections witnin the scope agreement rate over of the CSR attributable to Comstock inspections.
Since tne CSh incluceo ranoom samples from eacn of the W
six construction categories covered by Comstock inspections, it was possible to procuce an ob3ective statistical estimate of this population parameter.
In tne terminology of survey sampling, tne six rancom samples from eacn of tne six construction categories may De vieweo as a single, stratifiec, probability sample from tne entire combined population.
Each of
~
tne CSR construction categories or populations is referred to as a stratum in the terminology of survey sampling tneory.
h(
t L
I I
O t
Tne fact tnat tne sample qualifies as a probability sample follows immediately from the fact that a randam sample of construction items were selected, witn known probabilities, from each of the six electrical construction categories in tne population.
Tne prob-tn ability of selection for an item in tne n
CSR population or stratum (h takes on values from 1 to 6) w ere N ano n are the sizes is i =n /Nh, n
h n
g of tne population ano tne random sample respectively.
Since this is a probability sample, it is possible to produce unbiaseo estimates of population totals from tne sum of eacn sampic value weighted by the inverse of its probaollity of selection.
Tnis proceoure was followed for producing the component parts of the sample estimates cescribec below.
For eacn rancomly selected item in the CSR sample, Edison provided me witn the counts of tne number of inspection points and the number of agreement points (inspection points -
oiscrepancy points = agreement points).
Tnese counts were provided separately for Comstock inspections prior to July 1, 1982 and for Comstock inspections on or after July 1, 1982.
Using the inverse probability of selection for each item, I appliec weignts to tnese counts in order to obtain projections of the total number of inspection points ano the total number of I
'C )
(
agreement points for the entire population of Comstock QC inspections from which the CSR samples were drawn.
The agreement rates described in my previous answer were the percent of projected CSR agreement points relative to the projected total number of inspection points for the entire population.
These estimates, which are termed ratio estimates in survey sampling theory, are described mathematically in Attachment 2C (Frankel-1).
Q.26.
What did you do for any items for which no Comstock QC
(
inspector could be identified, or more than one Comstock QC inspector was identified?
A.26.
Since the purpose of this sample projection was the estimation of an inspector level agreement rate and not a present hardware observation rate, only inspection points that could be definitively linked to Comstock inspectors and times were included in the estimate.
If the inspector could not be identified, then the counts were not included.
If more than one Comstock inspector and/or more than one date was associated with inspections for a particular item, the counts for the corresponding inspection points and agreement points were included in the appropriate time period or periods (even if this resulted in O.
double-counting).
(
s
=
(
Q.27.
Dr. Frannel, dio you evaluate wnetner the L.K. Comstocn agreement rate in visual welo inspections differed over time, based on tne CSR data?
A.27.
Yes.
Tne agreement rates in visual weld inspections followeo tne same pattern snown by tne overall agreement rates.
ine agreement rate on an inspection point basis for visual welds was nigner in tne secono time period examined (in one instance post July 1, 1982 ano in tne secono post August 1, 1983).
Tnis difference was not statistically significant.
0.28.
Dr. Frannel, dio you evaluate wnetner the error rate differed over time for eacn of tne six construction categories?
A.26.
I attempted tnis evaluation but was unable to ao so.
0 29.
Kny not?
A.29.
Wnen a percentage estimate is based on a ratio of two separate sample estimates, tne percentage is, in tne terminology of statistical survey sampling, a ratio type.
knen ratio type estimates are useo, certain conditions must be satisfied.
One of these conditions l
involves tne coefficient of variation of tne denominator of tne ratio.
More specifically, tne coefficient of variation of tne cenominator snoulo be less tnan 0.20 (20%) 1/
Examination of the lCE) 1/
Tnis conoition may be founo in stanoard texts on probabil-I ity sampling.
See, for example Kish, L.,
Survey Sampling.
~
New Yorn: Jonn Wiley anc Sons, 1965 pp. 207-209.
(
appropriate coefficients of variation for tne separate construction categories revealed tnat the sample sizes witnin most categories woulo not support separate analyses of tne type carried out for tne entire sample.
It snoulo be noted, nowever, tnat in tne several instances wnere separate analyses were possible, the test conclusions were the same as tnose for tne total population.
More specifically, for tnose construction categories ano time period divisions where testing was possible, no statistically significant differences in agreement rates were founo between tne two time s
i perioos.
Q.30.
Dr. Frankel, are you familiar with tne results of tne PTL overinspections of welos for tne perioo July,1982 to June, 1966, as described in Mr. Marcus' testimony?
A.30.
Yes.
Q.31.
Ascume tnat per we16 agreement rate between the PTL overinspections and tne Comstocn QC inspections is a measure of tne effectiveness of Comstock QC inspections.
Knat, it anytning, can be saio about the effectiveness of Comstoex QC inspections over time, baseo on these PTL overinspection results?
A.31.
Since tne PTL sampling process was not a probability process, inferences from tnese sample results to tne entire population must be based on subjective expertise.
However, it we consider tne sample as an entire population, it is possiole to examine tnis -_
'O i
(
population for a linear tren6 over time.
Attacnment 2C (Frankel-2) snows tne agreement rates (without override) for tne PTL overinspections of the L. K.
ComstoCK visual weld inspections for the period July, 1982 througn June, 1986.
It snould be notea tnat no Overinspections were reported for the montns of October and November, 19'82.
Tnese montns are eliminateo from tne computations tnat follow in order to prevent oistortions.
It is clear fron. tnis grapn tnat tnere are variations in agreement rates over time.
Furtner, tnere does not appear tc be a strong trend over time, but tnere is some inoicatior. of increasing agreement rates.
In order to provide a more ob]ective quantification of a possiole trenc over time, it is possible to apply a technique Known as linear regression analysis.
Basically, in linear regression analysis, a straight line is matnematically fit to tne actual data points using a criterion Known as "least Squares."
For the data snown in Attacnment 2C (Frannel-2) tne
()
least squares line is Y = 90.247 +.117295X, wnere Y =
t agreement rate and X = time (1, 2, 3, etc.)
(
This means that if we mathematically fit a straight line to this data we find a slight positive trend in the agreement rate over time.
(i.e., the inspections got slightly better over time).
It should be emphasized that this rate is very slight.
In conjunction with this best fitting straight line a quantity known as coefficient of determination or R squared was also calculated.
The coefficient of determination tells us the extent to which the fitted straight line can, in fact, explain variations in the data over time.
R squared will fall somewhere between 0.0 and 1.0.
A value of 1.0 would indicate a pefect explanation of the variation in the agreement rate on the basis of time.
A value of 0.0 would indicate the total lack of a relationship between the agreement rate and time.
For this data set, the value of R squared was 0.0702.
This is quite small and indicates only a slight linear relationship between time and the agreement rate.
Q.32.
Does this conclude your testimony?
A.32.
Yes.
O
(
ATTACHMENT 2C (Frankel-1)
The population consists of H-6 strata.
Within each stratum a random sample of n
items was selected using simple random (without replacementfsampling from the population of N items.
Tggs the uniform probability of selection for items wikhin the f ""h "h'
/
h stratum is h
the number of inspection points minus the number Let y hi of discrepancy points found durinEh*
time perggd associated with the i sample item in the h stratum. This quantity is ggefined as the o
agreement p ints for the i sample item h[h number in the stratum.
the number of inspection points dyging the first Let x hi time perggd associated with the i sample item in the h stratum.
-x d
i denote the agreement and inspection point counts Let yhi x [r s t an I
period weighted by the inverse of their during the probability of selection into the sample.
- Thus,
/
yhi "I hi
- h "h "h "h hi ~
- hi
- X The proj e c te d totals y and x for the above variables are defined as the sample sums of the weighted values yhi hi" and x H
nh y-I I
yhi h-1 1-1 H
nh x-r z
xhi h-1 1-1 I
l l
a o%.
\\
The agreement rate during the first period' is ratio
- estimate, r,
defined as H
nh hi h-1 1-1 r
H nh hi h-1 1-1 In a corresponding fashion, define x',
y' and r'
as the projected total agreement points, inspection points and agreement rate for the second time period, c
The statistic of interest is the difference between agreement rates between time periods r r'.
The estimated standard error of the difference r -
r' is given by f
[ var (r-r') ]I/
se (r-r')
where 1
H 1
H VAR (r-r')
z d z
+
z d
z' h
x'2 h-1
~~~
x h-1 O'
2 H
t r
dz dz' h
N.
xx' h-1 M:
k 3
The term d z is defined as h
fh)
"h 2
2 (1
~
2
- hi
- h I'
h l "h dz
~ ''''
~
1) i"1 (n
~
h where Yhi ' ' *hi' Z
~
{
hi "h
and z
h
- hi
~
i-1 are defined as above using z'g in place of z'g.
The ' terms d z'3 are defined as The terms dzh*h f)
"h (1
h h
h
- M
- M
- h**h I
[ n dz dz'3
~
1) bl (n
h
(
(
These formulas may be found in Chapter 6 of survev samoline by
I l'
Leslie Kish (John Wiley and Sons. 1965).
The coefficient of variation for x is computed as
..x I
"Q H
nh 2
2 1/2 (x
/nh) I 3
[
z
[
r x
h-1 1-1 cv(x)
H I
X h h-1 where c
"h
- h
- hi
~
3..
1-1
-s.
d%
Y.' L,.
'p.
E.*
I,**;.
6.
3 e
?.
h-se
,A*
k*
y 4.
+. t:,.
efs-g.
(,.*
~
Revised Fra.nh
.o o
mH N
^., : : 5 -
3 5
8 C
,L K
, W L
F O
E M
NI T p
, 4 EV
, 8 R
I TO
,L AE CE
,W EA Y-T H
R T
P N
T O
SN M
E NM EE-I RR G
EA V
,,38 O
,L,W L
T P
, 2 i8 0
L M
0 0
0 0
0 0
0 0
0 0
O 7
0 9
8 7
6 5
4 3
2 1
1 wke Zd $th4 d
i ll l
i..
1!i I
17083 bO 1
MR. MILLER:
Your Honor, during the course of 2
Mr. DelGeorge's examination, there were a number of 3
questions posed to him about what Dr. Frankel did or did 4
not take account of.
5 In addition, there was at least one exhibit 6
introduced on cross examination of Mr. De1 George.
7 I don't know whether Mr. Guild is going to pursue 8
those subjects with Dr. Frankel.
9 If not, at the conclusion of the cross examination, 10 I would ask leave to conduct a very brief supplemental 11 direct examination of Dr. Frankel on those subjects.
Ih 12 MR. GUILD:
Mr. Chairman, I have no 13 objection.
14 I'd ask that Mr. Miller do that at this time before 15 my examination.
16 JUDGE GROSSMAN:
Yes, I think that would be 17 appropriate.
18 If you're prepared to do that, Mr. Miller, 19 certainly you ought to.
20 DIRECT EXAMINATION 21 (Continued) 22 BY MR. MILLER:
23 0
Dr. Frankel, do you have before you a document that has 24 been received into evidence as Intervenors' Exhibit No.
O 25 188?
l t
i l
v l
Sonntag Reporting Service, Ltd.
I Geneva, Illinois 60134 j
(312) 232-0262
17084 O
1 (Indicating.)
2 It consists of two tables and a page that's 3
entitled " Sample Calculation."
4 A
(Indicating.)
5 0
That's the one.
6 Just by way of background to this examination, Dr.
7 Frankel, will you describe briefly what use, if any, you 8
made of agreement rates in your statistical evaluation 9
of this data.
10 When I say "this data," I'm not referring to the 11 exhibit.
I'm referring to the subject of the inquiry in 12 general.
13 A
In analyzing the BCAP data, I tested whether or not 14 before and after agreement rates differed sufficiently 15 to be judged to be statistically significantly 16 different.
17 0
When you say "before and after," sir, before and after 18 what?
19 A
Well, there were two dates that were used as cutting 20 points, if you will, and the first date was July 1, 21 1982.
22 I looked -- and I tested whether or not the 23 agreement rate on reinspections was -- the agreement 24 rate calculated before 7/1/82 and the agreement rate i
25 af ter 7/1/82 were statistically significantly different.
Sonntag Reporting Service, Ltd.
Geneva, Illinois 60134 (312) 232-0262
17085 A
t
'%J 1
I also tested whether or not the agreement rate 2
bef ore August 1,1983, and af ter August 1,1983, were 3
different statistically, and I found that they were not, 4
either.
5 0
And what difference would have been significant 6
statistically, as a result of your evaluation, between 7
the agreement rates for those two periods?
8 A
As I mentioned -- as I said in my prefiled testimony, 9
had the difference been as small as six tenths of a 10 percentage point, if one were doing a 95 percent test or 11 so-called 5 percent test level of significance, one I
12 would have found that difference to be statistically N.~.))
13 significant and statistically different.
14 0
All right.
15 Now let's go to Exhibit 188.
16 There are numbers calculated in accordance with the 17 sample calculation that's found on Page 3 of that 18 exhibit, and the numbers that are calculated -- well, 19 the numbers that are calculated, which are shown in l
l 20 pencil, are, in fact, agreement rates between a l
l 21 first-line inspection and an overinspection.
You'll l
22 have to accept my representation as to that being the l
23 calculation that's made.
l l
24 Assuming that these are agreement rates that are
)
25 shown in pencil on the first page of this document,
(
l l
l Sonntag Reporting Service, Ltd.
l Geneva, Illinois 60134 I
(312) 232-0262 I
17086 O
1 where would the agreement rates that you calculated for 2
your before and after periods fall?
3 A
On both Table 1 and Table 2, they fall in the upper 4
lef t-hand corner of the table.
They were in the 98, 99 5
percent range.
6 In the left-hand margin, I see something labeled QC 7
Inspector accuracy rate of 80 percent or 90 percent; and 8
at the top of the table, I see craft error rates in the 9
neighborhood of 10 and 20 percent in association with 10 the kinds of agreement rates that I was working with.
11 0
Can you specifically state for the record which sets of (a) 12 agreement rates that are shown on Intervenors' Exhibit 13 188 correspond to or are in the range of the agreement 14 rate that you calculated?
15 A
Well, the agreement rates that I calculated were 16 approximately 98.8, 98.9, 98.7.
17 Do you want the numbers in this Table 1 that 18 correspond most closely to that?
19 0
What coordinates on the chart, in terms of craft error 20 rate and QC Inspector accuracy rate, would produce an 21 agreement rate comparable to the ones that you --
22 A
Craft error rate in the range below -- craft error rate 23 below 20 percent and QC Inspector accuracy rate above 80 24 percent.
25 0
Thank you.
v Sonntag Reporting Service, Ltd.
Geneva, Illinois 60134 (312) 232-0262
i I
17087
.p
' d 1
Now, assume, Dr. Frankel, that there is a I
2 10-percent decline in the Quality Control Inspector 3
accuracy rate from the rate, whatever it was, that led 4
to the results that you achieved in your statistical 5
analysis.
6 Are you with me so far?
7 A
Okay.
8 Q
And a craft error rate -- well, I'm going to have to 9
start over.
10 JUDGE GROSSMAN:
That's all right.
Take your 11 time, Mr. Miller.
12 BY MR. MILLER:
13 0
Let me start with a somewhat different kind of thing.
14 Let's suppose that there was a 10-percent change in 15 the Quality Control Inspector accuracy rate --
16 A
Okay.
17 Q
-- and that that change took place at a craft error rate 18 of 10 percent.
19 A
Yes.
20 0
What change in the -- well, would that change, a change i
i 21 of 10 percent, result in a statistically significant 22 difference on the basis of the calculations that you 23 made in your direct testimony?
2 24 A
In Table 1 --
i 25 MR. GUILD:
Objection, Mr. Chairman.
Sonntag Reporting Service, Ltd.
Geneva, Illinois 60134 (312) 232-0262 j
-e-y--m'"-
eam-- - -
4 17088
(-~5
\\- /
l 1
Point of clarification:
2 Are we talking about an individual inspector 3
accuracy rate?
Is that the premise for Mr. Miller's 4
question -- his question is vague -- or are we talking 5
about an aggregation of all inspectors' work for the 6
period of time that Dr. Frankel studied?
Is that the 7
premise for his question?
8 MR. MILLER:
Well, I'm taking these 9
statistics as simply standing for numbers.
10 I'm asking Dr. Frankel to evaluate changes in these i
11 numbers on the basis of the work that he did, which i
- (
12 aggregated the agreement rates for all the inspectors 13 that were captured in the CSR data base.
14 MR. GUILD:
So the understanding is the 15 question assumes an aggregated change of 10 percent 16 inaccuracy rate?
)
17 JUDGE GROSSMAN:
That's what we understand.
18 Fine.
19 Dr. Frankel?
l 20 A
As I read this table -- and I'm looking at Table 1 -- a 21 10-percent decline in Quality Control Inspector accuracy 22 rate would have been accompanied by approximately a i
23 1-percent decline in the agreement rate had the craft 24 error rate remained at 10 percent.
25 Had such a change occurred, the tests that I did i
i Sonntag Reporting Service, Ltd.
Geneva, Illinois 60134 (312) 232-0262
17089 1
would have picked that up because the tests that I did 2
would have picked up a change as small as six tenths of 3
1 percent and called that significant.
4 BY MR. MILLER:
5 0
Your conclusion that a change is statistically 6
significant has what consequence in the analysis that 7
you did?
8 What would that tell the Board and the parties if 9
there, in fact, had been a statistically significant 10 change?
11 A
Well, a statistically significant change means that 12 there is an indication that, in fact, if one examined 13 the whole population rather than the sample, one would 14 have found a difference.
15 For example, we have in the sample that I looked at 16 actually an increase of the agreement rate in the first 17 period from -- or approximately two tenths of one 18 percent.
The agreement rate actually goes up over time.
19 The statistical test, however, says that you can't 20 take that difference of two tenths of 1 percent as 21 indicating that had you examined the whole population, 22 you really would have found an increase in the whole i
23 population in terms of the agreement rate getting 24 better.
25 Had the difference in the sample been as large as Sonntag Reporting Service, Ltd.
Geneva, Illinois 60134 (312) 232-0262
17090 v
1 six tenths of 1 percent, that would have meant that we 2
were 95 percent; that had we taken the whole population, 3
in fact, we would have seen a higher agreement rate in 4
the second period.
5 But the fact that it's not significant, 6
statistically significant, tells us that we should not 7
read that two tenths of 1 percent as indicating that 8
there really is an increase in the whole population.
9 0
And conversely, if the agreement rates were to decline 10 by more than six tenths of 1 percent, that would 11 indicate, with 95 percent confidence, that the entire I h 12 population's agreement rates declined?
13 A
That's correct.
14 Q
Now, let us hypothesize that in the overall agreement 15 rates that you calculated, that there was a 6-percent 16 decline in the Quality Control Inspector accuracy rate, 17 as that term is used in Intervenors' Exhibit 188.
18 How much would the craft error rate have to move to 19 mask the effect of that decline so that the rate 20 appeared to be constant?
21 A
I observed a two tenths of 1 percent increase in the 22 agreement rate.
23 Had, in fact, the QC Inspector accuracy rate gone 24 down by 6 percent, the craf t error rate f rom this table 25 would have had to have gone up correspondingly at least g
j Sonntag Reporting Service, Ltd.
Geneva, Illinois 60134 (312) 232-0262
=
17091 0
1 8 percent to have produced the result that I found.
2 MR. GUILD:
I'm sorry.
I just misunderstood 3
the midpart of your answer.
4 Could I perhaps have it clarified?
5 The question asked for a change in craft error 6
rate, and you inserted one other change in there, Dr.
7 Frankel.
8 THE WITNESS:
I observed in the sample a two 9
tenths of 1 percent increase in agreement rate.
The 10 inspector -- or the Quality Control Inspector accuracy 11 rate actually declined by 6 percentage points.
12 The result that I saw could have been produced if 13 the craft error rate had declined by 8 percent and had 14 gotten better by 8 percent.
15 It would have taken that big of a change in the 16 craft error rate to produce the result that I saw under i
17 the assumption that the Quality Control Inspector i
18 accuracy rate went down by 6 percent.
19 BY MR. MILLER:
20 Q
Dr. Frankel, let's hypothesize that there's a 30-percent 21 decline in Quality Control Inspector accuracy rate.
22 How much of an increase in -- or improvement in the 23 craft error rate would have to take place to mask that i
24 sort of a decline in Quality Control Inspector accuracy 25 rate on this table?
t Sonntag Reporting Service, Ltd.
Geneva, Illinois 60134 (312) 232-0262
17092 m
1 A
I can' t tell from -- I can't tell from the values shown 2
in this table.
I'm not sure that the functions are 3
linear, so I don't want to interpolate, and I'd be 4
outside the range of this table.
5 It looks like it would certainly have to change by 6
close to a possible percentage.
We would have to get a 7
craft error rate close to zero, which I guess in that 8
case would give you a fine plant no matter what the 9
inspectors did.
10 MR. GUILD:
Objection, Mr. Chairman.
11 Dr. Frankel clearly otherwise disclaims any ability Ih 12 to opine about the quality of construction.
V 13 I take it that he hasn't become an engineer since 14 he prepared his prefiled testimony.
15 JUDGE GROSSMAN:
I'm sorry.
You're 16 withdrawing what you said?
17 TH E WITN ESS :
Yes, I withdraw what I said.
18 I'm sorry.
19 BY MR. MILLER:
20 0
Now, Dr. Frankel, you have before you a bar chart that 21 has been marked and received into evidence as 22 Intervenors' Exhibit 190.
23 (Indicating.)
24 If you don' t have that one, I pass it up to you.
25 (Indicating.)
Sonntag Reporting Service, Ltd.
Geneva, Illinois 60134 (312) 232-0262
17093 1
Dr. Frankel, what this exhibit purports to show is 2
the distribution over time of various subpopulations of 3
the CSR data.
As the legend at the bottom of the chart 4
describes, the data is broken down by weld inspections 5
and non-weld inspections and by the population of 6
inspectors totally and by the subpopulation of the 24 7
inspectors who went to the NRC with their complaints on 8
March 29, 1985.
9 Now, the first question on this chart -- and again 10 it relates to your statistical evaluation of the CSR 11 data -- was:
Did you control for the different
(
12 distribution across time of the relative proportion of 13 the weld to the non-welding inspections that were 14 captured by the CSR data?
15 A
I'll answer that question using the word " control" in 16 the sense that I understand it as a statistician, as we 17 use that word.
18 0
Perhaps you'd better define that for us before we go any 19 further.
20 A
We refer to the process of controlling -- we normally 21 say we're controlling for certain kinds of variation, in 22 recognition of the fact that certain differences that 23 appear when one looks at totally -- at overall numbers 24 can, in fact, be the result of underlying changes for --
25 that are not directly accountable for when you look at Sonntag Reporting Service, Ltd.
Geneva, Illinois 60134 (312) 232-0262
l 17094 O
1 the aggregated data.
2 In particular in this particular instance, we have 3
welds and non-welding inspections.
4 When one compares pre and post periods, when one 5
cuts the data into two groups, either in 1982 or 1983, 6
the mixture of welding and non-welding work before and 7
after is different.
8 Overall in the random portion of the data base, 9
approximately 83 percent of the inspection points are 10 welding inspection points and 16-1/2 or 17 percent are 11 non-welding.
(V) 12 In doing the tests that I did, I first tested the 13 overall agreement rates, which take the welds and the 14 non-welds in the proportions that they occur.
15 In fact, in the pre period, be it pre '82, the 16 middle of the year in '82, or pre '83, there were welds 17 and non-welds in both groups.
18 So I tested overall the way the agreement rates 19 ran.
20 Then I separately tested and reported in my 21 prefiled testimony effectively controlling for this 22 difference.
I just tested welds separately.
I took out 23 the nonwelding.
24 I found that if we tested welds separately, we 25 found no statistically significant differences in the Sonntag Reporting Service, Ltd.
Geneva, Illinois 60134 (312) 232-0262
17095
)
v 1
first versus the latter period.
2 Subsequent to my prefiled testimony, I also tested 3
separately the other, the non-welding portion, and I 4
round that again there was no statistically significant 5
difference.
6 Then I put them both together, and I did something 7
called standardization.
8 Standardization involves taking separately an error 9
rate for welding and non-welding and weighting that so 10 that the impact of welding and the non-welding in the 11 first period is matched by the impact of welding and 12 non-welding in the second period.
13 I did indeed standardize and built a standardized 14 agreement rate, and I found again ne statistically 15 significant differences.
16 So in that sense I controlled as a statistician I
17 would control.
18 I might point out that I used only the random 19 sample portion, though.
I didn't use -- I believe this 20 chart includes all samples, the random and the 21 engineering judgment portions.
22 0
Dr. Frankel, would it have been possible to separately 23 evaluate, on a statistical basis, the CSR data that 24 related only to the 24 inspectors who brought their 25 complaints to the NRC?
l Sonntag Reporting Service, Ltd.
Geneva, Illinois 60134 (312) 232-0262 J
l 17096 O
N~-]
1 A
Actually, I tried to do that, and found that the sample 2
size was not sufficiently large.
3 I report in my prefiled testimony how that 4
determination is really not a subjective evaluation on 5
the part of a statistician.
6 It's looking at certain things called coefficients 7
of variation, and those coefficients of variation were 8
too large, they exceeded 20 percent, which indicated we 9
couldn't do a separate analysis and perform a 10 statistical significance test.
11 0
Dr. Frankel, you were involved, were you not, in the 12 sample selection process for the BCAP program?
13 A
Yes, sir.
14 Q
First of all, does the term " homogeneous" have any 15 meaning to you as a statistician?
16 A
The word " homogeneous" is used in statistical sampling, 17 but it's used in a context that has not been raised in 18 this proceeding, as far as I know.
19 0
Was the statistical concept of homogeneity used in the l
20 BCAP sample selection process?
l 21 A
No.
22 0
would you describe for us briefly your involvement in 23 that sample selection process?
24 A
My involvement in the sample selection process involved O
t J
25 working out the mathematical calculations required to s
{
1 1
Sonntag Reporting Service, Ltd.
l Geneva, Illinois 60134 l
(312) 232-0262
17097 p
V 1
find the sample size that would be able to support a 95 2
percent confidence of 95 percent reliability, assuming 3
no design-significant discrepancies were found.
I did the calculations of what sample sizes were requ' ired.
~
4 5
In addition, I was involved in-the actu,a1 use of 6
random numbers to select the sample and the sampling 7
plans.
8 0
Accept my representation that within any given 9
population CSR data base that is being considered in 10 this proceeding and that was a part of the BCAP program, 11 there are different kinds of items in the' sense that f.m) 12 some are more complex than others that are included V
13 within a single construction category; that is, in the 14 cable pan hanger population, for example, some, cable pan 15 hangers that were captured by the CSR are relatively 16 simple while others are relatively complex.
17 What effect, l'f any, does that variation in the 18 physical attributes, if you will, of the items that were 19 within the population to be sampled have, from a 20 statistical standpoint?
21 A
From a statistical standpoint, it doesn't really have 22 any impact.
If one takes a sufficiently large sample 23 and finds a certain result, one makes a statistical 24 statement about that particular population.
[
(
25 It's an overall statement about the whole Sonntag Reporting Service, Ltd.
Geneva, Illinois 60134 (312) 232-0262
17098
+,
1 population, recognizing that there may be diversity in 2
the population.
3 MR. MILLER:
That concludes my supplemental 4
direct examination of Dr. Frankel.
5 I thank the Board and the parties for the courtesy.
6 JUDGE GROSSMAN:
Mr. Guild?
7 MR. GUILD:
We may save us some time if we 8
can take about a 10-minute break here --
9 JUDGE GROSSMAN:
Certainly.
10 MR. GUILD:
-- and try to integrate this new 11 knowledge.
es
)
12 JUDGE GROSSMAN :
Okay.
That's fine.
13 (WHEREUPON, a recess was had, after which 14 the hearing was resumed as follows:)
15 JUDGE GROSSMAN:
Back on the record.
16 Mr. Guild?
17 MR. GUILD:
Thank you, Mr. Chairman.
4 18 Good morning, Dr. Frankel.
)
19 THE WITNESS:
Good morning.
l 20 CROSS EXAMINATION 21 BY MR. GUILD:
22 0
when were you first employed by commonwealth Edison 23 Company, Dr. Frankel?
24 A
I believe it was July or August of '84.
/O
( )
25 0
And for what were you first employed?
For what purpose l
l Sonntag Reporting Service, Ltd.
Geneva, Illinois 60134 (312) 232-0262
. 2
-(
17099 l
1 were you first employed by Commonwealth Edison Company?
2 A
Well, I should make it clear I'm not in the strict sense 3
employed by them.
I'm retained by them as a statistical 4
consultant.
5 0
You work for Commonwealth Edison Compnay and they pay 6
you a fee for your services?
7 A
Yes, sir.
8 0
In that capacity, when were you first employed by 9
Commonwealth Edison?
10 A
In '84.
11 JUDGE COLE:
For Braidwood?
12 BY MR. GUILD:
13 0
You didn't volunteer for them before that, I take it?
14 A
Pardon?
15 0
You didn't volunteer for them before that, I take it?
16 A
No, no.
17 0
For what purpose were you so employed in 1984?
18 A
Well, in 1984 I was asked to provide statistical 19 ex pe r ti se, and part of that included testimony in 20 coni. net ca with the Byron licensing hearings.
21 0
And you testified in the Byron hearing?
22 A
Yes, I did.
23 0
And thereafter did you continue in Edison's employ until 24 the present?
25 A
Yes.
Sonntag Reporting Service, Ltd.
s Geneva, Illinois 60134 (312) 232-0262
- =
17100 DU 1
Q And I take it that you continued to be remunerated for 2
your services to Edison from the time you were first 3
employed until the present?
4 A
Yes.
5 0
And what was the basis for your remuneration, sir?
6 A
An hourly basis.
7 0
What is your hourly rate?
8 A
$100 an hour.
9 Q
And how many hours have you been employed by 10 Commonwealth Edison Company, approximately?
11 A
Well, approximately it averages to about -- I believe in 12 the course of time it's been about three days a month.
13 So I guess 2-1/2 to 3 days a month, 8-hour days, so i
l 14 that would be 20 to 24 hours2.777778e-4 days <br />0.00667 hours <br />3.968254e-5 weeks <br />9.132e-6 months <br /> a month.
I 15 Q
And approximately what has been your total remuneration 16 for your services to Commonwealth Edison Company?
l 17 A
Well, if I multiply those figures --
i 18 Q
I'll not try.
19 A
Pardon?
20 0
I said I'll not try.
I'm glad you have a calculator.
21 A
Okay.
I 22 Roughly $50,000.
23 0
All right, sir.
24 And you anticipate perf orming services f or them 25 af ter you leave the stand today?
4 Sonntag Reporting Service, Ltd.
Geneva, Illinois 60134 (312) 232-0262
17101
/
I\\_.
1 A
I don't know.
2 0
You don't anticipate that?
3 A
Well, I don't know if they will be conducting 4
statistical programs at the plant.
5 I've been helping them with their statistical 6
programs at Braidwood.
7 Q
I see.
8 A
I'm not sure if any new ones are in the offing or if the 9
ones that I've been working on will come to an end.
10 0
I take it that your remuneration for preparing and 11 presenting testimony in this proceeding is the same as 12 for your other services; is that correct?
13 A
Yes.
14 0
Counsel for the company was kind enough to provide me 15 over the weekend some information that I sought with 16 respect to the data on which you relied in preparing 17 your testimony in this proceeding, a transmittal dated 18 November 10, 1986.
19 I take it the company did supply you the data on 20 which you base your testimony --
21 A
Yes.
22 0
-- is that correct?
23 A
Yes.
24 Q
Dr. Frankel, let me show you a series of two packages of
(
25 documents that appear to be computer print-outs.
The Sonntag Reporting Service, Ltd.
Geneva, Illinois 60134 f
(312) 232-0262
17102 3
1 first is entitled "CSR Data Base for Dr. Frankel, 2
Welding and Non-welding."
It's a document of 10 pages.
3 (Indicating.)
4 I ask you if you can identify that.
5 A
I'm used to seeing this on a computer screen because it 6
was provided in diskette form, but this looks like it.
7 Yes,-this certainly looks like it.
8 Q
All right, sir.
9 And a second document that appears to be Pages 11 10 through 20, entitled "CSR Data Base for Dr. Frankel, 11 Weld Only."
rm
)
12 (Indicating.)
13 Can you identify that, I take it, as a paper 14 print-out of data that was in a computerized form on 15 which you relied?
16 A
Yes, yes.
17 Q
Now, I notice that the data is displayed in the 18 print-outs made available to me on a time basis before, 19 during and after.
There are three columns c; data
(
20 points --
l 21 A
- Yes, i
22 0
-- in both sets of documents -- both documents, both 23 data print-outs.
24 I take it that the before, during and after, as O)
(
25 explained by counsel, refers to the period first before Sonntag Reporting Service, Ltd.
Geneva, Illinois 60134 (312) 232-0262
17103 IV 1
July 1, 1982?
2 A
Yes.
3 0
That's the same data date point, point in time, used in 1
4 your testimony?
5 A
Yes.
6 0
All right.
l 7
During referring to the period af ter July 1,1982, 8
but before August 1, 1983?
9 A
Yes.
10 0
And August 1,
'83, is again the second point in time 11 referred to in your testimony?
(
12 A
Yes.
13 0
And af ter, of course, is the data that f alls af ter 14 August 1,1983?
15 A
Yes.
16 0
There is no data in these documents indicating the 17 specific date for a particular data point except in the 18 aggregations before, during and after; correct?
19 A
That's correct.
l 20 0
And I take it that you did not rely on any disaggregated l
l 21 data that indicated the precise dates on which the data 22 was derived?
23 A
That's right.
24 MR. GUILD:
Mr. Chairman, I'd like to ask 25 that the two print-outs identified by Dr. Frankel be Sonntag Reporting Service, Ltd.
Geneva, Illinois 60134 (312) 232-0262
17104 U.
1 marked for identification only at this point as 2
Intervenors' Exhibit 192.
3 JUDGE GROSSMAN:
Both of them as 192?
4 MR. GUILD:
Yes.
5 (The documents were thereupon marked 6
Intervenors' Exhibit No.192 for 7
identification as of November 13, 1986.)
8 BY MR. GUILD:
9 0
It's also true, is it not, Dr. Frankel, that the data 10 that you relied upon was displayed on an inspection 11 point basis?
I '
12 A
Yes.
13 0
Not on a basis of an item basis or a weld basis?
14 A
Well --
15 0
Let me be more precise if I can.
16 A
Yes.
17 Q
Perhaps that last question was unclear.
18 For example, for welding, you have the welding 19 inspection points and the welding discrepancies points; 20 and that's the data bases that you relied on?
21 A
Well, the actual units that I have the reporting for are 22 the items.
23 What you see corresponding to each item --
24 0
Yes.
25 A
-- are the number of inspection points and then the Sonntag Reporting Service, Ltd.
Geneva, Illinois 60134 (312) 232-0262 g
17105 i\\/
1 number of discrepancies.
2 0
Of course, the items are identified separately in the 3
lef t-hand column by their CSR number --
4 A
That's right.
5 0
-- the sample number?
6 A
Yes.
7 0
But you don't know, for example, how many welds there 8
are in a particular item that contains one or more 9
10 You simply know an item and whether there were a 11 certain number of welding inspection points and a
(
12 certain number of welding discrepancies points --
13 A
That's right.
14 0
-- not the number of welds?
15 A
That's right.
16 0
And you didn't calculate on the basis of welds, as I've 17 just used the term?
18 A
That's right.
19 0
Now, Counsel's supplemental questions were somewhat 20 helpful in putting in context your testimony, Dr.
21 Frankel.
22 It's helpful to know that 83 percent of the random 23 data on which you relied consisted of welding inspection 24 points and 16.5 or 17 percent consisted of non-welding 25 inspection points.
Sonntag Reporting Service, Ltd.
Geneva, Illinois 60134 (312) 232-0262
17106
,)
1 Can you tell me for the periods before, during and 2
after what the respective proportions of welding and 3
non-welding inspection points were, sir?
4 A
Let me just consult this pad that I have here.
5 0
All right.
6 A
I don't think I have those separately.
I built into the 7
program making ase of that data, but I don't have 8
specific -- I don't think I have the numbers at this 9
moment.
10 0
Could that --
11 A
In fact, if you give me about two or three minutes, if
[V}
12 this is important, I can see if I can --
13 0
It may be, and I'd appreciate it if you'd take the time 14 to answer that question.
15 A
Subject to further checking, I have some percentages.
16 0
Yes, sir.
17 A
In the pre '82 period --
18 0
That's before?
19 A
The before.
20
-- my calculations that I've done now show -- now, 21 these are weighted projected counts.
This is af ter the 22 sample is weighted for the different population sizes.
l l
23 6.4 percent was non-welding in the bef ore period, j
24 and in the during-af ter combination, 2 8 percent.
25 0
Non-welding?
Sonntag Reporting Service, Ltd.
l Geneva, Illinois 60134 (312) 232-0262
17107
/~w l'
A Right.
2 0
Weighted average inspection points?
3 A
Right.
4 0
By " weighted," you mean weighted according to the 5
incidence of inspection points in each of the 6
subpopulations?
7 A
Right.
As I indicated in my prepared testimony, I L
8 projected to the population sizes.
9 0
All right, sir.
10 And I take it, then, that subtracting from 100 11 those values for non-welding inspection points would
(
12 give us the percentage of welding inspection points --
13 A
Yes.
I 14 0
-- for the period bef ore --
15 A
Yes.
16 0
-- and then during and after?
17 A
Yes.
18 0
Now, your overall agreement rate on an inspection point i
19 basis you calculated, using the weighting process that 20 you've referred to, to be on the order of 98 percent; 21 right?
i 22 A
I'm sorry, sir?
l l
23 0
You calculated the overall agreement rate, through using 24 the weighting process that you've referred to, to be on 1
25 the order of 98 percent?
l Sonntag Reporting Service, Ltd.
Geneva, Illinois 60134 I
17108
(~~
l A
Yes, 98, 99 percent.
2 0
All right, sir.
3 Now, that's on the basis, as you've testified, of 4
inspection points and discrepancies points?
5 A
Yes.
6 Q
That's the way the data was given to you to use?
7 A
Yes -- well, that was the way I determined, in 8
conversations with Mr. DelGeorge, would be appropriate 9
to provide information that might be useful.
3 10 0
All right, sir.
l 11 Do you know what the agreement rates, again 12 weighted in the same fashion, would be on a weld basis?
13 A
I have not done that calculation, I have not done that 14 calculation.
15 Q
Do you know what the agreement rates would be weighted, 16 as you've done, on an item basis; that is, using each 17 CSR sample item as a basis for determining whether or i
18 not an item had a rejectable condition, a defect, a 19 discrepancy, or was defect or discrepancy-free?
l 20 A
I didn't calculate it that way.
21 Q
I see.
22 I ask you to assume with me that on a weld basis, 23 the weighted agreement rate overall is 85 percent, 24 approximately; and I'd ask you to refer to Intervenors' 25 Exhibit 188.
That's the agreement rates as a function Sonntag Reporting Service, Ltd.
1 Geneva, Illinois 60134 (312) 232-0262
17109 V
1 of craft error rate, QC Inspector accuracy, 2
overinspector accuracy; the table.
3 A
Table 17 4
Q Table 1, Table 2.
5 A
Okay.
6 Q
Then there's a sample calculation.
7 Now, assuming that there's an agreement rate for 8
the CSR samples weighted, aggregated, on a weld basis, 9
on the order of 85 percent, can we agree that that would 10 fall on Table 1 in the lower right-hand corner of the 11 matrix?
I h 12 A
I'm sorry.
The number -- the number 85 is close to the U
13 number in the lower left -- lower right-hand corner of 14 the table.
15 Q
Yes; close to the values for agreement rates that would 16 be indicated, assuming the 90 percent accuracy of 17 overinspection, where there is a craft error rate of on 18 the order of 30 percent and a QC Inspector accuracy rate 19 on the order of 50 percent.
20 A
Yes, as this table shows.
21 O
Yes.
22 And similarly, if we assume, as does Table 2, an 23 overinspection accuracy rate of 80 percent, an agreement 24 rate of 85 percent, that value for an agreement rate O) 25 would fall again in the lower right-hand portion of the
(
Sonntag Reporting Service, Ltd.
Geneva, Illinois 60134 (312) 232-0262
17110
.(x-1 matrix?
2 A
It looks to be off the table.
3 0
Yes, it is off the table.
4 That would imply a craft error rate of in excess of 5
30 percent, a QC accuracy rate -- that ic, the vertical 6
axis -- of less than 50 percent; correct?
7 A
Yes.
8 0
Not having calculated agreement rates on the basis of 9
welds or items, you don't know whether for any of the 10 periods before, during or af ter the CSR data, the random 11 data that you looked at exhibited an agreement rate on 12 the order of 85 percent?
13 A
That's correct.
14 0
All right, sir.
15 Now, Mr. DelGeorge in his testimony, using the 16 aggregated random and non-random, the aggregated i
17 probability and non-probability CSR samples, looked at 18 data over time on a quarterly basis.
19 Are you aware of that, sir?
20 A
Yes, I am.
21 0
'that's the bar graphs that appear at the back of Mr.
l 22 De1 George's testimony.
{
23 You've looked at these, I take it?
l 24 A
Yes.
25 0
All right.
s l
i Sonntag Reporting Service, Ltd.
}
Geneva, Illinois 60134 (312) 232-0262
17111
/~'N
(
)
%j 1
You did not look at the data over time, 2
disaggregated, even on a quarterly basis, did you?
3 A
That's correct.
4 0
I take it it's also correct that you didn' t look at the 5
data disaggregated on a more disaggregated basis over 6
time, such as a month or a week or a day or an hour?
7 A
No -- well, sir, I should correct my answer to -- the 8
previous answer by saying I looked at the possibility of 9
doing it and made a determination that the sample size 10 would not support my doing it statistically.
11 0
Yes.
f\\J 12 A
So the answer is no.
I mean, I looked at it to the 13 extent of determining that I coul6n't look at it.
14 0
And determined that you simply couldn't, given the data 15 available, disaggregate the data as precisely --
16 A
Yes.
17 0
-- as I have inquired about?
18 I take it, then, that you don't know whether or not 19 the agreement rates for data disaggregated more finely 20 than before, during and after, the way you've used those 21 terms, exhibited agreement rates that were lower than 22 the agreement rates that you determined on an aggregated 23 basis, lower than 98 percent, approximately?
24 A
I don't want to get out of my area of expertise here, N
25 but I know that when you -- when you disaggregate data, Sonntag Reporting Service, Ltd.
Geneva, Illinois 60134 (312) 232-0262
17112 1
you produce variation.
Usually you get some things 4
2 higher and some things lower than the average.
3 So to the extent that one would expect that to 4
happen whenever you disaggregate data, I wouldn't be 5
surprised if we found some lower and some higher during 6
periods of time.
7 0
And if we were interested in looking at the data on a 8
disaggregated basis over time, we might find -- in fact, 9
using an average implies that we will find agreement 10 rates lower than the average agreement rate that you 11 computed; that is, on the order of 98 percent?
(}
12 We find lower than 98 percent?
13 A
You might, sir.
14 0
If we disaggregated data not on the basis of time but on 15 the basis of subgroups of inspection type and subgroups i
16 of inspectors, we might also find agreement rates lower 17 than the average agreement rate that you derived for the 1
18 aggregated data on average?
i l
19 A
In the same -- in the same way I answered that other 20 question, if you disaggregate data no matter what way, I
21 you might find things that are higher and lower than the 22 mean.
]
23 So in that sense I would answer that you could find 24 some lower.
25 0
Yes.
Sonntag Reporting Service, Ltd.
Geneva, Illinois 60134 i
(312) 232-0262
17113
)
v 1
And in performing your analysis, Dr. Frankel, did 2
you have reference to what was prepared by Mr. DelGeorge 3
and has been received in evidence as Intervenors' 4
Exhibit 1917 5
(Indicating.)
6 I show you that document.
7 A
No.
8 0
Do you recognize that document?
9 Have you seen it before?
10 A
I've seen it across the table, but I've not seen it to 11 look at it like this.
i I
h 12 0
All right.
O 13 Mr. De1 George, I'll submit, describes this as a 14 depiction of the evidence in this proceeding, concerns 15 expressed about harassment, intimidation and production 16 pressure over time.
17 A
Yes.
18 0
But you didn't rely on it in making your analysis?
19 A
No.
I relied on the two dates that Mr. DelGeorge 20 provided to me.
21 0
Now, I heard your testimony, your supplemental testimony 22 this morning, describing your role in the BCAP sample 23 selection process, Dr. Frankel.
Let me see if I 24 understood correctly.
()
25 I gather that you provided two critical pieces of Sonntag Reporting Service, Ltd.
Geneva, Illinois 60134 (312) 232-0262
17114 A
1 expert advice to Commonwealth Edison Company.
2 As I heard your testimony, the first was on the 3
sample size recommendation, and the second was on the 4
methodology for employing a random number table to 5
perform the cample selection itself.
6 A
Yes -- well, and the methodology for creating the l
7 sampling f rame in conjunction with the random number 8
table.
9 0
In conjunction with that latter point?
10 A
Yes.
11 0
All right, sir.
12 Now, what number did you supply to Commonwealth 13 Edison Company for the sample size value for the 14 electrical populations?
15 A
Well, I did a calculation basically to confirm some 16 calculations that they had already done, and the general 17 calculation showed that for populations in excess of 300 1
18 or 400, that if one had a sample size of 60 and one 19 found zero design-significant discrepancies in that 1
20 sample, one could make the -- or draw the inference of a 21 95 percent level of reliability at a 95 percent level of 22 confidence.
23 I relied on the binomial theorem to make that 24 calculation.
25 For smaller samples, I relied on the hypergeometric Sonntag Reporting Service, Ltd.
Geneva, Illinois 60134 (312) 232-0262
i 17115 f
Ns/
1 formula and the hypergeometric distribution formula.
2 0
All right, sir.
3 And for the electrical populations, can we agree 4
that there were no smaller populations than on the order 5
of 300 or 400?
6 A
That's my recollection.
7 0
All right, sir.
8 So that the sample size value that you derived and 9
the methods that you just referred to for the relevant 10 populations, the electrical populations, were 60?
11 A
Yes.
12 0
Now, is that the sample size that you relied on in 13 identifying the data that was the basis for your 14 testimony; that is, 60?
15 A
In certain electrical populations, there were more than 16 60 items selected on a random basis f rom the entire 17 population.
18 Where there were more items selected randomly from 19 the entire population, I included those as well.
i 20 0
All right, sir.
21 And those are all shown in what's been marked as 22 Intervenors' Exhibit 192?
23 (Indicating.)
24 A
That's the data that was provided to me?
b) 25 0
Yes.
That's the data that I showed you, the print-outs.
(
Sonntag Reporting Service, Ltd.
Geneva, Illinois 60134 (312) 232-0262
l r
17116 v(3 1
A Yes.
2 0
All right, sir.
3 Now, can you tell me -- do you recall --
4 A
In my prefiled testimony, in certain instances, there 5
may not have been any observations or any data values in 6
conjunction with a particular item.
7 0
I'm sorry.
I just didn' t understand that.
8 A
In my prefiled testimony, I indicated that there were 9
certain instances where there might not have been any 10 data points associated with a sample item if it were --
11 if it were impossible to make a determination of what 12 the date was of that inspection.
13 So in that case, in one sense the item was included 14 in the formula because the formula takes care of that.
15 In another sense, I could have excluded it and run the 16 formula and gotten the same results.
17 Q
I see.
18 I guess my point is:
if I looked at Intervenors' 19 Exhibit 192, the list of all of the items that you 20 considered, there would be shown the 60 random items 21 that were selected on the basis of your advice that a 22 60-item sample size was indicated; and in addition, the 23 more than 60 items where more than 60 items were 24 selected using that random number sampling method?
25 A
Right.
i Sonntag Reporting Service, Ltd.
Geneva, Illinois 60134 (312) 232-0262
)
17117 AL) 1 0
All right.
2 Absent, though, items that may have fallen out for 3
the reason you just referred to?
4 A
Right.
5 0
All right.
6 Well, given that caveat, can you tell me which 7
electrical populations -- for which electrical 8
populations there were in excess of sample size of 60 4
9 selected using the random number method?
10 A
I don't have that data in f ront of me right now.
4 11 I can possibly go through my materials and do that.
12 0
All right, sir.
13 Can you derive that information solely f rom 14 Intervenors' 192?
15 You cannot, can you?
16 A
No, I don't think I can.
17 0
And that's because certain items may have dropped out --
18 A
Right.
l 19 0
-- for the reasons you've referred to?
20 A
Yes.
21 MR. GUILD:
Could I ask, Mr. Chairman, that l
22 Applicant supply that information?
23 It doesn't need to be supplied now.
24 JUDGE GROSSMAN:
That's fine.
As long as Dr.
(
25 Frankel tells you that, Mr. Miller, then you can supply Sonntag Reporting Service, Ltd.
Geneva, Illinois 60134 (312) 232-0262
17118 in 1
us with the information.
2 MR. MILLER:
Just so I'm certain that I 3
understand the nature of the request, it is for 4
identification of each of the electrical populations in 5
which the sample size on which Dr. Frankel performed his 6
calculations that are found in his testimony exceeded 7
the number 60?
8 JUDGE GROSSMAN:
Yes; and I guess you want to 9
know how much that was.
10 MR. GUILD:
The numbers would be fine; just 11 the sample sizes that are random that were indicated for i (r 12 each of the electrical populations.
13 MR. MILLER:
Okay.
14 THE WITNESS:
You want, before eliminations, 4
15 what the sample size was?
i 16 MR. GUILD:
Indeed.
Thank you.
17 If I may have a moment, Mr. Chairman.
18 JUDGE GROSSMAN:
Sure.
i 19 MR. GUILD:
May I have a moment, Mr.
20 Chairman?
21 JUDGE GROSSMAN:
Yes.
22 BY Mo. GUILD:
23 0
Dr. Frankel, at Page 4 of your prefiled direct 24 testimony, in answer to Question 5, you provide some
)
25 definitions of terms.
Sonntag Reporting Service, Ltd.
i Geneva, Illinois 60134 (312) 232-0262
17119 v
1 May I ask you about a definition of a term that you 2
used in defining a term?
3 Do you see the word " element" referred to in Line 2 4
of your Answer 5, sir?
5 A
Yes.
6 0
Now, if I describe for you that each of the 7
subpopulations of electrical construction activity 8
consisted of a number of items, using the word " item,"
9 and " item" is equivalent to the number that you 10 recommended when you said 60 or 70 or 80 as a sample 11 size, would " item" also be equivalent to the term
(}
12
" element" that you use as you use it in this answer in 13 your prefiled testimony?
14 A
Are you saying in conjunction with the BCAP sampling?
15 0
Yes.
16 A
Yes -- well, item is one of the -- is one of the objects 17 that will qualify under the definition.
It's not the 18 only one.
19 0
Why don't you define " element" then as you use the term 20 in your testimony at that portion referred to?
1 21 A
Well, here I'm saying, "A probability sample is a sample 22 that is selected by a procedure that gives each element 23 in a defined population a known, calculable, non-zero 24 probability of being included in the sample."
(
If we consider one of the DCAP populations for a 25 Sonntag Reporting Service, Ltd.
Geneva, Illinois 60134 (312) 232-0262
17120 im
)
%/
1 moment and if we say that there are things that are 2
defined as items -- a synonym might be an assembly --
3 that assembly has on it different subassemblies, 4
different details.
5 The way the BCAP sample was selected, we've got a 6
probability sample of the items in that each one has an 7
equal probability of --
8 0
Each assembly?
9 A
Each assembly.
10 0
All right.
11 A
But, in fact, each assembly -- but, in fact, each (v) 12 subassembly in that population has an equal probability 13 of being selected even though some items may have four 14 or five subassemblies and some may have none, and the 15 same is true for sub-subassemblies.
16 So that in the broad sense, element is -- may be 17 used as item, but it may also be applied in other ways 18 in characterizing the BCAP sample as a probability 19 sample.
20 0
So element does not have a specific limited meaning, 21 limited only to item, as I used the term " item" in my 22 question?
23 A
Again, I want to give you an answer that is technically 24 correct.
25 The word element here in this definition is used in Sonntag Reporting Service, Ltd.
Geneva, Illinois 60134 (312) 232-0262
a l
17121 v
1 a generic sense.
When one makes a determination of 2
whether or not one has a probability sample, one says, 3
"Well, okay.
For this particular application, this 4
particular thing is the element, and now we will see if 5
it satisfies the definition of a probability sample."
6 0
The answer to my question is yes?
7 A
I thought it was, yes.
8 0
Now, you've used the term " design significance" in your 9
testimony.
10 Is that a term that was defined statistically?
11 A
That was a working definition that I was using as f3 12 something that the engineers determined.
G 13 0
Understood.
14 You didn' t make a design significance determination 15 yourself, did you?
16 A
No.
17 0
Of course, that's a product of work that is beyond your 18 expertise?
19 A
The definition of whether or not you have it or not, 20 yes.
21 0
The determination, the judgment --
22 A
Yes.
23 0
-- is a judgment that you are unable to make as a 24 statistician?
25 A
Yes.
All I can do is work with yes, there is one, no, Sonntag Reporting Service, Ltd.
Geneva, Illinois 60134 (312) 232-0262
17122 L/
1 there isn't one, and then work with that.
2 Q
Do you know what the attributes are that affect the 3
design significance of a defect?
4 A
No.
5 O
Do you know what the incidence -- that is, the chance of 6
occurrence -- of the attributes that affect design 7
significance within the electrical subpopulation is?
8 A
No.
9 0
If I were to suppose that a type of connection on a 10 cable pan hanger was an attribute that had an effect on 11 the significance of a weld defect or discrepancy within
( )/
12 the BCAP CSR program and I were to identify a particular
\\_-
13 detail or connection that had a bearing on -- or effect 14 on design significance and tell you it was, for example, 15 a DV-7 connection, could you tell me what the incidence 16 or chance of occurrence of a DV-7 connection would be in 17 the population of cable pan hangers?
18 A
No.
19 0
Could you tell me what the incidence or chance of 20 occurrence of a DV-7 detail was in the cable pan hanger 21 sample that was selected as part of BCAP?
22 A
I can tell you that -- to the extent that something 23 exists in the population, I can tell you that it has a 24 certain probability of being drawn into the sample --
25 roughly speaking, the probabilities of the sample size Sonntag Reporting Service, Ltd.
Geneva, Illinois 60134 (312) 232-0262
17123 O
1 over the population size -- whatever it is.
2 If it's a -- if it's something that is associated 3
with items, then it has a known probability of being in 4
the sample.
Some of them -- some of them in the 5
population no doubt come into the samples.
Some of them 6
don't come into the sample.
7 But each one of the population has a chance of 8
being selected in the sample.
9 0
Indeed, some known chance?
10 A
Right.
11 0
You just don't know what that chance is?
12 A
I do, but I have to calculate.
13 0
But you haven't calculated it?
14 A
It's approximately -- because we had a situation here 15 where the f rame had excess numbers on it, it is the i
16 total number of selections that were made divided by the 17 frame size.
18 Now, in not all cases did a selection actually 19 yield an in-scope item.
20 So, for example, if one made 100 selections and the 21 population size were 1,000, and out of that 1,000 one 22 found 60 that were indeed in scope, the probability of 23 selection is 100 over 1,000; in other words, 1 in 10.
24 That 1 in 10 probability of selection applies to every 25 in-scope item in the list, even though they' re i
Sonntag Reporting Service, Ltd.
Geneva, Illinois 60134 t
(312) 232-0262
17124
(~V) 1 interspersed among some non-in-scope items.
2 Any kind of attribute you want to consider, as long 3
as they are attached to one item in the population, then 4
that attribute has that same 1 in 10 chance of being in 5
the sample.
6 0
All right, sir; but I'm being more precise and asking 7
you about a particular attribute that in this case is, 8
let's say, a DV-7 detail on a cable pan hanger.
9 Do you know how many DV-7 details there are in the 10 population of cable pan hangers?
11 A
The point here is that --
I) 12 0
would you respond directly to that question, Dr.
V 13 Frankel?
14 Do you know?
15 A
I don't know how many there are in the population.
16 0
Do you know how many DV-7 details were included in the 17 CSR sample --
18 A
I don't know how many were included in the sample.
19 0
-- for cable pan hangers?
20 A
I don't; and I don't have to to make my other statement 21 correct.
22 0
Do you know whether or not the --
23 THE WITNESS:
Judge Grossman?
24 JUDGE GROSSMAN:
I'm sorry.
25 THE WITNESS:
I don't want to be Sonntag Reporting Service, Ltd.
Geneva, Illinois 60134 (312) 232-0262
. _ - -- ~.
4 em.
17125 N_ /
1 inappropriate.
2 May I --
3 MP. GUILD:
Excuse me, sir.
Could you speak 4
up?
5 JUDGE GROSSMAN:
The witness is asking 6
whether he can elaborate on the answer, and the answer 7
to that from me is that you'll have a chance to consult 8
with your counsel and he'll be able to ask you to 9
elaborate on redirect.
10 THE WITNESS:
Thank you, sir.
11 BY MR. GUILD:
12 0
You recognize, don't you, Dr. Frankel, that given the 13 sample size, there are configurations, say, in cable pan 14 hangers that may not -- that may exist in the population 15 that may not 'even have been sampled once?
16 A
I don't know enough about configurations to know that, 17 but I know that there are possibilities.
18 If, for example, there is one particular thing 19 called a unique configuration in the population and, 20 say, the population cize is 1,000 and one takes a sample 21 of 100, then it's indeed possible that it may not be 22 selected in the sample.
23 0
And being more concrete, Dr. Frankel, you don' t know 24 whether or not there are configurations -- I'm going to 25 use that term -- type, size, complexity -- that's an Sonntag Reporting Service, Ltd.
Geneva, Illinois 60134 f
(312) 232-0262
6 17126 rr (Jl 1
inclusive term, configuration, as I use it -- you don't 2
know whether there are configurations of, say, cable pan 3
hangers that were in the population that were not 4
included in the CSR sample, that are not among the items 5
that came up in the sample?
6 A
Would you repeat the way you asked the question?
You 7
said I do not know?
8 0
You don't know that, do you?
9 A
I don't know that, that's correct.
10 0
Fine.
11 And do you know, Dr. Frankel, whether that fh 12 configuration, the configuration that I supposed was in GI 13 the population but not included in the sample, is a 14 configuration in the population, because of the ef fect 15 of configuration on the engineering question of design 16 significance, that in the population unsampled has a 17 design-significant defect on it?
18 THE WITNESS:
Would you read back the 19 question, please?
20 (The question was thereupon read by the 21 Reporter.)
22 MR. MILLER:
As reread, I think the question 23 is --
24 JUDGE GROSSMAN :
Okay.
I think that the 25 Reporter couldn't get your pauses and, therefore, it may i
Sonntag Reporting Service, Ltd.
Geneva, Illinois 60134 (312) 232-0262
l 17127 pb 1
not come out the way it was stated with the appropriate 2
commas.
3 MR. GUILD:
Fine.
I'll rephrase the 4
question.
5 BY MR. GUILD:
6 0
It follows, doesn't it, Dr. Frankel, that making the 7
assumption that I've asked you to make, that you don't 8
know whether or not that configuration that didn't get 9
sampled, didn't come up in the BCAP CSR sample for cable j
10 pan hangers but is in the population, has a 11 design-significant defect?
(
12 A
I'm trying to answer the question.
13 I don't know whether any item that was not _ picked 14 in the sample -- whether or not -- I don't know whether 15 or not it has a design-significant qefect or not.
16 0
Yes.
17 A
I personally don't know.
18 I know about the inference that's made to the whole 19 population, but if yo'1 pick a specific one, no.
20 0
All right, sir.
21 MR. GUILD:
May I have one moment, Mr.
22 Chai rman?
23 JUDGE GROSSMAN:
Sure.
24 MR. GUILD:
That concludes my examination, (A) 25 Mr. Chairman.
Sonntag Reporting Service, Ltd.
Geneva, Illinois 60134 (312) 232-0262 t
17128
/sU 1
JUDGE GROSSMAN:
Miss Chan?
2 CROSS EXAMINATION 3
BY MS. CHAN:
4 Q
Dr. Frankel, do you recall, on your direct examination 5
by Mr. Miller, that you stated to the effect -- some 6
words to the effect that homogeneity was not used in the 7
BCAP sample selection process?
8 A
I said that as a statistician defines " homogeneity," it 9
wasn't used in the sample selection process.
10 I think some of the non-statistical -- some of the 11 thoughts that are associated with the non-statistical
}
12 use of the term " homogeneity" were used in creating the 13 populations themselves to be sampled, but in the 14 statistical sense, it wasn't.
15 0
By that do you mean that it was not -- that homogeneity 16 was not taken into account in your statistical analysis?
17 A
Right.
18 For example, to pick the sample, we took a 19 population that may -- the population -- the decision to 20 have a particular population may have been based on the 21 notion of homogeneity in a non-statistical sense, but 22 the population of elements were numbered from 1 to 23 whatever the number was, and then we used a random 24 number table, which is equivalent to picking numbers out 25 of a fair hat.
Sonntag Reporting Service, Ltd.
i Geneva, Illinois 60134 (312) 232-0262
I 17129
/"N
\\~-)
1 So whether or not one particular item might have 2
been more similar to another in that population was not 3
taken into account in the sampling.
4 The deck was shuffled and then the sample was 5
picked --
6 0
Following --
7 A
-- in the random portion of the sample.
8 0
Following up on your response on also the direct 9
examination by Mr. Miller, he asked you about items of 10 different complexity; in other words, different physical 11 attributes.
12 I believe you testified -- correct me if I'm 13 wrong -- that there was no impact if a significantly 14 large sample is taken?
15 A
I don ' t --
16 0
would you like me to --
17 A
Yes.
I don't quite understand.
18 0
On direct examination, Mr. Miller asked you about items 19 that have different complexity in the sample; for 20 example, items that had many physical attributes, 21 therefore making them more complex than another.
His 22 example was cable hangers, I believe.
23 A
Yes.
24 0
You said that having items of that nature, of differing 25 complexity, would have no impact on your analysis if you Sonntag Reporting Service, Ltd.
Geneva, Illinois 60134 (312) 232-0262
17130
/~
1 had a sufficiently large sample?
2 A
Right, right.
3 If, for example, one found no design-significant 4
discrepancies in the sample of 60, then you can make the 5
statement of 95 percent confidence and 95 percent 6
reliability for that population, including all the 7
complexities that exist in the population.
8 0
By your response, do you mean that the sample of 60 that 9
was taken is sufficiently large to cover any 10 discrepancies or variations in your statistical analysis 11 that might have been due to the complexity of various --
o) 12 the differing complexity of various items?
IN/
13 A
Yes.
I think, if I understand correctly your question, 14 yes.
15 When you have a sample, you're not making a 16 100-percent guarantee of 100-percent reliability.
The 17 only way to sample and do that is to take the whole 18 thing.
19 When you go through the probability calculations l
20 that are required to make the statements that you can 21 make, that takes into account the fact that the 22 population items or the elements that you' re sampling 23 may be very different from one another.
Some may be 24 very complex; some may be non-complex.
r-'
(j 25 As long as you shuf fle the deck before you pick the i
l Sonntag Reporting Service, Ltd.
Geneva, Illinois 60134 (312) 232-0262
17131 i
1 sample and you pick a sample of sufficient size, you can 2
still make that statement no matter how complex the 3
population elements are, as long as you assure that each 4
one is given an equal chance of being in the sample.
5 0
So that in this case the shuffling was adequate so that 6
even though items were -- the population was 7
non-homogeneous, a sample of 60 was sufficiently large 8
to not have an impact on your results; is that correct?
9 A
In the sense that on the statistical side it was.
10 I think that you've very well raised one of the 11 reasons why the engineering judgment samples were used 12 as well.
13 It was -- on a statistical mathematical basis, the 14 answer is -- the statement of 95 percent reliability 15 with 95 percent confidence can be supported with a 16 sample size of 60 no matter how complex the population.
17 However, the people who designed the full BCAP 18 program -- it was my understanding that they felt that 19 they wanted additional assurances past what you could 20 get mathematically, and that's the reason that they took 21 the non-probability samples.
22 0
can you explain what you mean by the additional -- the 23 use of engineering judgment in the sample size?
24 Did that affect the size of the non-homogeneous 25 sample taken by BCAP?
Sonntag Reporting Service, Ltd.
Geneva, Illinois 60134 (312) 232-0262
17132 O
1 A
My calculations of what sample sizes were required did 2
not look at what was going to be done on the non-random 3
portion.
4 In certain instances, the so-called non-random 5
portion turned out to be in some sense -- in some cases 6
the decision was made to pick that additional sample on 7
a random basis f rom the whole population.
If, in fact, 8
that was done, I included it.
9 But the calculations -- or any decisions that were 10 made about the non-random portion didn't enter into the 11 calculations of how large the sample should be for the A
12 random portion.
13 0
Can you explain in Kaushal Attachment 2C Kaushal-2 --
14 it's a table of the samples, and it gives the 15 construction categories, the number of population 16 items --
17 A
Can I get a copy of that, please?
t 18 0
Surely.
19 A
I don't think I have one.
20 0
-- and the size of the random cample.
l 21 (Indicating.)
22 A
Yes.
23 0
Can you please explain for the record why, under conduit 24 hangers, the sample size was 72 as opposed to 60?
t
,-~()
25 A
It was my understanding that one of the NRC Inspectors l
Sonntag Reporting Service, Ltd.
Geneva, Illinois 60134 (312) 232-0262
r.
17133 (3
i LJ l
wanted the sample size increased.
It had something to 2
do with definitions, that I'm not quite familiar with, 3
in terms of being safety-related and being in the scope 4
of BCAP.
5 But I believe that one of the NRC Inspectors wanted 6
the sample size increased, so it was.
7 0
So is it your understanding that within the 72 items, 8
there were still zero defects?
9 A
Yes.
10 0
Can you tell us who that NRC inspector was who requested 11 a larger sample size?
f) 12 Do you personally know?
v 13 A
I was told the name by Mr. Orlov.
14 If you want to let me consult with him, I can 15 report it to you.
16 JUDGE COLE:
But you don't remember it?
17 THE WITNESS:
I think I know who it was, but 18 I may say the wrong name, so I would pref er not to.
19 JUDGE GROSSMAN:
Mr. Orlov, you've been 20 sworn.
21 Would you volunteer the name, if you recall?
22 MR. ORLOV:
It was Mr. Ron Gardner.
23 JUDGE GROSSMAN:
Thank you.
24 MS. CH AN :
Thank you.
25 I have no further questions at this time.
g Sonntag Reporting Service, Ltd.
Geneva, Illinois 60T~4 (312) 232-0262
17134 p) iV 1
BOARD EXAMINATION 2
BY JUDGE GROSSMAN :
3 0
Dr. Frankel, I believe you indicated that you had no 4
input in determining what a design-significant 5
discrepancy is.
6 A
That's correct.
7 0
And you selected the sample size of 60 on the basis that 8
there would be a design significance determination made?
9 A
Yes.
10 0
All right.
11 Now, even though you don't have or didn't have any I h 12 input in what would be a design-significant discrepancy,
.)
13 as a statistician, don' t you, nevertheless, have to 14 assume that there is a fixed standard for determining 15 that there is a design-significant discrepancy?
16 A
That -- yes, yes.
17 0
In other words, whatever may be the standards when you 18 make statistical conclusions, you must necessarily 19 determine that there are fixed standards.
20 A
Yes, in the sense that the inference itself says, "If 21 this procedure were applied to all of the items in the 22 plant, then we can make this statement about what would 23 be found."
24 So yes, in that context, we assume that there's 25 a fixed standard.
Sonntag Reporting Service, Ltd.
Geneva, Illinois 60134 (312) 232-0262
17135 d
1 0
okay.
1 2
Now, what if, instead of having fixed standards, 3
the determination of what is design-significant can be 4
changed at various times af ter the study is begun?
5 Would that affect the validity of your statistical 6
conclusions?
7 A
As long as whatever -- assuming there's a way of doing 8
an evaluation on the sample that is the same as whatever 9
the standard happens to be, as long as what can be 10 applied to the sample could be applied to the whole 11 population at any instant in time, then the statistical 12 inference can be made.
13 In other words, if --
14 0
As of that instant in time?
15 A
As of that instant in time.
16 If it were changed and the sample were 17 re-evaluated, then it could be -- then it would refer to i
18 what would occur if the whole plant were re-evaluated l
l 19 with the same set of standards.
20 0
okay.
21 But what if you knew beforehand that the 22 design-significant standards were subject to a 23 subjective judgment along the way and that those 24 standards could change because of subjective judgments?
25 Would that affect the validity of your statistical J
Sonntag Reporting Service, Ltd.
Geneva, Illinois 60134 (312) 232-0262
17136 s
I conclusions?
2 A
Well, if you're saying if we -- if we had a sample and 3
if -- let's just focus on the sample for the moment.
4 If you say that in this sample one could change 5
Item 1 to be design-significant and then change it to be 6
nondesign-significant, you would have to tell me when I 7
should perform the inference.
8 You'd have to -- you'd have to lock in the 9
definitions at some point in time, and then I'd have to 10 perform the calculation.
11 0
Okay.
()
12 But now I'm adding an assumption there.
I'm not v
13 asking you whether at any one point in time, once the 14 standard is set, you can come to a valid statistical 15 conclusion.
16 I'm asking you to accept the premise that at any 17 time that an item is determined or an element is 18 determined to be design-significant, that would be 19 subject to change so that it can become 20 nondesign-significant.
21 Would that affect the validity of your conclusion; 22 in other words, knowing that the determination is 23 subject to change?
l 24 A
It depends what would make sense from an entire-plant i
25 standpoint.
1 l
Sonntag Reporting Service, Ltd.
l I
Geneva, Illinois 60134 l
(312) 232-0262 t
~
17137 O
1 In other words, if you forgot about sampling for 2
the moment and if you applied that question to all --
3 let's suppose you could inspect all the work in the 4
plant.
5 If it would make sense to be able to change if you 6
were looking at the whole thing rather than the sample, 7
you can do the same thing for the sample.
In other 8
words, whatever you can do for the population, you can 9
do for the sample and vice versa.
i 10 0
Okay, but again you' re still answering me with regard to 11 having a set standard at any one point in time; isn't
(
12 that so?
13 In other words, you're saying that the standard 14 could change.
15 But once it changes and is set, at that point in 16 time, you can make a valid statistical conclusion; isn't 17 that so?
18 A
Yes, but you could also -- you could do it the other 19 way.
I'm not sure if I'm really understanding your 20 question.
21 Half the plant, let's say, you were going to 22 evaluate this month, and the other half you were going 23 to evaluate next month.
24 I could possibly be using different standards when 25 I evaluate one-half of the plant versus the other half Sonntag Reporting Service, Ltd.
Geneva, Illinois 60134 (312) 232-0262
17138 1
of the plant.
As long as I took the sample items from 2
both halves and treated them the same way, then I could 3
do it.
4 0
What I'm talking about now is not an application to 5
different elements at different points in time but a 6
change in a design-significant standard with regard to 7
elements that were assumed to be the same or homogeneous 8
or subject to your sampling all the way through but --
9 let me see if I can clarify it.
10 Let's assume that you had certain 11 design-significant standards and you knew beforehand
(
12 that items might be considered to be subject to design 13
-- might have design-significant discrepancies but at 14 any point in time the person determining what is 15 design-significant could change his standards and 16 determine at any point in time that what is 17 design-significant now will not be design-significant 18 under new standards.
In other words, we have a moving 19 target here.
20 Would that affect the validity of any statistical 21 conclusions that you might arrive at?
22 A
The statistical conclusions would be conclusions about 23 applying the same -- the same procedures to the whole 24 population.
(%
(_ )
25 If you literally had a moving target moving through i
Sonntag Reporting Service, Ltd.
Geneva, Illinois 60134 (312) 232-0262
17139 (m
1 the sample, your inference would be about a moving 2
target moving through the population.
3 0
Okay.
So you would just be testifying with regard to 4
applying your sample to the population, but you' re not 5
going any further and making any conclusions as to 6
statistical validity with regard to any standards that 7
may be established at the beginning.
8 It's only with regard to your final point of 9
standard determination; is that it?
10 A
If you would change the wording " statistical validity" 11 to " substantive engineering validity" -- with a sample, 12 I can tell you or I can give you an inference about what 13 would occur if you applied the same procedures to the 14 population.
I can do that in a statistically valid way.
15 Whether or not those procedures, when applied to 16 the population, are valid f rom an engineering standpoint 17 is something I wouldn't know about.
18 So as long as you'll change the word " statistical" 19 to " substantive," I can agree with what you said.
20 0
Okay.
21 Now, I believe in response to an early question by 22 Mr. Guild, you indicated that you had not run your l
23 calculations on an item basis.
{
24 Later on, in response to another question, you i
f~)
25 distinguished in your answer between situations in which l
l i
l Sonntag Reporting Service, Ltd.
Geneva, Illinois 60134 l
(312) 232-0262
17140
/O
\\~ I '
1 you may have run a preliminary determination and then 2
ultimately decided that you couldn't use that 3
information in your final statistical analysis, not 4
relating to this item versus inspection point.
5 But I want to make sure that we have your correct 6
answer with regard to whether you had ever been asked to 7
run calculations on an item basis, as distinguished from t
8 deciding not to do it ultimately.
9 I'm giving you all this as a preface to my question 10 just to get it establishe'd on the record.
As part of 4
l 11 the preface, let me indicate that it was my 12 understanding that originally the BCAP CSR sampling was j
13 tc be done on an item basis, and the NRC in some form 1
[
14 had insisted on that.
15 Now, with that in mind, could you tell me whether 16 you actually were ever asked to make your statistical 17 determinations on an item basis?
18 A
For design-significant discrepancies, my understanding 19 was, when I made my calculations, that we were dealing 20 with an item basis in the sense of it's all or nothing; 21 if there's one thing which is defined as 22 design-significant, then the item is, in statistical 23 parlance, a reject.
I 24 MR. GUILD:
Excuse me, Dr. Frankel.
Could i
25 you keep your voice up?
You're trailing off.
1 Sonntag Reporting Service, Ltd.
i Geneva, Illinois 60134 (312) 232-0262
1 17141
(~'h s- /
l t
1 THE WITNESS:
Yes.
Sorry.
2 A
(Continuing.)
When I was asked to do these 3
calculations, it was explained to me that no 4
design-significant discrepancies were found, and the 5
kind of analysis that was being done was fundamentally 6
different.
I had to understand what the formulas were i
j 7
in order to know what kind of estimate it was.
l 8
In this case we have -- associated with each item 9
we have inspection points and discrepancy points.
- But, 10 in fact, the statistic of interest was this ratio of 11 inspection points minus discrepancy points divided by I
h 12 inspection points, the so-called discrepancy rate.
V 4
13 I was never asked to do a calculation which would 14 involve calling an item all or none if it had one 15 discrepancy.
I was never asked to do that.
16 BY JUDGE GROSSMAN:
17 0
Okay, though you are aware of the X, Y, Z categories, 18 are you not, with regard to significant discrepancies?
19 A
Yes.
20 0
And even with regard to Z items, you were never asked to i
21 make any analysis on an item basis; is that what you' re 22 telling me?
23 A
I did do some early analyses -- and I don't have any of i
24 that material with me.
25 I did do some early analyses that broke things up Sonntag Reporting Service, Ltd.
Geneva, Illinois 60134 (312) 232-0262
I 17142 i
O 1
into two categories, and I don't know how they 2
correspond to X, Y and Z.
3 Again, if you want to let me consult with Mr.
4 Orlov, I can tell you what I did early on.
But I think 5
I didn't take that any further.
6 JUDGE GROSSMAN:
Why don' t you just check 7
with him now?
8 (There followed a discussion outside the 9
record.)
10 A
(Continuing.)
Judge Grossman, I believe I made a 11 calculation involving looking at the X and Y categories (v) 12 aggregated together and separately the Z categories 13 aggregated together.
That was in a much earlier 14 version, a preliminary version of the data base.
15 I did some calculations initially, and again I 16 don't remember exactly what I found.
But I know that i
17 the data base had changed since then.
18 I was then informed by Mr. Orlov and Mr. DelGeorge 19 that they wanted to group everything together as the 1
20 statistic of interest, so I didn't do any more on that.
21 MR. MILLER:
Excuse me.
Before Judge Cole 22 begins, Judge Grossman --
23 JUDGE GROSSMAN:
I'm sorry?
24 MR. MILLER:
-- I'd just like to inquire.
l 25 You asked a series of questions of Dr. Frankel with Sonntag Reporting Service, Ltd.
Geneva, Illinois 60134 (312) 232-0262
17143 OV 1
respect to his reliance on the design significance 2
determination and asked him to assume certain facts with 3
respect to changes in the design significance standard 4
and the way in which the design significance 5
determination was made over time.
6 I've assumed that those are, in fact, hypothetical 7
questions to test Dr. Frankel's involvement in the 8
process and don't require Applicant to make a further 9
evidentiary presentation with respect to changes in the 10 design significance standards, 11 JUDGE GROSSMAN:
You want to know what my 12 frame of reference is, Mr. Miller?
13 MR. MILLER:
les, sir.
14 JUDGE GROSSMAN:
The testimony by Mr. Kostal 15 and Mr. Thorsell with regard to design-significant 16 determinations.
17 I don't want to paraphrase their testimony, because 18 I'm sure it would be inaccurate, but that was my frame 19 of reference.
20 I don't see that Dr. Frankel would establish or 21 clarify anything further along that line.
I'm just 22 asking him where he comes in as a statistician as far as 23 that goes; and he's indicated what the limitations are, 24 I believe, in his role.
25 JUDGE COLE:
Just a couple questions for Sonntag Reporting Service, Ltd.
Geneva, Illinois 60134 (312) 232-0262 g
17144 m
\\/
1 clarification, Dr. Frankel.
2 BOARD EXAMINATION 3
BY JUDGE COLE:
4 Q
On Page 4 of your prefiled testimony, in response to 5
Question 5, about the middle of the page, you state 6
that, "The term ' random sample' is often used three 7
different ways."
Then in the next two or three -- the 8
next two paragraphs, you describe the ways in which the 9
term " random sample" is used.
10 I want to make sure I've identified the three ways 11 in which you said it's used.
m V) 12 In the first paragraph, do you identify one way or
(
13 two ways in which it's used?
14 Specifically, when you say it may be defined as 15
" selected without replacement" or "with replacement," is l
16 that two or one?
l l
17 A
That's one.
18 The second way is treated as independent, t
19 identically distributed random variables.
20 0
All right, sir.
l 21 Then the third is on Page 57 l
22 A
Yes, sir.
That's the general population definition.
t l
23 0
The last --
l 24 A
Yes.
25 0
-- paragraph?
l Sonntag Reporting Service, Ltd.
l Geneva, Illinois 60134 (312) 232-0262
17145 x
1 All right, sir.
Thank you.
2 MR. GUILD:
Just to be clear -- excuse me, 3
Judge Cole.
4 JUDGE COLE:
Sure.
5 MR. GUILD:
I see two, and I'm trying to find 6
the third.
7 JUDGE GROSSMAN :
The three paragraphs 8
correspond to the three different ways.
9 MR. GUILD:
The general population is the 10 third way?
Is that your answer?
11 THE WITNESS:
Yes.
12 JUDGE GROSSMAN:
I believe so, isn't it?
13 THE WITNESS:
Yes, sir.
14 BY JUDGE COLE:
15 0
On Page 8 and 9, you talk about the difficulty in making 16 inferences to a large population when you use 17 non-probability samples.
We've had that at least in l
18 part in this case, as you've stated.
i l
19 If the non-probability sample is selected on the 20 basis of some characteristic that would make that item 21 or characteristic more likely to have a failure, can we
(
22 make any general statements about the relative 23 significance of equal size samples on a probability 24 basis and a non-probability basis where in both cases we t
25 find no error?
l Sonntag Reporting Service, Ltd.
Geneva, Illinois 60134 l
(312) 232-0262
17146
\\,
i 1
A I think if -- let me -- let me answer that by relating 2
what you would have to assume statistically.
In this 3
way I can keep my -- keep my answer in my area of 4
expertise.
5 If you assumed that the non-probability samples 6
were at least as likely to find errors, discrepancies, 7
whatever you want to call them, and whatever you' re 8
interested in finding, as long as you're willing to 9
assume it's at least as likely to find them as the 10 probability sample, then you certainly can strengthen 11 your inference.
s
)
12 To the extent that you feel that the J
13 non-probability sample can be treated as a random 14 sample, except you've increased the chance of finding 15 any problems if there are some, then you could use that 16 to add to your confidence as well.
17 0
Well, sir, if I could then, looking at the strictly 18 probability-based sample, I can make certain 19 mathematically-based inferences --
20 A
Yes.
21 0
-- with certain levels of confidence and reliability.
22 Now, can I then, given that with the probability l
23 sample, I have the exact same size sample in the 24 non-probability side, non-probability samples -- can I (O) 25 then compare what sort of inferences I might be able to Sonntag Reporting Service, Ltd.
Geneva, Illinois 60134 (312) 232-0262
17147 x
.Q) 1 make with the non-probability sample with the 2
probability sample on a relative basis?
3 A
Yes, I think you can.
4 As long as you' re willing to assume that you had a 5
higher chance of finding any problems in the 6
non-probability -- if you want to assume that you had an 7
equal chance of finding them, then you can make at least 8
the same inference independently in a replicated sense.
9 You've made another observation, and you make the same 10 inference.
11 If you feel you've increased the chance, then you
()
12 can make the inference even stronger; instead of 95/95, s-13 make it 96/96.
14 Q
But without attaching a specific statistical number to 15 it --
16 A
Right.
17 0
-- because you can't --
18 A
Right, but you can say it's higher than.
19 Q
All right, sir.
Thank you.
20 In response to Question 12 at the top of Page 12, 21 you state, sir, that as a statistician, you're not 22 qualified to discuss the subjective inferences that may 23 be made f rom the non-probability sample -- the 24 non-probability portion of the sample.
)
25 What qualifications would be required, sir, to make Sonntag Reporting Service, Ltd.
Geneva, Illinois 60134 f
(312) 232-0262
17148 (wJ l
-- to discuss these subjective inferences in the 2
non-probability portion?
3 A
Someone would need to have an understanding of the 4
nature of the data that was being collected and what it 5
meant.
One would also have to have an understanding of 6
the process used to select the sample.
7 One would have to be able to evaluate those two 8
things.
9 0
So a knowledge of the subject -- would this be in the 10 category of, say, something like an engineering 11 evaluation?
(. ;
12 A
I have been of that opinion.
13 0
All right, sir.
14 On Page 20, what is the origin of the numbers in 15 the second full paragraph; specifically, the numbers.8 16 and.6?
17 You say, "This power level in excess of 99 percent 18 holds for differences as small as.8 percent 19 (absolute)."
20 What does that number mean, sir?
21 A
Okay.
If you look back to the formula in my testimony 22 on Page 18 --
23 0
All right, sir.
24 A
-- I have agreement -- the quantity called the test l
25 statistic is the agreement rate pre and agreement rate l
l l
Sonntag Reporting Service, Ltd.
Geneva, Illinois 60134 (312) 232-0262
17149 rx s
1 post divided by the standard error of difference.
2 Now, if that test statistic had been greater than 3
1.96 or less than minus 1.96 -- roughly, 2 -- had it 4
been larger than plus 2 or smaller than minus 2, we 5
would have judged the difference to be statistically 6
significant.
7 Recognizing that the standard error of the 8
difference is.3 -- in other words, the value that I'm 9
dividing by is.3 -- if I had a difference of
.6, when I 10 divided.6 by
.3, I would have gotten 2.
Then the test 11 statistic would have been over 2.
f) 12 Had the difference been minus
.6, when I divided by V
13
.3, I would have gotten minus 2; and then I would have 14 had a test statistic that was less than minus 1.96.
15 That's the origin of that calculation.
16 0
Let's say you got the
.6.
17 A
Yes.
18 Had the difference --
l 19 0
Wouldn't it also be true if you had a difference of.45?
20 A
Well, if I had a difference of.45, I would have had.45 1
21 divided by.2962, and that would have been 1.519.
l 22 Q
All ri;P' sir.
I see.
Thank you.
I 23 Now, with respect to Attachment 2 Frankel-2 --
l 24 A
Yes.
-s s i
25 0
-- the PTL overinspection of LKC agreement rate over 1
i Sonntag Reporting Service, Ltd.
Geneva, Illinois 60134 l
(312) 232-0262 1
f 17150
/T V
1 time --
2 A
Yes.
3 0
-- back in your tectimony you provide us with the 4
equation of the line of best fit.
5 A
Yes.
6 0
I'm looking at the chart, sir; and there does, in fact, 7
appear to be an upward trend.
8 If we look at the equation of your line of best 9
fit, the slope of that line is something just a little 10 bit less than 1 and a half percent increase per year.
11 Do you agree with that, sir?
12 A
Let me just get the equation, sir.
I believe.1 is the 13 slope.
14 0
.117.
15 A
Yes.
Then that -- yes, I agree with what you said.
16 0
Now, looking at the plotted points on the chart, there 17 appear to be three, possibly four -- certainly at least 18 three points that seem to depart significantly f rom the 19 rest of the population.
20 Do you agree with that, sir?
21 A
I believe that there -- yes -- well, I'm not sure if 22 it's three or four, but there are some that are clearly 23 larger in deviation.
24 0
At least certainly the three back in ' 82 and ' 83 --
O)
(
25 A
Right.
Sonntag Reporting Service, Ltd.
Geneva, Illinois 60134 (312) 232-0262
17151
[v\\
1 0
-- that are below 80 percent?
2 A
Right.
3 0
If there were valid reasons why those points might not 4
be representative of the data, the line would take a 5
much different shape, wouldn't it, sir -- not a shape, 6
but a different slope, a different elevation?
7 A
Well, I think -- I think when we calculated it, the 8
slope would probably turn out to be somewhat different.
9 I've got a feeling, though, that the other i
10 statistic that I calculated, the coefficient of 11 determination, would also -- would remain weak.
But I fN 12 think the slope would certainly --
13 0
Are you referring to the R value, sir?
14 A
Yes.
15 0
Now, let's say that we knew nothing about those points; 16 all we had was the raw data.
17 A
Yes.
18 0
Are there any statistical tests that we might apply that 19 would permit us to exclude those data points?
20 A
There are some statistical tests we could use if we 21 wanted to make certain assumptions that would allow for 22 the possibility of throwing out data if we felt it 23 deviated too f ar f rom the average.
24 I didn't do any of those, but there are some that 25 could be done.
The reason I didn't do any was, because Sonntag Reporting Service, Ltd.
Geneva, Illinois 60134 f
(312) 232-0262 I
17152 (o~b 1
this wasn't a random sample, I didn't want to overstep 2
my bounds.
3 0
I understand, sir.
4 But just looking at the information, if we exclude 5
the four lowest points that are plotted on this figure, 6
what would be the result in a line of best fit?
7 Could you make an approximation of that, sir?
8 A
(No response.)
9 0
Well, first, let me ask you a question:
Would it be 10 your opinion that the slope of the line would change to 11 a lesser slope or can't you tell from the data?
(
12 A
I really -- I think -- I think you may be right, but I 13 really feel uncomfortable testifying to that f act.
14 0
I understand your position, sir.
15 But in any event, would the line be at any point 16 higher than where it was in your original line of best 17 fit?
18 A
In the sense --
19 0
Would it show generally a higher agreement rate?
20 A
Yes, oh, yes.
The whole average would move up.
21 0
All right, sir.
Thank you.
22 JUDGE COLE:
That's all I have.
23 BOARD EXAMINATION 24 BY JUDGE CALLIHAN:
O)
(
25 0
Carrying that point just a bit further, did you look Sonntag Reporting Service, Ltd.
Geneva, Illinois 60134 (312) 232-0262
17153 1
into those four points or three points which are mostly 2
in disagreement with the majority of the population and 3
establish whether there was a legitimate cause for 4
including or excluding them?
5 A
I believe that there was -- there was some discussion i
6 that went on about those data values.
7 My recollection is that, in fact, they were 8
legitimate situations; that, indeed, there had been a 9
problem with certain inspectors.
10 I think it could have been -- it was -- it was 11 traceable to certain inspection activities.
I can't
()
12 recall the specifics, but I believe that there's some 13 other witnesses available that could answer those 14 questions.
15 But I believe that was looked into.
16 0
So for conservatism and to be on the safe side, you 17 didn't include them all --
18 A
Yes.
1 19 0
-- in determining the slope?
20 A
Yes.
21 JUDGE CALLIH'N:
Thank you.
A 22 BOARD EXAMINATION 23 BY JUDGE GROSSMAN:
24 0
You're not offering any conclusion as to that now, as to
)
25 whether there was a legitimate or not legitimate reason l
Sonntag Reporting Service, Ltd.
Geneva, Illinois 60134 (312) 232-0262
17154 O
1 for excluding that; you're just relating to us that 2
there was discussion by others who made that 3
determination?
4 A
Yes.
There were -- there was discussions, by the other 4
5 people that turned this data over to me, of whether or 6
not those data values were legitimate; and I believe 7
that the determination was that they were legitimate for 8
whatever reasons they came up with.
9 Q
Right, but not your conclusion?
3 10 A
Right.
11 BOARD EXAMINATION h
12 BY JUDGE COLE:
5 13 0
Are you saying, sir, that based upon what you know about 14 it, they had reasons for the departure f rom what I might 15 consider to be the norm and they were legitimate reasons 16 that would not cause you to exclude the data?
17 A
Yes.
My understanding was that there was no data in 18 data collection; that, in fact, what had happened was, 19 in fact, real and there had been a lower rate.
20 The only thing I can add is that I don' t -- I don' t 21 want my conclusions to be really interpreted that there 22 is a strong trend in this data.
If there's any trend at 23 all, it's very, very weak.
24 I'd like to -- I think I said that, and I'd like to 25 emphasize that I would not like to be cited as a source l
Sonntag Reporting Service, Ltd.
Geneva, Illinois 60134 (312) 232-0262
~.. _
17155 O
1 for the opinion of a strong trend in that data.
2 JUDGE COLE:
I understand your point.
Thadk 3
you.
4 JUDGE GROSSMAN:
Mr. Miller, do you want to 5
recess now for 10 minutes or so?
6 MR. MILLER:
Yes.
I think that would be very 7
helpful.
I don't think I have very much, but I would 8
like to recess.
9 (WHEREUFON, a recess was had, after which 10 the proceedings were resumed as follows:)
11 JUDGE GROSSMAN:
Mr. Miller?
I\\
12 MR. MILLER:
Thank you, Judge Grossman, b
13 REDIRECT EXAMINATION 14 BY MR. MILLER:
/
15 0
Dr. Frankel, you were asked by Mr. Guild on cross 16 examination whether you had done agreement rates for the 17 periods that are described in your prefiled direct 18 testimony on a weld basis.
19
.Do you recall that?
20 A
Yes.
21 0
All right.
22 I think you testified that you had not?
23 A
That's correct.
24 0
During the break were you able to perform such a 25 calculation?
Sonntag Reporting Service, Ltd.
Geneva, Illinois 60134 (312) 232-0262
17156 r~
1 A
Yes.
I was able to examine the agreement rates on a 2
weld basis before and after, using the same dates I had 3
used for the inspection point examination.
4 0
And what is the source of the information on the welds 5
from which you derived that calculation?
6 A
The print-out that was provided to Mr. Guild.
7 0
And that's been marked as Intervenors' Exhibit 192; 8
correct?
9 A
I believe so, yes.
10 0
Now, Dr. Frankel, can you describe for us what the 11 results of that calculation are?
12 A
I calculated the before and the after agreement rates 13 based on using a date of July 1,
'82.
14 Before July 1,
'82, the agreement rate was 83.31 15 percent; and af ter July 1,
'82, it was 90.28 percent.
16 At this point I ca7't tell you whether or not that 17 difference, that increase in agreement rate, is 18 statistically significant.
But I can tell you that if 19 we tested to see if there was a decline in agreement 20 rate, we would have not found a decline in agreement 21 rate.
22 In other words, it's either -- this difference of 23 seven points, the increase in agreement rate of seven 24 points, percentage points, is either not significantly O)
(
25 different or it is significantly different in the Sonntag Reporting Service, Ltd.
Geneva, Illinois 60134 (312) 232-0262
17157 O
V 1
positive direction.
It was an upward change.
I'd need 2
a computer program to do that.
3 Looking at the date August 1,
'83, before the 4
agreement rate is 85.06 percent; and af ter it's 90.08 5
percent.
The same conclusion holds:
Either it's not 6
statistically significant or it's significantly better.
7 It's definitely not significantly worse.
8 0
Dr. Frankel, assume that for both the calculations that 9
you made, that the hypothesis is that there was a factor 10 present in the population being sampled after each date 11 that would operate to cause the agreement rate to 12 decline.
G' 13 Is your statistical analysis consistent or 14 inconsistent with that hypothesis; that is, the analysis 15 that you just testified to?
16 A
My conclusion would either be there was no difference or 17 the agreement rates went up.
My conclusion would not be 18 that the agreement rates went down.
19 Q
So your statistical analysis would be inconsistent, 20 then, with a hypothesis that said that this f actor --
21 let's call it production pressure -- would cause the 22 agreement rates to decline?
23 A
If that -- if that is related to the way I stated it in 24 my prefiled testimony, if that could be translated into
('
25 that statement with numbers, that there was no gw Sonntag Reporting Service, Ltd.
Geneva, Illinois 60134 (312) 232-0262
l 17158 3
1 difference or there was a difference, yes.
2 0
All right, sir.
3 Now, Dr. Frankel, could you describe what, if any, 4
analysis of inspector performance, as measured by 5
agreement rates over time, you performed in connection 6
with your assignment for the BCAP -- connected with the 7
BCAP program?
8 A
With the BCAP program?
9 0
Yes.
10 A
Inspector performance over time?
11 That didn't have anything to do with the BCAP
()
12 program, as I understood it.
I mean, we are now looking 13 at the BCAP data to deal with questions like that, but 14 the BCAP -- initial BCAP assignment had nothing to do 15 with inspector performance over time; only to the extent 16 of looking at the work again.
17 0
When you suy "iooking at the work," what work are you 18 referring to?
19 A
Well, the BCAP selected items, and a determination was 20 made as to whether or not there were design-significant 21 discrepancies in those items.
22 Again, I'm out of my area, but my understanding is 23 that those items had to be OC-accepted to be in the 24 definition of BCAP program; and by that I took it to y j 25 mean that they had been inspected by an inspector.
4 Sonntag Reporting Service, Ltd.
Geneva, Illinois 60134 (312) 232-0262
17159
{G 1
But in terms of looking at the particulars of the 2
inspector performance over time getting better or 3
getting worse, that was unrelated to BCAP.
4 0
And what was the statistical hypothesis that was being 5
tested by the probability sample that was selected for 6
the BCAP program alone?
7 A
The BCAP program was -- as I understand it, was -- the 8
sampling was perf ormed to see if any design-significant 9
discrepancies would be found.
10 0
And what part, if any, did that analysis of the 11 statistical data play in the preparation of your
(
12 testimony in this very proceeding?
( ')
13 A
Well, because I knew that the BCAP samples were 14 probability samples about known probabilities, I was 15 able to use the data that was collected in conjunction 16 with doing the BCAP program to make the inferences that 17 I've spoken about today, although at the time that the 18 BCAP program was going on, I certainly had no idea that 19 the use that we're talking about today would be made of 20 that data.
21 Q
Perhaps I wasn't clear in my question, but what I'm 22 asking is whether the statistical statement about the 23 likelihood of a design-significant defect, one that you 24 derived as a result of your analysis for the BCAP
, q )
25 program, played any part at all in the testimony that Sonntag Reporting Service, Ltd.
Geneva, Illinois 60134 (312) 232-0262
l 17160 O(h 1
you presented here today.
2 A
In the calculation per se, no.
3 In both cases I relied on the fact that we were 4
dealing with probability samples.
5 0
Now, Mr. Guild asked you a question with respect to your 6
knowledge of how many of a particular connection were in 7
the cable pan hanger population.
8 Do you recall that?
9 I think you responded that you didn't know how many 10 DV-7's, which I think was the letters and numbers that 11 he used, were in the population.
12 Do you recall that line of examination --
13 A
Yes.
14 0
-- Dr. Frankel?
15 Assume that there's a number of these DV-7's in the 16 population.
17 What can you tell us about the use of a probability 18 sample on the likelihood that such a hanger will be 19 included in the sample?
20 A
If I understand what a DV-7 is, it's a part of a hanger; 21 it's a detail.
Again, this is not in my area as a 22 statistician, but my understanding is that an element 23 sampled might have one or more or zero of these things.
24 If we talk about these things, these DV-7's, if I
(
25 have a probability sample sample of elements or of Sonntag Reporting Service, Ltd.
Geneva, Illinois 60134 (312) 232-0262
17161 V
1 items, then I also have a probability sample of DV-7's.
2 If the probability of selection for any particular 3
item in the population is one in 100, then any 4
particular and specific DV-7 detail anywhere in that 5
population has a one-in-100 chance of being selected; 6
therefore, I've got a probability sample of those as 7
well.
8 It's not a simple random sample; it's a clustered 9
sample, but it's a probability sample.
10 0
Mr. Guild also asked you whether or not you knew whether 11 a specific installation had a design-significant defect.
12 I think you answered that you did not know.
13 A
Right.
14 0
What inferences about the unsampled population is one 15 able to make as a result of the sampling process and the 16 statistical results that were accumulated?
17 A
When one makes the inferential statement that the sample 18 sizes we were using had a 95 percent confidence with a 19 95 percent reliability, clearly we' re talking about 20 numbers bigger than the sample.
We're talking about the 1
21 entire population.
22 If the population had 10,000 in it, we're talking 23 about -- we're talking about 95 percent of the 10,000.
l j
24 So we're going certainly past the sample.
That's 25 the reason we have probability samples.
l
\\
Sonntag Reporting Service, Ltd.
l Geneva, Illinois 60134 (312) 232-0262
17162 O
1 Q
Now, Dr. Frankel, you were retained by Commonwealth 2
Edison Company to assist'in the statistical aspects of 3
the BCAP program, as you testified in response to Mr.
4 Guild's questions; is that right?
5 A
Yes.
6 0
were you informed that a definition of " design 7
significance" had been adopted in connection with the 8
BCAP program?
9 A
Yes, I was aware that there was a definition that had 10 been adopted.
11 Q
What significance, if any, would there have been to your
[)
12 work if the definition of " design significance" had been V
13 changed during the course of the BCAP program?
14 A
Had the definition been changed, it would have been 15 necessary to apply that new definition to the sample 4
16 again.
17 0
Were you ever informed that such a change had taken 18 place?
19 A
No.
4 20 0
Assume with me -- again, this is strictly hypothetical 21
-- well, strike that question.
Let me move on.
22 Are you aware of any change in the calculations for 23 design significance for any specific BCAP item?
24 A
I was -- I was aware that definition -- that
(
25 calculations could go through various stages of what the 4
1 Sonntag Reporting Service, Ltd.
4 Geneva, Illinois 60134 2
(312) 232-0262
.~..
17163
//
i LJ 1
engineers refer to as " refinements."
2 0
Did you understand that those refinements were for the 3
purpose of determining whether or not a defect was 4
design-significant or not?
5 A
Yes.
6 0
In the ef fect that these refinements -- well, what 7
effect, if any, did these refinements have on your 8
calculation of the statistical inferences that could be
~ 9 drawn f rom the results of the analysis?
10 A
I'm sorry.
Would you repeat the question?
11 0
Yes, sir.
()
12 What effect, if any, on the statistical inferences v
13 that you drew from the results of the BCAP program were 14 there as a result of these refinements in calculations?
15 A
None.
16 It was my understanding that these refinements were 17 similar to what goes on in things called " group 18 testing," where if you're looking for a particular 19 disease, you may throw a lot of blood samples together; 20 and if you find that there's no presence of that disease 21 in any of these -- in the whole lot mixed together, then 22 you effectively have performed the test on each one of 23 them.
24 If you find that the disease is present in the lot, 25 then you go back and make specific tests on each -- on Sonntag Reporting Service, Ltd.
Geneva, Illinois 60134 (312) 232-0262
L 17164 O
t Nj 1
each sample.
2 Now, again it was not my understanding that this 3
was the way the calculations were made, but it was my 4
understanding that it was done this way in the sense of 5
the way the errors could go.
6 As you refined the calculation, you really weren't 7
changing the definitions you were using as to whether or 8
not something passed the test or did not pass the test; 9
but you were refining the calculation to see if it would 10 fail.
11 MR. MILLER:
I have no further questions.
I )
12 JUDGE GROSSMAN:
Mr. Guild?
N.J 13 RECROSS EXAMINATION 14 BY MR. GUILD:
15 0
Let's talk a little bit about sample size, Dr. Frankel.
16 My text is "How to Lie with Statistics," Darrell Huff, 17 37th printing.
18 Are you familiar with it?
19 (Indicating.)
20 A
I've seen the book.
21 0
All right, sir.
Well, this has some sort of 22 illustrative examples not recommended either by the 23 author, I take it, or by the witness.
In the chapter on 24 sample selection, he has an example of coin tossing.
25 Let's talk about that a second.
Sonntag Reporting Service, Ltd.
Geneva, Illinois 60134 (312) 232-0262 i
17165 i
v 4
1 Now, there are two chances -- two possible outcomes 2
in flipping a coin if we eliminate the outcome of the 3
coin landing on its edge, and that is a head and a tail; 4
correct?
5 A
Uh-huh.
6 0
You've got to say yes or no or the Reporter can't pick 7
you up.
8 A
Yes.
9 0
All right, sir.
10 And translated into probabilities, there's a 50 11 percent probability that you'll get a head?
(
12 A
Theoretically you may want to assume that there's a 50 13 percent probability.
l 14 If you take any particular coin, if you toss it 15 enough, you won't find that it's 50 percent.
16 Q
But theoretically it's 50 percent?
17 A
Theoretically, yes.
l 18 0
And that probability -- if one sampled through coin i
19 tosses, that probability should be borne out within the 20 limits of statistical inferences, the limits of 21 precision of statistical inferences, by a sampling, 22 should it not?
23 A
Well, again, when I made that first comment about the 24 theory, if you take any particular coin and indeed i
25 actually run experiments, you find that indeed the true Sonntag Reporting Service, Ltd.
Geneva, Illinois 60134 (312) 232-0262
I 37166 OV 1
probability of getting a head is not exactly
.5.
2 0
Well, coins may not be uniform in weight and size, and 3
there may be a different chance.
4 But assuming a statistically perfect coin, we've 5
got a 50 percent chance of a head?
6 A
At any particular toss, yes.
7 0
Now, Huff, "How to Lie with Statistics," Page 39, says, 8
" Science proves that tossed pennies come up heads 80 9
percent of the time"; and Huff supports that conclusion 10 by a sample of ten tosses of the coin, an actual 11 empirical study, in which case eight of the tosses of f) 12 the coin came up heads.
\\>
13 You'd agree that that sample supports the 14 statement, " Science proves that tossed pennies come up 15 heads 80 percent of the time"?
16 A
No, and I'll tell you why.
17 Whenever one does -- makes a sample estimate, one 18 should either conduct a statistical test -- one could 19 conduct a statistical test to see whether or not the 20 probability was
.5.
21 In fact, I'm not -- I haven' t done the 22 calculations.
I think there's some problems with the l
23 sample size.
24 But you can do a statistical test or you can put a 25 margin of error, standard error, around your result.
l Sonntag Reporting Service, Ltd.
Geneva, Illinois 60134 i
(312) 232-0262 I
[
17167 O
1 0
Sure.
2 What would you do to express the result that you l
3 got of the example that I just gave; that is -- you 4
don't exclude the possibility that it could happen that 5
one could toss a coin ten times and eight of the ten i
6 times it would come up heads?
7 A
That's right.
8 0
How would you describe the results of that sample, the 9
sample that showed that eight out of ten times you got 10 heads?
11 A
one has to say the sample came out this way.
If one is l
12 doing a confidence interval, we calculate a 95 percent 13 confidence interval.
We say, "The confidence interval 14 ranges from this percent to that percent, and we're 95 15 percent sure that the truth -- the true underlying 16 probability is within that range."
17 Q
How would that range run?
What would be the limits of 18 that range for the example that I gave?
19 A
Well, ir. the example that you gave, the sample size is l
l 20 too small to use what's called the standard normal 21 approximation.
l 22 So, in fact, you'd have to go and do a computer 23 simulation and calculate what that confidence interval l
24 would be.
l l
25 0
You just can't calculate it?
l Sonntag Reporting Service, Ltd.
Geneva, Illinois 60134 (312) 232-0262
17168
(
t
%s 1
A I can't do it.
2 0
Because the sample size is too small?
3 A
Well, the sample size is below the sample size that's 4
usually required to use the f aster-to-use theorem.
5 0
Now, if I replicated my sample and I did it 1,000 times 6
-- instead of doing ten tosses of the coin, I did 1,000 7
tosses of the coin -- would you expect that my results 8
would come up closer to 500 heads or 50 percent heads 9
than I found when I only flipped it ten times?
10 A
Yes.
The expectation is when you increase the sample 11 size, you come closer to a correct estimate of the 12 population parameter, assuming that the true percent 13 with that coin was
.5.
14 Q
Right.
15 Assuming that it's not a rigged coin, assuming that 16 there really is a 50 percent chance of heads, I'd get 17 closer.
18 If I did a large enough sample -- in this case, 19 1,000 tosses -- I still wouldn' t get 500 heads and 500 20 tails, but I'd get closer to 50 percent the probability 21 of getting a head?
22 A
In general you would.
23 0
In the case of my result, the change in my result is a 24 function of sample size, is it not?
25 A
Yes.
Sonntag Reporting Service, Ltd.
Geneva, Illinois 60134 (312) 232-0262
J 17169 i
i 1
Q Now, you supported Edison's theoretical position that as 2
a matter of statistical principle for large numbered 3
populations, 300 to 400, a sample size of 60 was 4
adequate in order to achieve a 95 percent confidence 5
level, 95 percent reliability?
6 A
Yes.
7 Q
You've got to speak up a little bit.
i 8
A Yes.
j 9
Q All right, sir.
10 And you did that without regard to what was being 11 sampled; it didn't make any difference to you whether we
()
12 were sampling apples, oranges or cable pan hangers?
13 A
That's right.
14 0
All right, sir.
l 15 It is an immutable statistical mathematical 1
l 16 principle that you were supporting there that didn't 17 have any -- wasn't rooted in the character of the items 18 being sampled?
{
19 A
That's right, that's correct.
20 0
All right.
Now, let's talk about opinion sampling.
l 21 I take it that's somewhat in your line of work, is 3
22 it not, public opinion survey sampling?
23 A
I do some of it, but I don't do a lot of it.
24 0
Well, I read about it in the newspapers more than 25 anything else, Dr. Frankel.
Sonntag Reporting Service, Ltd.
Geneva, Illinois 60134 (312) 232-0262
17170 (3
i i
v 1
Now, I take it that you'd agree that if we were 2
sampling voters and their preference for candidates for 3
public office, that if the population of voters were all 4
the voters in the United States, we'd agree that that 5
represents a large numbered population in excess of 300 6
or 400; correct?
7 A
Yes.
8 0
Do you read the commonly reported results of the Gallup 9
poll, for example --
10 A
Yes.
11 0
-- reported to the press?
12 A
Yes.
13 0
All right.
14 Now, Mr. Gallup or Dr. Gallup -- I guess it's Mr.
15 Gallup; I'm not sure -- he --
16 A
I believe it's "Dr."
17 0
Dr. Gallup.
Let's call him "Dr. Gallup."
18 You expect that Dr. Gallup and his organization are 19 familiar with the same immutable principle of sample 20 size for large populations that you are, wouldn't you?
21 A
Dr. Gallup doesn't use simple random samples.
22 0
That's not exactly my question.
I'm going to get to 23 that.
24 But you'd expect that Dr. Gallup is also familiar 25 with that immutable principle of sample size that you --
Sonntag Reporting Service, Ltd.
Geneva, Illinois 60134 (312) 232-0262
17171 IV 1
A It's not --
2 MR. MILLER:
Excuse me.
3 A
(Continuing.)
The characterization of " immutable 4
principle" -- there's a formula.
I'm familiar with the 5
formula.
I'm sure Dr. Gallup is.
6 BY MR. GUILD:
7 0
All right, sir.
8 It's a formula that, in your judgment, supported a 9
sample size of 60 being sufficient, as you've testified, 10 for a large population, those in excess of 300 or 400?
11 A
Yes.
f) 12 O
All right, sir.
V 13 Now, if Dr. Gallup is familiar with that princip]e 14 and voters are a population we all understand to be in 15 excess of 300 or 400, you recognize, don't you, that Dr.
16 Gallup does more than -- samples a larger sample size 17 than 60 voters when he expresses his survey -- when he 18 performs his survey sampling of voter preference?
19 MR. MILLER:
Your Ilonor, excuse me.
I object 20 to the line of questioning.
21 It seems to me to be well beyond the scope of any l
22 questions that were asked prior to -- or subsequent to 23 Mr. Guild's first cross examination.
Indeed, the 24 example seems to me to be straying very, very far from b) 25 anything having relevance to this proceeding.
(
Sonntag Reporting Service, Ltd.
Geneva, Illinois 60134 (312) 232-0262
i 17172 (U
1 JUDGE GROSSMAN:
Well, I'm not sure that the 2
answer wouldn't indicate something relevant to the 3
proceeding.
4 It may well be that there is some reason why Dr.
5 Gallup selects a larger sample that might be appropriate 6
to our samples; I don't know.
But that's certainly the 7
gist of the question, and I think we ought to hear the 8
answer.
9 MS. CH AN :
Your Honor, I believe that Mr.
10 Guild's questions follow up on my questions to the 11 witness about the number of items in random samples.
[)
12 MR. GUILD:
May I proceed, Mr. Chairman?
%./
13 JUDGE GROSSMAN:
Yes, please.
14 BY MR. GUILD:
15 0
Dr. Gallup does use larger sample sizes than 60, does he 16 not?
17 A
He uses different kinds of samples and tries to estimate 18 different kinds of things, and he uses samples that are I
19 larger than 60.
20 0
And if he used only a sample of 60, you would expect 21 that he would understand that the results of that samplo 22 might not prove an effective measure of what he seeks to 23 sample; that is, voter preference?
i 24 A
I'm not sure whether or not he would find 60 cases i
25 useful for what he wants to do with his sample.
I Sonntag Reporting Service, Ltd.
Geneva, Illinois 60134 (312) 232-0262
.. _. _ _ - ~ _ _
i 17173 i
1 Q
All right, sir.
2 Well, Dr. Gallup recognizes, don' t you expect, Dr.
3 Frankel, that there are a variety of factors that 4
influence voter preference that occur within the 5
population?
6 A
Yes, I'm sure he does.
7 0
Party affiliation might be one of the factors that 8
influences voter preference, wouldn't you expect, for 9
example?
10 A
In my role as an expert statistician?
11 I think you' re asking me to become a political 12 scientist.
L 13 0
As a human being, wouldn't you expect --
14 A
As a human being, sure.
)
15 0
Okay.
16 And wouldn't you expect that race, race of the 17 voter, might affect one's preference?
18 A
Just about anything might affect one's preference, so 19 race qualifies as anything, so I guess so.
20 0
All right, sir.
i 21 And wouldn't you expect that age might influence 22 preference?
23 A
It might, it might, it might.
24 0
It might.
25 Wouldn't you expect that income level might Sonntag Reporting Service, Ltd.
Geneva, Illinois 60134 (312) 232-0262
i 17174
(
,v) t 1
influence voter preference?
2 A
It might.
3 0
Now, don't you recognize that Dr. Gallup, in doing his 4
voter preference samples, takes those variables that in 5
his experience over years of sampling he judges to 6
influence voter preference, to affect voter preference 7
-- that he takes those variables into account in 8
constructing his sample?
9 A
I believe that Dr. Gallup quit using quota sampling in 10
-- that's what you're describing taking into account --
11 in 1948 when Dr. Gallup was one of the people that
(
12 predicted that Tom Dewey would be the next President of 13 the United States.
14 In fact, one of the boosts that probability random 15 sampling got was when most of the pollsters were wrong 16 in 1948.
17 0
Well, sir --
18 A
I think Dr. Gallup now recognizes that one should use 19 not a quota sample but should try to have a probability 20 sample.
21 0
But that one should make sure that your sample size is 22 adequate to assure that you've represented in your 23 sample the variables that do influence, in this case, 24 voter preference?
25 A
That's not the basis f or the sample size.
Sonntag Reporting Service, Ltd.
Geneva, Illinois 60134 (312) 232-0262
I 17175
%)
1 The basis for the sample size is what is the 2
statistic you're looking at and what do you want to 3
predict.
4 0
That's right.
5 A
And Dr. Gallup is looking to estimate something that's 6
close to 50 percent.
He is trying to find whether or 7
not the preference for one party or the other will be 8
over 50 or under, when he knows that the probable range 9
of values is from, say, 40 to 60 percent.
10 In the BCAP program, we're looking for something 11 completely different.
We're looking for very small
\\
(J 12 numbers, and the formulas are different there.
13 Q
Well, sir, my "How to Lie with Statistics" talked about 14 the Dewey-Truman call, too.
15 That error, that misforecast, if you will, based on 16 sampling, statistical sampling, was a product of bias in 17 the sample, was it not?
18 A
Well, there was actually a Congressional investigation 19 of that that occurred.
20 0
How about answering the question directly and then you 21 can explain.
22 A
What do you mean by " bias"?
23 0
What do you mean by " bias"?
24 Don't you understand " bias," Dr. Frankel?
(
)
25 A
Well, if I -- the most familiar definition to me is the NJ Sonntag Reporting Service, Ltd.
Geneva, Illinois 60134 (312) 232-0262
17176
/
(w 1
mathematical statistical definition of " bias."
2 0
And regardless of what that definition might be, won't 3
you agree with me, sir, that tne misprojection of the 4
Dewey-Truman result was a product of sample bias?
5 That's the question, sir.
6 A
I don't know that for a fact.
7 0
You don't know?
8 A
No.
9 0
All right, sir.
Well, then, we'll just leave that 10 question because you don't know the answer.
11 Now, isn't it appropriate to determine whether or f ')
12 not, in the case of an opinion sample of the sort that V
13 I've been hypothesizing -- whether or not your sample 14 adequately represents the relevant influences on voter 15 preference as they occur in the population if one is to 16 make inferences f rom the sample to the population?
37 A
One doesn't go through that sort of determination when 18 one makes inferences from an opinion sample.
19 0
Well, whether one does or one doesn't, isn't it I
20 appropriate to do so, in your opinion?
21 A
No, it's not.
22 0
It's not appropriate?
23 A
That's what I'm saying.
I 24 0
It's not appropriate, fine.
25 Wouldn' t you expect, Dr. Frankel, that if one were 1
Sonntag Reporting Service, Ltd.
Geneva, Illinois 60134 (312) 232-0262
17177
,\\
]
1 sampling attitudes among voters -- let's say 100 in my 2
community -- and that the attitude being sampled was, 3
"Do I support or not support President Reagan," that if 4
that fact, whether one supports or does not support 5
President Reagan in answer to that question, is 6
influenced by whether one is a Republican or a Democrat, 7
that one would want to know whether one's sample 8
adequately represented the incidence of Republicans in 9
the population?
10 A
The beauty of probability sampling and random sampling 11 is that you don't have to do that.
You shuffle the deck
[)
12 and you draw the sample; and you are assured by V
13 mathematical theory that if you go through the process 14 of shuffling the deck correctly and drawing the sample, 15 you can make the inference statements that the formulas 16 tell you you can make.
It's all taken care of.
17 That's why a sampling statistician who may know 18 nothing about a peculiarity of a population of items in 19 the plant is able to select a sample of items to look 20 at.
21 0
I see.
22 Now, sir, you don't know anything about welding or r
23 design significance in nuclear power plant construction; 24 you concede that, don't you?
25 A
That is correct.
Sonntag Reporting Service, Ltd.
Geneva, Illinois 60134 (312) 232-0262
17178 1
0 So your inferences, your judgments, are based solely on 2
mathematical statistics?
3 A
That's right.
4 Q
All right, sir.
5 Now, if I were to tell you that I got eight heads 6
in my ten flips of the coin, you wouldn't sit here and 7
tell me that as a matter of confidence, a matter of 8
certainty, that the likelihood of getting a head is 80 9
percent, would you?
10 A
I would not do that.
11 0
No, you wouldn't do that, and you wouldn't do that 12 because you'd know that in my case the sample of ten 13 flips of the coin was an inadequate sized sample?
14 A
Just as I didn't tell you that the agreement rate --
15 when I saw a difference of
.2, I didn't tell you that 16 that proves that it went up by
.2.
17 I simply said that I did a statistical test which 18 takes into account the sample size; and from that 19 statistical test, I could not conclude that that was a l
20 big enough difference to judge it as a real difference.
21 0
All right, sir.
I 22 But when I reach into that hat there and flip 23 coins, it's a random sample whether that coin is going 24 to come up heads or tails, is it not?
25 A
That's right.
l l
l l
Sonntag Reporting Service, Ltd.
Geneva, Illinois 60134 l
(312) 232-0262
17179 s__J l
0 All right, sir.
2 And yet because I only did it ten times, I got a 3
sample that was not representative of the probability of 4
flipping a head as that probability occurs in the 5
population?
l 6
A You got an estimate of the probability that departed 7
from what you're hypothesizing the truth to be.
8 0
And that's not because I cheated; that's not because I 9
fudged and made sure that the heads came up eight out of 10 ten times.
11 I closed my eyes, I flipped my coin, and I got
(
12 eight out of ten heads; right?
13 A
Had you taken one coin and done it once, you would have 14 concluded that they were all heads or all tails.
15 0
And that would have been wrong, too?
16 A
Because of a small sample size, absolutely.
17 0
Now, what we did in BCAP -
"we" meaning you, 18 Commonwealth Edison Company -- is they sampled -- let's 19 take cable pan hangers.
I'm looking at Dr. Kaushal's 2.
20 Now, there may have been some corrections to this.
21 It may have altered these numbers.
I don't have them 22 immediately at hand, but Dr. Kaushal's Attachment 2 in j
23 my version, which is unrevised, says that for cable pan 24 hangers, we had a random sample of 60 and an engineering 25 sample that included a random portion of 30.
Sonntag Reporting Service, Ltd.
Geneva, Illinois 60134 (312) 232-0262
17180 O.U 1
Now, assuming those numbers are correct, that would 2
suggest that there was a total random sample size of 90, 3
would it not?
4 A
Yes.
5 0
All right.
6 Do you know whether it's 90 for cable pan hangers?
7 A
No, I do not.
8 0
Let's assume it was 90 or close to that.
9 Now, do you know what the total population of cable 10 pan hanger items was?
11 A
I'm not looking at that exhibit.
I 12 Does that exhibit show that?
\\_-
13 0
It says 3,769.
14 A
Okay.
15 0
That population, of course, is the population of cable 16 pan hangers that were final QC-accepted -- installed and 17 accepted as of June 30, 1984.
18 A
Yes, sir.
19 0
You understand that to be the case?
20 A
Yes.
21 0
You've got to say yes or no, Dr. Frankel.
22 A
Yes.
23 0
Now, you are aware, aren' t you, that that's not the 24 total population of cable pan hangers in the Braidwood 25 Station, aren't you?
Sonntag Repor ting Service, Ltd.
Geneva, Illinois 60134 (312) 232-0262
17181 OV 1
A It's my understanding that there are others, yes.
2 0
And do you know how many others?
3 A
No, I don't.
4 Q
Do you know whether or not they had 90 percent of the 5
cable pan hangers final QC-accepted and installed by 6
that date?
7 A
No.
8 0
Do you know whether it was -- whether there were fewer 9
than 50 percent of the cable pan hangers final-installed 10 and QC-accepted as of that date?
I 11 A
No.
}
12 0
But you assume, as I do, that the number of actual cable 13 pan hangers at the Braidwood Station, when it's 14 complete, will be greater than the number that's 15 depicted in the population figure here?
16 A
It's my understanding that that's correct.
17 Q
All right, sir.
18 Let's assume that we did 100 random cable pan 19 hangers, and let's assume for purposes of discussion 20 that the population is only as it was as of June 30, 21
'84; that is, 3,769.
22 Will you agree that our sample represents only 3.7 23 percent of the population?
24 That's how many -- that's the proportion of items 25 in the population that we picked.
Sonntag Reporting Service, Ltd.
Geneva, Illinois 60134 (312) 232-0262
17182
\\
-s 1
A 100 out of 3,769?
2
.Q Yes.
3 A
I get 2.65 percent.
4 0
Oh, I did it wrong, okay, 2.65.
When you have a 5
calcule. tor, it helps.
So we have 2.65 percent.
Let's 6
take your number.
7 Now, your testimony is that if DV-7 details -- I'm 8
just using that as an example -- have an effect on the 9
design significance of a weld defect, that when we close 10 our eyes and reach into that hat, when we use your 11 random number method for sample selection, that those
[b
\\
12 DV-7 details, regardless of their incidence in the 13 population, have an equal chance of getting picked in 14 our sample?
15 A
The first part of your question I disagree with.
16 You said that -- I'd like the question read back, 17 but I believe -- I don't think I agreed to the first 18 part of your statement that said I had agreed to 19 something.
20 JUDGE GROSSMAN:
Could you read that back, 21 Miss Reporter?
22 (The question was thereupon read by the 23 Reporter.)
24 A
(Continuing.)
I don't know that they have an effect on 25 design significance.
Sonntag Reporting Service, Ltd.
Geneva, Illinois 60134 (312) 232-0262
17183 7s i\\
1 MR. GUILD:
Right.
2 A
(Continuing.)
But I do know that each and every one of 3
your details in your example has a.0265 chance of being 4
picked to be looked at in the sample.
5 MR. GUILD:
Okay.
All right, sir.
6 BY MR. GUILD:
7 Q
Now, let's assume that our detail of interest, whether 8
it's a DV-7 or some other detail, a detail that affects 9
design significance -- let's assume that it occurs in 10 only one out of 100 hangers in the population.
11 A
Yes.
12 0
Isn' t there a chance in our sample that we just missed 13 the cable pan hangers where there was the 1 percent i
14 incidence of the DV-7 details?
15 A
You're saying that if they only occurred in 1 percent of 16 the population, could you select a sample and totally 17 miss them?
18 0
Yes.
19 A
Yes, the answer is yes.
20 0
All right, sir.
21 And if, of course, that was the detail that 22 affected design significance and there was -- and I ask 23 you to assume -- a design-significant flaw in a DV-7 24 detail, if you didn't pick it in the sample, any C
(
25 statement that you made about the sample would not Sonntag Reporting Service, Ltd.
i Geneva, Illinois 60134 (312) 232-0262 l
17184 1
accurately reflect the existence of that defect in the 2
population on the sample?
3 THE WITNESS:
Would you repeat that question?
4 (The question was thereupon read by the 5
Reporter.)
6 A
I think that's a tautology.
7 If you have a problem someplace, if you have a 8
design-significant discrepancy and it doesn't get picked 9
into the sample, then the sample doesn't pick it.
10 BY MR. GUILD:
11 O
And your statements about the non-existence of a
(
12 design-significant def ect in the population --
13 A
The statements --
14 0
Excuse me, sir.
15 Your statements about the non-existence of a 16 design-significant defect in the population would be an 17 error, would it not?
18 A
The statistical statements that are being made -- let me 19 make that clear -- are probabilistic statements.
One is 20 making 95 percent probability.
One is not making --
21 statistically making statements with 100 percent 22 certainty.
There's always a chance that the statistical 23 statement can be wrong.
24 O
There's one chance in 20 that the statistical statement 25 is wrong?
Sonntag Reporting Service, Ltd.
Geneva, Illinois 60134 (312) 232-0262
17185 1
A That's right, there's one chance in 20 at 95 percent.
2 0
Now, if there, in fact -- if configuration -- that is, 3
the shape, size, number of components -- of a cable pan 4
hanger, by example -- if that is an attribute that has 5
an influence on design significance as e matter of 6
engineering and I tell you that there are 300 different 7
kinds of configurations of hangers but our sample is l
8 only 200, doesn't it follow that you simply missed 100 9
different kinds of configurations that just don't fall 10 into your sample?
11 A
Sure, yes.
Oi 12 0
It does?
U 13 A
Yes.
14 0
Even though they had the same equal chance of being 1
15 picked as the 200 that you picked?
16 A
That's right.
17 0
And I could use the same example about any other number
(
18 of attributes that are occurring in the population.
19 There are attributes that you know that, while they 20 had an equal chance of being picked, may not have been 21 included in your sample?
22 A
That's right, yes.
23 0
And to the extent that those things that weren't picked 24 in your sample have a bearing on design significance, 25 the statements you can make based on your sample results l
I Sonntag Reporting Service, Ltd.
I Geneva, Illinois 60134 l
(312) 232-0262 l
17186 l
7"'s k-m I
will not accurately describe the incidence of 2
design-significant defects in the population?
3 A
The statements that I make statistically are made with 4
certain probabilities attached, and I stand by those 5
statements.
6 If you ask, "Can you find a situation where s 7
statistical statement is wrong a certain percentage of 8
the time," the answer is yes, of course.
9 0
You reckon they had some reliability and confidence 10 levels attached to their projections that Mr. Dewey won 11 the '48 election?
fh 12 A
That was an instance where they were not using random J
13 sampling.
It was using a non-random sample.
So I'm not 14 sure exactly what they were doing.
15 0
You don' t know whether they announced with some 16 confidence, 95/95 or some other level of confidence, 17 that --
18 A
It would have been --
19 Q
Excuse me, sir.
Let me complete my question.
20 A
Yes, sir.
21 O
You don't know whether or not those statisticians that 22 misprojected the Dewey-Truman election in '48 came up 23 with some statistical description of the level of 24 confidence and the reliability of their results, do you?
y J
25 A
They may or may not have used incorrectly fora:ulas.
Sonntag Reporting Service, Ltd.
Geneva, Illinois 60134 (312) 232-0262
17187 v
1 Q
But they were wrong, whatever statements of confidence 2
and reliability they made, since they mispredicted the 3
outcome?
4 A
They were wrong because they didn' t have the sample 5
selected in a way that would allow them to use the 6
formulas.
7 0
Yes, all right, sir.
8 MR. GUILD:
May I have one moment, Mr.
9 Chairman?
10 JUDGE GROSSMAN:
Sure.
11 BY MR. GUILD:
12 0
Dr. Frankel, if I increase my sample size, don't I, by 13 definition, increase my likelihood that I will include 14 in my sample an attribute that occurs in the population 15 with some known probability but didn't occur in my 16 sample when my sample was of a smaller size?
l l
17 A
If you use the same sample design, use simple random 18 sampling the way we did in BCAP, had we increased the l
l 19 sample size, we would have increased the probability of 20 selection of the element in each population, each item.
21 To the extent that there were certain kinds of 22 attributes on items that were relatively rare, we would l
23 have increased the probability of selecting them.
24 0
All right, sir.
25 MR. GUILD:
I have no further questions, Mr.
Sonntag Reporting Service, Ltd.
Geneva, Illinois 60134 f
(312) 232-0262
l 17188
,.U 1
Chairman.
2 JUDGE GROSSMAN:
Miss Chan?
3 MS. CHAN:
Staff has no questions.
4 JUDGE GROSSMAN:
Mr. Miller?
5 MR. MILLER:
I just have very few.
6 REDIRECT EXAMINATION 7
(Continued) 8 BY MR. MILLER:
9 0
Dr. Frankel, you stated, in response to Mr. Guild's 10 questions, that the Dewey-Truman race was mispredicted 11 because of a misapplication of the statistical formulas;
[
}
12 is that correct?
'w/
13 A
I said that they did not use probability sampling.
I 14 believe the conclusion of the Board that investigated 15 why they went wrong was that they did not use 16 probability sampling.
17 There was one organization that did use probability 18 sampling, the University of Michigan Survey Research i
19 Center, and they predicted the election the right way.
20 0
I think we've established that the BCAP sample is, in l
21 fact, a probability sample?
22 A
Yes.
23 0
A statistical statement that you make as a result of 24 your analysis of the BCAP data with respect to design
(~h
()
25 significance -- would you restate that, please, for the Sonntag Reporting Service, Ltd.
Geneva, Illinois 60134 l
(312) 232-0262
~ -.
17189 A
(G' 1
record?
2 MR. GUILD:
Objection, Mr. Chairman.
The 3
record reflects his previous statement.
To take another 4
bite of the apple --
5 MR. MILLER:
Your Honor, this is really a 6
predicate to a response to a line that Mr. Guild went on 7
and on --
8 JUDGE GROSSMAN:
Do you want to use it as a 9
foundation for a further question?
10 MR. MILLER:
Yes.
11 JUDGE GROSSMAN:
Go ahead.
I hope you give Ih 12 it the same way you gave it the first time.
O 13 (Laughter.)
14 A
For each population we have a 95 percent reliability 15 level at a 95 percent level of confidence.
16 BY MR. MILLER:
17 0
For what, sir?
18 I understand it's 95/95, but for what; that there 19 are no defects?
20 A
Well, that's built into the reliability statement, yes.
21 When you say you have the reliability statement, 22 that indicates that we're 95 percent sure that the rate 23 of reliability is at least 95 percent regarding 24 design-significant discrepancies.
O 25 0
All right.
(
j Sonntag Reporting Service, Ltd.
Geneva, Illinois 60134 (312) 232-0262
17190 1
That does not purport to make a statement, does it, 2
that there are no design-significant deficiencies in the 3
remaining population?
4 A
No.
5 0
Now, in many of his questions, Mr. Guild asked you to 6
assume that a specific kind of connection, his f amous 7
DV-7, was particularly of interest with respect to 8
design significance.
9 What feature, if any, of the BCAP program as a 10 whole, to your knowledge, was designed to zero in on 11 those kinds of components?
(
)
12 A
My understanding of --
U 13 MR. GUILD:
Objection, Mr. Chairman.
The 14 witness has already conceded what the limits of his 15 expertise are.
16 If Mr. Miller is satisfied that the record already 17 establishes this point, it's certainly beyond the 18 competence of this witness and it's irrelevant what his 19 understanding is.
20 JUDGE GROSSMAN:
Overruled.
But to the 21 extent that he doesn't have his own opinion on it, we're 22 going to strike the answer.
23 We certainly aren' t going to let him talk about 24 someone else's understanding as evidence in the case.
25 If he wants to -- we'll just stand by that ruling.
If Sonntag Reporting Service, Ltd.
Geneva, Illinois 60134 (312) 232-0262
17191 0
1 he answers about someone else's understanding, we're not 2
going to take that as evidence.
3 A
(Continuing. )
In my prefiled testimony, I indicated 4
that one of the strengths of the program was that it 5
used a probability and a non-probability engineering 6
judgment component of sampling.
7 In discussions with the people responsible for the 8
BCAP program, when we talked about what kind of sampling 9
was appropriate and what sample sizes were appropriate, 10 I indicated to them that if there were specific pieces 11 of work, specific items that had specific things on them
(}
12 that were of particular importance to someone looking at 13 safety aspects, then by using a non-probability judgment 14 sample, one could directly focus in on those.
So they 15 might not come into a randomly selected sample.
16 By having an engineering judgment sample, a person 17 who had knowledge about potential defects could go in i
18 and say, "I want to look at a specific assembly or a 19 specific item because I know it has this kind of
)
20 assembly, which is subject to a higher chance for having 21 a problem with it."
22 That was the strength, from an objective and 23 subjective standpoint, of the dual approach.
24 MR. MILLER:
I have no further questions.
25 BOARD EXAMINATION Sonntag Reporting Service, Ltd.
Geneva, Illinois 60134 (312) 232-0262 i
17192 in (m) 1 BY JUDGE GROSSMAN:
2 Q
Now, you could also have expanded the sample and 3
determined the random sample of that particular item, 4
had you chosen, couldn' t you?
5 A
Well, yes, you could.
6 If you have an item that occurs in the population, 7
say, with an incidence of one-half of 1 percent -- it's 8
a relatively rare kind of item -- and you know it may be 9
subject to particular things that might increase the 10 likelihood of finding a problem, you could raise the 11 random sample quite a bit and still not really be
[V) 12 assured of getting at least one of those.
13 If you have a person who is knowledgeable about the 14 population, they can specifically say, I want to go 15 after one of these items and look at it or two of them 16 or three of them and go after it."
17 So it's a much more efficient strategy, unless you 18 want to inspect the whole plant.
19 JUDGE GROSSMAN:
Mr. Guild?
20 RECROSS EXAMINATION 21 (Continued) 22 BY MR. GUILD:
23 0
Let's follow up on that last question and answer, Dr.
24 Frankel.
25 All other things equal, what sample size would we Sonntag Reporting Service, Ltd.
Geneva, Illinois 60134 (312) 232-0262 ym+,y-"T
-= = - - -
-'N
--r
~~
17193 b(x 1
have to have to achieve a 99 percent confidence level 2
with 95 percent reliability?
3 A
I don't have the calculations with me, but I think it's 4
something in the neighborhood of 300.
5 0
All right, sir.
6 And what sample size would we have to have to have 7
a 100 percent confidence level with 100 percent 8
reliability?
9 A
That's very simple.
You take the whole population.
10 This is what Arthur Nielsen told people when they 11 told him that they didn' t believe in sampling.
He said,
)
12 "The next time you go to your doctor for a blood test, 13 ask him to take it all."
14 (Laughter.)
15 0
The next time we license a nuclear power plant on the 16 confidence that there are no design-significant defects, 17 maybe we ought to inspect it all.
18 JUDGE GROSSMAN:
Okay.
The objection is 19 sustained, Mr. Guild.
20 NR. GUILD:
Mr. Chai rman --
21 JUDGE GROSSMAN:
That's outside the witness' 22 competence to testify to.
23 MR. GUILD:
Then I'd ask that his last 24 volunteered crack be stricken, and I will withdraw mine 25 as well.
Sonntag Reporting Service, Ltd.
Geneva, Illinois 60134 (312) 232-0262
17194 Us 1
THE WITNESS:
Well, I think it's --
2 MR. MILLER:
Excuse me, Dr. Frankel.
3 I don't believe -- I believe Dr. Frankel was 4
illustrating a point which was called for by the 5
previous question, and Mr. Guild's gratuitous comment 6
about licensing nuclear power plants was simply uncalled 7
for and argumentative.
8 MR. GUILD:
Then let me rephrase the 9
question, Mr. Chairman.
10 BY MR. GUILD:
11 0
If one wanted to have 100 percent confidence, if we were 12 doing something like, in this case, deciding whether or 13 not an inherently hazardous instrumentality, such as a 14 nuclear power plant, has a defect in it that occurs a 15 very, very small percentage of the time but does occur 16 and we wanted to have 100 percent confidence that there 17 were no such defects of that sort, we would have to look 18 at every item, wouldn't we?
19 MR. MILLER:
Your Honor, I object.
20 First of all, it seems to me that Mr. Guild -- if 21 that's simply a hypothetical, perhaps it's all right.
22 But the statutory standard, after all, is 23 reasonable assurance.
That's what we' re here to decide, 24 not 100 percent confidence caused by 100 percent 25 reinspection.
Sonntag Reporting Service, Ltd.
Geneva, Illinois 60134 (312) 232-0262
17195 m
1 JUDGE GROSSMAN:
Well, Mr. Miller, you're 2
suggesting an answer to the witness.
He posed a 3
question of 100 percent confidence level, and that's all 4
that was before the witness.
5 We'll overrule the objection, but it really goes a 6
little far in giving that answer.
7 You may answer that.
8 A
If one wanted to have 100 percent confidence, then in 9
theory one would have to conduct a census, look at it 10 all.
11 As I pointed out in my prefiled testimony, there
~ h 12 are many instances where a sample is certainly better in
[J 13 practice than looking at it all.
14 BY MR. GUILD:
15 0
Yes.
16 I don't want to have you repeat what you've already 17 said, so we will stand --
18 A
If you need 100 percent at 100 percent, then the only 19 way to do that is to take it all.
20 0
Is to take it all, all right, sir.
21 1
I take it that it's beyond your competence to 22 know whether or not that level of confidence, that 23 reliability level, is what is appropriate in this case; 1
l 24 that's a matter of judgment that's beyond a
! O) 25 statistician's confidence?
(
l l
Sonntag Reporting Service, Ltd.
Geneva, Illinois 60134 (312) 232-0262
17196 m(o 1
A Yes, it is.
2 MR. GUILD:
Thank you.
3 JUDGE GROSSMAN:
Miss Chan?
4 MS. CHAN:
No questions f rom the Staff.
5 JUDGE GROSSMAN:
Mr. Miller?
6 MR. MILLER:
Nothing.
7 JUDGE GROSSMAN:
Well, fine.
Thank you.
We 8
have concluded.
9 Thank you very much, Dr. Frankel, for testifying, 10 and you're excused.
11 (Witness excused.)
I\\
12 JUDGE GROSSMAN:
We might as well adjourn 13 until 2:00 o' clock on Tuesday, unless there's something 14 further, Mr. Berry.
15 MR. BERRY:
Yes.
Thank you, Mr. Chairman.
l 16 Earlier this morning, Mr. Chairman, I had an l
17 opportunity to consult with my colleagues back in 18 Bethesda; and I'm informed, Mr. Chairman, that Mr.
19 McGregor appears to be available to testify on Thursday 20 of next week.
21 I understand, Mr. Chairman, that OIA's report is l
22 scheduled to be issued by close of business tomorrow and 23 certainly should be transmitted to Region III by Monday, 24 certainly Tuesday.
So Mr. McGregor should have an 25 opportunity -- he and his lawyer should have an Sonntag Reporting Service, Ltd.
Geneva, Illinois 60134
[
(312) 232-0262
17197 j
\\
1 opportunity to f amiliarize themselves with that report.
2 So consequently I understand that Mr. McGregor's 3
attorney has been contacted, and he's indicated that he 4
has no objection to his client being available to 5
complete his testimony next week; next Thursday, in 6
particular.
7 We have alerted Mr. McGregor to be with us at 9:00 8
o' clock next Thursday morning.
9 JUDGE GROSSMAN:
Fine.
Let's try and work 10 all the witnesses in around that schedule.
11 Mr. Miller, is there a problem with that?
U)
I 12 MR. MILLER:
Well, I will consult with Mr.
13 McGregor's attorney.
14 It would probably -- given the schedule of the 15 other two witnesses, Mr. Laney and Dr. Hulin, if we 16 could arrange for Mr. McGregor to be available in the 17 afternoon on Wednesday, assuming that Mr. Laney begins 18 at noon on Tuesday and that his examination ends at 19 approximately -- that it's approximately a day, Dr.
20 Hulin is not available until Thursday morning himself, 21 and it would be highly desirable if Mr. McGregor could 22 be available that afternoon in the event that we have 23 time af ter Mr. Laney's examination is concluded.
24 MR. BERRY:
I don't believe that would
,)
25 present any problem, Mr. Chairman.
(
Sonntag Reporting Service, Ltd.
Geneva, Illinois 60134 (312) 232-0262 1
17198
\\
Us.
1 What I would point out, though, is Mr. McGregor has 2
expressed the interest that his attorney be present 3
during his testimony.
I have not been in touch with Mr.
4 Geocaris, and I don't know if his schedule would permit 5
him to be with us next Wednesday afternoon.
6 MR. MILLER:
I think Mr. Gallo has.
7 MR. BERRY:
I would suspect that if that's 8
the case, I don't anticipate there would be any problem.
9 JUDGE GROSSMAN:
Mr. Miller, who is your 10 third expert next week?
11 MR. MILLER:
Well, there's Mr. Laney and then 12 Dr. Hulin.
13 JUDGE GROSSMAN:
Wasn't there a third one?
14 MR. MILLER:
No, sir.
15 JUDGE GROSSMAN:
Oh, so just two experts and 16 Mr. McGregor and Mr. Gardner, and that's going to cap it 17 all off?
18 MR. MILLER:
I certainly hope so.
19 I did inform the Board and the parties that Fr.
20 Kafcas' testimony was withdrawn.
21 JUDGE GROSSMAN:
Yes, yes.
We're aware of 22 that.
23 Well, then, we may actually finish by the end of 24 next week.
m 25 I would urge counsel to consult.
Let's not just Sonntag Reporting Service, Ltd.
Geneva, Illinois 60134 (312) 232-0262
17199
("A 1
leave this until next Tuesday afternoon to discover that 2
we have a gap there again.
So let's try and get 3
everything set up so we can move ahead.
4 Okay, fine.
5 Mr. Berry?
6 MR. BERRY:
I have two other matters that I 7
just want to keep the Board advised on.
8 You'll recall the Board Chairman raised a question 9
in connection with REG GUIDE 161.
I believe it was the 10 damping values.
11 We've referred that matter to NRR, and they're l
l 8
}
12 looking into that.
We expect to have a response that we l
13 can communicate to the Board early when we resume next 14 week.
l 15 Finally, you'll recall, f rom the in-camera briefing j
1 16 that we had, that there was a couple matters that the 17 Board had asked the Staff to look into.
18 We're on top of that as well.
Hopefully we should 19 have some more information on that for the Board next 20 week.
21 You'll recall that those were two matters that the l
22 Board Chairman directed Mr. Schapker to look into --
I 23 JUDGE GROSSMAN:
Yes.
I I
24 MR. BERRY:
-- and hopefully we'll be in a f
25 position to report back to the Board next week as to Sonntag Reporting Service, Ltd.
Geneva, Illinois 60134 (312) 232-0262
17200
\\
r 1
whether there will be a need for a further in-camera 2
briefing or maybe something that Staff counsel may be 3
able to share with the Board either in camera or in open 4
session.
5 JUDGE GROSSMAN:
Fine.
We appreciate your 6
report to us on that, Mr. Berry.
7 MR. MILLER:
One last question.
8 JUDGE GROSSMAN:
Mr. Miller?
9 MR. MILLER:
I just wanted to make certain 10 whether or not a copy of the pertinent portions of the 11 FSAR has been made available to the Board with respect 12 to the damping values.
13 JUDGE GROSSMAN:
Has it been made available 14 to me?
15 MR. MILLER:
Yes, sir.
16 JUDGE GROSSMAN:
I've looked at it myself, 17 but I'm not sure those are the pertinent portions.
So 18 I'm leaving it up to you or Mr. Steptoe to tell me.
19 I have a table that I think probably is, but I 20 don't know for sure, and I don't want to jump to any 21 hasty conclusions.
22 i:R. GUILD:
Mr. Chairman, did I hear the time 23 of noon on Tuesday for the resumption or was that --
24 JUDGE GROSSMAN:
No; 2:00 o' clock on Tuesday.
25 So we're adjourned until then.
Sonntag Reporting Service, Ltd.
Geneva, Illinois 60134 (312) 232-0262
i 17201 1
(WHEREUPON, at the hour of 12:45 P.
M.,
2 the hearing of the above-entitled matter 3
was continued to the 18th day of 4
November,1986, at the hour of 2:00 5
o' clock P. M.)
]
6 7
8 9
10 11 7
12 4
13 14
[
15 16 17 l
l 18 19 i
20 21 22 23 i
24 l
25 Sonntag Reporting Service, Ltd.
L Geneva, Illinois 60134 (312) 232-0262
NO PAGE NUMBER CERTIFICATE OF OFFICIAL REPORTER l
S This is to certify that the attached proceedings before the UNITED STATES NUCLEAR REGULATORY COMMISSION in the matter of:
NAME OF PROCEEDING:
BRAIDWOOD STATION UNITS 1 62 COMMONWEALTil EDISON (IIEARING) 50-456/457/0L DOCKET NO.:
PLACE:
CilICAGO, ILLINOIS
/)v' DATE:
TilURSDAY, NOVEMBER 13, 1986 were held as herein appears, and that this is the originai transcript thereof for the file of the United States Nuclear Regulatory Commission.
(sigt)
/AN (TYPED) f b
Official Reporter Reporter's Affiliation 8
-