ML19343D384

Forwards SM Alexander Critique of Section Iii,Re Statistical Basis for Testing Program,Of SL-3753,Revision 1.Evaluation of Test Error Should Be Made.Concrete Specimens Should Be Tested to Evaluate Analyst Ability to Discern Flaws

ML19343D384
Person / Time
Site:	Marble Hill
Issue date:	03/04/1981
From:	Cassaro M LOUISVILLE, UNIV. OF, LOUISVILLE, KY
To:	Fiorelli G NRC OFFICE OF INSPECTION & ENFORCEMENT (IE REGION III)
Shared Package
ML19343D383	List: ML19343D382 ML19343D384
References
NUDOCS 8105040382
Download: ML19343D384 (10)

Text

(

UNIVERSITY OF LOUISVILI.E Q

sFEED SCIENTIFIC SCHOOL LOUISVILLE,KENITCKY 40208 EFAR OF CIVi L March 4, 1981 9 Y t!,,

Mr. G. Fiorelli, Chief V.

Projects Branch 2 8 U.S. Nuclear Regulatory Comission _ _,,__2_

"f, ".

1 3

Region III g- ,

799 Roosevelt Road -

APR 2 2 M ' r Glen Ellyn, IL 60137 y. cc :,

ys -p /

Dear Mr. Fiorelli:

During the Marble Hill meeting on February 9 and 10, afirmed Cordelle Williams that an analysis was being prepared as a follow-up to my letter and report to D. W. Hayes dated September 26, 1980. The analysis has been prepared by S. M. Alexander, Ph.D., PE, Assistant Professor of Engineering Management and Industrial Engineering at the University of Louisville. A copy of Prof. Alexander's report is attached.

The objective of Prof. Alexander's analysis was to investigate the actual confidene.e of the S&L/ PSI evaluation study of in-place concrete at Marble Hill performed by Construction Technology Laboratories. His analysis establishes a specification which considers the risk associated with the sampling plan used at Marble Hill involving 60 test sites.

Interent in the S&L plan, which is intended to assure with a confidence level of 95% that no more than 5% of the concrete volume is defective, is the assumption that no test error exists. The analysis is intended to determine the risk associated with this assumption. Thus, the probability that defective concrete will be arroneously accepted is not conjoined with the probability of finding defective concrete in the S&L plan. This omission results in a plan that inevitably provides less than the required 95%

confidence.

The probability of test error which will result in accepting defective concrete has not been evaluated or considerad by Construction Technology Laboratories in their report for this test program. However, there is ample evidence that error of this type exists. The following is a sumary of some factors which prcvide clear indication of the need to conside: the probability of testing error of the type cited above.

The qualification test was performed on June 28, 1979 on concrete test blocks which' contained several different types of flaws including representative conditions for honeycomb, cracks, air voids, and embedments.

l Although no photo record is available to verify the results, the test was witnessed by several persons including representatives of NRC. The l

8105040382 l

h- e, Rn

Mr. G. Fiorelli -

March 4, 1981 Page Two conclusions of the qualification test was that the microseismic procedure was able to distinguish the several types of defects and accurately describe their condition. Presumably this qualification also included the ability of the analyst to identify honeycomb and to distinguish between honeycomb and dispersed air of 1/8- to 1/2-inch diameter size. On June 27, 28, and 29 of 1979 several tests conducted on the in-place concrete produced the following results indicating the presence of honeycomb. The results are cited in the test data supplied with Mr. Robert T. Bartczak's letter to Mr. Donald Stegemoller dated January 29, 1981. The data represent raw field data and test photos in several areas at Marble Hill not included in Volume II of Report SL-3753, Revision 1.

Date Area Page No._ Grid point Photo No.

6/27/79 1 4 B-3 8 6/27/79 2 7 A-1 9 6/28/79 8 16 A-7 No Photo 6/29/79 9 14 B-3 49 6/29/79 Rad Waste 1 C-9 1&3 These interpretation results were changed during the February 9 & 10, 1981 meeting from honeyccmb to dispersed air or entrapped air. Such revision which may be appropriate for the tests leads to serious doubt concerning the accuracy of the qualification test. No matter how the responses were interpreted at the time of the qualification tests, it would appear reasonable to expect that a consistent basis for interpreta-tion was being applied over the three-day period of June 27-29, 1979.

A change of definition with respect to the test results for in-place con-crete may also be interpreted as a corresponding change in the interpreta-tions applied during the qualification tests. Then how are we to interpret the difference between a honeycomo reflection and dispersed air in the qualification tests vis-a-vis the field tests. No photographic data exist in the qualification tests record, and what record does exist is uncertain l with respect to size of defect and corresponding reflection signal.

i Other situations involving i .terpretation of wave reflections also raise concern about the need to consider test error. For example, on July 6,1979 a test reported in Area 16 on pages C-43, photo 61 of Volume II of Report SL-3753, Revision 1, indicates a discontinuity. The field notes for the test report honeycomb, separation and entrapped air. The un-certainties of the definitions being applied require that some provision I

be made to evaluate the probability of test error.

Other situations involving different thicknesses of wave reflections that are termed discontinuities raise the question concerning extent of

~

Mr. G. Fiorelli -

March 4, 1981 Page Three the opening. An example of this is in connection with tests of area 50 reported on page C-160 of Volume II of Report SL-3753, Revision 1. Photo 289 reports discontinuities for two wave reflections in which the space for one is twice as wide as for the other. Further, the member thickness is 54 inches, making the larger reflection about 2 inches wide.

When these considerations are applied to the S&L sampling plan the result is that the probability of test ermr must be considered. Applying equation (c) in Prof. Alexander's report (typical results are given in the accompanying table) for a test error of 15% with 60 samples the confidence level will be 92% rather than 95% as required by NRC. Actually, there is reason to believe that test errors may be greater than 15% resulting in a confidence level less than 90%.

Since no qualification test data are available to evaluate the test error to be applied, it is recem. ended that such an evaluation be made.

A series of concrete test specimens with various types of flaws should be tested with appropriate photography to both evaluate the analyst's ability to discern differences in response signals and to identify large voids.

When an appropriate test error has been determined, it may then be applied to the apprcpriate sampling plan to determine the number of test sites needed to satisfy a condition of 95% confidence with more than 5% defective concrete.

Sincere

/ " d/ ,

j Michael A. Cassaro, Ph.D., PE Professor of Civil Engineering lw cc: D. W. Hayes-NRC C. Williams-NRC A. Parte i R. Ha::n T. Datillo i

l l

l l

.n

This is a critique of the section

• Statistical Basis i for Testing Progra=" (Section III - Fgs. 4-8) included in the report SL-3753 - Revision 1. This critique was prepared by Suraj M. Alexander, Ph.D., P.E., Asst. Professor, 1

Engineering Management / Industrial Engineering, Speed Scientific School, University of Louisville, Louisville, KY.

This critique addresses three =ain areas of the above section. They are as fellcws:

(i) The lack of specification of risk associated with Statistical Quality Assurance Prcgra=.

(ii) Incorrectness of the theoretical background provided as related to the statistical program actually used.

(iii) Possible misinterpretation of termincicgy.

A. DEFIIIITIClis l

Sc=e standard Quality Assurance definitions are previded belcw in order to clarify the use of these ter=s in the cri-tique which follows.

(i) Sa=pling Plan - A specific plan which states ,

a) the sa=ple si:es and b) the criteria for accepting, rejecting or taking ,

anc:her sa ple, to be used in inspecting the lot.

(ii) Single Sa=pling: Sa=pling inspection in which a e

f decisien Oc accept er Oc reject is reached after f the inspection of a sin =le sa=ple.

f (iii) Multiple Sampling: Sampling inspection in which t

1

aftor cach semplo, tho dccicion may bo to cecept, to reject, or to take another sample but in which there is usually a prescribed maximum number of samples, after which a decision to accept'or reject is reached. Note: Multiple sampling as defined -

here is sometimes called see.uential sampling or .

group sequential sampling. The tern multiple sampling is preferred.

(iv) Sequential Sampling: Sampling inspection in which, after each unit is inspected, the decision is made to accept, to reject, or to inspect another unit.

Note: Sequential sampling as defined here is some-times called unit sequential sampling.

(v) Acceptance Number: The largest number of defectives (or defects) in the sample or samples under con-sideration that will permit the acceptance of the I

inspection lot.

I B. CRITIQUE (i) The lack of soecification of risk associated with the Statistical Quality Assurance Program.

In deriving the sampling plan, Sargent & Lundy Engineers (S & L) define r (Np)(Ng )

C=1-I x n-x (1) x=0 '

(N) n where they define C = Confidence level n = Sample size l

N = Size of population 1

p = Acceptable maximum fraction defectives in the population.

2

Thoy go on to stato that, "by tasting a numbcr of rondom samples (n out of a population N), we can establish with confidence C that the max. fraction defective is p if the observed nt=ber of defectives is r."

The above equation for C is infact the probability of rejecting a lot of quality p (fraction defective p), when a single sampling plan is used with a sample size n and accept-ance namber r. Since NRC expects the sa=pling plan to reject ' defective' concrete volumes 95% of the time (95%

confidence level), the presu=ption here is that concrete i

volumes with pt defective are ' defective.' The above equa-tion (i) , which represents the probability of having sore than : defectives in a lot is derived from the Hypergec=etric distribution. S&L uses the Binomial approximation to the Hypergeccetric distribution in deriving a value for n using a value of r equal to 0. The Binomial approximation to the l Hypergec=etric is shown below in equation (2) .

I w

C=1- 5 (U)p X g n-x (2) l x=0 l

l

• be above Binomial approximation is valid if n is small com-1 pared to N.

S& L uses equation (2) with a value of C = 95. p = .05 and : = 0 to solve for n at.d obtains a value of a = 59. They

then suggest a sampling plan with sample size n = 59 and acceptance numb'er r = 0. In their words, "as a first step, a sample of 59 areas is to be tested. In order to establish with a confidence level of 95% chat no more than 5% of the l

population is defective (i.e. reliability of 95%) none of the I

3 I

camplac chould fail'tho toct". If tha sampling plon colccted was used as a single sampling plan, (i.e. Sample size n = 59, Acceptance number r = 0); it would reject a population with 5% defective 95% of the time. In order to understand what happens to populations with larger and fewer defective per-centages an Operations Characteristic (O.C.) curve must be constructed for the sampling plan. As an example the O.C.

Curve shown in Figure (1), which was plotted after some rough calculations were made by me, detail the erobabilities of acceptance of different quality populations using a single sampling plan with n = 59 and r = 0.

... . .s ...... .---.... . .-.-.. .

i 3

.,, O.C. Curve P ro b ..i . n=59, r=0 o,

5 Acceptance , , , _ , , _ . _ ,

t

.g.

S-4'

• r .- t rfrc1 s 1 d p%

e

w. e- e .- . .

Figure 1 4

l I, . - - -_ . _ _ - . . .. -

Thus if the sampling plan was a single sampling plan (tha equations used to derive n are only valid for a single sampling plan) the risks associated with using the plan is best illustrated by the O.C. Curve Figure (1) . These risks ".

have not been specified by 5 & L in their statistical program.

(ii) Incorrectness of the theoretical background pro-vided as related to the statistical program actually used.

S & L's theoretical backing for its sampling procedure (equa-tions (1) & (2)) are for a single sampling plan, but at the end of the section on Methodology (See Pgs. 5 & 6 of the i

report) they state that "the sampling program is sequential in that if a defective area is encountered in the first 59 samples, the sample size is increased to a total of 93 units and the acceptance number is increased to 1, if another defective is found the sample size is increased to 124 with acceptance number 2 and so on (See Table III-l on pg. 8 of the report SL-3753). No explanation is provided as to how I

these numbers are obtained, though a quick check reveals that they are obtained from the same two equations (1) and (2) .

These equations as mentioned before, determine the probability of rejection of a population of fraction defective p by a ,

' single sampling plan of size n and acceptance number .

However the procedure followed in the testing program is that of pseudo sequential sampling (see pages 5 and 6) no theoretical background or references is provided for this sampling pro-cedure in the report. The sampling procedure is neither a i

l unit sequential sampling plan, (which is generally used to minimize the number of samples taken to make a decision for l 5 l

l

{ _. . _ _ _ - _ _ _ . _ . _ _ _ _ _ _ . . - . _ _ . _ _ _ _ . _ _ _ _

\ .

acceptance or rejection), nor is it a multiple sampling plan (group sequential; see definitions provided at the beginning -

of the critique).

The main point to be noted here is that the program suggested by S & L is not a single sampling plan hence it is incorrect to derive paranaters for the plan from equations (1) and (2) . Moreover, since the plan is not a standard plan and sincq no references are provided the determination of risks associated with the plan beccme difficult. In my opinion no sampling procedure for testin.g should be used without a proper indication of the risks associated with the procedure. Therefore S & L should either change'their test-ing program or provide correct theoretical backing for their sampling program and indicate the associated risks.

(iii) Possible misinterpretation of terminologv.

S & L defines a population with 5% defective as 95% reliable.

In other words they state that if 5% of the concrete volumes existing are defective then they are 95% reliable. This in-terpretation of reliability is not clear from the letter from NRC (Exhibit 1) which states, "Public Service of Indiana will:

1. With its contracted organizations, centinue surface and volumetric examination of existing concrete volumes to estab-lish its adequacy and test a statistical sample, representative of both congested and other concrete volumes to assure with 95% reliability and 95% confidence level, that concrete volume meets recuirements." The term reliability used above is quite ambiguous, it could for example relate to the accuracy of the test, whereas S & L has interpreted reliability to be synonomous with requirements and hence attempted to design a sampling

plan which rojccta populctionc of 5% dofcetivo (95% rollability according to S & L interpretation, See bottom of pg. 6) with a probability of .95 ,(Confidence level 95%) .

t

.%,'*1F 6

7

s. .

THOMM R. D1.TrTLO ATTORNEY AT LAW 311 EAST MAIN STREff MACl1oN. IN0iAMA 47200 DwoNS 882 26$4353 March 26, 1981 c 9  %

United States co e g truclear Regulatory Commission g, --

Region III 799 Roosevelt Road

", ' APR 2 21981 * ~

r 2

Glen Ellyn, Illinois 60137 1. CNs t. 2; Lq C: M --; 5.-4::3 ATTit: i:r. James G. Keppler, g d Director a p Re: STIl 50-546 STri 50-547

Dear I!r. Keppler:

Thank you for your response to Dr. Cassaro da:ed .'!2rch 20, 1981. Find encloccd an Exhibi; "A' herein Dr. Cassaro's specific reuponse to your:..

This wri:er in conjunction with :he enclosed Exhibi; "A" categorically states that the !!RC has not responded to the I: arch 4, 1981 letter of Dr. Cassaro ar.d the specific c:atistical and quali:y control eno:.neerin o conments therein. I; is our position :ha / car alleo ed e.<piar.a:ic is are simply ir.curricient to manifes :o :he I:RC : h . i t. its own criteria of 95% reliabil :y with 95% confidence has been me; and, or exceeded by the tes:

program devised by 360. 5 Lundy and conduc ted by For land Comen:

Ansociation.

In addition, it was our specific unders tandin , e :ha'. no f.RC responses would be made until :here was time to review wha; the independen; engineers 3: aced in a final written report subject to :he addi ional input af Dr. Cassaro, From all indications, i; is no: necessarily correct : hat the !!RC haa approved :he :cstin o standards and cri:eria speeific A;1/

se: out by the IEC in its prior orders herein. For tha; reason, we awai the final written repor:n of the independen eno incers and the apecific repor:s cf :he I RC. ,

Very t.ruly yours,

. '!a>w /" ;) N.k Thomas M. Da:cilo TI:D/mjb Enclosure

~ p ,rW Yl ,

Bb%@@#% Mk% .

Michael A. Cassaro, Ph.D. PE

\' Professor of Civil Engineering

( Speed Scientific School

\ Dept. of Civil Engineering University of Louisville Louisville, Kentucky 40208 March 26, 1981 9I :-

e /?'

' x Uni:cd States 7.. ,_ sy" .

iuclear Rebulatory Commission .-

Re' ion III 8-I "

799 Roosevelt P.oad AM 2 01%1 y Glen ~.ilyn, Illinois 60137

%. [c ATTil: -tr . Ja:nes G. Keppler, 7

s/ .ff Director 4 \ '

ic, I t t >/

Re: STN 50-546 STN 50-547

Dear Mr. Keppler:

ihe f:RC le t te r o f !? arch 20, 1981 L3 qu i: r de tailed 0:.d rea::u .ab iy comprehenn ive .

I:va - o f the points in my Le t ter of Geptecber 26, tv60 have been discucced and handled. It is the !!aren 4 L9di

! c t.te r o f m ine that has not been co:r.pletely anaac red and is the ;ubj ec t of this correspondence.

.he i;RO muc t. reiy on microce!.:.nic i n v e. . . c: a t . a ;c de '.e r: . ;:.e _f volds, aeparations or hcneyev::.oc cxis . .n :he ecn re:.e sinco all abservable volds are considered "curface con:re te de fee tivos" . Since the !.RC has estabiishou the criteria, 95% reliabill:y with 95% confidence, using :.qua:icn i ., .t must be recc onteed that this is a pratat.4 is:!c equation l implyind no inctru.T.ent or human error associa :cd with the aratistical approach.

'. h e l' arch 4 Letter giv?u an acceptib;e procedure fe r de termining an'd in: Lud ing t.he human error tnd the inc. ; ru ne n t

• rror in :he test program. Seve ral example:, o f this type of er. ar are cited in the March a 'etter. Fo r e xx:ple . if the

.: .erpre tor declared a " honeycomb" the an:.e da,' he tar:cd :he qua!ification tes; and the honeyco'.b tur:1ed .. 'c ha*.e a b:t i f ican t bubbics in the :cncrete, ther. O h .-r e a p p<: a r s :o te roa::anable room for error, m :n:r a-

).

(_p. Lt&

Qs  :

v-

=5b + p d>S3 t .

Mr. James G. Keppler Nuclear Regulatory Comm.

March 26, 1981 Page 2.

'de do not question Mr. Muenow's qualifications. I!owever, everyone is capable of making an error. It appears that no written qualification exists in the record at Marble Hill. A qualification record must be performed to evaluate if we have 95% reliability with 95% confidence.

As it now stands, there is certainly less than 95% reitability as outlined in the March 4 letter, and we believe that no verbim,,e will crase that reality, only clear statistics.

This writer awaits the furtr'.er responso o f the URC.

Sincerely,

\_g ,n o.. . b_g ' C% W ' '- ,% My .h.)

. .g

%  ;

Micnaet A. Caccaro, ?h.D.. PE I'/,C , :n j b P00R BRlBld-t

1. ,

5come of Work: -

Provide NRC-IE assitance as a technical consultant in the review of 1) concrete deficienci'es, known as "honeycombing," found at the Marble Hill ,

l facility through visual observation, 2) the program to determine if -

internal voids exist, 3) the repair p,recedares, 4) the completed repairs, .

and 5) evaluation of the affected structures to meet the original design ,, ,

intent. '

The effort will involve a review o' the techniques used to locate voids, discontinuities, etc., to determine if all significant deficiencies have

.in all probability beer. detected. These techniques included coring and .

puTse echo. '.The repair procedures and repairs made as a result of the ,

• findings of the investigations are also to be reviewed for adequacy. . The

___ main . basis of the consultant's review effort will be the report submitted .

by the licensee and its references as prepared by Sargent &,Lundy, Report

. SL-3753,11/20/79, ." Evaluation of_In-place Concrete, Marble Hill Generating ---- -

Station, Units 1 and 2," dated November 20, 1979.

• /,

.~ -

2. Obiectiyes of Tasks- -

e

/ a. To provide an independent assessment of the type and exten?o deficiencies in concrete construction defined as honeycombing ' '

and/or voids that could have safety significance;

_ b. to provide an independent assessment of any needed repairs or*

remedial actions; - -

c.

to provide independent conclusions regarding the capability of the affected struttures to perform the intended ' design functions.

3. Statement of Work: -

1 .

i a. Determine thaE any structurally significant honeycombing and/or l

i voids visually detectable have been located and identified.

b. Evaluate the .need for.,and adequacy of the nondestructive techniques .

used by the licensee in the investiga-ion of possible internal l voids. If other techniques or additional investigation are neces- --*

sary to assure structural adequacy, recommendations should be provided. This evaluation and any necessary reco=nendations should ,

j consider sample location and size. '

e%,

? '

_ e ,. e \

~-

, "g \  !

g-

.b9%S g @ ' rE J

• o

~

y g.%,.;ga:.~

g -

., tg

(

^

QQ