ML20137P725
| ML20137P725 | |
| Person / Time | |
|---|---|
| Site: | Comanche Peak  |
| Issue date: | 01/31/1986 |
| From: | Dignan T ROPES & GRAY, TEXAS UTILITIES ELECTRIC CO. (TU ELECTRIC) |
| To: | |
| References | |
| CON-#186-962 OL, NUDOCS 8602050310 | |
| Download: ML20137P725 (38) | |
Text
_
.o Filed: January 31, 1986.
e ~,
0)
Doct'Tra 'g UNITED STATES OF AMERICA p NUCLEAR REGULATORY COMMISSION vb u%gypCII98 srcy4.j
% U before the /g ,
ATOMIC SAFETY AND LICENSING BOARD
)
In the Matter of )
TEXAS UTILITIES GENERATING
) Docket Nos.50-446 50-445 Q
)
COMPANY et al. )
) (Application for an (Comanche Peak Steam Electric ) Operating License)
Station, Units 1 and 2) )
)
APPLICANTS' MEMORANDUM IN RESPONSE TO BOARD'S MEMORANDUM
' Statistical inferences from CPRT Sampling)
Introduction The Applicants submit this memorandum in response to the " MEMORANDUM (Statistical Inferences from CPRT Sampling)" issued by the Licensing Board on November 11, 1985 (" Memo"). In that memorandum the Board noted several questions and " concerns" that were apparently raised during the Board's review of the Program Plan J
0
%20 G
l So3
' C published by the Comanche Peak Response Team ("CPRT")
Senior Review Team ("SRT") on Jurs 28, 1985 (Revision 2), as well as by the Board's revie< of the transcripts of several meetings at which aspects of the Program Plan were discussed.
Neither the CPRT Program Plan nor any of the comments that the Board may have read are in evidence in these proceedings. Indeed, the Program Plan and several of its constituent attachments, appendices and action plans have been modified several times since the publication of Revision 2, and further revisions are possible (if not likely). Given that nothing is yet before the Board, we assume that the statements contained in the Board's memorandum are neither findings nor conclusions. See Memo at 3 n.3. The purpose of this memorandum is not to serve as testimony or evidence, but rather to indicate to the Board that the Applicants and the SRT have the Board's questions in mind.
This response is structured in parallel with the comments in the Board's memorandum, as follows:
- 1. The R.Sle, Use and Design of Sampling in the CPRT Program;
7.
> c
- 2. The Relationship Between Problems and their Resolution;
- 3. Stratification of Samples;
- 4. The " Level of Safety Chosen" by the CPRT and Relationship to Appendix B and FSAR Commitments; and
- 5. Exemption from Appendix B.
' e .
- 1. The Role, Use and Design of Sampling Studies in the CPRT Program The role of sampling in the CPRT Program is to serve as one of several approaches for adding rigor and consistency to final judgments of the CPRT. Sampling can also help CPRT assure that a systematic process is used in the search for potential deficiencies and the detection of potential programmatic weaknesses and potential root causes. The CPRT Program does not use sampling alone to draw aggregate conclusions about design, construction or testing activities.
Consistent with the CPRT Sampling Policy, which is set forth in Section 1.0 of Appendix D to the Program. Plan (Rev. 1, 1/30/86) (a copy of which is attached for the convenience of the Board), action plans are not formulated to rely solely on sampling studies or to result in purely statistical statements; action plans, rather, use sampling techniques to augment analysis, knowledge of plant conditions and testing results, knowledge of program and procedure provisions, and knowledge of inspector or other personnel qualifications and performance and engineering judgment. The following information is i e included, either directly or by reference to Appendix D of the CPRT Program Plan, in each action plan that involves the use of random sampling:
- 1. Population definition, including any stratification (to achieve homogeneity),
or any engineering bias relative to the sample selection.
- 2. A statement of the statistical " screen" to be applied (i.e., the minimum deficiency rate that will be detected at the stated confidence level).
- 3. A statement of the accept / reject criteria, including the relevant test statistics and critical region.
Computations will be summarized, and conclusions will be presented in the results reports.2 1The Board quotes from Introduction to Statistical Analvsis (4th ed., 1983) at 85, where the authors make some general statements regarding statistical studies in the context of a specific discussion of the hypothesis on the mean test. The SRT has no quarrel with those general statements; it points out, however, that more commonly employed in the ISAPs employing statistical analysis is a test for the existence of a non-parametric population extreme, in this case the existence of a safety- related construction or design deficiency. This is a type of analysis described, briefly, in the same reference at 401:
"In quality inspection it is sometimes desired to guard against or to detect individuals at only one extreme of the observational scale.
Table A-25e gives samples sizes corresponding to
- 2. The Relationship Between Identified Problems and their Resolution once a potential problem area has been identified.
by either an external or internal source, a plan of action is formulated to confirm or rebut the existence of a programmatic problem and to resolve any actual problems encountered. Tasks are identified in the action plan that, when completed, will provide reasonable assurance that a programmatic problem does not exist, or if one is detected, that it is properly dispositioned. If any of these tasks require the probability . . . of finding (a given] .
proportion . . . of the population at the specified end of the distribution; thus the single extreme provides one-sided tolerance limits."
Id. Non-parametric tests are often used in engineering evaluations of construction quality because the reliability of the statistical inference does not depend upon the validity of any a priori assumption about population distribution:
"The level of significance for a non-parametric method is not affected by the population distribution. Therefore the experimenter can be sure of knowing the chance of rejecting the null hypothesis when it is true."
Iji. at 408.
s' a .
reinspections and/or documentation reviews within a particular population, the CPRT may deem sampling and inference to be an appropriate investigation tool.
Before any sampling is done, the objective of the sampling must be clearly stated with a logical relationship to the action plan objective. The appropriate population or stratum of items or quantity of material is identified. This population must be reasonably homogeneous with respect to the parameter or attribute thought to be the problem. Data are then gathered from the population of items, the nature of which (e.g., a particular population parameter) is unknown. These data are used directly to infer the value of the unknown parameter. In most cases the parameter is the percentage of the population which is deficient and the inference will be statistical. For most of the ISAPs, the CPRT has adopted a 95/95 one-sided upper-bound confidence limit or probability interval screen on the percentage deficient in order to make judgments about the existence of programmatic problems.2 If the 95 percent upper-bound confidence limit (or probability interval) calculated from the sample is equal to or greater than 5 percent, the population of items is said to have failed the screen and further investigation is necessary.
The statistical inference approach used in the CPRT effort can be related to hypothesis testing, though there is a large (perhaps infinite) number of hypotheses that could be associated with the upper-bound confidence limit approach. One hypothesis test is based on the assumption that any of the programmatic 2The "95/95" test (which is sometimes referred to as a "95/5" test, the two being interchangeable in this context) is believed to be one commonly employed in nuclear cases and has been accepted as providing the requisite reasonable assurance. See, e.g., Public Service Electric and Gas Co. (Salem Nuclear Generating Station, Unit 1), ALAB-650, 14 NRC 43, 54-55 (1981):
"Despite the Colemans' skepticism, Exxon's guaranty of 95 percent leak-tightness with a 95 percent confidence level amply satisfies the public health and safety standard of the Atonic Energy Act."
(Footnote omitted.)
- o 4
type of problems asserted to have existed with the CPSES QA/QC program would be expected to result in a deficiency rate of at least 5% or more, if it existed in fact; one would then have the null hypothesis state that the population deficiency percentage is less than 5 percent. The alternative hypothesis is that the percentage of deficient items in the population is equal to or greater than 5 percent. The level of signi'ficance would be 0.95. The test statistic for this particular hypothesis test is the number of deficient items found in a sample of 60, and the critical region is one or more deficient items in the sample. ,
Where this form of statistical investigation is employed, however, the CFRT is not attempting to test any particular hypothesis. Rather, it is using a confidence limit (or probability interval) statistical inference approach to screen potential programmatic problems in any given population. If a population passes the 95/95 sample screen, the CFRT is quite confident that the population of items is free from programmatic deficiencies. (Passage also indicates a low level of non-programmatic (or random) errors, as
_9_
L-
i well.) If a population does not pass the 95 f 95 sample screen, the deficiency rate is considered high enough for the CPRT to be concerned that a programmatic problem may exist, and that further evidence is needed to verify that-an actual problem exists and, if it does, to determine the extent of the problem. This signals the CPRT that further investigation is needed to determine the root cause and generic implications of such a deficiency. This would lead to sample expansion ;
as discussed in Attachment 4 to Appendix D.
- 3. Stratification of Samples The CPRT is indeed structuring its investigations carefully with regard to deficiencies and adverse trends of deviations detected or suggested by either an external or internal source. These identified problems, which are considered to be potential programmatic deficiencies, are pursued with more rigor than an initial self-initiated exploratory sampling.
Generally, after evaluating a deficiency or a trend of ,
deviations, a root cause will be hypothesized to exist in the entire population or'a subset (etratum) of the population.
k F
i
' e If a deficiency or adverse trend of deviations is detected or suggested by an external source or in the CPRT sample, and no root cause associated with a particular stratum of the population can be isolated, sample expansion into the entire population will continue until it is determined that either the deficiency was a random occurrence of very low frequency (i.e., no programmatic problem exists), or a potentially programmatic deficiency is identified in a stratum. If deficiencies continue to be detected, or two or more deficiencies are detected in the initial minimum CPRT sample and they cannot be associated with a specific root cause (stratum), 100 percent of the population will be inspected or reviewed with all deficiencies detected and dispositioned.
The CPRT requires sampling (either initial or expanded) in a stratum when it has been hypothesized that the stratum contains and bounds the deviations or deficiencies of the type already detected. Such a stratum may be suggested by external sources, by a deviation trend in the initial CPRT sample, or by a root cause evaluation originating inside or outside the population of items being inspected or reviewed. In L
l o 4
addition, unless otherwise justified, sample expansion in the general population, minus the identified stratum, is required to verify that the deficiency is not associated with the remaining strata.
In general, the CPRT has recognized the importance of stratification on the overall investigation effort and has spent considerable time and effort in identifying and defining homogeneous strata in order to enhance the inference process (see Appendix D for further discussion of stratification).
In a nuclear power plant, as with all complex systems, stratification into homogeneous populations for sampling investigations must be based on the particular information sought. Note that if carried to the extreme, this process could inadvertently end up with single items being contained in a given stratum.
Therefore, from a practical engineering point of view, certain characteristics are considered unimportant in determining the boundaries of a homogeneous stratum.
If, however, after a sample is found to contain deficiencies or an adverse trend of deviations asscciated with items possessing a certain characteristic and not with items that do not possess
this characteristic, further stratificatier. would be carefully considered.
As this discussion reveals, unstratified samples are not intended to be persuasive regarding the non-existence of already found (or subsequently found) deficiencies. In cases where programmatic deficiencies have been or may be found, corrective action will be taken, often involving 100% reinspection of the potentially affected item. In cases where identified deficiencies are not programmatic, a root cause analaysis is generally combined with a focused statistical investigation to determine the nature and scope of the problem.
- 4. "The Level of Safety Chosen" by the CPRT and Relation to Appendix B and FSAR Commitments The Senior Review Team respectful]y does not concur in the Board's suggestion that the CPRT program poses any "hard question . . . about the level of safety that must be assured,'" if by this statement the Board is suggesting the need to " calibrate," with some sort of mathematical precision, the CPRT sample reinspection efforts to some assumed " level of safety" that would be expected to result f rom t he implementation of an unquestioned QA/QC program per 10 C.F.R. App. B. The CPRT Program Plan is intended to stand on its own as a demonstration of a basis for reasonable assurance that, as designed and constructed, Comanche Peak Steam Electric Station is free from undetected and uncorrected safety signficant deficiencies that would preclude the facility from being operated safely.3 3
If, in fact, the results reveal no safety significant deficiencies in the original or first-l expanded sample in any particular area of
It is undeniable that absolute perfection of construction (or design) is neither achievable in connection with a project of this magnitude nor required.* A fortiori there is neither requirement nor assumption that a QA/QC program implemented in strict accordance with Appendix B will catch each and every construction (or design) error that might be made over construction or design, then the implementation of the Program Plan will also provide assurance that the Appendix B QA/QC program in that area was free of the sort of inadequacy that would tend to permit the occurence of undetected safety significant construction or design deficiencies.
- The establishment and effective implementation of a QA/QC Program that is fully compliant with Appendix B can be expected to substantially contribute to the reduction of undetected and uncorrected defects.
Due to the number and complexity of the activities associated with the design and construction of a nuclear power plant, however, it is impractical to expect that a "cero defect" standard (while a desirable goal) can be met. Accordingly, nuclear pcwer plant design and construction is predicated on a philosophy of " defense in depth," which utilizes principles of diversity, redundancy and conservative safety margins.
This philosophy assumes that defects can occur and, therefore, provides the capability to safely withstand the effects of such defects.
L_
the course of the projects -- that is to say, discovery of a safety-significant design or construction deficiency does not, by itself, negate the existence or implementation of a fully compliant Appendix B program.
The CPRT can and does make no claim of absolute perfection for itself, either. The Senior Review Team knows, however, of no way to quantify the level of detection of an Appendix B Program for random errors, to quantify the level of detection of either type of program for systematic errors (except to note that, qualitatively, the more wide-sweeping the hypothesized systematic error, the more closely the level of detection approaches 100%), or to " calibrate" one to the other.
5"We note that, in theory, a design quality assurance program will provide 100 percent review of the design work. The record is clear, however, that such a program can never assure that there will be no design errors. . . . Indeed, Appendix B only provides that the purpose of a quality assurance program is 'to provide adequate confidence that a [ safety-related) structure, system or component will perform satisfactorily in service.'" Pacific Gas and Electric Co. (Diablo Canyon Nuclear Power Plant, Units 1 and 2),
ALAB-763, 19 NRC 571, 593 n.86 (1984). There is, for this point, no valid distinction between design and construction.
u
CPRT is not simply a program of statistically-based sample reinspections. CPRT was formulated to respond to assertions of systematic or programmatic QA/QC breakdowns during the CPSES construction and design and to test, with reasonable assurance, the validity of the hypothesis that such a breakdown occurred. CPRT takes into account all that is known about program and procedure quality, inspector qualifications, the results of NDE and functionality testing and any other data that may be available about the extent to which the as-constructed hardware is free of safety-significant construction errors or design errors that will preclude achievement of the design bases set forth in the FSAR and applicable regulations and codes.
Exemption from Appendix B '
The Applicants respectfully disagree with the Board's legal assertion (Memo. at 4) to the effect that, in the absence of a demonstration that sampling programs yield the same " level of safety" that is assertedly implicit in Appendix B, then "the plant would appear to fail to meet Appendix B requirements and to require the granting of an exemption from Appendix B pursuant to 10 CFR 2.758." Compliance with the construction QA/QC program required by Appendix B is not, either in terms or as applied, a condition precedent to the granting of an operating license. The only aspect of Appendix B that is a prerequisite to 1
issuance of an operating license is the inclusion in the FSAR of "information pertaining to the managerial and administrative controls to be used to assure safe operation." 10 C.F.R. Part 50, Appendix B,
" Introduction." Cperational QA is not a subset of the sole remaining contention in these proceedings.
It is of course true that the failure to implement aspects of Appendix B applicable to the design and construction of the facility might be the grounds for enforcement action by the Staff, but no
, - n. . . . . . - - . - . , ,. .--
" exemption" is required (even assuming such failures) in order for an operating license to issue. The only relevant licensing standard applicable to a construction QA/QC contention (such as Contention 5 in this case) is that set forth in 10 C.F.R.
S 50.57(a)(3)(i) (i.e., that there is " reasonable assurance" or " adequate confidence" that the facility has been designed and constructed such that it is capable of being operated safely). See Pacific Gas and Electric Co. (Diablo Canyon Nuclear Power Plant, Units 1 and 2), ALAB-763, 19 NRC 571, 587-88 (1984), modified on a different coint, CLI-84-84, 20 NRC 285 (1984).6 8"This argument [that no statistically valid error rates could be determined for the design sample reinspection there involved] essentially overlooks the standard by which the applicant's program is to be judged. We must determine whether the nonseismic verification program provides ' adequate confidence' that the nonseismic design of the safety-related structures, systems and components is proper so that such structures, systems and components will perform satisfactorily in service. This qualitative standard is not numerically quantifiable into expressions of probability or errors or error rates . . . .
" Pointing to the Ccmmission's regulations, 10 C.F.R. 5 50.57(a)(1), the Governor and the t
See also Philadelphia Electric Co. (Limerick Generating Station, Units 1 and 2), ALAB-819, 22 NRC 681, 729 (1985); Louisiana Power & Light Co. (Waterford Steam Electric Station, Unit 3), ALAB-812, 22 NRC 5, 14-15, joint intervenors repeatedly assert in their proposed findings that the applicant's verification program, in order to be sufficient, must demonstrate that the design of Diablo Canyon meets its license application
.equirements or licensing criteria. The application requirements and licensing criteria for Diablo Canyon, like any nuclear power plant, are spelled out in the various documents comprising of the operating license application including, most prominently, the applicant's Final Safety Analysis Reports (FSAR). The FSAR is a multivolume descriptien of the entire facility containing literally thousands of so-called
' licensing criteria' ranging from safety significant ones to insignificant and extreme minor specifications or descriptions of details that have no safety implications. . . . In their proposed findings of fact, the Governor and the joint intervenors do not distinguish between safety significant and nonsafety significant licensing criteria. For example, the Governor and joint intervenors argue, relying on the staff's and the applicant's witnesses, that the nonseismic design does not meet licensing criteria because it is a virtual certainty that there remain undetected design errors in the unreviewed portions of the design. . . . But the witnesses relied upon by the Governor and joint intervenors all testified that not only was it likely thers remained some design errors, but that it was extremely unlikely any of the errors were safety significant. . . The central issue with respect to the proper design of Diablo Canyon, or any
44 (1985)7; Duke Power Co. (Catawba Nuclear Station, Units 1 and 2), ALAB-813, 22 NRC 59 (1985); Union other facility, is the conformance of the design to the significant and substantive safety requirements and licensing criteria. To conclude otherwise would ignore reality and substitute ' perfection' for the regulatory standards of ' adequate confidence' and ' reasonable assurance.'" Id. at 587-88 & n.68.
7"LP&L, however, cannot turn back the clock and enlarge the QA staff that oversaw construction at the Waterford site. Our focus, then, must be on whether any significant construction deficiencies resulted and remain as a consequence of LP&L's inadequate staffing, and whether LP&L has taken steps to prevent understaffing in the future." Id. at 21. See also id.
at 26 (trending inadequacies), 27 (audit inadequacies)
("There is little doubt that the implementation of i LP&L's audit program was lacking and led to the 1982 QA breakdown. . . Given these serious deficiencies, two pertinent questions arise. Are the auditing failures responsible for actual hardware or workmanship deficiencies that may remain unidentified and uncorrected? What assurance is there that these auditing failures will not recur?"), 31 (NCR deficiencies). Despite the failure to comply with various aspects of Appendix B, there was no sugge; tion by the Appeal Board that an exemption was required as a condition precedent to authorization of an operating license.
Instructive is the Appeal Board's resolution of the audit problemr " Audits are an important element of an applicant's overall QA program and are required by
- - , - . e .----, - w -
Electric Co. (Callaway Plant, Unit 1), ALAB-740, 18 NRC 343, 346 (1983). All that is required is reasonable assurance that the as-constructed facility is constructed such that it is capable of being operated safely, and the results of the implementation of a construction Appendix B QA/QC program is but one way of providing evidence in support of that assurance.
Sample reinspection programs are another accepted way of provided the same assurance. E.g., Waterford, ALAB-812, supra; Commonwealth Edison Co. (Byron Nuclear Power Station, Units 1 and 2), ALAB-793, 20 NRC 1591 (1984). In no case of a found or hypothesized breakdown of the construction QA program, however, has 10 C.F.R. Part 50, Appendix B, Criterion XVIII.
Nonetheless, as the staff has explained, through the systematic sampling of various work and the QA documentation for it, audits provide but a third level of assurance. The principal levels of assurance are provided by, first, quality craftmanship and, second,
- quality inspections. . . . But as to [the work for l which the audit program was determined to have been ,
deficient], the first and second levels of assurance were, in fact, provided. This is demonstrated by the absence of significant safety deficiencies in the l improperly audited Mercury work, as revealed by the i
major reinspection of that work that was undertaken by qualified personnel." 22 NRC at 28.
i i
O o*
an exemption been required as a condition precedent to authorization of an operating license.
Respectfully submitted, Robert A. Wooldridge Worsham, Forsythe, Sampels &
Wooldridge 2001 Bryan Tower - Suite 3200 Dallas, Texas 75201 Telephone: (214) 979-3000 Nicholas S. Reynolds William A. Horin Bishop, Liberman, Cook, Purcell & Reynolds 1200 Seventeenth Street, N.W.
Suite 700 Washington, D.C. 20036 Telephone: (202) 857-9800 Roy P. Lessy, Jr.
Morgan, Lewis & Bockius 1800 M Street, N.W.
Washington, D.C. 20036 Telephone: (202)872-5000 1
Thomas G. Dignan, Jr.
R. K. Gad III Ropes & Gray 225 Franklin Street Boston, Massachusetts 02110 Telephone: (617) 423-6100 R ./
j .
/
x '
By s 1-s Thomas G. Dignah, Jr.
R. K. Gad III Dated: January 31, 1986.
[
i l
Revision: 1 Page 1 of 12 APPENDIX D CPRT SAFTLING POLICY, APPLICATIONS AND GUIOELINES 1.0 POLICY STATEMENT The Senior Review Team (SRT) has determined that, in general, it is l unnecessary to examine an entire population of items or quantity of j material in ceder to datermine whether programmatic problers exist, ;
By sampling a portion, inferences can be made regarding the entire population. The basis for CPRT decisione on design, construction, testing and QA/QC adequacy will be supported by sound engineering evaluation techniques, which often may include principles of j sampling. Sampling and resulting inferences can be used as a '
powerful tool in identifying programmatic safety-significant deficiencies in programs and processes. Although the process of drawing inferences from sampling is not the sole means of reaching reasonable assurance that the plant design and construction are adequate, sampling may be a significant centributor to that evaluation. l It is also recognited that, for sampling to result in meaningful information about a process or program, the items in the population to be sampled must be similar (i.e., he=ogeneous) in the i significant traits or attributes associated with that process or >
program. Some populations will be homogeneous by virtue of the '
work process by which they were made (e.g., ASME pipe support welding), others will be similar by virtue of the design activity that created them (e.g., containment isolatien valve elesure time),
and se forth. Since sampling is utilized for a variety of purposes !
in the CPRT program it is essential that, when sampling it used, the population te be sampled is homogeneous and the objectives of the sampling are clearly tied to those cf the action plan under consideration.
There are two basic ways to sample: one 1.* to use judgcent to select from a population those items that are likely to be the most critical, the other is to randomly Felect items from the g6neral ,
population (e.g., tandem selection of welds in a support i structure). In the first method, called " biased sampline." the !
validity of one's inferences depend te a censiderable extent on the validity of the investigator's prejudgment. In the second methed, the samples are drawn randomly; and the resulting inferences depend on little or no bias from human prejudgment. Within the scope of the CPRT, both of these approaches are used to investigate various areas of interest and are justified within the context of their applications. In many cases, both approaches are used in the j investigation of a single area of interest ,
o-r. c r s. e mr ,4 v r.,ug uo Revision:
1 Page 2 of 12 APPESDlX D (Cont'd) 2.0 APPLICATIONS The pureosa of this appendix is to:
- Delineate the various applicatiens of sampling within the CpRT progran.
- Set forth consistent guidelines for the mechanics of selecting samples wherever random sampling technicues are used in 15APs and DSAPs (including the TRT issues, the Design Adequacy Program and Quality of Construction Program).
2.1 Ceclity of Construction (00C)
The construction process produces hardware by execution of a 4
number of relatively uniform construction activitfee.
Therefore, the construction process is inherently susceptible i
to isolated hardware discrepancies. The overall frequency of deficiercies relarive to the total number of opportunities is typically low, unless a programmatic problem exists.
To obtain c censistent sampling afproach in the QOC Reinspec tion / Document Review (Issue-Specific Action Plan VII.c), descrthed as the selt-initiated investigatien in Appendix B, the SRT believed that an initial sample screen should be based on a specific standard. The SRT has concluded that a 95/3 sample plan, when used in the context of hemogeneous populations cf attributes, would provide a reasonable screen to detect progrannatic or systematic deficiencies *. Such a screen vould ensure a sufficient initial sample size to evaluate the adequacy of the safety-significant attributes associcted with each of the homogeneous work activities (HWAs) in the VII.c investigaticn.
Accordingly, an initial random sceple of at least 60 itens i .e required for each homogeneous population (see Attachment 1 Table 1).
2.2 Other ISAP_3 Many of the other ISAPs (i.e., TRT issues) utilite snapling techniques to investigate specific areas of concern. In general, the SRT reeuires that the sample sizes in each of these cases be consistent, at a minimum, with that required by the use of a minimum 95/5 sample screen. An> exceptions to this general prir.cipal tre approved by,the SRT. based on a case-specific review, and are reflected in the associated ISAP. ,
I
- A deficiency rate as low as 5% in a peputation will be detected by
- e. 95/5 sampling plan with a probability or confidence level of 0.95.
Revision: 1 Pare 3 of 12 APPENDIX D (Cont'd) 2.0 APPLICATIONS (Cont'd) 2.3 Design Adequacy _ Program (DAP_)_
The focus of the Design Adequacy Program (CAP) is on the verification of the end products of the engineer 1rc and design process (i.e., designs represented by drawings, evaluations, or design specifications). In contrast to the construction process, where relatively few KWAs apply to large numbers of
- individual hardware items, the engineering and design process is characterized by a large number of homogeneous design activities (FDAs) with comparatively few design cutputs being covered by each one.
The important aspect of the HDAs is that they include items for which a high degree of correlation exists in the design criteria, methodology, and procedures. Accordingly, evaluation of the adequacy of each HDA can be based on evaluating a representative selection of items within nach HDA. The number of selected items will be r.ufficient to justify inferences and extrapolations that are appropriate for all items within each HDA. Attachcent 4 of Appendix A to tne CFRT Program Plan presents further details en the establishment of HDAs and the criteria for selecting iters for evaluation.
If, in the event the DAP uses statistically-based sarpling in the verification of any HDA, the sampling will be conducted in accordance with the provisions of this appendix.
3.0 GUIDELINF.S FOR RANDOM SAMPLING The purpose of the atenched guidelines is to:
- Aesist in the development of non-parametric sample screens for Issue-Specific Accien Plans (IFAPs) er Discipline Specific Action Plans (DSAPs) where randem sampling is used (Attachcent '),.
Out]'ne the use of one-sided tolerance lirits for evaluating special cases of parametric ettributes (Attachment 2),
- Outline the basic rethods to be used in generating randon samples from a population '
of items er attributes (Attachment 3),
- Cutline the methods to be used for expanding samples (Attachment 4). :
Rsvision: 1 Pcgs 4 of 12 APPENDIX D (Cont'd) 3.0 GUIDELINES FOR RAND 0M SAMP'.1NG (Cont'd)
These guidelines apply to sampling screens for r est ISAPs and DSAPs. If other types of sampling applicatione arise, they must be considered on a case-by-case basis.
m
- =
- Rcvision: 1 Page 5 of 12 APPENDIX D (Cont'd)
ATTACHMENT 1 CUIDELINES FOR SAMPLE INSPECTION OR REVIEk' 0F ATTRIUCTES Table 1 of this attachment is generally used by CPRT to determine the sample stres and corresponding detection numbers which'nre consistent with a 95 percent confidence level (or 0.95 probability) on the 5. 2.5, and I percent upper bound population percentage screens. Unle ss otherwise justified and specifically approved by the SRT, the number cf deficiencies ellowed in the sample screen will be no more char one (see Attachment 4 for discussion of saeple expansion where ore deficiency is identified). These saepling plans are based on the assumption of an infinire population size and are conservative when ecmpared to set:p:Inc plcns based un finite populations.
The minimum sacp}c size for a 95/5 screer is 60 with a detection nurber of zero (i.e., the critical rction is one or more detected itent). This
= cans that out of a random sample of 60 items inspected, if no 1*c 3 are found to belong to the classificerion of interest (e.g., deficient),
there is a 95 percent confidence (or 0.95 probability) thae los3 than 5 percent of the population will be in this classification. If itens belerging to the classification of interest are detected in a miniru sample ($.e., the number detected is in the critical region), the C5 percent upper-bound confidence limit (or 0.95 probability intervni) will be greater than 5 percent. It is still possible that the populatirn percentage is less than 5 percent, but based only en the initial tample evidence, the probability that this is so is less than 0.95.
A root cause evaluation of the deficiency is performed in order te isolate a potential *y deficient stratun from the population. If such a stratum is identified, sample expansien into that stratum is used r-verffy that indeed the deficiency is asscciated with the identiffed stratum. Sample augmentation in the remaining population (cinu= the potentially deficient stratum) is used to verify that the deficience is not associated with the remaining stratun. Sample expansion is curther discussed in Attachr.ent 4 to this appendix.
- RGvision: 1 Page 6 of 12 APPENDIX D (Cont'd)
ATTACHMENT 1
. (Cont'd)
TABLE I SAMPLING PLANS FOR DETECTING UPPER-BOUND POPULATION PERCENTAGES (p")
AT 95 PERCENT CONFIDENCE I.EVEL*
SAMPLE SIZE ** DETECTION CRITICAL p = 5.0% p = 2.5% p = 1.0% Nt?.BER REGION 60*** 120 300 0 1 er note 95 190 474 1 2 or more 126 252 63C 2 3 or mere 310 775 3 4 er r. ore 155 183 366 915 4 5 er more 210 421 1051 5 6 nr more
= or 0.95 probe.bility level.
- Sample sizes are determined free A. H. Bewker, and G. J. Lieberman, Engineering Statistics, 2nd Edition, Prentice-Hall, 1972, page 536, hote that these same sample plans may also be derived f rer. Bayes' theorem, and are therefore applicable for sample a:<pansion, using Bayes' theorem (see A. Boissonnade "CPRT Sampling 91nr.s-Addendum,"
Civil / Structural / Mechanical CPRT File No. 11.1-0C5, or Fox and Tiao, Pavenian Inference in Statistical Analysis, Addison 'a'esley, 1973).
- For populations of 100 or fewer items, the =inimum r.ampic size may be reduced to 45, with a detection number of zero. This is based on the hypergeometric distribution.
Reference:
Lieberman, and Owen, Tables of the Hypergeotr.etric Probebility Distribution, Standard University Press, 1961.
i I
I l
l
.. Revisien: 1 Page 7 of 12 AFpENDIX D (Cont'd)
ATTACHMENT 2 5AMPLING GUIDELINES FOR ONE-SIDED TOLERANCE LIMITS In . wore specia' cases ISAPs or DSAPs (or an evaluation of an adverse trend) may require the determination of a parametric tolerance limit of a particu)nr attribute associated with items of a population. The acceptable quality rf a population of iters or quantity of material is often specified by setting a ]cwer (upper) bound value based on the criterion that a certain percentage of the population fall above (below) this value (e.g., the concrete code specifies that at least 90 percent of the Je-day cylinder strengths fall above the required design strength). A lower (upper) bound population percentage is then inferred from.a sample, compared with criterion value and the population either accepted 6s ir., or corrective acticn taken. When n lower (upper) bound populatien percentage is specified in statistical terms, it is called a tolerance limit. A cre-sided tolerance limit has the property that a certain percentage of the population of values (e.g., 90 percent) may be expected to tall above er below this beund with some level of confidence (e.g., 95 percent confidecee).
A ene-sided tol.crance limit is defined as X - KS (X + KS), where X !s the sample averacn, and S is the sample standard deviation. The tolerance factor, K, is dependent upon the sample sire, the specified population percentage abcve (below) the limit, and the desired !cvel of confiderce (e.g., the 95 percent confidence le~ vel). 'Once the confidence level has been sele.cted and the population percentage specified, the sample size is only a function cf the tolerance factur V. To lowe* the tolerance factor, it is necessary to increase the sample size. The relationship for several population percentages is listed in Table 2.
For 1.APs c or DSAFs requiring the use of one-sided tolerance limits, sampling plans are developed by first determining, through engireering, mat erials, er other types of evaluetions, that the enderlying populatien cistribution is either normal or log-normale. Then, as a minir.um, a sample size of 50 is obesined. The actual sample site selected, however, takes into account the difficulty in obtaining the sample and how sensitive the resulting conclusions are to the actual tolerance limit.
Th ere is no unique seeple si7e to be used for any particular colercree limit problem. However it is obvious from Teble 2 that it becomes frereasingly difficult to lower the telerance fnetor as the sample size increases. From a practical point of view, serple sizes between 50 and 100 provide reasonable tolerance factors for the sampling effort.
- A goodners-of-fit test should he used to aid in evaluatirg the reasonableness of the assumed underlying distributions. Any tolerance lieit applications for which the underlying population distribution cannot be reGFonably assumed to be normal or log-normal will be handled on a case-by-case basin.
+'
.- - Revision: 1 Page 8 of 12 APPENDIX D (Cont'd)
ATTACHMENT 2 (Cont'd)
TABLE 2 ONE-SIDED TOLERANCE LIMIT ,
FACTORS, K, FOR 95 PERCENT CONFIDENCE LEVEL First* Fifth* Tenth
- Fiftieth **
(ninety-ninth) (ofnety-fifth) (ninetieth)
Sample Site Percentile Percentile Percentile Percentile 5 5.75 4.21 3.41 0.90 10 3.98 2.91 2.36 0.56 15 3.52 2.91 2.36 0.45 20 3.30 2.40 1.93 0.38 25 3.16 2.29 1.84 0.34 30 3.06 2.22 1.78 0.31 35- 2.99 2.17 1.73 0.28 40 2.94 2.13 1.70 0.26 50 2.86 2.07 1.65 c.24 2.77 1.99 1.5B (.20 70 100 2.68 1.93 1.53 0.17 2.52 1.80 1.42 C 10 300
Reference:
D. B. Owen, Handbook of Statistical Tables, Addiser 'r'enley, 1962, page 126. Note that the firrt percantile rteans that 09 percent ot the population falls above, one percent fall.* below.
- F.ef e rence : F. A. b'ebster, " Developing Sampling Plana for TRT 1ssues",
Civil / Structure.1/ftechanical CPRT, File No. !!.1-001, 3/12/85.
9
,~ > ..
- Revision: 1 Page 9 of 12 APPE!;DlX D (Cont'd)
ATTACHMENT 3 GUIDELISES FOR GENERATING RANDCM SAMPLES I
The procedure for generating a random sample begins by first deficing I the unit to be sampled (e.g., truckloads of concreta, conduit runs, conductor terminations, etc.), then determining the total number of these units or items in the population. Note that the population, se defined, may actually be a subpopulation which has certain specified i engineering attributes (i.e., a stratum). Each unit in the populatien j
(or stratum) must be assigned a unique sequential number 1 through N, where N is the total number at units. A table of randem digits or e random number gener.: tor is then used to develop a random sequence of units from the population. Table 3 outlines the corplete procedure.
9
mv neg wa
- Revieten: 1 Page 10 of 12 APPEt: DIX D (Cont'd)
ATTACliMENT 3 (Cont'd)
TABLE 3 l'ROCEDURE FOR CENERATING A FANDOM SAMPLE FROM A POPIT ATLOY
- 1. Deternine population size, N, and number each item sequentially, 1, 2, ...N.
- 2. Start at a randem position in a table of random digits or use a random seed in a random nurber generator and perform the folios. ire steps for each random five digit decimal fraction in sequence, until desired sample size is obreined.-
- 3. If using a table of random digits, place a decfral point in frcnt of each set of five digits
- and multiply by the populaticn uize.
If uning a rander, number generator which produces five dicit decimal fractions *, simply multiply by the population site.
- 4. Retain only the ig d add
- 2 h i r.
will define the i'geger part item to of bethe aboveinproduct included an vatple.
the random
- 5. It is usually a gced idea to generate a longer liste of rar.dcr.ly selected items in case a particular item is inacce.sfble in rle field, or in case the same item is scleeted more than once.
Example: Generate a sample of 300 items from a population n*
size 3791.
RN1 - .04146** x 3791 = 157.17486 == 1.57 + 1 - 158 x 3791 - 868.30712 == 888 + . - 159 P.N 2 = .23437 Rh3 - .74381 x 3791 - 2,819.78371 =- 2.819 + 1 - .',6:0 2,245.06810 == 2,245 + -
-,246 RN300 = .59221 x 3791 - .
\
- A five digit randoe decimal fraction is only useful on petulationc of 10,000 items or less. Additional randen digits rus; be used in ,
the decimal fracticn if larger populationt ere sanpled.
Reference:
The Rand Corporatien, A Million Random Digi _ts, Free Press, 1955, p. 355.
.
- RGvision: 1 Pcg2 11 of 12 APPENDIX D (Cont'd)
ATTACHMENT 4 GUIDELINES ON SAMPLE EXPANSION The primary reason for continuing a sampling investigation is to i determine if detected deficiencies are systematic or random, and aid in their evaluation. It may also be used as an aid in evaluating adverse trends and their root cause(s).
If the 95 percent upper-bound confidence limit (or probability interval) calculated from the sample is greater than 5 percent, the population of items is said to have failed the screen and further investigation is necessary. If one deficiency is detected in the initial minimum sample of items (or one or more different attributes in the case of VII.c) and no root cause can be identified, sample expansion in the entire population, including all attributes, will continue until it is determined that either the deficiency is a random occurrence of very low frequency, or a trend or programmatic deficiency is identified in the If a deficiency is population (i.e., a potentially deficient stratum *).
detected in the initial sample and a root cause that implicates only a subset of actributes is identified, a reduced set of attributes will be considered in the sample expansion. If a sample is found to contain deficiencies or an adverse trend associated with items possessing a certain characteristic and not with items that do not possess this '
characteristic, a subset of items possessing these certain characteristics will be considered in the sample expansion. If deficiencies contfaue to be detected in the expanded sample, or two or more deficiencies of the same type are detected in the initial minimer sample, and they cannot be associated with a specific stracun, 100 percent of the ,;opulation will be inspected or reviewed. ,
Sample expannicn into a stratum will be required when it has been determined or lysothesized that the stratum contains and bounds the ,
adverne trends or deficiencies o. -he type detected in the initfal sample. Such a stratum may be identified by an adverse trend in the initial sample or by a root cause evaluatice originating inside or outside the population of items being inspected or reviewed. Sample expansion into a stratum proceeds in one of the following ways:
- The stratum is identified ccepletely, separated from the ;
general population, items numbered sequentially from one to the total number and then rendcoly selected, or i
- As used in the CPRT program, stratum will fefer to either
- 1) a subset of itens in the population,
- 2) a set of attributes of items in the population, i
or 3) a set of attributes for a subset of items in the !
population.
r'* Revision: 1 Paga 12 of 12 APPENDIX D (Cent'd)
ATTACHMENT 4 (Cont'd)
- The strctum is left within the general population and random snepling is continued in the general population until the required sample size is obtained in the stratum.
11 there is no idertified root cause for the initial deficiency or adverset trend, sample expansion in the general population is required to verif> that the deficiency detected in the initia1' sample was a randem occurrence of low frequency. When sample expansion is performed only for a stratum, the sample in the general population (minus the identified stratum) will be augmented with additional items to bring the gancral population sample back to the einimum 95/5 sample size.
For example, if one deficiency is found in an initial sarple using the einimum 95/5 corple screen, and no stratum can be identified, then an additional 35 randomly selected items is needed (note: this is based on i Bayes' theoremi. If a deficiency is found in an initial minimum 95/5 i sample., and a stratum is identified and removed for separate -
investigation, encugh additional items must be randomly selected from the general population minus the identified stratum to bring the total sample beck up to the minimum 95/5 sample size. In addition, the sample uiza in the stratum must total 95. Any items that were selected from the stratun in the initial sample are included as part of the sample expansion in the stratum. If no more deficiencies are detected, then the sample will pass the 95/5 screen and the conclusion will be made that the deficiencies are random and of very low frequency in the population (i.e., there was no progran=atic breakdown).
Generating an expanded sample in the general population follows the same ru,le s tor generatinF the initial random sample. The sampling will start whe-e the initial sample ended (see Table 3 of Attachment 3). Table 1 '
I in Attecha.ent I should be used as a guide for sample expansion in those cases where ISAP sarple plans deviate from the general minimum 95/5 i sample screen.
e
~ __w
~
r i Q3 4 4 t-
-e e DOaxny FEB r4193 p r CERTIFICATE OF SERVICE ( gg og D
, , MTkW b /j. .g\
I, Robert K. Gad III, one of the attorneys for the Ap ' nts herein, hereby certify that on January 31, 1986, I made service of the within " Applicants' Memorandum in Response to Board's Memorandum (Statistical Inferences from CPRT Sampling)," by mailing copies thereof, postage prepaid, to:
Peter B. Bloch, Esquire Mr. James E. Cummins Chairman Resident Inspector Administrative Judge Comanche Peak S.E.S.
Atomic Safety and Licensing c/o U.S. Nuclear Regulatory Board Commission
. U.S. Nuclear Regulatory P.O. Box 38 Commission Glen Rose, Texas 76043 Washington, D.C. 20555 Dr. Walter H. Jordan Mr. William L. Clements Administrative Judge Docketing & Services Branch 881 W. Outer Drive U.S. Nuclear Regulatory Commission Oak Ridge, Tennessee 37830 Washington, D.C. 20555 Chairman Chairman Atomic Safety and Licensing Atomic Safety and Licensing Appeal Panel Board Panel U.S. Nuclear Regulatory U.S. Nuclear Regulatory Commission Commission .
Washington, D.C. 20555 Washington, D.C. 20555
! Stuart A. Treby, Esquire Mrs. Juanita Ellis i Office of the Executive President, CASE Legal Director 1426 S. Polk Street
! U.S. Nuclear Regulatory Dallas, Texas 75224
. Commission l 7735 Old Georgetown Road l Room 10117 i Bethesda, Maryland 20814 4
i i
,,.=
~
i g<
1 i
l Renea Hicks, Esquire Ellen Ginsberg, Esquire j Assistant Attorney General Atomic Safety and Licensing
. Environmental Protection Division Board Panel
, P.O.' Box 12548, Capitol Station U.S. Nuclear Regulatory Commission Austin, Texas 78711 Washington, D.C. 20555 i Anthony Roisman, Esquire Joseph Gallo, Esquire
- Executive Director Isham, Lincoln & Beale I Trial Lawyers for Public Justice 1120 Connecticut Avenue, N.W.
I 2000 P Street, N.W., Suite 611 Suite 840 Washington, D.C. 20036 Washington, D.C. 20036 Dr. Kenneth A. McCollom Mr. Lanny A. Sinkin i Administrative Judge Christic Institute Dean, Division of Engineering, 1324 North Capitol Street
- . Architecture and Technology Washington, D.C. 20002 Oklahoma State University Stillwater, Oklahoma 74078
)
i Ms. Billie Pirner Garde Mr. Robert D. Martin Citizens Clinic Director Regional Administrator, 4
, Government Accountability Project Region IV 1901 Que Street, N.W. U.S. Nuclear Regulatory Commission
! Washington, D.C. 20009 Suite 1000 j 611 Ryan Plaza Drive Arlington, Texas 76011 i
Elizabeth B. Johnson Geary S. Mizuno, Esquire Administrative Judge Office of the Executive Oak Ridge National Laboratory Legal Director P.O. Box X, Building 3500 U.S. Nuclear Regulatory Commission Oak Ridge, Tennessee 37830 Maryland National Bank Bldg.
Room 10105 7735 Old Georgetown Road Bethesda, Maryland 20814 Nancy Williams Cygna Energy Services, Inc.
101 California Street Suite 1000 San Francisco, California 94111 f._ _ s, g
~~~ ~ _
~.
("_ '
Robert K. Gap',III