ML20199E076
| ML20199E076 | |
| Person / Time | |
|---|---|
| Site: | Comanche Peak  |
| Issue date: | 02/28/1985 |
| From: | TEXAS UTILITIES ELECTRIC CO. (TU ELECTRIC) |
| To: | |
| Shared Package | |
| ML20199D912 | List:
|
| References | |
| FOIA-86-36 PROC-850228, NUDOCS 8606230176 | |
| Download: ML20199E076 (15) | |
Text
.-
O Guid2 for Sampling Plans DRAFT: C Random Scaplos R: vision: 0 02/28/85 Page 1 of 14
1.0 INTRODUCTION
The purpose of this document is to provide guidance in the development of sampling plans for CPRT Action Plans requiring sampling, and to outline the basic method to be used in generating random samples from a population of items. What is discussed in this guide applies to sampling plans for CPRT Action Plans related to either classifying attributes in a population or tolerance limits on the value of an attribute in a population. Most of the CPRT Action Plans requiring sampling fall into one of these two categories. If other types of sampling problems arise, they must be dealt with on a case-by-case basis.
t
2.0 BACKGROUND
\
The CPRT is addressing a number of specific design and construction concerns regarding whether safety requirements have been appropriately implemented in CPSES. In conducting its review, the CPRT is, in part, assessing the degree to which the TUCCO engineering, construction and QA/QC programs for CPSES provide the necessary confidence that safety requirements have been met. It is clear that many aspects of the design and construction programs at
, CPSES have been successful and that confidence exists that safety requirements have been implemented. For any specific system or activity, his judgement may be based upon such things as previous inspections of installed hardware, successful completion of start-up testing of safety systems, past NRC inspections and T evaluations, and general industry experience.
8606230176 860609 PDR FOIA GARDE 86-36 PDR
. - 4-
(N .l
, Guido far Sampling Pitns DRAFT: C Random Scxplaa Rr,visien: 0 02/28/85 Page 2'of 14
2.0 BACKGROUND
(Cont'd) -
Notwithstanding this judgement and in most situations in light of it, the only practical vehicle to further investigate whether a safety- significant deficiency exists is to reinspect or review the outputs of the design, construction and quality programs. In so doing, the SRT recognizes that in order to meet the CPRT objective of providing reasonable assurance that no safety- significant deficiencies remain uncorrected, complete reinspection, reanalysis, or reevaluation is not practical and largely unjustified at this time. Sampling is therefore used as a tool to aid in investigating and resolving concerns related to:
project design and construction, practices the effectiveness of the QA/QC program in existence at CPSES, and .
safety-significant construction or design errors that remain undiscovered and uncorrected.
In selecting sample sizes, the CPRT will generally apply a 95/5 criteria, which means that for classifications of interest (e.g.,
deficient or non-deficient), a population percentage as small as 5 percent will be detected with 95 percent confidence. Samples from populations of design activities will not necessarily employ the 95/5
, i, level of sampling effort when evaluating the adequacy of design n
e ]
. s . . , . .
- n o ~
. Guid2 for Sampling Pirns DRAFT: C Random Samplsa Rzvision: 0 02/28/85 Page 3'of 14
2.0 BACKGROUND
(Cont'd)
- assumptions since these assumptions are believed to apply to much more than 5 percent of the populations of design activities. The 95/5 sampling criteria represents a minimum requirement adopted for CPRT sampling activities associated with potential deficiencies which may occur infrequently. A more stringent criterion may be applied, on a case-by-case basis, if approved by the SRT.
The substantial engineering experience and judgement available to the CPRT will typically be employed to focus the sampling efforts on areas and populations where deficiencies are more likely to be safety significant. Within these focused populations the technique of random sampling will ensure that potential biases do not influence the sample selection. Based upon the foregoing, the CPRT will generally select samples as follows:
either a random sample will be selected from a population identified by engineering ju6gement to contain deficiencies which are more likely to be safety significant, or two separate random samples will be selected from a population, one based on the general population of items or attributes of items under review and the other based on a
~
subset of this population in which any deficiencies found are more likely to be safety significant.
~' '
e
C' .
Guid2 for Sampling Plans DRAFT: C Random Stuplca Rsvicinn: 0 02/28/85 Page 4'of 14
2.0 BACKGROUND
(Cont'd)
- Each specific Action Plan using sampling will provide a clear statement of the sampling approach to be utilized, the criteria for delineating the classification of items inspected or reviewed and the criteria for expansion of the sample where appropriate. Sample expansion will be triggered by the identification of deficiencies with potential for being safety significant.
The objective of sample expansion will be to refine the estimate and confidence level of the population percentage of items belonging to the classification labeled deficient, and to determine if the deficiency is isolated in a specific subset of the
( population. Therefore, if deficient items are detected in the general population and, through a root-cause evaluation, cannot be isolated to specific subset of the population, an engineering evaluation will be conducted to determine the safety significance of the type of deficiency detected which considers this type of deficiency in more critical locations. This may include a sample expansion to refine the estimate of the population percentage through the use of Bayes Theorem to estimate the likelihood of this type of deficiency occurring elsewhere. If the deficiency is determined to be isolated in a specific subset of the population, the sample will be expanded into that specific subset to assure that similar deficiencies do not exist elsewhere in the subset and j to disposition any that are detected. In addition, the sample will be expanded in the rest of the population in order to meet the 95/5 criterion.
. Guida fer Sampling Pltn3 DRAFT: C Random Samplos R1 vision: 0 02/28/85 Page 5 'of 14 3.0 SAMPLING PLANS FOR INSPECTION OR REVIEW OF ATTRIBUTES -
(
Sampling inspection is concerned with sorting items, or attributes associated with items into different classifications (e.g.,
deficient or non-deficient). The intent of this type of sampling is to establish, with some level of confidence, a bound on the population proportion or percentage of items or attributes belonging to one classification or another. It is therefore essential that the sample be drawn in a random manner. Note that a sample is random if every item in the population has an equal chance of being selected. This may easily be accomplished by using a table of random numbers or a random number generator on a computer to generate the sample (see Section 4.0). Human attempts to randomize a sample, without such aids often result in biased samples.
If sampling is to be used rather than 100 percent inspectiota or review, certain risks must be assumed in conjunction with the I inferred population percentage. After observing the sample result, we can only make confidence level or probability statements about the population percentage. One method cf displaying this risk for a specific sample plan [i.e. sample size and detection (acceptance) number] is to plot the probability of observing the detection l number or fewer items from the sample belonging to the l
classification of interest, given the population percentage is known. A plot of this probability as a function of the population
( percentage is known as the operating characteristic (OC) curve.
... /
o
- Guida far Sa pling Plcna DRAFT: C Random Sampisa R;visicn: 0 02/28/85 Page 6 'of 14
{ 3.0 SAMPLING PLANS FOR INSPECTION OR REVIEW OF ATTRIBUTES - (Cont'd)
One minus this probability is defined as the confidence level for a given upper bound population percentage . In other words, a 95/5 sampling plan is interpreted as meaning that the confidence level is 95 percent that the population percentage belonging to the classification of interest is less than 5 percent. By specifying a confidence level and upper bound percentage, a number of sampling plans can be developed. The minimum sample size corresponding to a given confidence level and upper bound percentage is dictated by specifying a detection number equal to zero.
Another method of representing the risk ascociated with sampling is
- ( to determine the probability that the population percentage is less than any specified value. This probability is based on Bayes' Theorem in conjunction with the observed sample outcome and is defined as the confidence level. In other words, a 95/5 sampling plan is interpreted to mean that there is a 0.95 probability that the population percentage is less than 5 percent. This interpretation is particularly useful when sample expansion is desired to update the probability distribution on the population percentage.
- 5 e
O
Guida far Sampling Plenn DRAFT: C Randon Sampica Revisign: 0 02/28/85 Page 7 of 14 3.0 SAMPLING PLANS FOR INSPECTION OR REVIEW OF ATTRIBUTES - (Cont'd)
An example of a set of generally accepted sampling pl is which are based on OC curves is the MIL-STD 105D.* This standard is oriented toward the manufacture of large quantities of similar items which are separated and inspected by lots. By using the Limiting Quality Table VII-A of the standard to determine the sample size code letter and AQL consistent with the desired percentage beund, MIL-STD 105D single or double sampling plan tables can be used to determine a limited number of appropriate sample sizes and corresponding detection (acceptance) numbers. For example, specifying a 95 percent confidence level (i.e., 0.05 probability of acceptance) and a 5 percent bounding population percentage, and going to Table VII-A, a sample size code letter, K, and an AQL of 0.65 is found. Taking this information into Table II-A, a single sample pIr? with a sample size of 125 and detection (acceptance) number of 2 is indicated.
As an alternative to using MIL-STD 105D to develop sampling plans.
Table I may be used to determine the sample sizes and corresponding detection numbers which are consistent with a 95 percent confidence level on the 5, 2.5, and 1 percent upper bound population percentages. These sampling plans are based on the assumption of an infinite population size and are conservative when compared with w
Reference:
Military Standard 105D, U.S. Government Printing Office e
. . 4 . , . .
- . e .
Guida for Sampling Plens DRAFT: C
. Rcndom Samp12a
- R visitn: 0 i 02/28/85 Page 8 of'14
{ 3.0 SAMPLING PLANS FOR INSPECTION OR REVIEW OF ATTRIBUTES - (Coht'd) sampling plans based on finite populations. Note that for a 5 percent upper bound percentage at the 95 percent confidence level the minimum sample size is 60 with detection a number of zero.
This means that out of a random sample of 60 items inspected, if no items are found to belong to the classification of interest (e.g.,
deficient) there is a 95 percent confidence that no more than 5 percent of the population will be in this classification. If items belonging to the classification of interest are detected in the minimum sample, the confidence level of 95 percent on the 5 percent upper bound population percentage no longer holds. It is still possible that the population percentage is less than 5 percent, but
( the confidence level that this is true is lower than 95 percent, based on this sample size. At this point, the sample may be expanded (increased) in order to refine the confidence bounds on the population percentage for engineering evaluation purposes, or to detect all such items in the population belonging to this classification for disposition. In the laccer case, 100 percent inspection or review is required.
It is suggested that Table I be used as a guide, even for small populations. Use of MIL-STD 105D for small populations will generally result in sample sizes which are even larger that those listed in Table 1.
1
.. ./
\ e e *
(. _ _ _
. Guids fcr Sampling Pirna DRAFT: C
. Randon Sampica R visicn: 0 02/28/85 Page 9 of 14
[ .
TABLE 1 -
SAMPLING PLANS FOR UPPER BOUND POPULATION PERCENTAGE DETECTED (PD) 95 PERCENT CONFIDENCE LEVEL
. SAMPLE SIZE NUMBER OF DETECTED 5.0 PD 2.5 PD 1.0 PD ITEMS IN SAMPLE 60** 120 300 0 95 19.0 474 1 126 252 630 2 155 310 775 3 183 366 915 4 210 421 1051 5 237 474 1184 6 263 526 1315 7 339 67t 1696 10 t
Reference:
A. H. Bowker, and G. J. Lieberman, Engineering Statistics, 2nd edition, Prentice-Hall, 1972, page 538.
- For populations of 100 or fewer items the minimum sample size may be reduced to 45, with a detection number of zero. This is based on the hypergeometric distribution.
Reference:
Lieberman and Owen, Tables of the Hypergeometic Probabiliev Distribution, Stanford University Press, 1961.
4.0 SAMPLING PLANS FOR TOLEPANCE LIMITS The acceptable quality of a population of items or quantity of material is often specified by setting either an upper or lower bound value based on a
( certain percentage of the population falling below or above this bound.
When an upper or lower bound is specified in statistical terms it is called i a tolerance limit. A tolerance limit has the property that a certain tercentage of the population of values (e.g. 90 percent) may be expected to fall above or below this bound with some level of confidence (e.g. 95 percent confidence). Tolerance limits are usually based on the assumption that the underlying population distribution is either normal or log-normal.
There exist tolerance limits which are independent of the form of the underlying population distribution, however they are of. limited practicability since they require very large samples in order to make reasonable confidence statements. Sampling plans outlined here are based l
on the assumption that the underlying population distribution is reasonably 5,
,x approximated by the normal or log-normal distribution.
3 l n I e
. ~
. , , , . - _ ,, _ -,,_._._...-,..r ..
~
~
Guida fer Sampling Plcro DRAFT: C Randon Simp 1Go R4 vision: 0 02/28/85 Page 10'of 14 4.'O SAMPLING PLANS FOR TOLERANCE LIMITS - (Cont'd) -
A tolerance limit is defined as X t KS, where" X is the sample average and S is the sample standard deviation. The tolerance factor, K, is dependent on the sample size, the specified population percentage above or below the limit, and the desired level of confidence (e.g. the 95 percent confidence level). Once the confidence level has been selected and the population percentage specified, the sample size is only a function of the tolerance factor, K. To lower the tolerance factor, it is necessary to increase the sample size. The relationship for several population percentages is listed in Table 2.
Note that there is no unique sample size for any particular problem.
However, it is obvious from Table 2 that it becomes increasingly difficult to lower the tolerance factor as the sample size increases. From a practical point of view, sample sizes between 50 and 100 provide reasonable tolerance factors for the sampling effort.
It is recommended that, for CPRT Action Plans related to tolerance limits, sampling plans be developed by first determining through engineering, materials, or other types of arguments whether it is reasonable to assume the underlying population distribution to be normal or log-normal. If this assumption is reasonable, then it is recommended that, as a minimum, a sample size of 50 be obtained. The actual sample size selected, however, should take in.co account the difficulty in obtaining the sample and how sensitive the resulting conclusions are to the actual tolerance limit.
f .
Any tolerance limit problems, for which the underlying population ,
distribution cannot be reasonably assumed to be normal or log-normal, must be handled on a case-by-caso basis.
I u* .
6 '
- +
. e .
Guida fcr Sampling Pirne DRAFT: C
, Random Scapico Revisign: 0 02/28/85 Page 11'of 14 4.0
[ SAMPLING PLANS FOR TOLERANCE LIMITS - (Cont'd) -
TABLE 2 SAMPLE SIZE VERSUS TOLERANCE LIMIT FACTORS, K. FOR 95 PERCENT CONFIDENCE LEVEL (Tolerance Limit = X 1 KS)
First Fifth Tenth Fiftieth Sample Size Percentile
- Percentile
- Percentile
- Percentile **
5 5.75 4.21 3.41 0.90 10 3.98 2.91 2.36 0.56 15 3.52 2.57 2.07 0.45 20 3.30 2.40 1.93 0.38 25 3.16 2.29 1.84 0.34 30 3.06 2.22 1.78 0.31 35 2.99 2.17 1.73 0.28 40 2.94 2.13 1.70 0.26 50 2.86 2.07 1.65 0.24 70 2.77 1.99 1.58 0.20 l
100 2.68 1.93 1.53 0.17 I
300 2.52 1.80 1.42 0.10 2.33 1.65 1.28 0.00
Reference:
D. B. Owen, Handbook of Statistical Tables, Addison-Wesley, 1962, paga 126.
- Based on approximate relationship K 5.0 GENERATING RANDOM SA.T LES It is generally impractical or undesirable from a time and expense point of view to examine an entire population of items or quantity of material. One may, however, sample part of it and, on the basis of this limited investigation, make inferences regarding the entire population. The question then is how the sample should be selected.
Guida f&r Sampling Pltna DRAFT: C Randon Scapica Revicisn: 0 02/28/85 Page 12'of 14 j 5.0 GENERATING RANDOM SAMPLES (Cont'd) -
There are basically two ways to select samples, one is to use engineering judgement and select items that are likely to be the most critical, the other is to randomly sample the population. Inferences based on the former appr ach must also be based on engineering judgement or arguments.
Inferences based on random sampling are statistical. In either case, the inferences cannot be 100 percent certain without 100 percent sampling.
The procedure for generating a random sample begins by first defining the attribute and unit si:e of interest (e.g., the concrete strength which is representative of a truck load of concrete, the seismic capacity of a conduit run, the functionality of a conductor termination, etc.), then i determining the total number of these units or items in the population.
Note that a population, so defined, may actually be a subpopulation which has certain specified engineering attributes (i.e. a focused population).
Each unit or item in the population must be assigned an unique sequential number, I through N, where N is the total number of units in the population. A cable of random digits or a random number generator is then used to develop a random sequence of units from the population. Table 3 outlines the complete procedure.
\
, Guide f.;r Sampling Pirns DRAFT C Randon Samplec Revicien: 0 02/28/85 Page 13'of 14 TABLE 3 -
f Procedure For Generating a Random Sample From a Population
- 1. Determine population size, N, and number each item sequentially, 1, 2
.....N.
- 2. Start at a random position in a table of random digits, such as that referenced below, or use a random seed in a random number generator and perform the following steps .for each random five digit decimal fraction in sequence, until the desired sample size is obtained.
- 3. If using a table of random digits, place a decimal point in front of each set of five digits
- and multiply by the population size. If usir.g a random number generator which produces five digit decimal fractions *, simply multiply by the population size.
- 4. Retain only the integer part of the above product and add 1. This will define the 1 3d! item to be included in the randem sample.
- 5. It is usually a good idea to generate a longer list of randomly selected items in case a particular item is inaccessible in the field, or in case the same iter. is selected more than once.
Example: Conerate a sample of 300 items from a population of size 3791.
s .
- 6 . q * .
= . - . .
Guida far Sampling Plena DRAFT: C
. Random fiamp103 R visign: 0
/ 02/28/85 Page 14'of 14
[ Random Digits ** Population Size Random Sample
. RN g = .04146 x 3791 = 157.17 == 157 + 1 = 158 RN 2
= .23432 x 3791 = 888.31'== 888 + 1 = 889 RN = .74381 x 3791 = 2,819.78 == 2,819 + 1 = 2,820 3
R!l 300
= .59221 x 3791 = 2,245.07 == 2,245 + 1 = 2,246 A five digit random decimal fraction is only useful on populations of 100,000 items or less. Additional random digits must be used in ttie decimal fractica if larger populations are sampled.
Reference:
The Rand Corporation, A Million Random Digits, Free Press, 1955, P. 355.
t.
l
J ,
OFFICE MEMORANDUM *
\ -
TO: Review Team Leaders FROM: Senior Review Team DATE: January 30, 1985
SUBJECT:
Developing Sampling Plans and Random Samples for TRT Issues The Program Plan requires that the third party engineering statistics consultant.
review sampling plans for each Action Plan that uses sampling. Fred Webster has developed the attached guide to air! in the preparation of sampling plans.
Review Team Leaders should use the attached guide in the development of sampling plans. Any difficulties encountered in its use should be brought to the ettention of John Reed or Fred Webster of Jack Benjamin & Associates, the third party engineering statistics consultant.
L
, gg $15'J4
~ _ -
- - - - _ _-____ _- N'JO