ML051150046

Subject: Request for Additional Information (RAI) on the Expert Elicitation Process Used in Electric Power Research Institute (EPRI) Product No. 1009325, Risk Impact of Extended Integrated Leak Rate Testing (ILRT)

ML051150046
Person / Time
Issue date:	04/22/2005
From:	Tschiltz M NRC/NRR/DSSA/SPSB
To:	Pietrangelo A Nuclear Energy Institute
References
Download: ML051150046 (11)

Text

April 22, 2005 Mr. Anthony Pietrangelo Senior Director, Risk and Performance Based Regulation Nuclear Energy Institute Suite 400 1776 I Street, NW Washington, DC 20006-3708

SUBJECT:

REQUEST FOR ADDITIONAL INFORMATION (RAI) ON THE EXPERT ELICITATION PROCESS USED IN ELECTRIC POWER RESEARCH INSTITUTE (EPRI)

PRODUCT NO. 1009325, RISK IMPACT OF EXTENDED INTEGRATED LEAK RATE TESTING (ILRT)

Dear Mr. Pietrangelo:

The Nuclear Regulatory Commission (NRC) staff has completed its review of the expert elicitation process and data which forms the basis for EPRI Product No. 1009325, Risk Impact Assessment of Extended Integrated Leak Rate Testing Intervals, referenced in Draft Revision 1 to Nuclear Energy Institute (NEI) 94-01, Industry Guideline for Implementing Performance-Based Option of 10 CFR Part 50, Appendix J. The staff comments and RAIs are enclosed. Please give priority attention to the first two items, since satisfactory resolution of these items is necessary for the staffs acceptance of the report. Additional RAIs associated with the use of the expert elicitation process in the risk model and the referenced reports will be provided at a later date, as we have previously discussed.

If you would like to further discuss these comments and RAIs before responding formally to us, please contact me at (301) 415-3183 or Lynn A. Mrowca at (301) 415-4061.

Sincerely,

/RA/

Michael D. Tschiltz, Branch Chief Probabilistic Safety Assessment Branch Division of Systems Safety and Analysis Office of Nuclear Reactor Regulation

Attachment:

As stated cc via e-mail: Rich Luckett, Nuclear Energy Institute

April 22, 2005 Mr. Anthony Pietrangelo Senior Director, Risk and Performance Based Regulation Nuclear Energy Institute Suite 400 1776 I Street, NW Washington, DC 20006-3708

SUBJECT:

REQUEST FOR ADDITIONAL INFORMATION (RAI) ON THE EXPERT ELICITATION PROCESS USED IN ELECTRIC POWER RESEARCH INSTITUTE (EPRI)

PRODUCT NO. 1009325, RISK IMPACT OF EXTENDED INTEGRATED LEAK RATE TESTING (ILRT)

Dear Mr. Pietrangelo:

The Nuclear Regulatory Commission (NRC) staff has completed its review of the expert elicitation process and data which forms the basis for EPRI Product No. 1009325, Risk Impact Assessment of Extended Integrated Leak Rate Testing Intervals, referenced in Draft Revision 1 to Nuclear Energy Institute (NEI) 94-01, Industry Guideline for Implementing Performance-Based Option of 10 CFR Part 50, Appendix J. The staff comments and RAIs are enclosed. Please give priority attention to the first two items, since satisfactory resolution of these items is necessary for the staffs acceptance of the report. Additional RAIs associated with the use of the expert elicitation process in the risk model and the referenced reports will be provided at a later date, as we have previously discussed.

If you would like to further discuss these comments and RAIs before responding formally to us, please contact me at (301) 415-3183 or Lynn A. Mrowca at (301) 415-4061.

Sincerely,

/RA/

Michael D. Tschiltz, Branch Chief Probabilistic Safety Assessment Branch Division of Systems Safety and Analysis Office of Nuclear Reactor Regulation

Attachment:

As stated cc via e-mail: Rich Luckett, Nuclear Energy Institute DISTRIBUTION: ADAMS PUBLIC SPSB R/F SCBlack (RidsNrrDssaDpr) OGC (RidsOgcRp)

ABuslik HAshar RPalla JPulsipher EImbro DLew ADAMS ACCESSION #: ML051150046 G:\DSSA\SPSB\Mrowca\ ILRT RAIs (F).wpd NRR-106 OFFICE SPSB SC:SPSB SC:SPSB DE/EMEB RES/DRAA BC:SPSB NAME LAMrowca RLDennig /RA/ J. MRubin /RA/ D. KManoly HHamzehee MTschiltz Lee /For/ Harrison /For/

DATE 03/ 10 /05 03/ 11 /05 04/ 18 /05 03/ 18 /05 04/ 15 /05 04/ 22 /05 OFFICIAL RECORD COPY

REQUEST FOR ADDITIONAL INFORMATION (RAI)

ON THE EXPERT ELICITATION PROCESS USED IN EPRI PRODUCT NO. 1009325, RISK IMPACT OF EXTENDED INTEGRATED LEAK RATE TESTING (ILRT)

1. Please address the issues described below associated with the determination of the probability for an extremely large leak. Provide the entire community uncertainty distribution for the probability, including the mean and 95th percentile values for this distribution.

Section 6.0: Issues associated with the determination of probability for an extremely large leak have been identified by NRC staff based on independent calculations using a methodology that addresses these issues and uses the estimates of the four experts from the report. Using an alternate methodology, the staff obtained a mean value of 8.8E-3 for the probability of an extremely large leak (A > 100 La) in a large containment. Even though the staff estimate did not include the fatigue failure mode, the staff estimate is a factor of over 30 higher than the estimate given in the report (2.47E-4 from Table 6-1 of the EPRI report). Using the estimates of all six experts and including the fatigue failure mode may further increase the discrepancy between the estimate obtained and the estimate given in Table 6-1. The issues in the approach used in the EPRI report are described below:

a. Since one leak size range for which expert opinion elicitation was performed was the extremely large leak size A > 100 La, the results of the expert elicitation should be used for that leak size only, instead of attempting a fit of the probability of a leak versus leak size to a Weibull distribution (Section 6.3). In other words, uncertainty distributions should be generated for the probability of the leak size range A > 100 La for each expert, and arithmetically average these uncertainty distributions. This becomes the community distribution, or aggregated distribution.

b. The results for leaks greater than 100 La are based on a Weibull model (page 6-4).

Because the Weibull is fitted to the estimated frequencies for small and medium leaks as well as large leaks, the estimated frequencies for large leaks are sensitive to the results for smaller leaks. In effect, the results for small and medium size leaks are extrapolated to large leaks. In view of the fact that there is no known relation between leak frequency and leak size, please provide the justification for using the Weibull distribution. In addition, use other distribution shapes to analyze the sensitivity of the results to the different distribution shapes.

c. Each expert was asked to provide his low, best, and high estimates (Sections 5.5.1 and 6.3.2, pages 5-7 and 6-5). It is generally accepted that, in expert elicitation, the experts best estimate refers to the median of the distribution.

However, it is clear (page 6-5) that the experts best estimate was treated as a mean. Please provide the guidance given to the experts as to the definition of the terms low, best and high estimates. Please describe the methods used to insure that the responses of the various experts had a common meaning.

Please explain how the experts inputs were interpreted in calculating the results.

d. Tchebycheffs theorem (Section 6.3.2, page 6-5) is used to derive a standard deviation from the low, best, and high estimates. However, even if the best estimate

2 were the mean value, this theorem would give only a lower bound on the standard deviation. When Tchebycheffs theorem is used to estimate tail areas given the standard deviation, it is conservative, but when estimating the standard deviation given the tail areas, it is non-conservative. For example, for a normal distribution, estimating the variance from the upper and lower 5 percent tails, the variance is estimated to be 0.27 times the true variance, and the estimated standard deviation is about half of the true standard deviation. Please justify the use of the Tchebycheff approximation.

e. The EPRI report reduced the problem of fitting the probability of the leak rate size range to the leak rate to a linear regression problem. The estimates of the exceedance frequencies for the probabilities of leak rates of different sizes are dependent variables. The procedure of linearizing the problem requires approximations to be made in the treatment of the dependence of the variables.

Equations 6-19 (Section 6.3.3, page 6-7) are only valid if the errors in the yi are linear functions of the errors in the Pk (i.e., it is assumed that a first order Taylor series expansion is an adequate approximation). These errors are in addition to the errors made by using Tchebycheff's theorem to estimate the variances of the Pk, and the errors made by assuming that the expert's best estimate is the mean of his uncertainty distribution.

f. A logit transformation of Q(a) is introduced, X=logit(Q), and X is assumed to be normally distributed (Section 6.3.4, page 6-8). It is difficult to see the justification for using a normal distribution for logit (Q). The minimization of D2, given in Equation 6-15 (Section 6.3.3, page 6-7) is consistent with assuming that the yj have a joint normal distribution, with covariance matrix as given by Equation 6-18. Then the most logically consistent approach would be to assume that the estimates b^ 1, b^ 2 of Equation 6-22 are jointly normally distributed, not that logit(Q) is normally distributed.

Then ln(lnQ(a)) is normally distributed. Please justify the assumption that logit(Q) is normally distributed.

Moreover, the uncertainty analysis approach is questionable because information on the uncertainty distributions for each expert are not included, except incorrectly by the use of Tchebycheff's theorem (Section 6.3.2, page 6-5, Equation 6-7). In addition, if the expert's best estimate corresponds, as is generally true, to the median and not mean, then the treatment of the best estimate as a mean is questionable.

g. The EPRI report (Section 6.3.6, page 6-10), notes that one expert used zero several times in the assignment of the probability of ILRT failure, and that zeros are difficult to treat in the statistical evaluation of expert opinion. This expert was not included in the community distribution, and therefore it was considered prudent to not include the highest expert as well. Although it seems that the expert who assigned zero to the probability of ILRT failure exhibited an extraordinary amount of overconfidence, there is no difficulty mathematically in including his results, when one uses only the estimates for the extremely large leak size range, and does not fit the probability of leak rates of different sizes to a Weibull distribution in the way done in the EPRI report. The reason for including this expert is that then there is no basis for discarding the estimates of the highest expert in the development of the community distribution.

3 Alternate Methodology: To eliminate these issues, the staff determined each expert's uncertainty distribution for the probability of an extremely large leak in the following way:

The expert's low, best estimate, and high probability estimates for the extremely large leak were fitted to a distribution, where the best estimate was taken as the median value of the distribution. A split lognormal distribution, truncated at the 99.9th percentile, was used and the remaining 0.1 percent of the distribution mass was placed at the 99.9th percentile. To define the bottom half of the distribution, the best estimate was used as the median value and the low estimate, treated as a 5 percent lower bound, was used for the second parameter defining the lognormal distribution. Similarly, the distribution for the upper part of the lognormal distribution was defined using the best estimate for the median value and the high estimate, treated as a 95 percent bound, was used for the other parameter. Then, the non-conservative errors from using the Tchebycheff distribution to estimate the variance are eliminated (Item d, above), and the expert's best estimate is the median instead of the mean (Item g, above). The low, best, and high estimates of the leak rate probability were given for only four experts, and the staff derived uncertainty distributions for only these four experts. The distributions for all six experts should be used (Item g, above). The results were aggregated using Equation 6-29 of the EPRI report.

In principle, the issues in Section 6.3 of the EPRI report (Statistical Analysis of the Expert Elicitation Input) could be at least partially removed by using a non-linear least squares regression technique with a variety of functional forms of probability of leak rate versus leak rate size. Uncertainty distributions based on the split lognormal distribution with truncation could be used for the uncertainty in the expert's estimate of the leak rate of a specific size.

However, when a variety of functional forms for the probability of leak rate vs. leak rate size are used, the uncertainties will likely be so large that there is no benefit from using this approach in comparison to the approach used by the staff (i.e., where the aggregate estimate of the probability of an extremely large leak is estimated by using only the experts' estimates of the probability of an extremely large leak and no attempt is made to use a fit to a probability of leak rate vs leak rate size curve).

2. Documentation is a key aspect of the expert elicitation process (per References 11 and 12 on EPRI report page 10-1). Only through documentation can the analysis and results be reviewed by others and provide a credible, defensible basis for the purposes of licensing.

There is insufficient information provided in this report to conclude that the expert elicitation process is acceptable as conducted. As an example, the staff attempted to reproduce the EPRI results using the stated method in the report. The staff obtained different values in many cases. Please provide additional documentation regarding the basis for selection of the experts, the actual elicitation process, training provided to the experts, each individual experts judgments and reasoning bases, and detailed calculations used to obtain the results documented in the report. (See Items 4, 5, 8, 10, 12, 13, 17, 18, 19, 31, 32, and 33 below for additional details where documentation is requested.)

Report Summary

3. Page v: The document is intended to be a demonstration that the generic risk impact assessment for optimized ILRT intervals of up to 20 years is very small for all containment designs and sites. However, the number of example calculations is very limited and biased and there is not enough evidence to support a conclusion that the assessments described

4 are either representative or bounding for all designs and sites. Provide additional justification and analyses to support a conclusion that the risk impact is very small for all containment types and sites.

4. Pages v and vi: It is stated that there is a very small risk associated with the extension, provided that the performance bases and defense-in-depth are maintained. On page vi, it is stated that the expert panel also considered defense-in-depth approaches, such as alternative inspections that supplement the testing programs. Alternative means of leakage detection are briefly mentioned in Section 3.5, but do not appear to have been further pursued as part of the risk study or expert elicitation process. Provide the following in this regard: (1) a description of the performance bases and the defense-in-depth measures that are considered essential to assuring the very small risk associated with the extension; (2) further details on the deliberations and conclusions of the expert panel in this area; and (3) an explanation of how safety principle 3 of Regulatory Guide 1.174 (related to the use of performance measurement strategies) would continue to be met if these bases/measures are not expected to be enhanced as part of the proposed extension of the ILRT interval (e.g., through use of indirect containment monitoring techniques or enhanced inspections as a condition of receiving the 20-year extension).

Section 2: Problem Statement

5. Section 2.2 (page 2-2): Please show the mathematical expressions for LERF and population-dose as a frequency-weighted sum over all accident classes rather than a simple product.

6. Section 2.2 (page 2-2): Footnote 1 defines an ILRT failure. A practical definition of ILRT failure is one which does not meet the Type A test performance criterion as defined in ANSI/ANS 56.8-2002. This definition should be used for risk-informed analysis. Please provide justification for using the less conservative definition of Footnote 1.

Section 3: ILRT Data Applicability

7. Section 3 (page 3-1): The report states, In fact, no failures that would result in a large early release have been found. There is no direct connection between ILRT failures and large early release. Early release is likely to occur under accident conditions which would generate high pressures or if the containment is not properly isolated. The ILRTs are performed at relatively low pressures and the duration of a test is articulated to ensure an essentially leaktight containment. If ILRTs at DBA pressure indicate a leakage rate greater than 1 La to 10 La, it can be considered as a precursor to an early release under challenging accident pressures. (See Sandia Report SAND90-0119, Insight into the Behavior of Nuclear Power Plant Containments during Severe Accidents.) This concept should be incorporated into the elicitation report.

8. Section 3 (page 3-1): The report states that the testing data alone does not, without expert opinion, support the development of realistic values for the probability of a large containment leakage event. Please explain why alternative statistical models beyond those identified in Table 3-1 can not be used to derive more realistic probability values (e.g., mean or 50 percentile values at some lower confidence level) in lieu of

5 an expert elicitation process. Also, provide a comparison of the probability values (for Class 3a and 3b leakage) obtained from the expert elicitation process with values derived from alternative statistical models or model assumptions (e.g., in the form of an expanded version of Table 3-1).

9. Section 3.1 (page 3-1): It is misleading to state: The first ILRT survey was performed in early 1994-NUMARC/NEI, when NUREG/CR-4220 (June 1985): Reliability Analysis of Containment Isolation System, provides an extensive database of isolation failures that occurred between 1965 and 1983 based on licensee event reports (LERs). NUREG-1273, Technical Findings and Regulatory Analysis for GSI II.E.4.3, Containment Integrity Check, subsequently moderated the conservative bias of NUREG/CR-4220, and analyzed potential alternatives to ILRTs. Please provide justification for not using the combined database (old and new) and appropriate ILRT failure criterion (See Item 5) in the elicitation process and eventually in the risk-informed analysis.

Section 4: Expert Elicitation Process

10. Section 4.3.2.1 (page 4-3), 4.4, and Table 4.2: Based on the credentials of the experts (Table 4.2 of the EPRI report), there was one expert in probabilistic risk assessment, three in-service inspection (ISI) and testing experts, and two containment leakage rate testing experts (one of which co-authored the report). Since most of the experts had little knowledge or experience in the field of containment leakage rate testing, training was provided during the meeting that may have influenced their proffered leakage rate probabilities. It is also stated (page 4-3) that experts will be chosen for their knowledge of the mechanisms that can result in containment leakage events, and therefore provide additional assurance that their judgement is only moderately significant to the overall result.

Except for one expert from the NRC, the experts all have associations with industry rather than including experts from the National Laboratories or academic institutions. Please explain why there were not more containment leakage rate testing experts chosen for expert elicitation. Provide additional information and credentials to justify that the experts have expertise regarding the mechanisms that can result in containment leakage events (including relevant containment degradation mechanisms). Justify why the expert choices that were made would not lead to biased results, especially since the input from only four experts was used to determine the leakage rate probabilities.

11. Section 4.3.2.1 (page 4-3): The EPRI report notes that the results of the expert elicitation process are very significant to the results of the analysis necessitating an assignment of a Degree III importance (i.e., highly contentious issue; very significant to the overall result of the analysis; and/or highly complex), but that this is mitigated by the availability of significant amounts of data. As a result, the issue was assigned a Degree II importance.

Assigning the issue a Degree II importance appears to have enabled the use of a technical integrator approach, wherein the technical integrator plays the role of evaluator. In contrast, assigning the issue a Degree III importance would require the use of a technical facilitator/integrator approach, wherein multiple evaluators would be involved. One could argue the converse with regard to the availability of significant amounts of data, i.e., that there is insufficient data regarding large leakage events. Please discuss the implications on the expert elicitation process and results if the issue were assigned a Degree III rather than Degree II degree of importance.

6 Section 5: Expert Elicitation Input

12. Section 5.3.3 (page 5-5): It is stated that the database of found degradations was included in the presentation, and that the effects of aging on potential containment failure modes was emphasized in the expert elicitation. Please provide a description of the technical content of the presentation, a copy of the slides/handouts from the meeting, and a characterization of the expert inputs regarding the significance of various degradation mechanisms to the probability of containment leakage derived from the expert elicitation process.

13. Section 5.5 (pages 5-6 and 5-7) and Section 6.2 (page 6-3): It is stated that the actual frequencies elicited from the experts were all relative to available data or to frequencies of smaller leaks. However, all of the results presented in the report are in terms of absolute frequencies. Please provide the raw elicitation inputs of the relative values provided by the experts. If best, low, and high relative values were elicited, please provide the guidance given to the experts as to the definition of these terms. Please describe how the relative values provided by the experts were used to calculate the absolute frequencies.

14. Section 5.5.1 (page 5-7): Please explain why a multiplier of 2.5 (i.e., 1000 / 400) was used rather than a multiplier of 5.5 (i.e., 1000 / 182).

Section 6: Expert Elicitation Results and Analysis

15. Section 6.0 (and other sections): It is well-known that experts tend to underestimate the uncertainties of their subjective assessments. For example, a rule of thumb is that the actual coverage of an uncertainty interval is about a factor of two smaller than the nominal coverage. Thus, for almanac-type questions where the answers are known, only about half of the nominal 90% coverage intervals supplied by a panel of experts actually contain the correct answers. There is no discussion of this phenomenon in the report. Please explain why no adjustment was made to the expert responses to compensate for possible overconfidence.

16. Section 6.1 (page 6-1): Separate input was collected for small containments and large containments. Please explain the decision for delineating based on containment size rather than containment type, especially since the applicability of various corrosion mechanisms or failure modes may be more a function of containment type than size.

Please discuss the potential impact on results if the inputs were grouped based on containment type.

17. Section 6.1 (page 6-2): The EPRI report states that significant changes to failure modes were made by the experts. It is not quite clear which failure modes were considered in the expert elicitation process. One of the major failure modes that is relevant to the risk analysis is the corrosion of the bottom liner plate of concrete containments, and corrosion of the bottom head or plates of steel containments. These areas are not conducive to ISI, but an ILRT failure could indicate if any gross degradation is occurring. The staff recognizes that no data is available to characterize the probability of such a failure.

However, the elicitation process needs to consider the feasibility of such a failure mode in estimating the leakage rates. Please provide a summary of failure modes considered

7 in the elicitation process.

18. Section 6.2 (page 6-2): Significant areas for deliberation were said to include: (1) the effects of aging on the containments and the resulting failure modes; (2) the fact that not all potential containment failure modes may appear in the current data; and (3) different containment types having the potential for different failure modes with potentially different failure rates. Please provide a discussion of the key points deliberated in each of these areas, and the view(s) of the expert group with regard to each key point.

Include any views expressed by dissenting experts, including the two experts whose inputs were not used in developing the community distribution.

19. Section 6.2 (page 6-2): Expert input was solicited for each of five containment failure modes: (1) construction errors or deficiency; (2) human error associated with testing or maintenance; (3) human error, design error, or other deficiency associated with modifications; (4) corrosion; and (5) fatigue. Based on the information provided in Section 6.2 and Appendix C, it is not possible to discern the key issues considered by each expert for each of the five failure modes, and ultimately, whether the experts were able to provide meaningful input for each of these failure modes. Please provide a brief description of the key issues considered by each expert for each of the five failure modes, and a characterization of their views regarding the relative importance of each failure mode to the overall failure probability.

20. Section 6.2 (page 6-3): Corrosion is mentioned only briefly as a potential failure mode to be considered by the experts. In support of one-time, 15-year ILRT test interval extensions, licensees were requested to evaluate the potential contribution to large early release frequency (LERF) from shell/liner degradation mechanisms. Although the contribution from corrosion was typically less than 1E-8 per year in these analyses, under alternative assumptions (e.g., relating to the increase in flaw likelihood, the likelihood of breach given a flaw, and likelihood of failure to detect a flaw developing from the liner back side) this contribution could be in the range of 5E-8 per year to 2E-7 per year, and would be even greater for a 20-year interval extension. In this regard, please provide a quantitative evaluation of the potential impact of corrosion on LERF based on a 20-year interval extension and a methodology similar to that used in support of the one-time, 15-year test interval extensions.

Section 7: Technical Approach

21. Section 7.1 (page 7-4): Please further describe containment endstates/release modes that contribute to Intact CDF. All CDF that does not lead to LERF/bypass (e.g.,

intermediate and late containment failures) should be considered for inclusion in this accident class. Additional guidance is also needed on whether/how events with containment sprays (for PWRs and BWRs) and suppression pools (for BWRs) are to be handled.

22. Section 7.1 (page 7-4): The last sentence of the second bullet seems to indicate that the existing empirical data for small leaks has been disregarded and replaced by the probability developed by the expert elicitation process. Please explain why it is appropriate to supplant empirical data with the results of expert elicitation.

8

23. Section 7.2 (page 7-4): The plant IPE or PRA are identified as possible sources for offsite dose estimates. Please expand this discussion to recognize that population dose estimates may have also been developed subsequent to the IPE, e.g., as part of the severe accident mitigation alternative analysis performed in support of license renewal. Plant-specific values can also be approximated based on offsite consequence analyses for similar plants, and appropriate scaling to account for site-to-site differences in meteorology and demography.

24. Section 7.2 (page 7-4): A leak rate for Class 3b of 100 La is said to be conservative.

While 100 La may be conservative as a threshold for when a leak is considered to be large, it is non-conservative as the basis for quantifying the magnitude of a large leak. As stated on page 3-2, a leakage of 600 %/day (or 600 to 6000 La) is more representative of a large early release. Accordingly, the doses for Class 3b should be assigned a value closer to 1000 La. Please justify the use of 100 La as conservative for Class 3b.

25. Section 7.5 (page 7-5): The change in probability of leakage detectable only by ILRT was determined by multiplying the baseline probability by the ratio of the new to the old test interval. Please justify this approach, since it presumes a constant containment degradation rate, which may not be reasonable over a 20-year test interval.

Section 8: Application of Technical Approach

26. Section 8.1 (page 8-1): The hypothetical PWR CDF characteristics (i.e., breakdown of CDF by accident class) presented in Table 8-1 is not very representative of PWRs.

Specifically, in the example, 53% of the CDF is associated with a large release (Class 7 and 8) and only 47% of the CDF is associated with an intact containment. This will tend to understate the relative impact of the ILRT interval extension. (A more typical CDF breakdown would involve less than 10% early/bypass and 70 to 80% intact containment.)

Please provide a justification for the basis of the example calculation or provide a calculation more representative of CDF characteristics to avoid biasing the results.

27. Sections 8.1, 8.1.3, 8.2 (pages 8-1, 8-4, 8-11): The doses presented in Table 8-1 for a hypothetical PWR are extremely large and not representative of values typically derived from PRA methods (as claimed on page 7-2). In general, dose values are expected to be on the order of 1E+3 person-rem per event for Class 1, and on the order of 1E+6 to 1E+7 person-rem per event for Class 7 and 8. The dose values in Table 8-1 result in an annual population dose for the hypothetical plant of 68,000 person-rem per year, which is about three to four orders of magnitude greater than obtained from recent risk analyses submitted in support of license renewal, and is also 4 orders of magnitude greater than the value for the hypothetical BWR in Section 8.2. The example calculation should be based upon more representative dose values to avoid biasing the results. Finally, the sum of the population dose estimates for each accident class (presented as the total dose in Tables 8-1, 8-3, and 8-10) is not meaningful. The total dose is only meaningful if it is a probabilistically-weighted sum. Please delete reference to the total doses.

28. Section 8.1.5 (page 8-6): The equation for population dose does not yield the increase in population dose. Rather it yields the annual population dose (rate) corresponding to the extended test interval. Please express the population dose as:

9 population dose rate 3a = population dose 3a x [frequency 3a, 1/10 - frequency 3a, 3/10 ]

29. Section 8.1.6 (page 8-7): The information displayed in Table 8-6 (percentage values for each accident class) is not very useful for showing the impact of the ILRT extension on total population dose. Rather than reporting percentage values, please consider showing the population dose (per year) for each accident class, and the sum of these values over all accident classes in the table.

Section 9: Results Summary and Conclusions

30. Section 9 (page 9-1): The role of the EPRI report is unclear, i.e., whether it is intended to establish a methodology that can be used by licensees to support plant-specific license amendment requests to extend the ILRT test interval to 20 years, or whether it is intended to be a generic demonstration that the risk impact of extending the ILRT test interval to 20 years is very small for all containment designs and sites. The document appears to be oriented toward the former objective, and does not meet the latter intent for numerous reasons, including a very limited and biased number of example calculations, and the lack of any rationale or logic to support a conclusion that the assessments described are either representative or bounding for all designs and sites. Please provide a detailed discussion of how the EPRI document is to be used by individual licensees and the NRC to support a permanent extension of the ILRT test interval to 20 years, including a description of any plant-specific analyses that would be required as part of plant-specific license amendment requests.

B: Expert Elicitation Input Data

31. Table B-2 (page B-3): If the experts supplied rationale for the probability estimates, please supply the documentation giving this rationale. If there is no documentation of the rationale, please justify the decision not to require such rationale to be documented.

C: Expert Elicitation Results

32. Table C-5 (page C-7): In Table C-5, the aggregate upper bounds for La $ 5 are all zero while the corresponding upper bounds for Experts A, D and E are all greater than zero.

Please explain how the entries in the tables in Section C were calculated.

33. Figures C-1, C-2, C-3 (pages C-15 and C-16): Page 7-3 indicates that the data used to develop these curves was Detectable by ILRT Only (Failures), but the data set used to produce Table 6-1 is not identified. Please identify whether the set of data used to produce these figures was based on Total Degraded ILRTs or Detectable by ILRT Only (Failures).