ML081700323

Citizens' Response to Commission Order Dated May 28, 2008

ML081700323
Person / Time
Site:	Oyster Creek
Issue date:	06/11/2008
From:	Webster R Eastern Environmental Law Ctr, Grandmothers, Mothers & More for Energy Safety, Jersey Shore Nuclear Watch, New Jersey Environmental Federation, New Jersey Public Interest Research Group (NJPIRG), Nuclear Information & Resource Service (NIRS), Sierra Club, New Jersey Chapter
To:	NRC/OCM
SECY RAS
References
50-219-LR, RAS-H-40
Download: ML081700323 (54)

Text

'R ks !4- L(i, UNITED STATES OF AMERICA DOCKETED NUCLEAR REGULATORY COMMISSION USNRC ATOMIC SAFETY AND LICENSING BOARD June 11, 2008 (4:05pm)

OFFICE OF SECRETARY BEFORE THE COMMISSION RULEMAKINGS AND ADJUDICATIONS STAFF In the Matter of )

) Docket No. 50-0219-LR AMERGEN ENERGY COMPANY, LLC )

)

(License Renewal for the Oyster Creek )

Nuclear Generating Station) )

.)

CITIZENS' RESPONSE TO COMMISSION ORDER DATED MAY 28, 2008 Richard Webster Eastern Environmental Law Center 744 Broad Street - Suite 1525 Newark,.NJ 07102 Counsel for Citizens June 11, 2008 T NPAi tSPC~z {- DŽ,&

UNITED STATES OF AMERICA NUCLEAR REGULATORY COMMISSION OFFICE OF THE SECRETARY BEFORE THE COMMISSION In the Matter of )

) Docket No. 50-0219-LR AMERGEN ENERGY COMPANY, LLC )

)

(License Renewal for the Oyster Creek - ) June 11,2008 Nuclear Generating Station) )

CITIZENS' RESPONSE TO COMMISSION ORDER DATED MAY 28, 2008 This response to the Commission's May 28, 2008 Order, is filed on behalf of Nuclear Information and Resource Service, Jersey Shore Nuclear Watch, Grandmothers, Mothers and More for Energy Safety, New Jersey Public Interest Research Group, New Jersey Sierra Club, and New Jersey Environmental Federation (collectively "Citizens"). The Commission has asked first whether AmerGen Energy Co. LLC

("AmerGen") has committed to do an analysis of the factor of safety that matches or bounds the sensitivity analyses that Judge Baratta would impose and second whetheradditional analysis is needed.

The short answer is no and yes.

At present AmerGen has made a vague commitment to perform a three dimensional finite element structural analysis of the drywell shell using modem methods and the current drywell shell thickness. The analysis will include some sensitivity studies to determine how uncertainties in the size of severely corroded areas affect the margins. In contrast, Judge Baratta would impose a requirement to carry out a series of sensitivity analyses, at least one of which would use all the measured thickness data in an extrapolation scheme, that could be similar to that employed by Citizens' expert Dr. Hausler, to determine drywell thicknesses. To date AmerGen has not committed to use all the data, use such an extrapolation scheme, or carry-out "a series of sensitivity analyses."

Judge Baratta's additional requirements will lead to extremely valuable information about the compliance of the drywell with the Current Licensing Basis ("CLB") requirement that there be a factor of safety of 2.0 during refueling, which is the requirement that is most stringent in terms of drywell thickness. However, these requirements are likely to further illustrate that the uncertainty about the current state of the drywell makes it impossible to assure compliance with the CLB with any certainty without further measurements. AmerGen should therefore be required to explicitly estimate the full range of uncertainty in the predicted factors of safety using a series of sensitivity analyses. Then, if compliance with the CLB is not demonstrated with a high level of certainty,' AmerGen should be required to perform additional measurements of both the thickness and the shape of the drywell to reduce the uncertainty in the model. AmerGen should then be required to do a re-analysis using the additional thickness measurements and the actual shape of the drywell to attempt to establish compliance with the CLB at the Required Level of Certainty. Armed with these results, the Commission could then take an appropriate decision on whether there is reasonable assurance that the drywell shell at the Oyster Creek Nuclear Generating Station ("Oyster Creek") would meet the CLB on the first day of any extended period of operation and would continue to do so.

ARGUMENT I. AmerGen's Commitments Are Vague And Inconsistent With Reasonable Assurance AmerGen has made a vague commitment to perform a three dimensional finite element structural analysis of the drywell shell using modem methods and using the current drywell shell thickness. NRC Staff Ex. I at A-30 to A-3 1. The analysis will include some sensitivity studies to determine how Based on legal and general technical arguments, Citizens have stated that the required level of certainty is at least 95%. However, it is possible that a higher degree of certainty is required to remain consistent with assumptions made in the risk assessments regarding accidents at Oyster Creek, which is an issue that Citizens have not examined. Citizens note that in the context of initial licensing 10 C.F.R. § 50.46 requires explicit estimates of uncertainty in calculated results and the analyses must show "a high level of probability that the [acceptance] criteria would not be exceeded." Furthermore, when initially licensed, the predictions of the factor of safety were much more certain than they are now, because the wall thicknesses were known to be at or very close to nominal design thicknesses. Moreover, compliance was assured by walls that were almost double the current thickness of the drywell in the areas that have corroded most. Because the Commission has not yet decided what level of certainty is required to establish reasonable assurance of compliance with the factor of safety requirements at Oyster Creek, Citizens will refer to the certainty required as the "Required Level of Certainty."

2

uncertainties in the size of thinned areas affect the margins. Id. at A-3 1. AmerGen committed to notify the NRC if "the analysis determines that the drywell shell does not meet required thickness values." Id.

During this proceeding, the Atomic Safety and Licensing Board (the "Board") found that the CLB includes a requirement to meet the safety factor of 2.0 during refueling. LBP-07-17, 66 NRC 327 at n.

20. Thus, Citizens believe that AmerGen has committed to notify the NRC if the model predicts a factor of safety of less than 2.0 during refueling or fails to meet other similar requirements.

The notification aspect of this commitment is straightforwardly inadequate because the modeling should be required to affirmatively establish compliance with the CLB with reasonable assurance. As a matter of policy, uncertainty must be resolved against licensees where they control the level of certainty provided through decisions on scope and frequency of measurements. Thus, at minimum, AmerGen should be required to notify the NRC if the outcome of the modeling is indeterminate and fails to establish compliance with the factor of safety requirements of the CLB to the Required Level of Certainty . Futhermore, because this modeling will serve as critical evidence concerning the resolution of the contention, AmerGen should be required to provide a copy of the analysis to Citizens irrespective of the outcome.

II. Judge Baratta Correctly Concluded That More Work Is Needed As the Commission has correctly recognized, Judge Baratta believes that the licensee failed to "fully" show that "there is reasonable assurance that the factor of safety required by the regulations will be met throughout the period of extended operation. . . ." LBP-07-17, 66 NRC 327, 373 (Additional Statement of Judge Baratta at I). This is because "to date ... no analysis of the actual condition of the drywell has been done." Id. at 4 (emphasis in original). Therefore, "we do not know what the actual safety factor is." Id. Adding to the uncertainty caused by this lack of analysis is "a very limited knowledge of the actual thickness of the shell" because "there are large areas of the drywell that do not have any recent measurements or any measurements at all." Id. at 5. Because further corrosion of the drywell cannot be ruled out, "it is essential to have a conservative best estimate analysis of the drywell 2 As discussed below, this modeling should be done prior to any decision on relicensing.

3

shell before entering the period of extended operation." 1d. at 4. In addition, that analysis must take account of the uncertainty caused by the lack of knowledge regarding the thickness of the drywell. Id. at 5.

Judge Baratta recognized that AmerGen was going to use~a three dimensional finite element model, which would have as inputs the measured thicknesses. Id. Based upon AmerGen's oral testimony he also concluded that the model will use the actual geometries of the drywell. Id.; See also Tr. at 659-60 (Gallagher). 3 He then stated "I would impose an additionalrequirement on the.., applicant." Additional Statement of Judge Baratta at 6 (emphasis added). The additional requirement is that the applicant should perform a series of sensitivity analyses. Id. One of these analyses should include the use of an extrapolation method to determine the thickness between the measured locations. Id. This could be similar to the approach suggested by Citizens' expert and use contour plots generated from known thickness points measured from both the interior and the exterior. Id.

Plainly, Judge Baratta would not have suggested these additional requirements if he thought AmerGen had already committed to this analysis. In fact, AmerGen's commitments are much less detailed than the requirements spelled out by Judge Baratta. Furthermore, AmerGen documents show that AmerGen changed its mind on how to derive the thickness values for the three dimensional analysis, but has never proposed using all the measured results. See Citizens Ex. 65 (base the three dimensional analysis on the external measurements); Citizens Ex. 45 (use primarily the internal measurements).

Moreover, AmerGen vigorously opposed the admission and then the use of Dr. Hausler's contour plots during this litigation. E.g. AmerGen Ex. C part 3 at 30-32.4 Thus, Judge Baratta's additional 3 Contrary to the plain meaning of Mr. Gallagher's statement, information available to Citizens indicates that AmerGen intends to use the idealized geometry model it used in the past by modeling'a geometrically idealized drywell and then applying a capacity reduction factor to take account of the real imperfect shape of the drywell.

NRC Staff Ex. 3 at 5-4 to 5-5; Citizens Ex. 65 at 1.

4 Part of the objection is that some of the external measurement points were selected to be at the visually identified minima and some may have been overground during surface preparation. Citizens recognize that the data set is less than ideal, but have analyzed these issues in detail and have found that these effects are insignificant.

Citizens Proposed Finding of Fact and Conclusions of Law at 18-20. Thus, as Judge Baratta recognized, all the thickness data should be included in the further modeling by using contour plotting or a similar approach.

4

requirements not only go beyond what AmerGen has committed to do, they also go well beyond what AmerGen would do in the absence of any further regulatory requirements.

III. Existing Data Are Probably Insufficient To Provide Reasonable Assurance All indications are that the drywell is, at best, marginally above the factor of safety requirements, but that there is a chance that it could already have violated the standards. For example, on August 17, 2007, Dr. Hartzman of the NRC Staff stated baldly that "Based on the currently available corrosion data of the sand bed region, the Staff estimates that the EFS [effective factor of safety] in the sand bed shell is 1.9." Affidavit of Mark Hartzman, dated August 17, 2007. However, the Staff then amended the pre-filed testimony to read: "Assuming that the corrosion is as extensive and severe as depicted by Dr.

Hausler's contour plots in Citizens Exhibit 13, the Staff estimates that the EFS in the sand bed shell is 1.9." See NRC Staff Ex. C at A28. Subsequently on September 24, 2007, Dr. Hartzman testified orally that based on the average thickness of the shell, the current factor of safety is "probably about two, even greater than two." Tr. 453:12-16.

NRC Staff have shown no errors in Dr. Hausler's.contour plots and Judge Baratta has recognized their value. On sur-rebuttal, Dr. Hausler showed that the plots he had presented previously did not show the full extent of corrosion because they were confined to the measured area as provided by AmerGen.

Dr. Hausler then presented additional plots utilizing an additional extrapolation method showing even more extensive corrosion. Citizens' Ex. 61 at 14-17. Furthermore, Dr. Hausler showed that his plots were merely more refined versions of AmerGen's estimates regarding severely corroded areas. Citizens Ex. 61 at 4. Thus, if Dr. Hartzman had continued to rely on Dr. Hausler's plots or, had relied upon AmerGen's latest interpretation of the data, he would have predicted a factor of safety of less than 1.9.

Furthermore, the NRC Staff repeatedly stated they had not reviewed AmerGen's latest analysis of the severely corroded areas in detail. NRC Staff Ex. B at A9 (page 13); Tr. 415:16-21; Tr. 420:4-10.

Therefore, Dr. Hartzman's estimate that the factor of safety is greater than .2.0 based on the average thickness is undermined by his estimate that it is approximately 1.9, based upon more detailed data analysis.

.5

Somewhat similarly, Dr. Mehta of GE testified that the factor of safety is probably "greater than two," but not much greater than two, but failed to state how he had interpreted the thickness measurements to reach this conclusion. Tr. 441:11-24. Finally, the best available analysis of the drywell shell was carried out by Sandia National Laboratories. Based on non-conservative assumptions, the Sandia Study concluded that the factor of safety was approximately 2.15 and the current buckling strength is approximately 44% lower than when it was built. Citizens' Proposed Conclusions of Fact and Law at 34-35. Sandia warned that its analysis was intended to focus on the relative reduction in design margin, rather than the absolute stress, and was insufficient for licensing purposes. NRC Staff Ex. 6 at 12. This was because many assumptions had to be made and thickness measurements were "limited to a few selected regions in the sandbed [region of the drywell shell]." Id. Judge Baratta also warned that the Sandia Study is "based on a very limited knowledge of the actual thickness of the drywell." Additional Statement of Judge Baratta at 3.

During the hearing, Judge Abramson also commented on the "unknown information" highlighted by Sandia and asked AmerGen if it was going to do "a lot of measuring." Tr. at 657:13-16. Surprisingly, even though AmerGen agrees that there are "an insufficient number of UT measurements to evaluate a representative average thickness over each area," e.g. AmerGen Ex. C at Part 3 A.38, AmerGen responded that it is going to base the required three dimensional analysis on the existing measurements.

Tr. at 657:17:18. As Dr. Hausler has repeatedly pointed out, while it is possible to interpolate and extrapolate to generate thickness values for unmeasured areas, this gives rise to large uncertainties, which make any prediction of the current factor of safety based upon the existing measurements highly uncertain. E.g. Ex. CR I Attachment 1 at 5.

Adding to this uncertainty is the proposed approach of using a capacity reduction factor to take account of imperfections in the shape of the drywell shell. The original General Electric ("GE") study utilized an enhanced capacity reduction factor to take account of the beneficial effects of hoop stress.

Citizens' Ex. 55, Report of Brookhaven National Laboratories at 3. The reviewers found that this approach may have double-counted these effects and recommended further evaluation. Id. at 4-5.

6

Similarly, Sandia reviewed the justification for using the enhanced capacity reduction factor, found that the references provided were inadequate, and so decided against using the enhanced factor. NRC Staff Ex. 6 at 67. Sandia confirmed this finding at the January 18, 2007 ACRS meeting. Transcript of ACRS Meeting on January 18, 2007 ("Trl") available at ML070240433 at 284:2-11. At that meeting, the NRC Staff also concurred with Sandia. Trn at 288:12-19. However, at the February 1, 2007 ACRS meeting, the Staff indicated that the enhanced capacity factor was acceptable, but did not ask. Sandia to provide any further presentation. On February 9, 2007, the supervisor of the Sandia study stated that Sandia's views "differ somewhat from the opinions presented by the licensee and the Staff' at the ACRS meeting on February 1, 2007. E-mail from Hessheimer toAshley, dated February 9, 2007 available at ML070430292. The e-mail further explained that the three-dimensional model already took explicit account of the hoop stress. Thus, while it might be appropriate to enhance the capacity reduction factor when using the formulae specified in the ASME code, it is not appropriate to enhance the capacity reduction factor when using a three-dimensional finite element model. Id.

The choice of the capacity reduction factor makes a major difference in the predicted factors of safety. The Sandia Study used a reduction factor of 0.207, while GE used a factor of 0.326. NRC Staff Ex. 6 at 67; TI at 96:15-17. Thus, if AmerGen predicts a factor of safety of 2.0 by multiplying the output of the three-dimensional model by 0.326, Sandia believes that the actual predicted factor of safety should be 1.27. See TI at 292:25-293:9. At minimum, this shows that significant uncertainty in the choice of an appropriate capacity reduction factor adds to the already large uncertainties that arise from the limited scope and frequency of the thickness measurements and the need to make other assumptions about drywell properties and behavior. It is therefore important to ensure that the uncertainty in the capacity reduction factor is fully reflected in the sensitivity analysis.

The studies to date show that the current factor of safety is highly uncertain and is, at best, right on the edge of what is required by the ASME code. Because no amount of computer analyses can eliminate the uncertainties about the thickness and the degree to which the shape of the vessel reduces its strength, it is highly likely that any sensitivity analysis that takes full account on the current uncertainties 7

will indicate that there is a considerable chance of both compliance and non-compliance with the safety factor requirements, failing to establish compliance to the Required Level of Certainty. 5 If this is the case, a more refined analysis would be needed prior to any decision to extend the license.

Finally, even if the sensitivity analysis based upon the additional requirements unexpectedly showed compliance with the CLB to a high degree of certainty, Judge Baratta recognized that further deterioration of the shell could reduce the safety factor to below 2.0 while AmerGen still passes the measurements as compliant with the acceptance criteria. Additional Statement of Judge Baratta at 3.

Similarly, Judge Abramson recognized during the hearing, "we don't have an analysis of how much ...

degradation this shell can take before it approaches buckling." Tr. at 510:19-21. Because there is currently no assessment of how much more corrosion, if any, would be acceptable, it is impossible to determine an appropriate monitoring scope or frequency. In addition, if future results show further thinning, there will be no way of knowing whether they are consistent with the C.LB. Finally, without this limiting margin, it is impossible to determine how accurate future estimates of thickness need to be.

Therefore, as AmerGen appears to have already recognized, the three dimensional analysis must also be used to determine the thickness margin above the safety factor requirements, by reducing the modeled thickness in steps until compliance is not predicted to the Required Level of Certainty. Citizens Ex. 65.

IV. State-of-the-Art Measurements And Analysis May Provide Sufficient Certainty For Licensing Citizens anticipated the need for a more refined analysis some time ago and engaged Stress Engineering Services, Inc. ("Stress"), a well-qualified firm of structural engineers, to determine if AmerGen's commitment to evaluate future UT results in the sand bed region "per the existing program" was adequate. This material, which is attached as Ex. CR 2, was presented to the Board on July 25, 2006 in support of Citizens comprehensive contention regarding the drywell. The resumes of the individual 5 As Dr. Hausler points out, one approach to ensure that the sensitivity analysis fully accounts for the uncertainties in the input parameters would be to automate the analysis by assigned probability density functions to, the thickness measurements and other inputs, then using a Monte Carlo simulation combined With contour plotting and extrapolation techniques to generate a probability density function for the predicted factors of safety. Ex. CR I Attachment I at 5.

8

engineers from Stress Who provided the opinion were provided as Exhibit NC II (availableat ML062140418).

In its expert opinion dated July 15, 2006, Stress pointed out that the applicable engineering code relates primarily to pressure integrity and governs construction of pressure vessels, not serviceability. Ex.

CR 2 ("Stress Opinion") at 2. Thus, in stark contrast to the current situation, the code assumes that vessel thickness values are determined by design with a known small uncertainty. Because the margins in the code are designed to protect against unidentified uncertainties, Joshua M. Reinart & George E.

Apostolakis, Including model uncertaintyin risk-informed decision making, 33 Annals of Nuclear Energy 354, 355 & 357 (attached as Ex. CR 3), it is unacceptable to rely upon the code-required margin of safety to offset the uncertainty about code compliance that stems from the lack of certainty concerning the thickness of the vessel wall or the uncertainty in the shape due to operational loading or environmental degradation. See e-mail from Hessheimer to Ashley, dated February 9, 2007.

Stress opines that much better techniques than those used by the licensee are now available, are code compliant, and provide the most accurate assessment of vessel integrity possible. Stress Opinion at

3. One critical advance is the use of lasers to map the actual shapes of pressure vessels, along with sophisticated UT techniques that measure the wall thickness. Id. Thickness measurement techniques using other technologies have also advanced. Ex. CR 1, Attachment I at 5-6. Stress also points out that the G.E. analysis used idealized geometries, such as a perfect sphere for the lower part of the drywell.

Stress Opinion at 1-2. The calculations were then adjusted by making assumptions about surface irregularities, plasticity, and local buckling using the capacity reduction factor. Id. at 2. Thus, the laser measurement technique described by Stress can greatly reduce the uncertainty associated with the capacity reduction factor. In this regard, AmerGen stated that it would use the "actual geometry" of the shell in its three dimensional analysis, apparently recognizing the value of this approach. Tr; 660:1-5.

Thus, a three dimensional computational analysis using input derived from state-of-the-art measurement techniques could provide a much more precise prediction of the ability of the drywell to withstand buckling. In addition, the model could then be used to predict the current margin and the 9

expected service life of the vessel, as anticipated by AmerGen using the current approach. Citizens' Ex.

65. Depending on the results, such a study could provide the basis for a licensing decision based upon affirmative knowledge of drywell safety now and during any extended licensing period, rather than a default decision based upon lack of knowledge. However, to facilitate detailed design of such a study, it would be ideal to have some indication from the Commission about the Required Level of Certainty.

V. Reasonable Assurance Must Be Established Prior To Licensing And Should Be Established Prior To Refueling There is no dispute that licensing may only proceed if there is reasonable assurance that the drywell. shell will meet CLB requirements on April 10, 2009, the first day after the expiration of the current license, and thereafter. 10 C.F.R. § 54.29; Citizens Petition for Review at 3-4. Because such assurance is currently lacking, id. at 11-12, the Commission cannot decide whether to renew y)st~er Creek's operating license until AmerGen submits a modeling study that establishes current and ongoing compliance with the buckling factor requirements of the CLB with reasonable assurance. 6 CONCLUSION For the foregoing reasons, the Commission should require AmerGen to conduct additional analyses of the factors of safety of the drywell shell prior to any decision on relicensing and grant any other relief as it may see fit.

Respectfully submitted, Richard Webster, Esq.

Eastern Environmental Law Center Attorneys for Citizens Dated: June 11, 2008 6 Although it is concededly beyond the scope of the proceeding below, Citizens also ask the Commission to consider whether there is reasonable assurance that the drywell will comply with the requirement for a factor of safety of 2.0 during the next refueling outage, scheduled for October 2008. If further analysis is needed to establish compliance with the CLB during refueling prior to expiration of the license in less than a year, that analysis must also be needed prior to the scheduled refueling in October 2008. Citizens recognize that this issue is primarily the responsibility of the Staff, but the Commission should consider whether to exercise its supervisory authority over the Staff because the issues concerning relicensing also have implications for current safety.

10

EXHIBIT CR 1 EXHIBIT CR 1 UNITED STATES OF AMERICA NUCLEAR REGULATORY COMMISSION BEFORE THE COMMISSION In the Matter of )

)

AMERGEN ENERGY COMPANY, LLC ) Docket No. 50-219-LR (Oyster Creek Nuclear Generating Station) )

)

)

)

DECLARATION OF DR. RUDOLF HAUSLER

1. My name is Dr. Rudolf Hausler. Citizens have retained me as an expert witness in proceedings concerning the application of AmerGen Energy Company LLC to renew its operating license for the Oyster Creek Nuclear Generating Station ("Oyster Creek") for twenty years beyond the current expiration date of April 9, 2009.

2. I am an expert on the corrosion of metals during operation.

3. The attached memorandum dated June 10, 2008 represents my current opinion regarding the topics it covers.

I declare under penalty of perjury that the foregoing and the attached memorandum, dated June 10, 2008 is true and correct.

Executed this /0 day of June, 2008 at Kaufman, Texas Rudolf Ha err,PhD 2

CORRO-CONSULTA Rudolf H. Hausler.

8081 Diane Drive Kaufman, TX 75142 Tel: 972 962 8287 Mobile 972 824 5871 Fax: 972 932 3947 Memorandum To Richard Webster, Esq. June 10, 2008 Legal Director Eastern Environmental Law Center 744 Broad Street, Suite 1525 Newark, NJ, 07102

Subject:

Response to NRC Commissioners' Request Regarding Additional Requirements of Judge Baratta The Commissioners ask the parties to address the following:

Explain whether the structuralanalysis that AmerGen has committed to perform, and that is reflected in the Staff's proposed license condition, matches or bounds the sensitivity analysis that Judge Barattawould impose. In any event, explain whether additional analysis is necessary

1. Background There is no dispute that the Drywell Liner at the Oyster Creek Nuclear Generating Station has been subject to severe corrosion in the (former) sand bed region. Several locations have been documented where the shell has lost in excess of 40 percent of its original wall thickness 2). I have in the recent past reviewed a broad spectrum of the documentation from GPU-Nuclear/Exelon/AmerGen surrounding this problem 3), and generated a number of expert opinions and affidavits 4)based on that documentation.

I have consistently argued that the external UT data acquired in 1993 following the removal of the sand were not sufficient to fully characterize the degradation of the drywell liner while the internal UT data did also not sufficiently account for all the corrosion that had occurred in the sandbed region. This leads to considerable uncertainty in our knowledge of the current state of the sand bed region.' Unfortunately, but understandably, the predictions of the residual safety factor did not attempt to estimate this uncertainty numerically.

') Commissioners' Order (Requesting additional briefs), 5/28/08

2) Original wall thickness is nominally 1.15 inches. Residual wall thickness of less than 0.7 inches has been documented.

3)See Citizens' Exhibit B, Attachment 2 and Citizens' Exhibit C, Attachment I

4) See Section VI: Additional References, Anthology of Memoranda, Affidavits etc.

One does not therefore at this point in time know with any kind of confidence whether the drywell liner still complies with the code. Repeating the same calculations again with the latest input data, would not reduce the uncertainty, but could allow the uncertainty to be better quantified; by performing sensitivity analyses 5). However, deriving some sort of confidence limits from the existing data either by refining the contour plots or similar methodology, as Judge Barrata suggests, may well lead to very large confidence intervals spanning 'a large range from compliance to non-compliance, which in the end would not be sufficiently determinative. The only real solution to the problem is the acquisition of additional data (in areas not previously examined by UT) that would permit a realistic representation of the corroded surface in the sandbed area, which could then be used in a 3-D model for the FEA and subsequent sensitivity study. Fortunately, advances in measurement techniques that have taken place since the existing monitoring regime was devised in 1993 have made such an approach totally realistic.

II. The Consequences of Weakening of the Shell Although I am not a structural engineer I understand from other testimony that the factor of safety during refueling is precariously near or below the minimum of 2.0 embedded in the CLB6 . If by modeling a safety factor of 2.15 were calculated, then the confidence limits could not be larger than +/-0. 15 or 7.5%. Judge Baratta seems to be suggesting that these confidence limits, or by implication the uncertainty in additional modeling results using available measurements, could be established by a series of sensitivity analyses7 ). I believe this is a pragmatic and sensible first step, but, if the full range of uncertainty is taken into account, is unlikely to produce a definitive result.

In 2006 NRC commissioned a study by Sandia Laboratories to analyze the structural integrity of the Drywell Containment at Oyster Creek 8). Very broadly the results showed that in the non-degraded (as built) state the safety factor in the sandbed region was 3.85 9) while in the degraded study the estimate was 2.15 .10). This corresponds to a reduction of structural strength of 44%. The Sandia result is based on the average remaining wall thickness in the sandbed region in each 36 degree bay, calculated using the 1992 external measurements, which on average showed thicker walls that the latest measurements.

Thus, the Sandia study probably overestimated the absolute factor of safety.

Furthermore, the Sandia Study did not address the uncertainty (confidence limits) in the prediction, although it did point to many sources of uncertainty and suggested that more attention should be paid to the degree of degradation rather than the absolute prediction of the factor of safety. Supporting the view that the Sandia Study probably overestimated the factor of safety, Dr. Hartzman of the NRC Staff has testified that if the contour plots

5) AmerGen proposes to repeat the past measurements, at the identical locations, for use in a new modeling effort. This may remove some uncertainty in the data, but does not remove the uncertainty relative to the areas that had never been measured.

6) See ASLB Initial Decision 12/18/07, page 19

7) See ASLB Initial Decision 12/18/07: Additional Statement by Admin. Judge Anthony J Baratta, PhD.

8) Sandia Report Sand207-055: Structural Integrity Analysis of the Degraded Drywell Containment at the Oyster Creek Nuclear Generating Station, Jan. 2007 9)Sandia Study (NRC Staff Ex. 6) at 70

10) Sandia Study page (NRC Staff Ex. 6) at 79 2

of the measured areas that I produced are correct, the factor of safety would be approximately 1.91) 111. The Nature of the Available Corrosion Data 12)

In order to perform not only the structural integrity calculations, but the sensitivity analysis as well, one needs to discuss first the uncertainties underlying the available data.

Before discussing our actual knowledge of the degree of corrosion damage of the drywell liner a fundamental principal of confidence interval (or confidence limits) needs to be understood. It is entirely reasonable (and customary) to assess the quality of a bin of say a thousand bolts by sampling a few, measuring the property of interest of the few, and defining from the results the average property of the lot as well as the probability that the individual will vary from the average by an amount given by the confidence limits defined by the desirable degree of confidence - 95%, 99% or even better. This is so because the bolts all were manufactured by the same machine and can therefore be assumed to belong to the same family of bolts, which can be characterized by a mean and a probability density customarily referred to as the Gaussian distribution curve.

It is difficult to apply this principle to the distribution of corrosion in the sandbed area for two main reasons. First, not all of the longitudinal sections (called bays) belonging to the former sandbed area have suffered the same degree of corrosion. Second it has been assumed that in the bays, where severe corrosion has been observed, these areas have been sufficiently characterized without the need to characterize the entire bay. In other words it had been assumed that the areas examined represented all the corrosion there was.

Because of this state of the current knowledge and related assumptions one needs to deal in depth with the following three concerns: .

• The measurements themselves (how accurate and reproducible are the measurements?)

" The paucity of measurements (are the few available measurements really representative of the state of the degraded shell?)

• How does one need to deal with the external point measurements (see below)

Accuracy and Reproducibility:

What is known about the residual wall thickness of the dry well liner has been obtained by means of ultrasonic thickness (UT) measurements. The accuracy (uncertainty) of UT measurements has basically two origins. The instrument and the operator account for a standard deviation of about 1.5to 2% of wall thickness for a single measurement (modern instruments may be slightly more accurate). If one adds in the reproducibility of

) NRC Staff response to initial Presentation and Response to Board questions, Aug 17, 2007, A28

12) The issues discussed here have been detailed in a number of previous submissions either as affidavits or direct testimony; (See list of submissions attached as Appendix A) 3

performing the same measurement repeatedly over time the overall standard deviation becomes 0.028 inches or in this case 3.5% of wall thickness. This means that for single measurements 13) the 95% confidence limits are of the order of +/- 7% of wall thickness.

For example, for a residual wall thickness of 800 mils, this would put the 95% confidence interval from 856 to 744 mils 14)

Two types of measurements have been made over the years: a) internal UT measurements 15) to monitor progressive corrosion and b) external UT measurements after the sandbed had been removed to assess the remaining integrity at points that were often located at visually identified minima in the external surface.

The Density of Measurements:

The internal measurements were made by using a 6 inch by 6 inch template with holes in it every inch so that within the 36 square inches 49 measurements could be made repeatedly over the years at the same spots. Considering that the surface area of the sandbed region for each bay is 73 square feet 16) it is unlikely that that measuring 0.3%

(36 square inches is 1/4 ft2) of this area on top of the sandbed would be representative of the entire sandbed, unless the corrosion in the sandbed were distributed uniformly.

Analysis of the data and the visual inspections show that the occurrence of corrosion is actually highly variable, as AmerGen's 17) testimony regarding external corrosion measurements confirms (see below).

A large number of measurements in a small are only improves the information about the small area but says nothing about the areas beyond.

The only quantitative knowledge of the areas beyond the grids comes from the external measurements. These measurements were made after the sand had been removed from the sandbed in order to confirm the extent of corrosion. It was established that in several instances corrosion damage increased in severity deeper down in the sandbed, thus showing that the internal grid data cannot be representative of the entire region. Again, the extent of the external UT measurements covered an area of varyingly about 10 - 30 %

of any one bay.

Typical results for Bay 1are shown in Figure 1 of Citizens Exhibit C1, Attachment 1I").

The area covered by the measurements is 27 ft2 or about 1/3 of the total area of the bay 13)AmerGen has very rarely repeated measurements in order to improve their accuracy.

14) It should be noted that neither AmerGen nor its predecessor ever examined the accuracy and/or reproducibility of the UT measurements.

15) Ultrasonic wall thickness measurements 16

) NRC Staff Ex. C at A28

17) AmerGen Ex. B Part 3 at A18: "portions of the exterior surface of the drywell shell of the sandbed region ... have a very uneven surface." Clearly corrosion in the sandbed region was not uniform.

1 The Figure shows the area where measurements had been made. Each measurement is represented by the black square. The vertical and horizontal coordinates are in inches from a reference point roughly located at the top of the sandbed. The values of the measurements (residual wall thickness) have been translated to contour lines representing the average residual wall thickness within the area contoured by the plots.

4

and it is clear that corrosion could have extended beyond -60 inches and further into the non-measured areas. Furthermore, corrosion did not occur only near the top of the sandbed (the reference point for the top of the sandbed being roughly at 0 inches).

Because the density of measurements is very low and clustered around two areas it is clearly difficult to depict what the surface might have really looked like. Drawing contour plots as shown was one way of presenting the data. However, the algorithm used to generate contour plots essentially calculates average wall thicknesses between one point and all the surrounding ones. No consideration is given to the fact that each measurement only represents the residual wall thickness at a specific point (z-direction), and no information has been gained about how far the degradation extends in the x-y dimensions. These are equally important with regards to integrity. This is probably why Judge Barrata suggested to perform a series of sensitivity analyses using a methodology similar (but not necessarily identical) to the one used to generate the contour plots.

IV. The Sensitivity Analysis There are a number of crucial inputs to the structural integrity analysis of the degraded Drywell containment. These deal with:

• Material properties;

• The geometry of the vessel;

• The extent of the degradation described above.

The sensitivity analysis aims at determining the degree of variation of the safety factor as a function of uncertainties in the any of the above parameters. With respect the extent of degradation for which standard deviations and confidence limits have been established it should be possible to-define probability densities for each measured point. This should be done for the external measurements in particular because they are single point measurements that are quite variable. By means of a Monte Carlo simulation combined with the extrapolation techniques I have proposed (or any other applicable statistical stochastic procedure), a series of sensitivity analyses could be calculated, which could be used to estimate the lower 95% confidence interval for the output.

It should be noted, however, that in light of the few data points, absence of information of corrosion in the x-y directions, the missing information about a large extent of the surface area, and the lack of any measurements of the actual geometry, the confidence limits will very likely be large and perhaps unreasonably so. That may mean that the lower confidence interval of the predicted factor of safety could be very much below two, leading to an indeterminate result.

V. How to approach the Problem For all of the above reasons we think that the question of whether the drywell meets the safety requirements can only be resolved by taking additional measurements in the areas of interest not previously explored in detail. Fortunately, since 1993, when the current 5

measurement regime was devised, technology has advanced considerably. It should be possible to. actually map the residual wall thickness over the entire area of the individual bays by means of automated instruments using UT, MFL (magnetic flux leakage), PEC (Pulsed Eddy Current) or a combination of these techniques 19). The digitized maps of the surface thickness could then be used directly as input to the finite element analysis of the residual strength of the vessel.

In addition, I understand that Stress Engineering Services has proposed using a laser technique that could measure the shape of the vessel.

VI. Additional References (Anthology of Memoranda, Affidavits, etc. in Support of Citizens' Contention re. Oyster Creek Dry Well Corrosion)

• Nov. 10, 2005, Memorandum R. H. Hausler to Paul Gunter, NIRS, re. Oyster Creek Drywell Liner Corrosion; in support of Citizens' Request for Hearing and Petition to Intervene, Nov 14, 2005

• Feb. 4, 2006, Memorandum R.H. Hausler to Paul Gunter, Richard Webster, Esq.,

re. Oyster Creek Drywell Liner Corrosion,

• March 16, 2006, Memorandum R. H. Hausler to Richard Webster, Esq., Paul Gunter, re. Oyster Creek Dry Well Corrosion:Additional Evidencefor Continued Corrosion

• May 3, 2006, Memorandum R.H. Hausler to Richard Webster, Esq., Paul Gunter, Oyster Creek Dry Well Corrosion: Comments re. "Audit Q&A (Questions No.

AMP -141, 210, 356) dated 4/5/06 Ref ML060960563

• June 9, 2006, Memorandum R. H. Hausler to Richard Webster, Esq., re.

StatisticalAnalysis ofAmerGen-Exelon UT Data,Dry Well Corrosion,

• June 12, 2006 Memorandum R.H. Hausler to Richard Webster, Esq., re Discussion of CorrosionMonitoring Methodologies at Oyster Creek Nuclear PlantDry Well 0 June 20, 2006 Memorandum R.H. Hausler to Richard Webster, Esq., re. Oyster Creek. Pitting Evaluation of Dry Well Liner in Sandbed Region 0 June 22, 2006, Memorandum R.H. Haulser to Richard Webster, Esq., re.

Discussion of CorrosionMonitoring Methodologies at Oyster Creek Nuclear PlantDrywell

• April 25, 2007, Citizen Exhibit B, Attach. 3, Memorandum to Richard Webster, Esq., Update of CurrentKnowledge regardingthe State Integrity of OCNGS Drywell Liner and Comments Pertainingto the Aging Management Thereof

• July 18, 2007, Citizen Exhibit B, Attach. 4, Memorandum to Richard Webster, Esq., Review of Fitnessfor Service Assessment of Oyster Creek Drywell on Basis of Extended Data Analysis

• Aug 16, 2007, Citizen Exhibit C, Attach. 2, Memorandumri, Response to ASLB re Ques.tions concerningStatistics

19) e.g. Monitoring Average Wall Thickness of Insulated or difficult to access objects with pulsed Eddy Currrrent, by R. Scottini, H.J. Quakkelsteijn - RDT Group, The Netherlands IV Pan-American Conference on NDT, Buenos Aires, October 2007 (attached) 6

Aug 16, 2007, Citizen Exhibit C, Attach. 3, Memorandum R. H. Hausler, to Richard Webster Esq., FurtherDiscussion of the nature of the CorrodedSurface and the Residual Wall Thickness of the Oyster Creek Drywell.

Sep 13, 2007, Citizen Exhibit C-1, Attach. 1, Memorandum R. H. Hausler to Richard Webster, Esq., FurtherDiscussion of the External Corrosionon the Drywell Shell in the Sandbed Region.

7

IV e * .rm

• ti*. , *1

,43 0rc Alies .0cliube 2007 Monitoring Average Wall Thickness of insulated or difficult to access objects with Pulsed Eddy Currrent R. Scottini, H.J. Quakkelsteijn - RTD Group, The Netherlands Abstract The method of Pulsed Eddy Current (PEC) has been successfully applied in corrosion detection for several years now. Whereas field experience on insulated objects has grown significantly, the technique's characteristics make it also highly suitable for other field situations where the object surface is rough or inaccessible. Because (surface) preparations can be avoided the tool provides a fast and cost-effective solution for corrosion detection. Due to the high repeatability accuracy PEC technology is specially of interest for monitoring purposes.

An overview of the fundamentals and the INCOTEST-- pulsed eddy current tool for corrosion detection is presented and application ranges are discussed. Several field applications other than insulated objects are presented. These range from the inspection of objects covered fire proofing, to rough or corroded surfaces, coated objects and objects covered with marine growth.

These spin-offs offer interesting possibilities in many areas of industry such as sub sea piping, offshore jackets, civil engineering and FPSO ship hull inspection.

Introduction Corrosion under insulation is a major concern for the owners and operators of almost'all carbon steel installations and structures. Periodic or continuous inspection of objects for occurrence of corrosion or monitoring the extent and severity of known corrosion areas should ensure operation of the installation within the safe zone.

To operate the installation at minimum cost, new techniques can be applied to minimise the overall maintenance and inspection costs. Such techniques can aim at reducing the total number of activities either by reducing the number of selected areas to look after or by reducing the overall costs per inspected area. The latter, for instance, is possible by reducing the peripheral costs of inspection (preparation, cleaning, access etc.).

The INCOTEST pulsed eddy current tool can assist by bringing down both the number of selected areas and the peripheral cost in several applications.

This tool was deve!oped for the detection of corrosion Fig 1: Example of under insulation (CUI). It allows the detection of wall insulated objects.

thinning areas without removing the insulation. Using this tool to indicate the affected areas can lead to significant cost reduction. Fewer areas need follow-up and less insulation needs to be removed. Also, in case of asbestos insulation the safety hazards are diminished.

INCOTEST applies pulsed eddy currents for the detection of corrosion areas. A pulsed eddy current technique uses a stepped or pulsed input signal, whereas conventional eddy currents use a continuous signal. The advantages of the pulsed eddy current technique are its larger penetration depth, relative insensibility to lift-off and the possibility to obtain a quantitative measurement result for wall thickness.

This leads to the characteristic which makes it suitable for the detection of CUI: no direct surface contact between the probe and the object is necessary. Also, this tool can be employed in other field situations where the object surface~is rough or inaccessible. After a brief introduction of the theory, some of these applications are discussed.

Pulsed Eddy Currents for corrosion detection The applied operating principle of pulsed eddy currents can vary from system to system. In order to obtain a quantitative reading for wall thickness INCOTEST uses a patented algorithm that relates the diffusive behaviour in time to the material properties and the wall thickness. It operates only on low alloy carbon steel.

InsultionMagnetic field Eddy Currents Fig 2: Principle of operation INCOTEST.

The principle of operation is illustrated in Figure 2. A pulsed magnetic field is sent by the probe coil. This penetrates through any non-magnetic material between the probe and the object undelr inspection (e.g. insulation material). The varying magnetic field will induce eddy currents on the surface of the object. The diffusive behaviour of these eddy currents is related to the material properties and the wall thickness of the object.

The detected eddy current signal is processed and compared to a reference signal. The material properties are eliminated and a reading for the average wall thickness within magnetic field area results. One reading takes a couple of seconds. The signal is logged and can be retrieved for later comparison in a monitoring approach.

75%

Fig 3: Display of INCOTEST showing AWT reading (top left), logged inspection grid (bottom left) and the decay of the eddy currents (bottom right).

The area over which a measurement is taken is referred to as the footprint. Probe design is such that the magnetic field focuses on an area on the surface of the object. The result of the measurement is a reading of the average wall thickness over this footprint area. The size of this area is dependent on the insulation and object thickness, as well as the probe design.

Roughly, the footprint can be considered to be in the order of the insulation thickness. Due to the averaging effect, detection of highly localised defects types like pitting is not reliable with this tool. This effect is illustrated in Figure 4.

rlwgtU loss:

Irregaear "W1sloss (eg. cmaratoh under fthwktdoa):

AlIrze wa*1 tphickness foo4"in iVry toaali~ed co'rrosIo (Tike pittnRJ Avemee wS.J tfuckn ts s M3inl=Wm wa khid=3 Fig 4: Difference between average and minimum WT within the footprint area.

Although the average wall thickness reading is not a direct replacement of the commonly used UT obtained minimum wall thickness a quantitative result is obtained that can be interpreted unambiguously.

The outer application ranges of the INCOTEST tool can be described by:

• Low alloy carbon steel

• Pipe diameter > 50 mm or 2"

• Nominal wall thickness between 6 mm and 65 mm

• Insulation thickness up to 200 mm

• Sheeting thickness up to 1 mm stainless steel, aluminum, galvanised steel

• Object temperature > -100°C to < +5000 C These ranges are determined on condition that a reliable signal can be obtained under regular field conditions.

Inspection approach As with any other NDT technique, the pulsed eddy current technique has its own merits and cannot be a direct substitute for an existing NDT technique in an existing NDT inspection program. The characteristics of INCOTEST result in the application of the tool with various intentions. Firstly, the reduction of surface preparations may be an incentive to use the tool.

No cleaning, grinding or removal of coating and insulation is required.

Secondly, on-stream screening for corrosion areas can be the objective. This means detecting defects is more important than sizing them accurately. It may be done to bring some ranking in a large number of structures or objects that would otherwise not get any attention because conventional inspection is too costly. Another application can be to select areas for follow-up.

For instance, in a pre-shutdown inspection the items that need follow-up during a shutdown can be identified.

On-stream monitoring of corrosion areas using INCOTEST is another approach that is of interest because of intrusion on the process is kept to a minimum. The data of previous measurements can easily be retrieved and compared.

Finally, in a risk based inspection approach a choice is made for the level of information required and the necessary certainty for inspection of a particular object. This leads to a choice for a non-destructive testing approach in which pulsed eddy current can be one technique.

Field applications Fireproofing Many foundations in installations, such as skirts of process columns and the supports of spherical storage tanks, are covered with a layer of fireproofing for obvious safety reasons. Small cracks or damages to the fireproofing may cause ingress of water, resulting in corrosion underneath the covering. The deterioration process can not readily be detected from the outside. Failing adequate condition monitoring, the deterioration process may eventually cause the object foundation to collapse with disastrous results.

As these fire proofing materials are non-magnetic and non-conducting, the magnetic field can freely propagate between the probe and the object under inspection. Hence, pulsed eddy current can be used to detect corrosion areas without removing the fire proofing material.

To obtain a picture of the foundation's Fig object: support on 5: Application concrete legs covered of spherical storage tanks.

condition, measurements are taken in several . .. . .. . ..

. s...

p o in ts of a d efin e d g rid . O n th e s u p p orts of spherical tanks a rapid screening is done by taking readings on four wind directions distanced 100mm-150mm apart axially and starting 300mm from the foot. This results in about 100 readings per support leg. In one inspection day all eight support legs of a tank can be.

screened and reported.

Walithickness West 90 S70 I--,-W est--- NomninýaI Fig 6: Graphical presentation of results on one sphere leg.

All average values meas ured are presented in a table together with a graph of the results indicating areas of interest for further action. These results can be used for strength calculations indicating the necessity whether or not to take action on the support leg.

Again, using INCOTEST the owners/operators of these structures can find out the current condition in a rapid and cost-effective manner.

Sub sea applications Many bank-protections, ports and waterworks in areas with a soft soil consist of steel sheet pilings. These sheet pilings have only a very limited protection againstthe elements. As a result the unshielded steel surface will be attacked by various forms of corrosion, among Accelerated Low Water Corrosion (ALWC). Similar situations occur for instance at risers and the support pillars of jetties. In all these situations both time and money are saved by using the ability of pulsed eddy current to penetrate dirt and marine growth Fig 7: underwater and through marine growth: sheet Maintenance including coating the surface is a costly action. The need to create a clean and dry environment below water level, and in the tide zone is, the most expensive.

The conventionally used methods of UT require extensive cleaning. Because no cleaning is necessary the use of INCOTEST in this situation leads to a faster inspection. The inspection can be carried out both above and below water level. Based on this result further maintenance can be done creating only localised clean and dry areas.

Even more inaccessible areas are sub sea piping and ship hulls of FPSO's. Many sub sea piping is covered with coatings or concrete. Inspection of these pipe lines is mainly done with intelligent pigs, however that requires that the line is taken out of service. Inspection of INCOTEST from the outside is possible with the use of divers or a ROV. Main aim it to obtain inspection data, without the associated inspection costs, in order to decide if immediate follow up is necessary or to determine the next inspection interval.

Fig 8: Inspection of ISO ship hull.

Monitoring Another advantage is also apparent in the relatively small lift-off ranges of several millimeters coating material or directly on the object itself. The repeatability of INCOTEST is 2% and thus an excellent tool for monitoring. Although UT is very accurate, it is commonly known that the repeatability of UT is relative poor.

Once a defect is identified, e.g. by mapping with UT to determine the weakest point, a INCOTEST probe is positioned and measures every day, week or month. At PEMEX, The "Ing. Antonio M. Amor" refinery in Salamanca, Mexico encountered an interesting application for INCOTEST; Pemex performed a lot of experimental inspectionsin 2005. Inspection programs on insulated tank walls were established where, in the past, no inspectionscould be performed because of accessibilityproblems. During a shutdown a severe wall loss was detected in the shell of a heat exchanger containingHydrochloric acid. Because a spare was not available, and in order to maintain safe production, Pemex found an interesting solution. "Afterbasic repairs,Pemex decided to monitor the shell thickness by performing INCO TEST with an interval of 15 days.

Due to the coating and operating temperatureit was not possible to perform UT A wall reduction of 16% was detected in only a few weeks. However, Pemex was able to continue production under safe conditions until a replacement was available. Pemex gained a lot of experience, which will be implemented in our inspection plans", says Ing. Jorge Galvan Pena, Superintendente de Inspeccion Tecnica Fig 9: Monitoring heat exchanger shells.

Conclusion Beside insulated objects INCOTEST proves a suitable application for situations where access to or preparation of the object surface is hampered. The application of this pulsed eddy current technique can be done with several different inspection approaches. Because of its unique characteristics it can play an important role in-the inspection strategy or RBI approach of an entire installation. Practical examples have been given for situations where dirt, corrosion, water, concrete or coating material hamper direct surface access. Because (surface) preparations can be avoided the tool can provide a fast and cost-effective solution for corrosion detection.

References

1. Lara P.F., "TEMP - An Innovative System to Measure the Wall Thickness of Pipes, Tanks, and Vessels through Insulation", ASNT Fall Conference, p.157, September 15, 1991.

2. Raad J.A. de, "Novel Techniques for Outside Inspection of Plant Pipework", Insight Vol 37 No 6, p409-p412, June, BINDT, 1995.

3. RTD, "Pulsed Eddy Currents - a novel method for accurate wall thickness measurement through insulation", Insight Vol 37 No 6, p452, June, BINDT, 1995.

4. Cohn M.J., Raad J.A. de, "Nonintrusive Inspection for Flow-Accelerated Corrosion Detection",

ASME 1997 PVP Vol. 359, Fitness for Adverse Environments in Petroleum and Power Equipment p.185-192, July, 1997.

5. Wolters J. Th., "Een Revolutionaire Wanddiktemeting", KINT newsletter 33, p. 4 , 1997.

6. Raad J.A. de, Wolters J.T., Vries R.P. de, "Assessment of the Pulsed Eddy Current Technique:

Detecting Flow-Accelerated Corrosion in Feedwater Piping", EPRI report TR-1 09146, December, 1997.

7. Stalenhoef J.H.J., Raad J.A. de, "MFL and PEC tools for plant inspection", Proceedings of ECNDT Copenhagen p. 1831, May, 1998.

8. Raad J.A. de, "PEC (pulsed eddy current) and MFL (Magnetic Flux Leakage) for NDT applications", International Pipeline Conference, Calgary, Canada, June, 1998.

9. Cohn M.J., "Comparison of Flow-Accelerated Corrosion prediction and field measurement results",

ASME PVP Vol 392 Service experience in fossil and nuclear power plant, p15-24, 1999.

10. Stalenhoef J.H.J., Raad J.A. de, "MFL and PEC tools for plant inspection", Insight Vol. 42, No. 2, February, BINDT, 2000.

11. Wassink, C.P., Robers M.A. "Condition monitoring of Inaccessible Piping" Proceedings of WCNDT Rome, Session Industrial plants and structures, Lecture IDN075, 2000.

12. Wassink, C.P., Robers M.A. "Condition monitoring of Inaccessible Piping" Insight Vol. 43, No. 2, February, BINDT, 2001.

13. Vries R.P. de, "Degradation of covered object foundations", KINT newsletter, 2001.

14. Vogel B., Wolters J., Postema F.J., "Pulsed eddy current measurements on steel sheet pilings",

Proceedings of Structural Faults and Repair 2001, London, UK, July 2001.

15. Whytock S., "Measuring remaining wall thickness of insulated / fireproofed / coated and bare components", Proceedings of NACE Central Area Conference, Corpus Cristi, TX, USA, October 2001.

16. Robers M., Scottini R., "Pulsed eddy current in corrosion detection", Proceedings of 8th ECNDT Barcelona, Lecture IDN251, June 2002.

About the authors Mr. Scottini earned his masters in material engineering at the University of Trento, Italy. Since 1997 Mr. Scottini is employed by RTD Group and has been actively involved in the technical development of INCOTEST and the development of new applications. Mr. Scottini is considered the leading technical expert on PEC.

Mr. Quakkelsteijn earned his bachelor in Technical Business Administration at the TH-Rijswijk, The Netherlands. Since 1997 Mr. Quakkelsteijn is employed in RTD Group and has been involved on on-stream leak tightness testing, advanced ultrasonic tube testing and is now overall responsible for INCOTEST in the RTD Group.

EXHIBIT CR 2 STRESS 13800 Westfair East Drive, Houston, Texas 77041 -11[

ENGINEERING Phone: (281)955-2900 Fax: (281) 955-2638 Website: www.stress.coi SERVICES INC. HOUSTON

• CINCINNATI

• NEW ORLEANS

• CHICAG(

SENIOR PRINCIPALS President Joe R. Fowler, Ph.D., PE.

Senior Vice President W. Thomas Asbil. PE, July 15, 2006 Vice Presidents Ronald D. Young, Ph.D., P..

Clinton A Haynes Jack E. Miller, P.E J, Randy Long, P.E Mr. Richard Webster SES Project No.: 131377 PRINCIPALS James W Albert PF. Staff Attorney Claudio Allevato. Corp. LIII Kenneth Bhalla, Ph.D. Rutgers Environmental Law Clinic Mark A Bennett, P.S.

Richard S. Boswell. P.R. 123 Washington Street Helen Chan, C.P.A.

John F. Chappoll. P.E. Newark, NJ 07102 S. Alien Fox, RE.

Andreas Katsounas; Tel: 973 353-5695 Paul J, Kovach, P.E, Terry M, Leclhinger rwebster(&kinov.rutgers.corn Douglas L. Marriott, Ph,D Christopher Malice. Ph D.. P .

Charles A. Miller, P.E.

George Ross, Ph.D.

Subject:

Cursory Check of Structural Analyses, Oyster Creek Drywell Vessel Ten Shackeltord David A Tekamp, P.E.

Kurt D.Vandervorn, Ph D., P.E.

Kenneth R. WMeber, P.E.

Dear Mr. Webster:

Robert ESWh*n, P.R.

SENIOR ASSOCIATES/

STAFF CONSULTANTS Recently, you requested that Stress Engineering Services, Inc. "consider several Christopher Alexander Glen A, Aucoin. P.E. documents that you provided and others that were made available to us through Richard C. Biel, PE.

Michael J Effenberger, PE. internet link references from the U. S. Nuclear Regulatory Commission. These Kimberly 0. Flesrer, P E Greg Garic. P.RE documents concern the license renewal of the Oyster Creek Nuclear Generating David L Garrett, Ph.D, Robert . Gordon. Ph.D., PE.

David P. Huoy, P.E.

Station.

Kenneth R. Riggs. PH.D, RPE.

Bobby W VWight. P.E SENIOR ASSOCIATES One issue of contention in the license renewal at hand is whether the corroded Rafik Boubenider, Ph.D.

Donnie Curington drywell shell retains adequate strength for continued service. Your specific Steven A Garcia Mark Hamilton instructions were to review the structural analyses and comment on the approach used Wiliam A Miller John M.Moore to assess their adequacy. Thus, we did not address any issues related to either the Ronald A Morrison. P.E.

Thomas L. Power, Ph.D.

preexisting corrosion damage or potential ongoing corrosion of the vessel, unless it Brian S. Royer Mahmod Samman. Ph.D.. P.E.

was salient'to our review of the structural analysis work.

Ramonr I, San Pedro, P.E.

Daniel A Plls, P.E.

Matthew J. Stahl. DEng, P.E.

Lane E. Wilson This report contains two sections. The first section addresses the general structural STAFF CONSULTANTS analysis methods and results. The second section addresses the ASME Code Ray R. Ayers, Ph.D, P.E.

J. KirkBrownlee, RE. provisions. In both sections, it is important to note that our comments and opinions Clinton H. (Clint) BriLtP.E.

Yusong Cao, Ph.D. are based on a severely limited review that only touches the highlights of the Joe Frey. RE.

Mike W. Guiltot, Ph.D., P.E. respective subjects. A more detailed review is needed to address these subjects with Lod C. Hasselbring, Ph.D.. P.E.

• Daniel Krzywicki. RE.. CSP the depth of study necessary to uncover the fundamental. differences between the Charlie Ribardo, Jr.. Ph.D.

Jackie E. Smith, P.E. work that was done in support of the license and the state-of-the-art in structural ASSOCIATES Lyle E. Breaux. P.E.

analysis.

P. James Buchanan Roger D. Cordes. Ph.D.

Nriperldu Dules, Ph.D.. P.E.

Kenny T. Farrow, Ph.D.

Structural Analyses Brett A. Hormberg Stuart J. Harbert, Ph.D.

At issue is the structural adequacy of the drywell shell, which has the shape of an David Renzi Chad Searcy. Ph. D.

inverted light bulb. The primary structural concern is the drywell shell's ability to Obaidullah Syed, P.E.

Leo Vega resist buckling with an adequate margin. for continued safe operation.

Kevin Wang. Ph.D.

SENIOR ANALYSTS HrfanBaig. PhD, The structural analysis results offered by AmerGen were obtained using typical Lixin Gong, Ph,.D Dilip Maniar, Ph D. techniques for the period of time in which the analyses were performed. Due to the Bo Yang, Ph.D.

ANALYSTS limited computational power. that was readily available at the time, the computer-Julian Bedoya Rhett Dotson aided analysis performed by General Electric (GE) utilized relatively small slices of

Mr. Richard Webster Rutgers Environmental Law Clinic July 15, 2006 the vessel, idealized geometries (perfect spheres, cylinders, etc.), and required computationally efficient calculation techniques. Calculated buckling load behaviors for the idealized geometries were subsequently adjusted using assumptions or "capacity reduction factors" for surface irregularities, plasticity, and local buckling; and the resulting adjusted values were taken as representative of the actual buckling load. GE compared the calculated buckling loads with the imposed loads, and safety margins were determined for comparison to ASME Code minimum requirements. Primarily because of these computational limitations, the finite element analysis performed by GE on the drywall vessel may not be adequate to capture its global behavior, which may be some combination of symmetrical and anti-symmetrical buckling.

The state-of-the-art has progressed far beyond the methods available to structural analysts in the early 1990s. Today, when reconstructing or reverse engineering existing structures, it is routine to use laser devices to generate "point clouds" that fully define the surfaces of pressure vessels, including any irregularities. The point clouds are digitalized, and the digitized information is converted into a mathematical representation of the actual surface shape, which is subsequently utilized for full three-dimensional, modeling. Since the resulting models account for actual surface waviness, unevenness, bulges, facets, and other potentially deleterious geometric surface conditions, there is no longer any need to resort to the use of "capacity reduction factors" to determine buckling loads, as the GE analysts were forced to do.

The digitized surface is converted into a form suitable for meshing and further processing using finite element analysis (FEA). The mesh areas are then assigned the corroded thicknesses at the specific areas where they actually occur, and any future corrosion allowance is subtracted from the thickness at this time. The FEA mesh density would then be generated as fine as needed to capture the stiffness that resists buckling. The simulated loads are then applied and the buckling load and shape are directly calculated without needing imposed perturbations or anything except the measured geometry and thicknesses.

Utilization of point cloud surface mapping techniques along with measurements that represent the actual wall thickness is thought to give the most accurate structural analysis results possible, with the fewest assumptions, using current technology. Three-dimensional thin shell analyses can be done today with few assumptions concerning stiffness and in a way that complies with Case N-284-1-1320.

ASME Code' Provisions At issue is whether the Code is the best tool available for determining the drywell's fitness for continued service.

In general, the Code establishes rules of safety relating only to pressure integrity and governing the construction 2 of boilers, pressure vessels, transport tanks, and nuclear components. Its ASME Boiler and Pressure Vessel Code, Section I11,Nuclear Components, and Section VIII, Rules for Constructionof Pressure Vessels, American Society of Mechanical Engineers, Three Park Avenue, New York, NY 10016 2 Construction,as used in the Code, is an all-inclusive term comprising materials, design, fabrication, examination, inspection, testing, certification, and pressure relief.

Mr. Richard Webster Rutgers Environmental Law Clinic July 15,2006 wording allows for some latitude in design and analysis methods, anticipates that deterioration of pressure vessels will occur, requires the use of engineering judgment, and recognizes the inevitability of technological progress in design and analysis methods. The following statements, which we excerpted from the FOREWORD of the current edition of the ASME Boiler and Pressure Vessel Code, support this contention.

"The Committee's function is to establish rules of safety, relatingonly to pressure integrity, governing the construction of boilers, pressure vessels, transport tanks and nuclear components, and inservice inspectionfor pressure integrity of nuclear components and transporttanks, and to interpret these rules when questions arise regarding their intent... With few exceptions, these rules do not, of practical necessity, reflect the likelihood and consequences of deterioration in service relating to specific service fluids or external operating environments. Recognizing this, the Committee has approved a wide variety of construction rules in this Section to allow the user or his designee to select those which will provide a pressure vessel having a marginfor deteriorationin service so as to give a reasonably long, safe period of usefulness. Accordingly, it is not intended that this Section be used as a design handbook; rather, engineeringjudgment must be employed in the selection of those sets of Code rules suitable to any specific service or need... The Committee recognizes that tools and techniques used for design and analysis change !astechnology progresses and expects engineers to use goodjudgment in the application of these tools."

Clearly, the authors of the Code never intended that its rules be used as the only arbiter of pressure vessel structural integrity. Neither did the authors intend the rules be used to extend, possibly unreasonably, the useful life a significantly corroded nuclear pressure vessel such as the drywell. Nonetheless,, some continue to rely on Code construction rules for these purposes.

They continue to do so despite the existence of tools such as three-dimensional thin shell analysis that have proven to be more than adequate for nuclear applications when applied in the presence of seasoned engineering judgment.

Respectfully Submitted, Richard C. Biel, P. E.

Staff Consultant Stress Engineering Services, Inc.

L J. Kirk Brownlee, P. E.

Staff Consultant Stress Engineering Services, Inc.

EXHIBIT CR 3 Available online at www.sciencedirect.corn SCIENCE cd)DIRECT- annals of NUCLEAR ENERGY Annals of Nuclear Energy 33 (2006) 354-369 www.elsevier.com/locate/anucene I ELSEVIER Including model uncertainty in risk-informed decision making Joshua M. Reinert, George E. Apostolakis

• Department of Nuclear Science and Engineering, Massachusetts Institute of Technology, Room 24-221, Cambridge, MA 02139-4307, USA Received 3 August 2005; received in revised form 28 November 2005; accepted 28 November 2005 Available online 19 January 2006 Abstract Model uncertainties can have a significant impact on decisions regarding licensing basis changes. We present a methodology to iden-tify basic events in the risk assessment that have the potential to change the decision and are known to have significant model uncertain-ties. Because we work with basic event probabilities, this methodology is not appropriate for analyzing uncertainties that cause a structural change to the model, such as success criteria. We use the risk achievement worth (RAW) importance measure with respect to both the core damage frequency (CDF) and the change in core damage frequency (ACDF) to identify potentially important basic events. We cross-check these with generically important model uncertainties. Then, sensitivity analysis is performed onl the basic event probabilities, which are used as a proxy for the model parameters, to determine how much error in these probabilities would need to be present in order to impact the decision. A previously submitted licensing basis change is used as a case study. Analysis using the SAPHIRE program identifies 20 basic events as important, four of which have model uncertainties that have been identified in the lit-erature as generally important. The decision is fairly insensitive to uncertainties in these basic events. In three of these cases, one would need to show that model uncertainties would lead to basic event probabilities that would be between two and four orders of magnitude larger than modeled in the risk assessment before they would become important to the decision. More detailed analysis would be required to determine whether these higher probabilities are reasonable. Methods to perform this analysis from the literature are reviewed and an example is demonstrated using the case study.

© 2006 Elsevier Ltd. All rights reserved.

1. Introduction determine by how much the probability of each event must be increased to have an impact on the decision. Analysis Very low-probability, high-consequence events are the must then be done to determine whether this increase is focus of reactor safety studies. Because of the limited num- reasonable.

ber of these events, there are large uncertainties regarding PRAs use models of a plant's structures, systems, and their probabilities of occurrence. Uncertainty in the core components (SSCs) to determine the probability of occur-damage frequency (CDF) and large early release frequiency rence of various events. Sometimes, however, there is no (LERF) can be separated into three classifications; param- consensus on the appropriate model to be used. It may eter uncertainty, model uncertainty, and completeness be that, because the system or process is not sufficiently uncertainty. This paper describes a methodology for the understood, there are differing opinions as to which model identification of basic events which have the potential to most accurately represents the system. This creates uncer-impact licensing basis decisions. We concentrate on appli- tainty in the model, which could be related to the structure cations of Level 1, at-power, internal-events probabilistic of the model, or its numerical assessments. This uncertainty risk assessments (PRAs) and on the decision-making pro- in the accuracy of the model introduces uncertainty in the cess related to licensing basis changes. Once these basic output of the model and, therefore, uncertainty in the out-events are identified, sensitivity studies are performed to put of the PRA. It i's this uncertainty that we refer to in this paper as model uncertainty (Mosleh et al., 1993).

Corresponding author. Tel.: +1 617 252 1570; fax: +1 617 258 8863. Nuclear power plant licensees may use PRA informa-E-mail address: apostola@mit.edu (G.E. Apostolakis). tion to apply for plant-specific licensing basis (LB) changes.

0306-4549/S - see front matter ©.2006 Elsevier Ltd. All rights reserved.

doi:10.1016/j.anucene.2005.1 1.010

J.M. Reinert, G.E. Apostolakis / Annals ofW uclear Energy 33 (2006) 354-369 355 Guidance for doing this is provided in regulatory guide sistent with the state-of-the-art, to support a risk-informed (RG). l.174 (USNRC, 2002) and includes a requirement regulatory framework (USNRC, 1995). RG 1.174 is a key to meet acceptance guidelines based on the plant's CDF document in this framework. It presents five principles of and LERF and the corresponding changes, ACDF and risk-informed decision making to be used for making deci-ALERF, resulting from the requested change. RG 1.174 sions regarding plant-specific changes to the licensing basis.

and the regulatory guidance on the definition and treat- These principles are:

ment of model uncertainty will be discussed in Section 2 of this paper. Model uncertainty is then discussed in detail 1. The proposed change meets the current regulations in Section 3. The intent is to establish a clear understanding unless it is explicitly related to a requested exemption.

of model uncertainty, its interpretation, theory, and how it 2. The proposed change is consistent with the defense-in-is handled in practice, as well as a review and discussion of depth philosophy.

model uncertainties identified as generally important in the 3. The proposed change maintains sufficient safety literature. The problem then becomes how to determine margins.

which uncertainties can affect the decision. The proposed 4. If the proposed change increases risk, the increase methodology for this is presented in Section 4. should be small.

This methodology begins with using the PRA of the 5. The impact of the proposed change should be monitored plant to determine the risk achievement worth (RAW) using performance measurement strategies.

importance measure (Cheok et al., 1998) of each basic event. The use of RAW to determine the effect of an event The first principle makes it clear that existing regulations on CDF is well understood. We use RAW in the same way, not related to the requested change must still be met. The but we also evaluate RAW with respect to ACDF. From second and third principles account for some of the com-the importance measure information, we determine which pleteness uncertainty that exists when assessing nuclear basic events could possibly affect the decision either power plant risk. This uncertainty is referred to as the through CDF, ACDF or both. "unknown unknowns," or the uncertainties that exist but Basic events that are identified as potentially important have not been identified. Because the uncertainty has not through the RAW analysis are cross-checked with those been identified, it cannot be quantified. Traditional that have been identified in the literature as having gener- defense-in-depth measures and safety margins (the "struc-ally important model uncertainty. This cross-check results turalist" approach to safety (Sorensen et al., 1999)) are in a list of basic events that have uncertainties that could requirements designed to protect against these uncertain-possibly affect the decision. These basic events are then ties. The fourth principle is the one we are concerned with analyzed qualitatively and quantitatively to determine their in this paper, requiringrisk increases to be small. Risk and impact on the specific decision.. risk increases are quantified using PRA. The fifth principle

. The benefit of this methodology is that important basic ensures that the results of a licensing base change are as events are identified with respect to the change-specific expected. Performance monitoring and measurement after decision at hand rather than through the use of a general the change provide feedback that can be used to identify definition of importance. This methodology also reduces and correct unexpected problems that result from the the number of uncertainties that must be analyzed exten- change and also to inform future changes. Also, the effects sively by allowing qualitative arguments based on of uncertainty on all of these principles must be considered, change-specific conditions. In its present, form, the pro- whether the uncertainty is explicitly included in the model posed methodology deals with the uncertainties associated or not.

with the modeling of events that appear in the PRA; it does Risk increases must be small. Small changes are defined not deal with model uncertainty that may affect the logical using the CDF and LERF acceptance guidelines of Figs. I structure of the PRA itself. and 2, respectively. The values of CDF, ACDF, LERF, and In order to illustrate the proposed method, a case study is provided in Section 5. In Section 6, we evaluate the important basic events from the case study. This helps us to understand how important the uncertainty might be in the context of this case study and whether it warrants an in depth review of its probability and the assumptions 10" underlying the calculation of its probability. Finally, Sec-tion 7 contains a summary and conclusions from this research.

2. Risk-informed decision making The US Nuclear Regulatory Commission (NRC) 10-1 1(r CDF -+-

encourages the use of PRA methods where practical, con- Fig. 1. CDF acceptance guidelines.

356 J.M. Reinert, G.E. Apostolakis I Annals of Nuclear Energy 33 (2006) 354-369 the associated uncertainties (USNRC, 2004a). Uncertain-U, ties in the PRA must be understood and accounted for, whether or not they are explicitly modeled.

X~ RG 1.200 also provides two ways to ensure the technical adequacy of a PRA in support of risk-informed regulatory decisions. The first is to meet the criteria of the American Society of Mechanical Engineers (ASME) PRA standard (ASME, 2002), as supplemented by the comments in RG 1.200. The second is to have the PRA peer reviewed using the Nuclear Energy Institute (NEI) peer reviewprocess 10 10"5 LERF-* (NEI, 2000), as supplemented by the comments in RG Fig. 2. LERF acceptance guidelines. 1.200.

2.1. Types of uncertainty ALERF to be used in these figures are supposed to be the mean values of the uncertainty distributions of these quan- Uncertainties can be categorized as either aleatory or tities. This paper focuses on Level I PRAs, so we will focus epistemic uncertainties (Apostolakis, 1993). Aleatory on the CDF/ACDF guidelines. Referring to Fig. 1, the hor- uncertainty reflects our inability to predict random obser-izontal axis represents the baseline CDF. This is the fre- vable events. For example, the flip of a fair coin is generally quency at which core damage is expected to occur at the accepted to yield heads with a probability of 0.50. How-plant if no plant changes are made. The vertical axis repre- ever, the number of times that heads will occur as a result sents the ACDF, the amount that the CDF is expected to of 10 coin flips in unknown. Only the probability distribu-increase, if the proposed LB change were made. tion of the number of heads is known. This is also referred Each nuclear power plant has an associated plant-spe- to as 'randomness' or 'stochastic uncertainty.' Epistemic cific CDF. Each LB change that a plant desires to make uncertainty represents our confidence in the model and has an associated plant-specific and change-specific ACDF. the numerical values of its parameters, e.g., that the coin These two risk metrics place a proposed change in one of is fair so that the probability of heads can be taken to be the three labeled regions in Fig. I. Uncertainty in the risk 0.50. A process may not be sufficiently understood and, metric calculations prevents an exact placement of a as such, a specific model may not be universally accepted change onto one of the three regions. Therefore, the values as being the right model. This type of uncertainty is also representing the dividing lines between the regions must be called 'state-of-knowledge' uncertainty or just 'uncertainty' viewed as indicative,; rather than definitive. Changes that (Zio and Apostolakis, 1996). We note that, unlike aleatory have a ACDF placing them in Region III are classified as uncertainties, epistemic uncertainties are associated with having a very small increase in risk and may be approved non-observable quantities, e.g., the parameters of models without the need for a quantification of CDF. Changes such as failure rates.

that are in Region II are classified as having a small The distinction of uncertainty into aleatory and episte-increase in risk and may be approved, but may require a mic is largely due to "practical aspects of modeling and more stringent review. Region II also sets an upper bound obtaining information" (Winkler, 1996). At their core, they of about 10-4 per reactor-year (ry- 1) on the baseline CDF. both refer to the problem of modeling real-world systems Changes in Region I do not meet the requirements of a with mathematical formulas, whether deterministic or small risk increase and will, in general, not be approved. probabilistic.

Fig. 1 also shows gradual shading, darkening as one Epistemic uncertainties can be roughly split into three moves upward and to the right, representing CDF and categories and is done in RG 1.174. These are: parameter, ACDF combinations that are closer to the boundaries model, and completeness undertainty. Parameter uncer-between regions. The darkness of the shading corresponds tainty is that which relates to the parameters of the PRA, to the level of review that the application will be given, such given a choice of model. Even with a known model, the that LB changes that have a representative point in areas of parameter values may still be unknown. In situations where darker shading, i.e., near the region boundaries, warrant a historical data is limited, this uncertainty may be, quite review that is generally more intensive. "The closer the esti- large. Examples of parameter uncertainties include equip-mates of ACDF and ALERF are to their corresponding ment failure rates, initiating-event frequencies, and human acceptance guidelines, the more detail will be required" error probabilities.

(USNRC, 2002). In many cases, there is limited knowledge and some dis-RG 1.174 requires that all sources of uncertainty be agreement on the proper model to represent a system. The identified and analyzed such that their impacts are under- result is that for a particular process, there are multiple stood at the technical element level, and on the CDF and competing models, each of which necessarily produces a LERF risk metrics. RG 1.200 states that "an essential different approximation of the same real-world system.

aspect of the risk characterization is an understanding of Because the correct model is unknown, there is additional

J.M. Reinert, G.E. Apostolakis I Annals oJfAuclear Energy 33 (2006) 354-369 357 uncertainty in the output of any model, representing the and Smith support this argument by measuring the model uncertainty in the model's itself. This uncertainty adds to uncertainty regarding the success criteria of Auxiliary the parameter uncertainty described above and is model Feedwater pumps and comparing it with the parameter uncertainty. The outputs of each reasonable model must uncertainty in the pump failure rates (Knudson et al.,

be considered, according to the degree of belief in the 2002). Model uncertainty is measured by varying the num-appropriateness of each model, to prevent exclusion of ber of pumps required, such that the system may be a 1-valuable uncertainty data from consideration. Several out-of-3, 2-out-of-3, or 3-out-of-3 system, assigning a methods have been proposed to accomplish this, including probability that each case is true. Looking at each uncer-the linear or otherwise combination of models weighted by tainty while ignoring the effects of the other, the parameter the analyst's belief that each model may .be correct (Apos- .and model uncertainty provide the following 90 percent tolakis, 1993), and the use of an adjustment factor on the confidence intervals for the system failure rate.

single most likely model (Zio and Apostolakis, 1996).

These and other methods will be discussed in Section 3. Parameter uncertainty 2.2 x 10-4 to 6.3 x 10-4 ry-r The PRA structure itself is model-dependent because Model uncertainty 1.9x 10-5 to 1.5x 10-3 ry-1 model uncertainty can affect the choice of success criteria.

For example, one model might say that two primary relief While the parameter uncertainty range spans about a valves are required to prevent core damage during a loss of factor of three, the model uncertainty range spans about offsite power event, while another might say that only one two orders of magnitude.

primary relief valve is required. In this case, while there still Bley et al. (1992) measure the impact of model uncer-remains (parameter) uncertainty in the failure rate of relief. tainty on CDF directly. In their work on a plant-specific valves, there is also uncertainty in how many relief valves PRA, they identified three model uncertainties that had are required. This latter uncertainty is model uncertainty the greatest potential to impact CDF: reactor coolant also. In cases where model uncertainty is treated by using pump (RCP) seal LOCA timing, low-end seismic fragility a single, conservative model, the effects of alternate curves for piping and D/C-electrical components, and seis-assumptions must be recognized. RG 1.174 recommends mically-induced relay chatter.' They chose two alternate using sensitivity studies to determine whether or not there assumptions for each model and assigned probabilities that are any assumptions or models whose results Would reduce each was true. This resulted in eight different sets of confidence in the conservatism of the chosen model. assumptions when all three models were inserted into the Completeness uncertainty is a type of model uncer- PRA. Each set of assumptions resulted in a different mean tainty, but is handled differently. It represents the uncer- value of CDF for the plant, which ranged from about tainty due to the portion of risk that is not explicitly 2 x 10-4 to 3 x 10-3 ry-1, with the most likely value being included in the PRA. It may be that, for a particular risk about 2 x 10-4 ry:1. The most likely value corresponds to contributor, the state-of-the-art has not evolved to the the low end of the range because the set of assumptions point where the risk can be modeled defensibly. This is with the highest probability of being true resulted in the the case with safety culture and organizational behavior lowest CDF. These results' show that modeling assump-in general. RG 1.174 states that "the influences of organi- tions can shift the mean value of CDF significantly.

zational performance cannot now be explicitly assessed."

Completeness uncertainty also includes risks that have 3. Model uncertainty not been identified.' This includes anything that has not been identified as a risk contributor, yet does contribute As stated in Section 2.1, there is no mathematical differ-to risk. Due to the nature of this type of uncertainty, it is ence between different types of uncertainty. They all refer impossible to quantify. Conservatisms, such as defense- to unknowns, the limit of knowledge about a real-world in-depth and safety margins largely exist to defend against phenomenon. "For the case of a finite.number of alterna-this type of uncertainty, as stated earlier. tive models, the model uncertainty is equivalent to param-Referring back to the acceptance guidelines of Fig. 1, eter uncertainty" (Buslik, 1993), with reference to Savage's the values of CDF and ACDF used are supposed to be epi- partition problem (Savage, 1972). The theoretical overlap stemic mean values, i.e., the mean values of the distribu- between model and parameter uncertainty can also be seen tions of CDF and ACDF. These distributions are largely by creating a parameter whose value is dependent upon the the result of propagating through the PRA the epistemic model used (Zio and Apostolakis, 1996).

distributions that represent the parameter uncertainties Methods to deal with model uncertainty include predic-that were explicitly included in the PRA model. Because tion expansion and model set expansion (Zio and Apostola-these mean values already include the effects of parameter kis, 1996). In prediction expansion, a single model is chosen uncertainty, as represented in the probability distributions as the best one to represent the system. However, it is recog-of the input parameters, they are fairly insensitive to nized that this model has uncertainties and may model some changes in these distributions. In contrast, model uncer- characteristics of the system better than others. Sensitivity tainties generally have a greater potential to affect these studies are performed on the various assumptions to ana-metrics thus affecting the approval decision. Knudson lyze the effects of the choice of assumptions on the model

358 J. M. Reinert, G.E. Apostolakis / Annals of Nucdear Energy; 33 (2006) 354-369 output. This uncertainty is dealt with by applying an adjust- expert given equal weight. The PACUA approach goes ment factor to the model results. The adjustment factor may one step further by including information about the confi-be multiplicative or additive, or both may be necessary. The dence in each expert. The experts are asked to produce dis-purely additive and multiplicative adjustment factor tributions for seed variables, for which data is known, and approaches can be seen in Eqs. (1) and (2), respectively: their opinions are compared to the known data. Experts y = y* +E (1) with superior performance when estimating the seed vari-y =y*

• E*m (2) able distribution are given higher weight when opinions regarding the system in question are elicited.

where y* represents the model prediction, E* represents the The final method under consideration here is the techni-adjustment factor, and y represents the adjusted model out- cal facilitator-integrator (TFI) approach, which takes put. E* may also have (aleatory or epistemic) uncertainty in advantage of many of the lessons learned from previous its value, due to limited data, for example. expert elicitation exercises. In this approach, the experts In model set expansion, the characteristics of the system are treated as a team, rather than individuals, each sharing under consideration are analyzed and models are created in their opinion separate from the consideration of the other an attempt to emulate the system based on goodness-of-fit experts. Individual elicitations are obtained. However, the criteria. The models may use different assumptions, and team works together with the TFI to integrate the data, require different inputs. Each model has its own advantages including the experts' knowledge of technical experts out-and disadvantages, including limitations on applicability. side of their own group, into a meta-model that attempts These models are then combined to produce a meta-model to represent the current total body of knowledge. Part of of the system. the TFI's role is to mitigate problems identified in behavior Several methods have been proposed regarding the con- science, such as the tendency for more dominant members struction of this meta-model. They include mixture (Apos- of the group to be given undue weight on their opinion.

tolakis, 1993), Bayesian updating (Winkler, 1993,), the With this background on model uncertainty, the distinc-NUREG-I 150 approach (Keeney and von Winterfeldt, tion between different classifications of uncertainty, the 1991), the joint US/Commission of European Communities' reason for these classifications, the theory behind the for-(EC) Probabilistic Accident Consequence Uncertainty malisms of model uncertainty, and practical applications, Analysis (PACUA) approach (USNRC, 1997), and the we now look at generic model uncertainties in PRAs that Technical Facilitator-Integrator approach (Budnitz et al., have been identified in the literature.

1996). Of course, all of these methods rely on expert opinion.

In the mixture approach, the set of plausible models is 3.1. Generic model uncertainties agreed upon from expert opinion and these experts agree on probabilities that each model is correct. The models A literature review provided a fairly extensive, yet man-are then combined linearly, with their weights correspond- ageable, list of major model uncertainties pertaining to ing to the probability of correctness. The result is a Level 1, at power, internal events PRAs. Insights from weighted average of the probability distributions that result the literature review will be used to learn more about the from each model. The multiple distributions should be pre- uncertainties that were identified as important in the case sented before they are combined, thus allowing an analyst a study. Much of the data comes from NRC-sponsored stud-more transparent look at the range of models that became ies. This review was not limited to NRC generated data, the meta-model. however, and a variety of sources was used. A discussion In the Bayesian approach, each model is integrated into of the results is provided here, with an emphasis on basic the meta-model using Bayes' theorem, using the following events relating to Level 1, at-power, internal events PRAs.

formula: NUREG/CR-4550 (USNRC, 1990) organized an expert fp(xlf,,.. ,fk) = f(x)

• g(f],..- ,fkJx) (3) panel to address several important uncertainties. Some were related to problems at individual plants and are where Jp(xV]i,. .. ,fk) is the posterior distribution resulting excluded here. These are:

from the combination of the individual models, fix) is the analysts' prior distribution, f.(x) is the distribution given " Probability of the failure of two check valves in series in by the ith available model, and g(f 1, ... ,fkkx) is the likeli- a PWR constituting a boundary between a high and a hood function. This method is theoretically very attractive low pressure system.

due its mathematical rigor and ability to incorporate all " Emergency core cooling system (ECCS) failure rates due types of information. However, it is difficult to implement to venting or containment failure. This refers to the "in large part due to the need to treat the thorny issue of operability of components in hostile environments.

dependence among models" (Winkler, 1993). PRAs normally assume 100% failure rate if equipment In the NUREG-1150 approach, multiple experts are are operated above their qualification limit. This data elicited to produce their own probability distribution of shows expected failure rates with respect to different the system in question, based on their own opinion. The types of components, different operating condition, individual results are then combined linearly, with each and different lengths of operation.

J.M. Reinert, G.E. Apostolakis / Annals of Xýuclear Energy 33 (2006) 354-369 359

• RCP seal LOCA probability. Results were given with Boiling water reactors:

respect to time after the initiating event and leak rate.

" Probabilities of innovative recovery actions for long- " Perform manual depressurization to allow injection with term sequences involving loss of containment heat low pressure injection systems. This is typically done by removal. The panel concluded that success probabilities operating the safety relief valves, are highly dependent on plant-specific features like cli- " Vent containment and align containment or suppression mate, location, staff training, plant design, and layout. pool cooling during a LOCA.

Results are given in terms of probabilities of repair ver- a Control vessel level during an ATWS in order to control sus time for various components. reactor power.

" Failure probability of using high pressure service water " Initiate standby liquid control during an ATWS.

spray in the dry well. " Inhibit the automatic depressurization system in order

" Battery depletion time. .to prevent instabilities that occur at low pressures.

• Diesel generator field flashing failure probability.

• Miscalibration of pressure switches that are important

" Hydrogen ignition probability on restoration of A/C for initiating and controlling the ECCS.-

power. " Initiate. isolation condenser in BWR plants of early

" Human actions to shutdown the plant failure design.

probability. " Control feedwater events. Control the feedwater system

• Secondary safety valve demand and failure rates. after a loss of feedwater event.

" Reactor coolant system depressurization failure " Recover offisite power.

probability. " Shed D/C loads after a Station Blackout in order to

" Common-cause fl-factor uncertainty ranges., extend battery life.

" Common-cause f-factor for air-operated valves.

Bley, Buttemer, and Stetkar argue that an adequate anal-NUREG-1764 (USNRC, 2004b) provided a list of ysis of event sequence tinling is, important in PRA analysis uncertainties in the reliability of human actions that were for a couple of reasons (Bley et al., 1988). Success criteria either known to be risk-important or had the potential to determination requires an understanding of sequence tim-be risk-important. They are broken into categories by plant ing. How plant parameterg change over time during an acci-type, pressurized water reactor (PWR) or boiling water dent sequence, etermines what equipment is necessary to reactor (BWR),.as follows. prevent core damage. Success criteria are often chosen Pressurized water reactors: based on deterministic thermohydraulic calculations using assumptions that are conservative. Calculation of the prob-

" Switch the ECCS from the injection mode to the recircu- ability of recovery is also dependent on the results of a lation mode in a LOCA scenario. sequence timing analysis. Human performance is highly

• Feed and bleed, particularly the use of pressurizer relief dependent on the time available for the operator to com-valves. plete actions, and the time available is calculated using an

" Provide water supply for auxiliary feedwater by moving analysis. of sequence timing. The authors analyze a number water from alternate sources into the auxiliary feedwater of risk-important parameter and model uncertainties and system when long-term cooling is needed. reach the following conclusions:, success criteria and recov-

" Trip the RCPs to prevent RCP seal LOCA on a loss of ery modeling are highly dependent on sequence timing; RCP cooling. determinations of the factors that affect operator perfor-

" Recover RCP seal cooling by aligning an alternative mance, including dependencies and competing demands, means of cooling. requires a detailed analysis; and simple analysis involving

" Recover emergency A/C or offsite power. mass and energy balance to determine their effect on

" Respond to an anticipated transient without scram sequence timing is often sufficient for PRA applications.

(ATWS) - failure of the reactor protection system, par- RG 1.200 (USNRC, 2004a) provides several examples ticularly the initiation of boron injection and including of key uncertainties when determining the technical ade-manual scram of the reactor and ensuring turbine trip. quacy of a PRA. Uncertainties in success criteria, human

" Depressurize during a steam generator tube rupture reliability, and the choice of a RCP seal LOCA model (SGTR). This includes the depressurization" of the pri- are included. In these cases, the choice of the model and mary and secondary systems and equalizing pressure how it is used may have a significant impact on risk.

between them. Sump performance was identified as important by the

" isolate steam generator during a main steam leak break NRC's Advisory Committee on Reactor Safeguards (Wallis, or a SGTR. 2004). Because of the nature of the sump, and the limited

" Shut power operated relief valve (PORV) blocking valve data that exist on how a sump might perform when needed, during a stuck open PORV event. it is difficult to estimate the probability that it will perform

• ,Isolate interfacing system LOCA during a LOCA in the successfully. Specifically, it is difficult to model how the strai-low pressure injection system. ner on the intake side of the pump will be affected by debris in

360 J.M. Reinert, G.E. Apostolakis I Annals of Nuclear Energy 33 (2006) 354-369

• the sump. There is some probability that the debris will clog ing basis change: The proposed change will change the the strainer and reduce the net positive suction head on the frequency at which some of the minimal cut sets occur.

pump sufficiently to effectively disable the pump. However, most of them will remain unchanged. Each min-o Interviews with NRC personnel also provided a number imal cut set that is unaffected will, therefore, appear in both of important model uncertainties (Interview). RCP seal terms on the right-hand side of Eq. (6) and drop out of the LOCA probability, battery depletion time, common-cause equation. It is clear that uncertainties affect the value of

• failure modeling, and modeling of sump plugging and pool CDF and ACDF. It is also clear that these uncertainties strainer plugging were identified. Emergency diesel genera- can change the outcome of a decision based on the accep-tor mission time and recovery modeling were identified tance guidelines in Fig. 1.

I also. This refers to how long the diesel generator is We propose a methodology for including these uncer-assumed to be needed in order to fulfill its mission, and also tainties in the decision-making process used to make risk-what mechanisms for recovery are credited in the PRA and informed licensing basis decisions in accordance with RG the probability of these recovery mechanisms. Success cri- 1.174. This methodology begins with using the PRA of a I teria determination is important, specifically with regard plant to determine the RAW importance measure of each to how many PORVs are required during a feed-and-bleed basic event.

evolution. Support systems may be important in ways that I are not obvious at first glance, especially when they have 4.1. RA W with respect to CDF the ability to cause comnon-cause failures across many systems. Sometimes, the risk-importance of these systems Importance measures are used in the ranking and cate-is missed and they are either not modeled adequately, gorization of basic events modeled in a PRA (Cheok introducing model uncertainty, or left out ofý the PRA, et al., 1998). The importance measure of most interest to introducing completeness uncertainty. Instrument air is us is the risk achievement worth (RAW). RAW is defined an example of a support system that many components as S in multiple systems depend on, but that may not seem risk-important itself unless attention is brought to these RAW,- (7) dependencies. The modeling of SGTR event tree was also considered important. where RAWj is the value of RAW for basic event j, R is the value of the model's baseline risk metric, and R+ is the

4. Proposed methodology value of the model's risk when basic event j is set to a log-ical TRUE. Each basic event, therefore, is assigned a value I The CDF is calculated as of RAW, by the PRA, that quantifies the factor by which a CDFbase Z fr(MCSbase,i) (4) plant's risk would increase if the associated basic event were assumed to be completely unreliable. RAW is a bounding measure that provides the maximum level of risk where CDFbase is the baseline CDF of the plant, as it is that a basic event could cause (Cheok et al., 1998).

normally configured. However, this calculation can be per-The meaning of RAW can also be viewed with respect to formed for any plant configuration. MCSbae,i is the ith the logic structure of the PRA. A basic event that is com-I minimal cut set, and fr(MCSbase,i) is the frequency at which pletely unreliable serves no risk function in the PRA. It is the ith cut set occurs in the baseline PRA model. as if the basic event were completely removed from the Uncertainties surround the value of the baseline CDF, logic structure. Therefore, the RAW of a basic event repre-

• since there are uncertainties in the frequency of the initiat-sents the factor by which a plant's risk would increase if the ing events and also in the conditional probabilities of basic event were removed from the plant. Since the risk occurrence of the basic events. These, same uncertainties metric in this case is the CDF, create uncertainties in the value of ACDF. The uncertain-ties in ACDF can have a significant impact on the decision .CDFIýas RAWj,CDF-b.se -

=~ bs whether or not to approve the change, as acceptance guide- CDFbaseý (8) lines are provided as a combination of CDF and ACDF.

I The significance of this impact can be seen by looking at The set of values for RAWj,cDFbase can easily be generated using the SAPHIRE program.

the definition of ACDF, ACDF = CDFanfcr - CDFbase, (5)

H where CDFafter is the CDF of the plant after the proposed licensing change. Inserting Eq. (4) into Eq. (5), we find that 4.2. RA W with respect to A CDF Importance measures can also be used to show areas in a PRA where uncertainty can have the greatest impact on the ACDF = Z fr(MCS,*af",i) - Z fr(MCSb,,,j) (6) change in risk that is proposed by the licensing basis change. To represent this importance measure, we start where MCSft~j is the frequency at which thejth cut set oc- with the definition of RAW above, Eq. (7), and note that curs in the PRA model as it exists after the proposed licens- the risk metric R in this case is ACDF. Therefore

J. M. Reinert, G.E. Apostolakis / Annals of Muclear Energy 33 (2006) 354-369 361 ACDF- CD Fthreshold RAWj,ACDF- ACDF (9) RAWCDFthrsh1d -

(12)

CDFbasc Noting the definition of ACDF, Eq. (5), we expand this where RAWCDFthreshold is the RAW value that will move equation to the CDF to the right and into a different region, and CDFthreshold is the value of CDF corresponding to the ver-CDF -CDF+

RAWjACDF -CDFfter -D ,base (10) tical lines between the regions of Fig. 1. The threshold CDFafte - CDFbas RAW value is dependent on the CDF of the plant and Inserting Eq. (8) into Eq. (10), we see that the ACDF of the proposed change and its value is change-specific. Remember that although the CDF has RAWj,ACDF uncertainty and is represented by a distribution of values,

- (RAWj,CDF-after) * (CDFafter) - (RAWj,CDF-base) * (CDFbase)

RG 1.174 calls for the mean value to be used in Figs. I CDFafter - CDFbase (II) and 2.

The threshold value for ACDF can be determined in a The values on the right-hand side of Eq. (11) are fairly easy similar fashion. Referring to Fig. 1, we see that there is a to generate. From the PRA, CDFbase is known directly. value of RAW with respect to the ACDF that will move From the application of Eq. (8), we calculate the set of val- ACDF upward in the figure until it enters a different region.

ues for RAWjCDF-base. To find the other values, we must It is this change between regions that changes the context modify the PRA to represent the plant as it would exist of the decision and possibly the decision itself. We, there-after the change. Using this model and repeating the steps fore, determine the value of RAW with respect to ACDF used to calculate CDFbase and RAWj,cDF-base, we calculate that will change the decision as follows:

CDFafter and the set of values for RAWj,CDF-after. Now, all ACDFthreshold of the variables on the right-hand side of this equation are 'RAWDACDF (13) known and the set of values for RAWjACDF can be gener-ated using, for example, sorting and arithmetic algorithms where RAWACDFthreshod is the RAW value that will in a Microsoft Excel spreadsheet. move ACDF upward and into a different region, and ACDFthreshold is the value of ACDF corresponding to 4.3. Calculating RA W thresholds the horizontal. line between the applicable regions of Fig. 1. Just as the CDFbas used when calculating At this point, we have a complete set of basic events with RAWCDFthreshold was a mean value, the ACDF value used their respective values for RAW with respect to CDF and here should be a mean value. This determination again RAW with respect to ACDF. Some threshold must be set differs from the traditional RAW threshold value of two to determine the value of RAW below which we deem used in licensing basis change decisions. The threshold the basic event to be not risk-important. It should be noted value of RAW used here is change-specific.

that traditionally in licensing basis change requests a threshold value of RAW is set at a value of two (NEI, 4.4. Cross-check of important basic events 1996). Using this method, basic events with a RAW of two or higher are deemed as potentially important and ana- Of course, not all basic events have large uncertainties lyzed further, while those with. a RAW less than two are in their reliabilities. For example, motor-driven pumps classified as not risk-important. The methodology in this have been used extensively in nuclear power plants for paper proposes a simple method for determining a some time. Because of this, a sufficient amount of histor-change-specific threshold value of RAW. ical failure data has been accumulated such that their fail-By referring to the acceptance guidelines in Fig. I and ure rates are known with a fair degree of certainty and the the position of the proposed licensing change's risk on mechanisms by which they fail are fairly well understood.

the figure, we see that there is some value of RAW that will The methodology, therefore, cross-checks the basic events move the plant's risk to the right on the figure until it enters whose uncertainty may be important as identified by the a different region. The RAW with respect to CDF of each method above to basic events that have been identified basic event indicates the maximum amount that uncer- in the literature review as having generally important tainty in this basic event can move the plant's CDF to model uncertainty. The basic events that remain after the right. A small RAW might indicate that regardless of the cross-check are those that are important to the the reliability of a particular basic event, the CDF would plant-specific licensing basis change decision at hand.

not be in Region I, and therefore would not affect the deci- They have model uncertainties identified in the literature sion. In other cases, the RAW may be large enough. It is as generically important, and are close enough to a only when the CDF moves into Region I that the uncer- threshold value in Fig. I that these uncertainties could tainty becomes important to the decision. We, therefore, possibly affect the decision. The generically important determine the threshold value of RAW with respect to model uncertainties and their descriptions are included CDF that will change the decision as follows: in Section 3.

362 J.M. Reinert, G.E. Apostolakis I Annals of Nu4ear Energy 33 (2006) 354-369 4.5. Making the decision program applied to 160 valves that made up various por-tions of 10 systems. These systems were:

Having identified the important basic events with respect to both CDF and ACDF, we must now investigate steam generator blowdown; their potential impact on the decision. Quantifying the heating and ventilation - purge air; model uncertainty would allow the decision maker to see compressed air - control air; how CDF and ACDF of the proposed change move and chemical and volume control; whether the change meets the acceptance guidelines. How- safety injection; ever, model uncertainty is quite difficult to quantify at this essential raw cooling water; time. Instead, our proposed methodology employs sensitiv- component cooling; ity analysis to determine the degree to which a basic event's core spray; failure probability would need to change in order to violate waste disposal; the acceptance guidelines, in effect changing the approval radiation monitoring.

decision. From this point, qualitative arguments remove some basic events from further consideration. This does, The risk-informed 1ST program proposed that, for these of course, still rely on expert opinion as a tool for estimat- valves, the IST frequency would be changed from once per ing plausible upper bounds for risk. However, expert opin- quarter to once per refueling cycle, or approximately once ion is used to a lesser degree because many uncertainties per 18 months. The licensee states that it is conservative to are eliminated from consideration because they are deter- assume that the failure probability of these valves increases mined to be unimportant. Remaining basic events must linearly, with time between inspections. Since the time be subjected to considerable scrutiny and the effects of between inspections increases by about a factor of six, uncertainty quantified before a decision can be made. Cal- the failure probability of each valve affected by the change culations needed for,.this paper were performed using the is increased by a factor of six when modeling the effects of Standardized Plant Analysis Risk (SPAR) model for this the change.

plant and the System Analysis Programs for Hands-On The point estimate baseline CDF and ACDF of the Integrated Reliability. Evaluations (SAPHIRE) computer plant were:

software. Model uncertainties that require a detailed quan- CDF = 6.8 x 10-' ry-'

titative evaluation may be handled by using the methods of Section 3. For example, one method was to use an adjust- ACDF = 6.9 x 10-7 ry-1 ment fac*tor. In this case, reasonable alternative assump- These values represent a point in the CDF versus ACDF tions to those used in the PRA are established, and their acceptance guidelines as shown in Fig. 3. This pair of val-effects on the PRA output quantified. Expert judgment is ues places the point representing the proposed LB change then used to determine the probability distribution of the in Region Ill of the acceptance guidelines. We note that adjustment factor. The adjusted CDF from the PRA is the CDF and ACDF reported were "point" estimates, then compared with the acceptance guidelines to determine i.e., the licensee did not propagate the distributions of the if the licensing basis change decision is sensitive to these input parameters to produce distributions for CDF and modeling assumptions. ACDF. As point estimates, the values of CDF and ACDF The benefit of the proposed methodology is that are sensitive to parameter uncertainties also. Since our important basic events are identified with respect to the objective is to investigate model uncertainties, we will treat change-specific decision at hand rather than using a general these point estimates as if they were epistemic means.

importance measure. This methodology also reduces the Following our methodology, we must first generate the number of uncertainties that must be analyzed extensively complete set of RAW values for the basic events at this by allowing qualitative arguments based on change-specific plant. This includes the RAW with respect to CDF and conditions. In its present form, however, this methodology is not sufficient when model uncertainty affects the logical structure of the PRA, as in the example provided earlier where uncertainty affected the success criteria.

at the RAW with respect to ACDF.

0L

5. The case study 1O To illustrate the methodology proposed in this paper, we present a case study. This case study was a licensing basis change request submitted to the NRC. The request applied to a commercial PWR Westinghouse four-loop design and proposed to establish a risk-informed in-service testing (IST) program that would replace the existing IST 10.5 10.4 CDF requirements for a portion of the plant's valves. The IST Fig. 3. CDF acceptance guidelines with representative point.

J.M. Reinert, G.E. Apostolakis I Annals of Nuclear Energy 33 (2006) 354-369 . 363

5. 1. Event RAW with respect to CDF of 10-6 ry- t and ACDFthreshold of 10- 5 ry- 1, yield RAWACDFthreshold values of 1.4 and 14, respectively.

The point representing the proposed change is in Region Therefore, any basic event with a RAW with respect to IIl and the decision would be affected if uncertainties the ACDF greater than 1.4 has the potential to change moved this point into Region I. Therefore, we are inter- the decision. If this RAW is 14, then the potential to ested in the horizontal threshold between Region I and change the decision is much higher because the representa-Region I1, which is about 10-3 ry-1. Although this is tive point would be in Region I.

not a "bright line" boundary, 10- 3 ry-1 is a reasonable Using SAPHIRE, we determined that there were 10 value. We note that the NRC has a goal of keeping the basic events that had a RAW with respect to ACDF greater CDF below 10-4 ry- 1 , thus making 10-4 ry- t another rea- than 14. They are listed in Table 3.

sonable boundary value; this is also the value it would have For each of these basic events, the RAW with respect to to exceed to be in Region I if the uncertainties were to bring CDF, as calculated in Section 4.1, is provided for compar-the ACDF up to Region It. Therefore, CDFthreshold is given ison. Out of these 10 events deemed important to the licens-two values, equal to 10-3 and 10-4 ry- t . The CDFnmean of ing basis decision because of their RAW with respect to the plant was 6.8 X 10- 5 ry-1. Therefore, the ACDF, only three were identified as important because RAWCDF,threshold values were about 14.6 and 1.46, respec- their RAW with respect to CDF exceeded 14. These are tively, which we truncated to 14 and 1.4. Any basic event emphasized with bold font in Table 3. This implies that with a RAW greater than 14 has the potential to change the licensing basis decision because the actual CDF could Table 2 be in Region 1. Also, a RAW greater than 1.4 indicates that RAW with respect to CDF: 1.4 < RAW < 14 uncertainty in the basic event probability is important to Basic event RAWcDF the decision because the actual CDF could be greater than (a) Diesel generator B fails 12.1 the NRC goal of keeping CDF less than 10-4, and it has the (b) Control rods remain energized 11.0 potential of being important to the decision, depending on (c) Diesel generator A fails 11.0 the ACDF value once uncertainties are included. (d) Operator failure to depress below steam generator 8.89 Using SAPHIRE, we determined that there were 12 relief valve setpoints (e) Failure to recover offsite power before battery 4.83 basic events with RAW greater than 14. They are listed depletion in Table 1. (f) Failure to isolate faulty steam generator 4.26 We also found that there were 32 additional basic events (g) Ruptured steam generator isolations fail 4.23 with RAW greater than 1.4. They are listed in Table 2. (h) Turbine boundary valves and condenser fail to 3.75 cooldown the reactor coolant system (i) Common-cause failure of RHR suction valves 3.66 5.2. Event RA W With respect to A CDF (j) Operator fails to initiate RHR - 3.66 (k) Operator fails to isolate reserve water storage tank 3.65 In our case study, the decision would be affected if (1) RHR hotleg discharge valve A fails 3.65 model uncertainties moved the representative point into (m) RHR hotleg discharge valve B fails 3.65 Region I or Region 1I; so we are interested in the vertical (n) PORV I fails to reclose 3.52 (o) Operator failure to initiate cooldown below RHR 3.40 boundary between Region III and Region II at about tolerances 10-6 ry- t and between Region I1 and Region I at about (p) RCP seals fail 3.30 10-' ry-'. ACDFmea, was given in the licensee's applica- (q) Common-cause failure of RHR to both high 2.93 tion as 6.9 x 10- ry-1. Our thresholds, ACDFthreshold pressure injection isolation valves (r) Common-cause failure of sump recirculation valves 2.93 Table I (s) Common-cause failure of reserve water storage 2.93 RAW with respect to CDF: RAW > 14 tank isolations (t) Sump failure 2.92 Basic event RAWcDF (u) Operator failure to initiate high pressure 2.89 (a) Control rods fail to insert 3050 recirculation (b) Common-cause diesel generator failure 271 (v) PORV I fails to open 1.96 (c) Failure to depressurize due to hardware failure 218 (w) Common-cause failure of auxiliary feedwater 1.87 (d) Scram breakers fail to open 202 motor-driven pumps (e) 4160V Bus IB fails 197 (x) Auxiliary feedwater steam supply valves fail 1.86 (f) Common-cause failure of residual heat removal 134 (y) Common-cause failure of turbine driven pump 1.85 (RHR) pumps steam supply valves to open (g) Common-cause failure of RHR heat exchangers 134 (z) Turbine driven pump 1.84 (h) Reserve water storage tank not available 112 (aa) Operator fails to identify a SGTR 1.77 (i) Common-cause auxiliary feedwater pump failure 26.6 (ab) Operator fails to initiate feed and bleed 1.77 (j) Common-cause failure of steam generator discharge 26.5 (ac) Operator fails to initiate reactor coolant system 1.76 valves to open depressurization (k) Common-cause failure of the steam generator 26.0 (ad) RHR motor driven pump B fails 1.59 inlet check valves (ae) RHR motor driven pump A fails 1.53 (1) 4160V Bus IA fails 18.4 (af) Operator fails to manually scram reactor 1.43

364 J.M. Reinert, G.E. Apostolakis / Annals of Nuclear Energy 33 (2006) 354-369 Table 3 Table 5 RAW with respect to ACDF: RAW > 14 RAW with respect to CDF and RAW with respect to ACDF > 14 Basic event RAWAcDF RAWcDF Basic events RAWACDF RAWCDF (a) Failure to isolate faulty steam 55.5 4.26 (a) 4160V Bus I B fails 44.2 197 generator (b) Common-cause failure 35.6 134 (b) Mechanical failure of steam generator 55.0 4.23 of RHR pumps isolations (c) Common-cause failure 32.8 134 (c) 4160V Bus IB fails 44.2 197 of RHR heat exchangers (d) Common-cause failure of RR pumps 35.6 134 (e) Failure to initiate high pressure 33.6 2.89 recirculation Table 6 (f) Common-cause failure of RHR supply to 33.6 2.94 RAW with respect to CDF > 14 and RAW with respect to ACDF > 1.4 high pressure injection isolation valves Basic events RAWACDF RAWcDF (g) Common-cause failure of sump 33.6 2.94 recirculation valves (a) Operator failure to isolate a faulty 55.5 4.26 (h) Common-cause failure of RHR reserve 33.6 2.94 steam generator water storage tank isolation valves (b) Ruptured steam generator isolation 55.0 4.23 (i) Sump failure 33.4 2.93 failures (j) Common-cause failure of RHR heat 32.8 134 (c) Operator fails to initiate high pressure 33.6 2.89 exchangers recirculation (d) Common-cause failure of RHR . 33.6 2.94 supply to high pressure injection valves (e) Common-cause failure of sump 33.6 2.94 Table 4 recirculation valves RAW with respect to ACDF: 1.4 < RAW < 14 (f) Common-cause failure of residual heat 33.6 2.94 Basic event RAWACoF removal reserve waster storage tank (a) Common-cause failures of high pressure 4.45 isolation valves injection flowpath (g) Sump failure 33.4 2.93 (b) High pressure injection cold leg injection valve 4.38 (h) 4160V Bus IA fails 2.79 18.4 fails (i) Reserve water storage tank not 2.35 112 (c) Common cause failure of reactor coolant 4.38 available system cold leg discharge check valves j) Scram breakers fail to open 2.34 202 (d) High pressure injection serial component 4.23 (k) Failure to depressurize due to 1.40 218 failures hardware failure (e),Common-cause failure of high pressure 4.16 (I) Operator fails to diagnose SGTR 1.94 1.77 injection discharge check valves (m) Operator fails.to initiate 1.93 1.76 (f) 4160V Bus IA fails 2.79 depressurization (g) Reserve water storage tank not available 2.35 (n) Operator fails to throttle high pressure 1.91 1.77 (h) Scram breakers fail to open 2.34 injection to reduce pressure (i) Operator fails to diagnose SGTR 1.94 (o) RHR motor driven pump 1B fails 1.45 1.60 Ij) Operator fails to initiate depressurization 1.93 (p) RHR motor driven pump IA fails 1.42 1.53 (k) Operator fails to throttle high pressure 1.91 injection to reduce pressure (1) Common-cause failure of chemical & volume 1.70 control discharge valves A and B although, individually, uncertainty in the remaining seven (m) Common-cause failure of chemical & volume 1.70 basic events cannot be sufficient to move the licensee's control discharge valves C and D CDF horizontally into Region I of the acceptance guide-(n) Charging system discharge check valves fail 1.70 lines, they each have uncertainty that may be sufficient to (o) Charging system suction check valves fail 1.70 (p) Common-cause failure of charging pumps 1.70 move the change vertically into Region 1.

(q) Common-cause failure of chemical & volume 1.61 We also found that there were 28 additional basic events control suction valves that had RAW with respect to ACDF greater than 1.4.

(r) Common-cause failure of VCT isolation valves 1.61 They are listed in Table 4.

(s) Common-cause failure of chemical & volume 1.60 control pump check valves (t) RHR motor driven pump B fails 1.45 5.3. Combined importance wvith respect to CDFand A CDF (u) RHR discharge valve B fails 1.43 (v) Sump isolation valve B fails 1.43 Basic events that have high RAW values with respect to (w) Reserve water storage tank isolation valve B 1.43 both CDF and ACDF are especially important because fails their uncertainty can move the representative point both (x) RHR discharge A fails 1.43 (y) Sump isolation valve A fails 1.43 horizontally and vertically in Fig. I simultaneously. In (z) Reserve water storage tank isolation valve A 1.43 our case study, a factor of 14 increase in CDF or a factor fails of 14 increase in ACDF was sufficient to move the point (aa) RHR motor driven pump A fails 1.42 into Region I. However, a factor of 1.4 increase in CDF (ab) Fail to depressurize due to hardware 1.40 in combination with a factor of 1.4 increase in ACDF

J. M. Reinert, G.E. Apostolakis / Annals of Nuclear Energy 33 (2006) 354-369 365 would also move the representative point into Region I. shorter than four hours, the calculated failure probability This is because the ACDF required to enter Region I would be conservative. If an event required longer than four changes, depending on the value of CDF. If CDF is greater hours of operation, the failure probability would be optimis-than about 10- 4 ry- 1, then ACDF must be below about tic. The mission time must therefore be chosen to bound rea-10-6 ry- 1 to remain out of Region 1. Otherwise, ACDF sonable mission time requirements in order to maintain may be as large as 10-5 ry-1. This threshold RAW value conservatism. However, the reasonableness of this bound of 1.4 is an order of magnitude lower than the previously may change if, for example, confidence in the reliability of required threshold RAW values of 14. Therefore, basic the municipal electric grid changes.

events with uncertainties that affect both the CDF and Modeling of diesel generator field flashing success proba-ACDF, but have a relatively weak effect on each, can still bilities have also been identified as generally important to affect the licensing basis decision. The fact that the factor diesel generator failure rates (USNRC, 1990). Because of of 14 increase required is the same for both CDF and diesel generator design, field flashing is a necessary compo-ACDF is purely coincidental. nent of a generator's ability to produce electricity and, there-For basic events that were important with respect to fore, has a large impact on diesel generator failure rates.

both CDF and ACDF, we divided them into three catego- During a station blackout, where offsite power and emer-ries. There were three basic events that had both a RAW gency A/C power have been lost, diesel generator field flash-with respect to CDF and a RAW with respect to ACDF ing power is drawn from station batteries. Therefore, the greater than 14. They are listed in Table 5. duration the battery is capable of supplying power before In addition to those listed in Table 5, there were II basic it is depleted is also a factor in determining diesel generator events that had both a RAW with respect to CDF and a failure rates. Battery depletion time has also been identified RAW with respect to ACDF greater than 1.4. They are as an important model uncertainty (USNRC, 1990).

listed in Table 6. Sensitivity to these model uncertainties can be found by varying the specific assumptions related to each. For exam-

6. Evaluation ple, the modeling of diesel generator field flashing is impor-tant. One could look at the model used to determine the Thus far, we have identified basic events whose proba- probability that field flashing would fail, and question the bility has the potential to adversely affect the decision. assumptions that it makes. In lieu of this, we vary the fail-Next, it must be determined whether there are model uncer- ure rate of the basic event, effectively using the failure rate tainties associated with these basic events that can actually as a proxy for the modeling assumptions that went into its lead to a different decision. We start by matching the gener- determination.

ically important model uncertainties from the literature The basic event that we are concerned with here is with the basic events identified in the tables and then deter- "Common-cause diesel generator failure". Since there are mine how far the probability of each basic event would two diesel generators, the failure rate of this event is calcu-have to shift in order to impact the decision. Expert opiný lated as follows:

ion must be used to determine whether or not the required 3

),= # (4,t +  ;,ft..

• t) (14) shift is reasonable. We provide some analysis here, but this is an area that requires further research, perhaps building where ), is the total common-cause failure rate, ,f1s is the upon the ideas presented in Section 3. rate of independent diesel generator failure to start on de-The basic events in Table]1 had high RAW with respect to mand, 2)ft. is the rate of independent diesel generator failure CDF, sufficient to move the representative point from the while running per hour, t is the mission time in hours, and case study horizontally into Region 1. Table I was only con- /3 is the beta-factor, defined as cerned with the effects of uncertainty on CDF, without regard to their effect on ACDF. Basic events (b), (f), (g), (i), (15)

/3= + ),

(j), and (k) in Table I are all similar in that they refer to a common-cause failure mechanism. Basic event (b) in Table where ), is the common-cause failure rate and ).j is the 1, "Common-cause diesel generator failure", is particularly independent failure rate (represented by the sum of fts interesting because it ranks second in importance with and Aftr Xt ). Looking at Eq. (14), we see that the com-respect to CDF and is described in the literature as generally mon-cause failure rate has two components, P3and the important. Specifically, the modeling of diesel generator mis- independent failure rate. In the PRA, #3is 0.038, )fts is sion time and recovery are important model uncertainties 3.0 x 10-2/demand, ),f, is 2.0 x 10- 3 /hour and the mission (Interview). These uncertainties are related to how long the time is four hours. The common-cause failure rate is, diesel is assumed to be needed in order to accomplish its mis- therefore, sion, and how probabilities of recovering a failed diesel gen-erator are calculated. In the model used for this analysis, a 2= 0.038 x [3.0 x 10-2/demand + (2.0 single bounding mission time of four hours was chosen and x 10-3/hour)(4 hour4.62963e-5 days <br />0.00111 hours <br />6.613757e-6 weeks <br />1.522e-6 months <br /> mission time)]

diesel generator failure probabilities were calculated using this value. Therefore, if an event required a mission time = 1.44 x 10-3 per mission. (16)

366 J. M. Reinert, G.E. Apostolakis / Annals of Nuclear Energy 33 (2006) 354-369 We tested the sensitivity of the CDF to variations in 2, and eled may not be long enough. Therefore, the basic event optimistic.

found that a factor of 35 increase would change the com- failure probability may be mon-cause failure rate to 0.051 and place the representative The important model uncertainties were diesel generator point in Region I of the acceptance guidelines. The ques- mission time and recovery modeling, diesel generator field tion now is whether this increase is reasonable. Since 2;, is flashing modeling, and battery depletion time modeling.

a product of two variables, we must question the values Mission time modeling assumptions may be optimistic in used for each variable in order to question to value used our case study because of the recent question of whether for 4c. mission times are long enough. Diesel generator field flash-We first look at the value of P3. In the PRA, fl is 0.038. ing and battery depletion time modeling may be optimistic The value of 0.10 is often used as a generic value for 3 . in our case study for the same reasons. Longer outages The NRC's Common-Cause Failure Database (CCFDB) require longer battery depletion times in order to prevent (USNRC, 2001) provides common-cause failure data from a station blackout. Also, the battery is required to §upply industry-wide operational experience. It provides failure- diesel generator field flashing power.

while-running and failure-to-start data for diesel We have concluded that a /3value of 0.037 is reasonable.

generators. In this database, the value of /3 for the So, in order for the common-cause failure rate to be 0.051 failure-while-running case has a mean of 0.0370 and a (sufficient to affect the decision), the independent failure 95th percentile of 0.0499. /3 for the failure-to-start has a rate would have to increase to mean of 0:'0263 and a 95th percentile of 0.0370. These are lower than the generic value of 0.10, indicating that diesel 4 0.051 - 1.34 per mission (18) generators are somewhat robust with regard to common- /3 0.038 cause failures. This is due to the fact that diesel generators This value is clearly unrealistic, therefore, model uncertain-are well known to be risk-important and focused efforts ties regarding the diesel generators do not appear to be have been made to minimize the fraction of common-cause capable of affecting the decision.

failures. Notably, the Station Blackout rule, 10 CFR 50.63, Looking at the basic events that were important with established the emergency diesel generator reliability pro- respect to ACDF in Table 3, we see that three of the uncer-gram. The value of /3 in the PRA is consistent -with the tainties involve known important model uncertainties.

CCFDB values. They are listed in Table 7 with their associated model We next look at the value of 2i, the independent diesel uncertainties.

generator failure rate. In our case study, ),i is quantified as Basic event (a) in Table 3, "Failure to isolate faulty Steam Generator" is recognized as having the potential A, = 3.0 x 10 2 /demand + (2.0 x 10 3 /hour) to be a risk-important human action after a Main Steam leak or a Steam Generator Tube Rupture initiating event.

x (4 hour4.62963e-5 days <br />0.00111 hours <br />6.613757e-6 weeks <br />1.522e-6 months <br /> mission time)

In the PRA, the conditional probability that this action will

= 0.038 per mission (17) not be done when needed is 10-3. For this analysis, we must increase this failure probability by a factor of 250 The failure rate is a function of the probability that the die- (thus making it 0.25) to achieve a RAW with respect to sel generator fails to start and the probability that it fails to the ACDF of 14, thus placing the representative point in run for the mission time. We compared the probabilities Region I. This same factor of 250 increases the CDF by used in the case study with the probabilities used in repre- only a factor of two. It is apparent, therefore, that the effect sentative PWR PRAs and found them to be consistent. of an uncertainty on the CDF and the ACDF can be quite However, these sources used the same NRC Accident Se- different.

quence Evaluation Program (ASEP) database, which uses Uncertainty in human reliability is well known to be industry-wide accumulated data. Plant-specific failure rates important. Inputs to human reliability models, such as per-may vary considerably from the industry averages. Also, formance shaping factors, are difficult to quantify, the the electrical grid outage of August 14, 2003 has raised is- models are sensitive to these inputs, and different human sues as to whether the current modeling assumptions are -reliability models with the same inputs may produce failure sufficient (Rasmusson, 2004), specifically, assumptions on rates that span orders of magnitude. In the European Com-the time to recover offsite power. Because of the extent of mission's Human Factors Reliability Benchmark Exercise this outage, recovery times at some plant were quite long, (Poucet, 1989), 15 teams of analysts from different coun-raising concern that current recovery times that are mod- tries were asked to calculate human reliability for the Table 7 Associating basic events with model uncertainties Basic event RAWAcDF Associated model uncertainty Failure to isolate faulty steam generator 55.5 Human reliability - failure to isolate faulty steam generator (USNRC, 2004b)

Failure to initiate high pressure recirculation 33.6 Human reliability - switch ECCS from injection to recirculate (USNRC, 2004b)

Failure of sump 33.4 Sump plugging and pool strainer plugging modeling (Interview)

J.M. Reinert, G.E. Apostolakis / Annals of Nuclear Energy 33 (2006) 354-369 367 crew's response to an operational transient at a nuclear Therefore, the basic event probability may be very conser-power plant. One team produced results using different vative. We next look at the value of independent RHR models ranged from about 1.5 x 10-2 to 3.5 x 10-1. Across pump failure rate. We compared the probabilities used in teams, results using the same model ranged from about the case study with the probabilities used in representative 6 x 10-3 to 3.5 x l0-1. In order for the "Failure to isolate PWR PRAs and found them to be consistent. Unlike the faulty Steam Generator" basic event to be important to diesel generator common-cause failure basic event, where the decision, the probability of not performing this action there were several model uncertainties, common-cause fail-would need to change from one in 1000 to one in four. ure modeling is the only model uncertainty that applies to To assess whether or not an error probability of 0.25 is rea- the RHR pump common-cause failure basic event. We con-sonable for this event, one would need to look at operator clude that the factor of 20 increase necessary to affect this training, time available, and other performance shaping decision is not reasonable.

factors. Such a high probability, however, does appear to There were two other basic events listed in Table 5.

be unreasonable. Basic event (a) "4160V Bus lB fails" had no associated Basic event (e) of Table 3, "Failure to initiate High Pres- model uncertainties that were identified in the literature sure Recirculation" is another human action and has the review as generically important. An analysis of basic event potential to be important to the decision. In the PRA, (c) from Table 5, "Common-cause failure of RHR heat the conditional probability that this action will not be done exchangers" produced similar results to that of the RHR when needed is 10-3. For the analysis, we increased this pump common-failure analysis and is probably not impor-failure probability by a factor of 400 to achieve a RAW tant to this licensing basis change decision.

with respect to the ACDF of 14, placing the representative point in Region 1. This corresponds to a failure rate of 0.4 7. Conclusions per demand. As was the case in basic event (a) of Table 3, an analysis is required to determine whether this error We have sought to identify basic events where the value probability is reasonable, although we expect it to be of their probability can change the decision, and are known unreasonable. to have significant model uncertainty. We focused on Level Basic event (i). of Table 3, "Sump failure" has model I, at power, internal events PRAs, and the decision-making uncertainties that have the potential to be important to process related to licensing basis changes. The acceptance the decision. There has been significant debate over even guidelines with respect to a plant's CDF and ACDF of a whether sufficient data exists to measure sump perfor- proposed change have been clearly defined by RG 1:174 mance. The PRA assigns a failure probability of 5 x 10-5. and the need to address all uncertainties in the decision-This would need to increase by a factor of 8500 to a failure making process has been established. Once the basic events probability of about 0.4 in order to impact the decision. of interest are identified, they are analyzed to determine Looking at the basic events that have high values of what their probability would need to be to affect the deci-RAW with respect to both CDF and ACDF from Table sion. Then, an analysis must determine whether this change 5, we see that basic event (b) "Common-cause failure or is reasonable. We referred to several methods to accom-RHR pumps" is risk-important. There are two RHR plish this and provided a case study.

pumps. In the PRA, #3is 0.15, fts is 3.0 x 10-3/demand, 4ftr In our case study, a total of 12 basic events had RAW is 3.0x 10-5/hour and the mission time is 24 h. The com- with respect to CDF showing that their uncertainty could mon-cause failure rate is, therefore, place the licensing basis change's representative point in Region I of the acceptance guidelines, in which case the c = 0.15 x [3.0 x 10-3/demand + (3.0 changewould generally not be approved. The model uncer-x 10- 5 /hour)(24 hour2.777778e-4 days <br />0.00667 hours <br />3.968254e-5 weeks <br />9.132e-6 months <br /> mission time)] tainties in one of these basic events have been found to be

= 5.58 x 10-4 per mission. (19) important in a review of the literature. 10 basic events have RAW with respect to ACDF showing that their uncertain-We tested the sensitivity of the CDF to variations in the ties could place the change's representative point in Region common-cause failure rate and found that a factor of 20 in- I. Of these, three were important in the literature. Two crease would change the common-cause failure rate to basic events were common to both lists, showing high 0.011 and place the representative point in Region I of importance with respect to both the CDF and ACDF.

the acceptance guidelines. Since 2, is the product of two Therefore, a total of 20 basic events were identified as variables, we must question the values used for each vari- important.

able in order to question the value used for 2., The decision appears to be insensitive to uncertainties in We first look at the value of /3. The CCFDB lists a / for all of these basic events. In order to move the representa-RHR pump failure-while-running having a mean of 0.0464. tive point into Region I, the probabilities of "failure to iso-and a 95th percentile of 0.0653. P3 for the failure-to-start late faulty steam generator", "failure to initiate high has a mean of 0.0362 and a 95th percentile of 0.0598. pressure recirculation", and "failure of sump" would need The value of P3used in the case study application, 0.15, is to increase considerably. An evaluation of the reasonable-considerably higher than the CCFDB database values. ness of these increases would be required.

368 J. M. Reinert, G.E. Apostolakis / Anndls of Nuclear Energy 33 (2006) 354-369 We also performed a sensitivity of success criteria Interviews conducted with the US Nuclear Regulatory Commission related to auxiliary feedwater pumps for illustration, where Research and Regulatory Staff.

Keeney, R., von Winterfeldt, D., 1991. Eliciting probabilities from experts we changed the assumption that one feedwater was suffi-in complex technical problems. IEEE Transactions in Engineering cient to ensure success to an assumption that either the tur- Management 38, 191-201.

bine-driven pump or both motor-driven pumps were Knudson, J., Smith, C., 2002. Estimation of system failure probability required. The alternative assumption produced a CDF of uncertainty including model success criteria. In: Bonano, E.J. (Ed.).

6.85 x 10-5, or 0.36% higher than the baseline case. Proceedings of the 6th International Conference on Probabilistic Safety Assessment and Management (PSAM 6), San Juan, Puerto Rico, 23-28 June, 2002, Elsevier Science Ltd., United Kingdom.

Acknowledgments Letter from G.B. Wallis, Chairman, Advisory Committee on Reactor Safeguards, to N.J. Diaz, Chairman, US Nuclear Regulatory Com-mission,

Subject:

"Safety Evaluation of the Industry Guidelines This work is a product of the project "Contributions to Related to Pressurized Water Reactor Sump Performance," October Risk-Informed Decision Making" and has been supported

.18, 2004. Available at: <http://www.nrc.gov/reading-rm/doc-collec-by the US Nuclear Regulatory Commission under a coop- tions/acrs/letters/2004/5162097.pdf>.

erative agreement with the MIT Department of Nuclear Mosleh, A., Siu, N., Smidts, C., Lui, C. (Eds.), 1993. Proceedings of Science and Engineering. The views presented here are Workshop on Model Uncertainty: Its Characterization and Quantifi-cation, Annapolis, MD, October 20-22, 1993. Center for Reliability those of the authors and do not necessarily represent the Engineering, University of Maryland, College Park, MD, 1993. Also views of the US Nuclear Regulatory Commission. We published as Report NUREG/CP-0138, US Nuclear Regulatory thank Hossein Hamzehee, Todd Hilsmeier, and Susan Commission, Washington, DC.

Cooper of the NRC Office of Nuclear Regulatory Research Nuclear Energy Institute, 1996. Industry Guideline for Monitoring the for their support and comments. We thank Patrick Bara- Effectiveness of Maintenance at Nuclear Power Plants. Report NUMARC 93-01, Washington, DC, April 1996.

nowsky, Michael Cheok, Mary Drouin, Christopher Nuclear Energy Institute, 2000. Probabilistic Risk Assessment Peer Grimes, and Gareth Parry of the NRC for discussing with Review Process Guidance. Report NEI-00-02, Revision A3, Washing-us issues related to risk-informed decision making. Finally, ton, DC, March 20, 2000.

Mary Presley of MIT reviewed the manuscript and gave'us Poucet, A., 1989. The European benchmark exercise on human reliability analysis. In: Proceedings of the International Topical Meeting on useful comments.

Probability, Reliability, and Safety Assessment' PSA '89, American Nuclear Society, La Grange Park, IL, pp. 103-110.

References Rasmusson, D., 2004. Station Blackout Reevaluation, -September 24, 2004. Available from: <http://www.nrc.gov/what-we-do/regulatory/

Apostolakis, G., 1993. A commentary on model uncertainty. In: Mosleh, research/mtg-archive/m1043280190.pdf>.

A., Siu, N., Smidts, C., Lui, C. (Eds.), Proceedings of Workshop on Systems Analysis Programs for Hands-on Integrated Reliability Evalua-Model Uncertainty: Its Characterization and Quantification, Annap- tion (SAPHIRE), Version 6.79. <http://saphire.inel.gov>, developed olis, MD, October 20-22, 1993. Center for Reliability Engineering, by Idaho National Engineering and Environmental Laboratory, Idaho University of Maryland, College Park, MD, 1993. Also published as Falls, Idaho.

Report NUREG/CP-0138, US Nuclear Regulatory Commission, Savage, L., 1972. The Foundation of Statistics. Dover, New York.

Washington, DC. Sorensen, J.N., Apostolakis, G.E., Kress, T.S., Powers, D.A., 1999. On American Society of Mechanical Engineers, 2002. Standard for Probabi- the role of defense in depth in risk-informed regulation. In: Proceed-listic Risk Assessment for Nuclear Power Plant Applications, ASME ings of PSA '99, International Topical Meeting on Probabilistic Safety RA-S-2002, April 5, 2002. Addenda to ASME RA-S-2002, ASME Assessinent, Washington, DC. American Nuclear Society, La Grange RA-Sa-2003, December 5, 2003. Park, IL, August 22-26, 1999, pp. 408-413.

Bley, D., Buttemer, D., Stetkar, J., 1988. Light water reactor sequence US Nuclear Regulatory Commission, 1990. Analysis of CDF from timing: its significance to probabilistic safety assessment modeling. Internal Events: Expert Judgment. Report NUREG/CR-4550, vol. 2, Reliability Engineering and System Safety 22, 27-60. Washington, DC, April 1990..

Bley, D., Kaplan, S., Johnson, D., 1992. The strengths and limitations of US Nuclear Regulatory Commission, 1995. Use of Probabilistic Risk PSA: where we stand. Reliability Engineering and System Safety 38, 3-26. Assessment Methods in Nuclear Activities: Final Policy Statement, Budnitz, R., Apostolakis, G., Boore, D., Cluff, L., Coppersmith, K., Federal Register, vol. 60 (60 FR 42622), August 16, 1995, p.

Cornell, C., Morris, P., 1996. Recommendations for Probabilistic 42622.

Seismic Hazard Analysis: Guidance on Uncertainty and the Use of US Nuclear Regulatory Commission and the Commission of European Experts, Report NUREG/CR-6372, US Nuclear Regulatory Com- Communities, 1997. Probabilistic Accident Consequence Uncertainty mission, Washington, DC. Analysis. Report NUREG/CR-6555 and NUREG/CR-6523, Wash-Buslik, A., 1993. A Bayesian approach to model uncertainty. In: Mosleh, ington, DC.

A., Siu, N., Smidts, C., Lui, C. (Eds.), Proceedings of Workshop on US Nuclear Regulatory Commission, CCF Parameter Estimations, 2001.

Model Uncertainty: Its Characterization and Quantification, Annap- Common-Cause Failure Database. Available from: <http://nrcoc.i-olis, MD, October 20-22, 1993. Center for Reliability Engineering, nel.gov/results/CCF/CCFParamWebHelp/CCFParamEst.htm>.

University of Maryland, College Park, MD, 1993. Also published as US Nuclear Regulatory Commission, 2002. An Approach for Using Report NUREG/CP-0138, US Nuclear Regulatory Commission; Probabilistic Risk Assessment in Risk-informed Decisions on Plant-Washington, DC. specific Changes to the Licensing Basis. Regulatory Guide 1.174 (RG Cheok, M., Parry, G., Sherry, R., 1998. Use of importance measures in 1.174), Revision 1, Washington, DC, November 2002.

risk-informed regulatory applications. Reliability Engineering and US Nuclear Regulatory Commission, 2004. An Approach for Determin-System Safety 60, 213-226. ing the Technical Adequacy of Probabilistic Risk Assessment Results Clemen, R., Winkler, R., 1993. Aggregating point estimates: a flexible for Risk-informed Activities. Regulatory Guide 1.200 (RG 1.200),

modeling approach. Management Science 39, 501-515. Washington, DC, February 2004.

J.M. Reinert, G.E. Apostolakis / Annals of Nuclear Energy 33 (2006) 354-369 369 US Nuclear Regulatory Commission, 2004. Guidance for the Review of published as Report NUREG/CP-0138, US Nuclear Regulatory Changes to Human Actions: Final Report. NUREG-1764, Washing- Commission, Washington, DC.

ton, DC, February 2004. Winkler, R., 1996. Uncertainty in probabilistic risk assessment. Reliability Winkler, R., 1993. Model uncertainty: probabilities for models? In: Engineering and System Safety 54, 127-132.

Mosleh, A., Siu, N., Smidts, C., Lui, C. (Eds.) Proceedings of Zio, E., Apostolakis, G., 1996. Two methods for the structured assessment Workshop on Model Uncertainty: Its Characterization and Quantifi- of model uncertainty by experts in performance assessments of cation, Annapolis, MD, October 20-22, 1993. Center for Reliability radioactive waste repositories. Reliability Engineering and System Engineering, University of Maryland, College Park, MD, 1993. Also Safety 54, 225-241.

UNITED STATES OF AMERICA NUCLEAR REGULATORY COMMISSION BEFORE THE COMMISSION In the Matter of )

) Docket No. 50-0219-LR AMERGEN ENERGY COMPANY, LLC )

)

(License Renewal for the Oyster Creek )

Nuclear Generating Station) ) June 11, 2008

)

CERTIFICATE OF SERVICE I, Richard Webster, of full age, certify as follows:

I hereby certify that on June 4, 2008, I caused Citizens' Response to the Commission's order dated May 28, 2008 to be served via email and U.S. Postal Service (as indicated) on the following:

Secretary of the Commission (Email and original and 2 copies via U.S Postal Service)

United States Nuclear Regulatory Commission Washington, DC 20555-0001 Attention: Rulemaking and Adjudications Staff E-mail: HEARINGDOCKET@aNRC.GOV Office of Commission Appellate Adjudication (Email and U.S. Postal Service)

United States Nuclear Regulatory Commission Washington, DC 20555-0001 Attention: Rulemaking and Adjudications Staff E-mail: OCAAMailgnrc.gov Administrative Judge E. Roy Hawkens, Chair (Email and U.S. Postal Service)

Atomic Safety and Licensing Board Panel Mail Stop - T-3 F23 United States Nuclear Regulatory Commission Washington, DC 20555-0001 E-mail: erhgnrc.gov I

Administrative Judge*

Dr. Paul B. Abramson (Email and U.S. Postal Service)

Atomic Safety and Licensing Board Panel Mail Stop- T-3 F23 United States Nuclear Regulatory Commission Washington, DC 20555-0001 E-mail: pba@nrc.gov Administrative Judge Dr. Anthony J. Baratta (Email and U.S. Postal Service)

Atomic Safety and Licensing Board Panel Mail Stop - T-3 F23 United States Nuclear Regulatory Commission Washington, DC 20555-0001 E-mail: ajb5@nrc.gov Law Clerk Emily Krause (Email and U.S. Postal Service)

Atomic Safety & Licensing Board Panel Mail Stop - T-3 F23 U.S. Nuclear Regulatory Commission Washington, DC 20555-0001 E-mail: DAWI (@nrc.gov Office of General Counsel (Email and U.S. Postal Service)

United States Nuclear Regulatory Commission Washington, DC 20555-0001 E-mail: OG CMAILCENTER(gNRC.GOV James E. Adler (Email and U.S. Postal Service)

U.S. Nuclear Regulatory Commission Office of the General Counsel Mail Stop: 0-15 D21 Washington, DC 20555-0001 E-mail: jeal(nrc. gov Mary C. Baty (Email and U.S. Postal Service)

U.S. Nuclear Regulatory Commission Office of the General Counsel Mail Stop: 0- 15 D21 Washington, DC 20555-0001 E-mail: mcbl @nrc.gov Alex S. Polonsky, Esq. (Email and U.S. Postal Service)

Morgan, Lewis, & Bockius LLP 1111 Pennsylvania Avenue, NW Washington, DC 20004 E-mail: anol onskvnamorganlewis.com 2

Kathryn M. Sutton, Esq. (Email and U.S. Postal Service)

Morgan, Lewis, & Bockius LLP 1111 Pennsylvania Avenue, NW Washington, DC 20004 E-mail: ksuttongmorganlewis.com Donald Silverman, Esq. (Email and U.S. Postal Service)

Morgan, Lewis, & Bockius LLP 1111 Pennsylvania Avenue, NW Washington, DC 20004 E-mail: dsilverman@(imorganlewis.corn J. Bradley Fewell (Email and U.S. Postal Service)

Exelon Corporation 200 Exelon Way, Suite 200 Kennett Square, PA 19348 E-mail: bradley.fewella Iexceloncorp.com John Covino, DAG (Email and U.S. Postal Service)

State of New Jersey Department of Law and Public Safety Office of the Attorney General Hughes Justice Complex 25 West Market Street P.O. Box 093 Trenton, NJ 08625 E-mail: john.corvinogdol.lps.stale.1nj.lus Valerie Gray (Email)

State of New Jersey Department of Law and Public Safety Office of the Attorney General Hughes Justice Complex 25 West Market Street P.O. Box 093 Trenton, NJ 08625 E-mail: valeri e.gray@dol. lps. state. nj. us.

Paul Gunter (Email and U.S. Postal Service) c/o Nuclear Information and Resource Service 6930 Carroll Ave., Suite 340 Takoma Park, MD 20912-4446 E-mail: paulcbeyondnuclear.org

'3

I Edith Gbur (Email)

Jersey Shore Nuclear Watch, Inc.

364 Costa Mesa Drive. Toms River, New Jersey 08757 E-mail: gburldcomcast.net Paula Gotsch (Email)

GRAMMIES 205 6 th Avenue Normandy Beach, New Jersey 08723 E-mail: paulagotsch@verizon .net Jeff Tittel (Email)

New Jersey Sierra Club 139 West Hanover Street Trenton New Jersey 08618 E-mail: Jeff.Tittelgsierraclub.org Peggy Sturmfels (Email)

New Jersey Environmental Federation 1002 Ocean Avenue Belmar, New Jersey 07319 E-mail: psturmfelsCa~cleanwater.org Michele Donato, Esq. (Email)

PO Box 145 Lavalette, NJ 08735 E-mail: mdonatoamiclieledonaloesq.con Signed:__ _ _ _ _ _ _ _ _ _ _ _ _

Richard Webster Dated: June 11, 2008 4