ML20345A003

TLR-RES/DE/CIB-2020-11, Basis for a Potential Alternative to Revision 2 of Regulatory Guide 1.99

ML20345A003
Person / Time
Issue date:	01/19/2021
From:	Carolyn Fairbanks, Matthew Gordon, Jeffrey Poehler, Dan Widrevitz Office of Nuclear Regulatory Research
To:
J. Poehler
Shared Package
ML20345A002:ML20345A009	List: ML20345A003
References
TLR-RES/DE/CIB-2020-11
Download: ML20345A003 (71)

Text

Technical Letter Report TLR-RES/DE/CIB-2020-11 Basis for a Potential Alternative to Revision 2 of Regulatory Guide 1.99 Technical Letter Report Jeff Poehler Dan Widrevitz Matt Gordon Carolyn Fairbanks U.S. Nuclear Regulatory Commission Office of Nuclear Regulatory Research January 19, 2021

DISCLAIMER:

This report was prepared as an account of work sponsored by an agency of the U.S. Government. Neither the U.S. Government, nor any agency thereof, nor any employee, makes any warranty, expressed or implied, or assumes any legal liability or responsibility for any third partys use, or the results of such use, of any information, apparatus, product, or process disclosed in this publication, or represents that its use by such third party complies with applicable law.

2

This report does not contain or imply legally binding requirements. Nor does this report establish or modify any regulatory guidance or positions of the U.S. Nuclear Regulatory Commission and is not binding on the Commission.

3

Table of Contents Table of Contents.......................................................................................................................... 4 List of Figures ............................................................................................................................... 5 List of Tables................................................................................................................................. 6 List of Abbreviations...................................................................................................................... 7 Executive Summary ...................................................................................................................... 9

1. Introduction ........................................................................................................................... 10

2. Motivation for the Evaluation Effort ....................................................................................... 11

3. Regulatory Guide Framework Elements ............................................................................... 13 3.1 Selection of the Embrittlement Trend Correlation ........................................................ 13 3.2 Use of Surveillance Data .............................................................................................. 32 3.3 Margins ........................................................................................................................ 41 3.4 Default Values .............................................................................................................. 48 3.5 Limitations .................................................................................................................... 51

4. Fleet Impact Evaluation......................................................................................................... 54 4.1 Methodology of Fleet Impact Evaluation ...................................................................... 54 4.2 Results ......................................................................................................................... 55

5. Conclusions........................................................................................................................... 63 5.1 Elements of Alternative ................................................................................................ 63 5.2 Fleet Impact Study ....................................................................................................... 64

6. References ............................................................................................................................ 66 Appendix A.................................................................................................................................. 68 Comparison of BASELINE T41J Values to REAP T41J ............................................................. 68 4

List of Figures Figure 3-1 Explanatory diagram of Type A through D errors ..................................................... 32 Figure 3-2 Data for BWR Plant A before refit ............................................................................. 35 Figure 3-3 Data for BWR Plant A after refit ................................................................................ 36 Figure 3-4 Data for PWR Plant B before refit ............................................................................. 37 Figure 3-5 Data for PWR Plant B forging after refit .................................................................... 38 Figure 3-6 Flowchart of surveillance data refit process ............................................................. 40 Figure 3-7 RMSD vs. TTS fit for U.S. Plate only ........................................................................ 43 Figure 3-8 Comparison of SD for Plate and Plate + SRM .......................................................... 43 Figure 3-9 Comparison of SD based on various data sets for Plate and Plate + SRM, and SRMs only ........................................................................................................... 44 Figure 3-10 RMSD versus TTS fit to U.S. weld data ................................................................. 45 Figure 3-11 Comparison of TTS determined using E900-15 and fit to U.S. welds only ............. 45 Figure 3-12 RMSD versus TTS fit for U.S. forging data ............................................................. 47 Figure 3-13 Comparison of forging fits for U.S. only, E900-15, and RG 1.99 base metals........ 47 Figure 3-14 Distribution of Cu (weight %) values for BWR forgings from BASELINE................ 49 Figure 3-15 Distribution of U.S. reactor temperature data from the BASELINE database versus Ni content ............................................................................................................ 53 Figure 4-1 Distribution of ESDs, all materials in fleet impact smart sample............................... 56 Figure 4-2 Distribution of ESDs for limiting materials only, at 1/4T location .............................. 57 Figure 4-3 Distribution of ESDs for limiting materials only at ID location ................................... 58 Figure 4-4 Distribution of ESDs versus fluence for all materials ................................................ 60 Figure 4-5 Distribution of ESDs versus fluence for limiting materials only ................................. 60 Figure 4-6 Distribution of ESDs versus fluence for all base materials ....................................... 61 Figure 4-7 Distribution of ESDs versus fluence for limiting base materials only ........................ 61 Figure 4-8 Distribution of ESDs versus fluence for all weld materials........................................ 62 Figure 4-9 Distribution of ESDs versus fluence for limiting weld materials only ........................ 62 5

List of Tables Table 3-1 Statistical Metrics Used To Determine Bias ............................................................... 15 Table 3-2 Determination and Interpretation of RMSD ................................................................ 16 Table 3-3 Equations Used for the T-Test ................................................................................... 18 Table 3-4 RMSD, Bias, and Ln(L) Results for ResidualsAll Baseline Data ........................... 20 Table 3-5 RMSD, Bias and Ln(L) Results for ResidualsU.S. Data Only a .............................. 21 Table 3-6 RMSD, Bias and Ln(L) Results for ResidualsInternational Data Only .................. 22 Table 3-7 Bias, RMSD, and Ln(L) Results for Residuals for BASELINE (U.S. plus International Data) .................................................................................................................... 23 Table 3-8 Bias, RMSD, and Ln(L) Results for Residuals for U.S. Data Only ............................. 24 Table 3-9 Bias, RMSD, and Ln(L) Results for Residuals for International Data Only.......... 25 Table 3-10 T-Test on Residuals Results for E900-15 and 10 CFR 50.61aBASELINE .......... 27 Table 3-11 T-Test on Residuals Results for E900-15 and 10 CFR 50.61aU.S. Data ............ 28 Table 3-12 T-Test on Residuals Results for E900-15 and 10 CFR 50.61aInternational Data 29 Table 3-13 Key for T-Test Tables .............................................................................................. 29 Table 3-14 Preliminary Type Testing Results with Unmodified BASELINE Data ...................... 33 Table 3-15 Preliminary Type Testing Results with Bias-Refit BASELINE Data ......................... 34 Table 3-16 Recommended C and D Values for Calculation .................................................. 42 Table 3-17 Recommended Default Chemistry Values (PWR and BWR) .............................. 50 Table 3-18 Chemistry, Temperature, and Fluence Limits of the E900-15 Database (BASELINE)

............................................................................................................................ 51 6

List of Abbreviations Abbreviation Definition location at one-quarter of the total thickness within the reactor pressure 1/4-T vessel as measured from the inner diameter location at half of the total thickness within the reactor pressure vessel as 1/2-T measured from the inner diameter location at three-quarters of the total thickness within the reactor pressure 3/4-T vessel as measured from the inner diameter ACRS Advisory Committee on Reactor Safeguards ADAMS Agencywide Documents Access and Management System ART adjusted reference temperature ASME American Society of Mechanical Engineers ASTM American Society for Testing and Materials BWR boiling-water reactor BWRVIP Boiling Water Reactor Vessel and Internals Program (EPRI)

C Celsius CF chemistry factor CFdefined chemistry factor defined as mean reality in simulation CFFIT chemistry factor refit to surveillance data CFSIM chemistry factor simulated from sampling mean and standard deviation CFR Code of Federal Regulations CMM correlation monitor material (also called a standard reference material)

EMA equivalent margins analysis EOL end of life EPRI Electric Power Research Institute Eq. equation ESD embrittlement shift delta ETC embrittlement trend correlation F Fahrenheit F forging fsurf fluence at inner diameter of reactor pressure vessel ID inner diameter ISP Integrated Surveillance Program Ln(L) Logarithm of Likelihood LWR light-water reactor MD management directive MRP Materials Reliability Program (EPRI)

MTR materials testing reactor NDT nil-ductility temperature NIIAR Research Institute of Atomic Reactors (Russia)

NRC U.S. Nuclear Regulatory Commission NRR Office of Nuclear Reactor Regulation P plate PFM probabilistic failure mechanism PSF pool side facility PSSP PWR Supplemental Surveillance Program PWR pressurized-water reactor RAMA Radiation Modeling Application (BWRVIP) 7

Abbreviation Definition REAP Reactor Embrittlement Archive Project RG regulatory guide RPV reactor pressure vessel RMSD root-mean-square deviation RTNDT reference temperature, nil ductility transition shift in reference temperature for a reactor vessel material measured at RTNDT the 30-foot-pound energy level SD standard deviation SLR subsequent license renewal SMR small modular reactor SRM standard reference material (also called a correlation monitor material) standard deviation standard deviation of T41J measurement reference temperature characterizing the onset of cleavage cracking at T0 elastic or elastic-plastic instabilities (or both)

T41J metric term for RTNDT TTS transition temperature shift U.S. United States of America USE upper-shelf energy USE(I) upper-shelf energy irradiated USE(U) upper-shelf energy unirradiated USE shift in upper-shelf energy VVER Water-Water Energetic Reactor (Soviet PWR design)

W weld 8

Executive Summary This report serves as a knowledge management tool to document work performed in support of a potential alternative to Regulatory Guide (RG) 1.99, Revision 2, Radiation Embrittlement of Reactor Vessel Materials, issued May 1988. RG 1.99 provides guidance to licensees of light water reactors in the United States to predict the change in the material reference temperature and the upper-shelf energy (USE) due to neutron irradiation. At this time, the staff is not pursuing revision of RG 1.99.

In 2019, the NRC staff completed an assessment of the continued adequacy of RG 1.99 in Assessment of the Continued Adequacy of Revision 2 of Regulatory Guide 1.99Technical Letter Report. The assessment identified several issues for further consideration. This report presents the technical basis for a potential alternative to RG 1.99, developed in response to the findings of the 2019 assessment. This report includes the elements for the potential alternative, their development, and the basis for the choices made by the staff in producing the potential alternative.

In RG 1.99, the reference temperature, adjusted to account for the effects of irradiation, and with margin added for uncertainty, is known as the adjusted reference temperature (ART). The potential alternative addresses the prediction of the ART but does not concern the prediction of USE. This is consistent with the findings related to USE from the RG 1.99 assessment, which indicated no change was warranted. The potential alternative is built around the standard American Society for Testing and Materials (ASTM) E900 15, Standard Guide for Predicting Radiation-Induced Transition Temperature Shift in Reactor Vessel Materials.

The potential alternative includes an embrittlement trend curve from ASTM E900 15, recommendations for use of plant-specific surveillance data, margins to account for uncertainty (both on initial properties and the shift in reference temperature due to irradiation), default values of input variables, and limitations on the ranges of input variables. This report also documents the method and findings of a study of the impact, in terms of the changes to the ARTs of the beltline materials, of implementing the alternative framework for the materials from a smart sample of 21 reactors.

This report serves a knowledge management purpose. As previously mentioned, the staff is not pursuing a revision to RG 1.99 to implement the proposed alternative. The decision to not pursue a revision is primarily based on the results of a risk study, documented in TLR RES DE CIB 2020 09 RG 1.99R2 Update FAVOR Scoping Study, dated October 26, 2020. The findings of this risk study are not discussed in this report.

9

1. Introduction The U.S. Nuclear Regulatory Commission (NRC) issued Regulatory Guide (RG) 1.99, Revision 2, Radiation Embrittlement of Reactor Vessel Materials (Ref. 1), in 1988. RG 1.99 provides guidance to licensees of light-water reactors (LWRs) in the United States to predict the change in materials properties due to irradiation. RG 1.99 provides a methodology to determine the reference temperature, nil ductility transition (RTNDT) of irradiated materials and the upper shelf energy (USE) of irradiated materials. In RG 1.99, the RTNDT adjusted for the effects of neutron irradiation is called the adjusted reference temperature (ART). RG 1.99 also provides guidance on the use of surveillance program data on plant-specific materials to adjust the prediction of RTNDT and USE.

In 2019, the NRC staff completed an assessment of the adequacy of RG 1.99, which it documented in Assessment of the Continued Adequacy of Revision 2 of Regulatory Guide 1.99Technical Letter Report (Ref. 2). The assessment identified a few issues for further consideration. The most significant of these is the performance of the embrittlement trend correlation at higher neutron fluences (greater than 6x1019 neutrons per square centimeter (n/cm2), (energy (E) > 1 mega electron-volt (MeV)).

The staff presented the assessment to the Metallurgy and Reactor Fuels Subcommittee of the Advisory Committee on Reactor Safeguards (ACRS) on August 22, 2019 (Ref. 3), and the ACRS full committee during its 668th meeting on November 6-8, 2019 (Ref. 4). The ACRS replied in its letter dated November 27, 2019 (Ref. 5). The staff responded to the ACRS by letter dated December 23, 2019 (Ref. 6).

The NRC held a public meeting on May 19, 2020, at which it discussed the technical basis supporting a potential alternative to RG 1.99, including the framework elements of the alternative RG, results of a fleet impact study of a smart sample of plants, and the results of a probabilistic fracture mechanics (PFM) analysis assessing the impacts of nonconservatisms associated with the RG 1.99 embrittlement trend correlation (ETC) (Ref. 7).

This report presents the technical basis for a potential alternative to RG 1.99, developed by the NRC staff working group and oversight group in response to the findings of the assessment report, the ACRS review, and its endorsement of the staffs effort to revise RG 1.99.

The NRC staff working group determined that, based on the assessment report in Reference 2, it was not necessary to update the USE model in RG 1.99 for the potential alternative RG, due to the relatively low safety significance and lack of regulatory need.

Section 2 of this report discusses the motivation for developing a potential alternative to RG 1.99. Section 3 describes the framework elements of a potential alternative RG, including the ETC, margins, use of surveillance data, default values for ETC inputs, and limitations.

Section 4 discusses the results of a fleet impact study related to the potential alternative RG, which determined the predicted changes in RTNDT associated with changing to an updated ETC for a smart sample of plants. Section 5 contains conclusions, and Section 6 lists the references.

10

2. Motivation for the Evaluation Effort Numerous alternative ETCs published since the original issuance of RG 1.99 use larger databases and more complicated mathematical forms. As a modeling exercise, creating statistical regressions from large databases has proven somewhat challenging, as many improved ETCs contain perceived weaknesses for particular subpopulations within the underlying database. The NRC has closely followed the evolution of American Society for Testing and Materials (ASTM) E900 and, with the advent of ASTM E900-15, Standard Guide for Predicting Radiation-Induced Transition Temperature Shift in Reactor Vessel Materials, gained access to the underlying BASELINE dataset. Access to this database supported the conduct of a high-quality preliminary assessment of RG 1.99. The results indicated statistically significant deviations between RG 1.99 predictions and measured data. Consequently, the NRC conducted a thorough assessment of RG 1.99 during its normal RG evaluation period.

In July 2019, the NRC staff completed an evaluation of RG 1.99, documented in Assessment of the Continued Adequacy of Revision 2 of Regulatory Guide 1.99Technical Letter Report (Ref. 2). The assessment identified a few issues for further consideration, of which the most significant is the performance of the ETC at higher neutron fluences (cited as greater than 3 to 6x1019 n/cm2, (E > 1 MeV)).

Other findings of the assessment included the following:

• The ETC is inaccurate for low-copper (Cu) materials.

• The standard deviation (SD) of the shift in the reference temperature due to irradiation (RTNDT) a is too small.

• The ETC has a conservative bias in the low-to-medium fluence range, which creates a potential burden on licensees, because predictions that are too high may narrow the operating window of pressure-temperature limits or increase the required hydrostatic testing temperature.

• The ETC lacks a specific input for irradiation temperature, which creates inaccuracy for conditions near the bounds of the data.

• The credibility criteria are fundamentally flawed due to a higher probability of rejecting new data as credible as more data become available. This is often caused by one outlier that does not meet the scatter requirements. In such cases, RG 1.99 defaults to the prediction based on the generic ETC rather than that based on the surveillance data, even if the surveillance data would result in a more accurate prediction of the material behavior.

• The USE model is nonconservative for 19 percent of materials; however, the safety impact of this nonconservatism is minimal.

a The term T41J is used synonymously with RTNDT in this report. RTNDT is defined as the change in the 30-foot-pound temperature. T41J is the metric equivalent. ASTM E900-15 uses the term transition temperature shift (TTS) synonymously with T41J.

11

• Several common practices not addressed in the RG should be addressed in a revision, such as use of sister plant data, implementation of credibility criteria, and degree-for-degree adjustment.

The NRC staff presented the assessment to the ACRS Metallurgy and Reactor Fuels Subcommittee on August 22, 2019 (Ref. 3), and the full committee during its 668th meeting on November 6-8, 2019 (Ref. 4). The ACRS issued a letter to the staff on November 27, 2019 (Ref. 5) related to this topic. The staff responded to the ACRS by letter dated December 23, 2019 (Ref. 6).

12

3. Regulatory Guide Framework Elements 3.1 Selection of the Embrittlement Trend Correlation 3.1.1 Background Since the publication of RG 1.99, various research and regulatory organizations have developed numerous ETCs. Many of these were based on considerably more data than the 177 pieces of data on which RG 1.99 was based. ASTM Subcommittee E10.02 has published standards containing an ETC several times since 1989. In 2014, the ASTM E10.02 subcommittee began an effort to update its E900 standard to a more modern ETC. The subcommittee evaluated nine different ETCs. It documented the results of these evaluations in the Adjunct for ASTM E900-15 Technical Basis for the Equation Used to Predict Radiation-Induced Transition Temperature Shift in Reactor Vessel Materials, dated September 18, 2015 (Ref. 8). The result was the publication of ASTM E900-15 (Ref. 9).

3.1.2 Available Embrittlement Trend Correlations The staff selected two ETCs for evaluation as potential replacements for the ETC of RG 1.99, considering the results of the ASTM ETC evaluation. The staff selected these because they are either already approved in an NRC regulation (Title 10 of the Code of Federal Regulations (10 CFR) 50.61a, Alternate fracture toughness requirements for protection against thermal shock events) (Ref. 10) or are approved in a consensus standard (ASTM E900-15).

10 CFR 50.61a (EONY)

The NRC sponsored the development of the 10 CFR 50.61a ETC (50.61a ETC) as part of an effort to update 10 CFR 50.61, Fracture toughness requirements for protection against pressurized thermal shock events (the Pressurized Thermal Shock Rule). This ETC was eventually incorporated into 10 CFR 50.61a, the Alternate Pressurized Thermal Shock Rule, published in 2010 (Ref. 10). The 10 CFR 50.61a ETC was fit to 855 RTNDT values encompassing U.S. LWR (boiling-water reactor (BWR) and pressurized-water reactor (PWR))

surveillance data through 2004.

The 10 CFR 50.61a ETC has 31 empirically fit parameters and is based on the following:

• three exposure variables: fluence, temperature, and flux

• four composition variables: Cu, nickel (Ni), manganese (Mn), and phosphorous (P)

• three categorical variables: product form, vessel manufacturer, and weld flux type ASTM E900-15 The ASTM E900-15 ETC (E900-15 ETC) was originally known as WRC(5)-R1. This was one of four ETCs that the ASTM chose for recalibration in 2014. The recalibrated version of WRC(5)-R1 is based on 1,878 RTNDT data points (the BASELINE database). BASELINE contains only commercial power reactor material data (BWR and PWR), not material test reactor data. The data include both U.S. and international surveillance data, with 1,033 data points being U.S. surveillance data.

The E900-15 ETC has 32 empirically fit parameters and is based on the following:

• two exposure variables: fluence and temperature 13

• four composition variables: Cu, Ni, Mn, and P

• one categorical variable: product form RG 1.99, Revision 2 For comparison, the RG 1.99 ETC was based on 177 RTNDT data points, as follows:

• one exposure variable: fluence

• two compositional variables: Cu, Ni

• one categorical variable: product form 3.1.3 E900-15 vs. 10 CFR 50.61a Statistical Comparison Statistical comparisons were made of the E900-15 and 10 CFR 50.61a ETCs, and also of both ETCs to the RG 1.99 ETC. The comparisons were made both for all data and for several data subsets or bins, including for PWRs and BWRs; product form (welds, base); low and high Cu; and low and high fluence. Low and high Cu bins were defined based on Cu 0.08 weight percent (%) (low) and > 0.08 weight % (high). Low and high fluence bins were defined based on neutron fluence values 3x1019 n/cm2 and > 3x1019 n/cm2 (E > 1MeV). This fluence value was selected because (1a) it is roughly the point at which the mean base metal predictions in RG 1.99 diverge from the mean of measured data, and (2) it ensured enough data in the high fluence bin to report useful statistical results.

The following statistical measures were evaluated:

• root-mean-square deviation (RMSD)a measure of scatter

• biasa measure of whether there is a mean overprediction or underprediction of the data by the ETC

• Ln(L)Logarithm of Likelihooda measure of goodness of fit

• Students t-testused to examine residual trends versus specific variables These methods are described below. Before developing the results in this report, researchers expected that the performance of both ATSM E900-15 and 10 CFR 50.61a would be acceptable for U.S. data as both were fit to largely the same data with additional results included in ASTM E900-15. As ASTM E900-15 was also fit to a broader set of data from international sources, it was expected to have superior performance with regard to these data. The results confirmed these expectations.

Caution should be used in interpreting the results here, as both trend curves are being compared to data that were used to fit the curves initially (entirely overlapping in the case of ASTM E900-15), and consequently these results provide no insight on any potential overfitting issues. Additionally, these results do not distinguish where issues arise from a paucity of data related to particular input variables as opposed to the inherent characteristics of the mathematical formulation used in the ETCs. Finally, these results do not (especially for ASTM E900-15) provide clear indications of the stability of the fits when extrapolated beyond the highest fluence data used to calibrate the ETC. Despite these cautions, the mathematical form-functions used for both E900-15 and 10 CFR 50.61a are expected to have superior extrapolation characteristics (i.e., predictions outside their basis data) relative to RG 1.99.

14

All data used to generate the results below are based on the BASELINE dataset as described in the RG 1.99 assessment.

3.1.4 Methodology of Statistical Tests Bias The values reported for bias in this evaluation are the Rmean values as defined in Table 3-1.

RMSD RMSD is defined as an estimate of the average deviation between predicted and observed RTNDT values in a particular data subset. Ideally, these values should be as small as possible.

Table 3-2 provides the equations that were used to determine RMSD.

Table 3-1 Statistical Metrics Used To Determine Bias Mean Residual =

(RMEAN) l l

=

/

T test on where RMEAN (TMEAN)

=1( )2

=

1 RMEAN is the mean value of all prediction errors in a RMEAN particular data subset. A value of zero indicates an unbiased prediction.

Metric TMEAN is a value of Students t-statistic that can be Interpretation used to assess the statistical significance of the TMEAN mean residual. If n exceeds 30, then values of TMEAN above 1.96 are generally considered significant.

Definitions y = RTNDT(PREDICTED) - RTNDT(MEASURED) or USE(I)MEASURED - USE(I)PREDICTED, as appropriate n = number of data records

=

=1 15

Table 3-2 Determination and Interpretation of RMSD Metric RMSD =

Definition An estimate of the average deviation between Metric predicted and observed RTNDT values in a particular RMSD Interpretation data subset. Ideally, this value should be as small as possible.

Definitions y = RTNDT(PREDICTED) - RTNDT(MEASURED) or USE(I)MEASURED - USE(I)PREDICTED, as appropriate n = number of data records

=

=

Logarithm of Likelihood, Ln(L)

Another statistical metric used in this report is called likelihood, which provides a quantitative answer to the following question:

Given a particular trend curve equation, and assuming that it is correct, what is the likelihood that a particular data set could have occurred?

Trend curve equations having higher likelihood values provide better representations of the data (i.e., have better goodness-of-fit) than those having lower likelihood values. The logarithm of likelihood, Ln(L), is typically reported and is defined in Equation (Eq.) (3-1) (Ref. 11):

n 1 () () 2 Ln (L) = Ln(2) =1 ln( ) + (3-1) 2 2 Where:

L Is the likelihood of a particular trend curve equation being correct, given the set of measurements being considered n Is the number of measured values of RTNDT RTNDT(measured)i Is a measured value of RTNDT RTNDT(predicted)i Is a value of RTNDT predicted by a particular trend curve corresponding to a particular measured value i Is the published standard deviation of the RTNDT(pred)i value. In some trend curve equations is the same for all conditions, while in others it may depend on variables such as the product form, Cu, or fluence.

Mathematically, the likelihood of a set of data is the product of probability densities of all data in the set; thus, it quantifies the probability of observing the set of data subject to the 16

assumption that the trend curve equation used to calculate the predicted values is correct.

Trend curve equations having higher likelihood are therefore more plausible models of reality than lower likelihood models, where reality is quantified by the set of data selected for evaluation. Examination of Eq. (3-1) makes clear that Ln(L) quantifies how well both the central tendency and scatter are represented by a particular ETC equation, as described below:

R() R()

• Mean: The term in Eq. (3-1) measures the difference between a predicted value and a measured value, normalized by the published standard deviation of the trend curve equation. Predictions that are inaccurate (i.e., far from the measured value) produce larger values of the highlighted term which, since this term is subtracted, decreases the value of Ln(L).

• Scatter: The ( ) term in Eq. (3-1) is the published standard deviation for an ETC equation corresponding to the ith measured RTNDT value. Predictions that are uncertain (i.e., have large values) decrease the value of Ln(L), since this term is subtracted.

Thus, predictions that are either inaccurate or highly scattered penalize (decrease) the Ln(L) metric. The units of Ln(L) may not seem as intuitive as the other statistical metrics. For example, both RMEAN and the RMSD have the same units as the quantity that is measured or predicted. However, as explained by Reference 11, least-squares fitting is a maximum likelihood estimation of the fitted parameters if the measurement errors are independent and normally distributed with constant standard deviation. Least-squares fitting is a tool familiar to many engineers.

Students T-Test (T-test on Slope)

Table 3-3 presents the methodology used to perform the t-test.

17

Table 3-3 Equations Used for the T-Test

=

2 For example, on the plot below, m = 0.869 Slope (m)

Metric Definition ll

=

where T-test on 2

=

slope 2 (TSLOPE) 1 2 = 2 2 ( 2 )

( 2)

Example: on the plot above, TSLOPE = 11.8 Any slope on a plot of prediction error vs. a composition or exposure variable indicates that the TTS equation assessed M does not fully describe the embrittlement trends associated Metric with that variable in a particular data subset. Ideally, the Interpretation value of slope should be zero.

TSLOPE is a value of a Students t statistic that can be used to TSLOPE assess the statistical significance of the slope value (i.e., is the slope statistically different from zero?).

Definitions x = variable being assessed for trends (e.g., Cu, Ni, fluence, temperature) y = RTNDT(PREDICTED) - RTNDT(MEASURED) or USE(I)MEASURED - USE(I)PREDICTED, as appropriate n = number of data records

=1 =1

=

=1

= 2 = 2

=1 =1 18

3.1.5 Statistical Test Results Bias, RMSD, and Ln(L) Results Table 3-4, Table 3-5, and Table 3-6 present the results for bias, RMSD, and Ln(L) for all data, U.S. data only, and international data only, for the ASTM E900-15 and 10 CFR 50.61a ETCs.

Table 3-7, Table 3-8, and Table 3-9 show the RMSD, bias, and Ln(L) for the same data subsets and also add the values for the RG 1.99 ETC and the RG 1.99 ETC with the degree-for-degree modification. Values are color coded as follows:

• BiasResults that indicate mean underprediction are shown in red, with intensity increasing with the magnitude of mean underprediction. Results that indicate mean overprediction are shown in yellow, with intensity increasing with the magnitude of mean overprediction. The shading intensity is greatest at the maximum underprediction and overprediction values.

• RMSDResults greater than or equal to 30 are shaded red, with increasing intensity up to the maximum reported value.

• Ln(L) [Log(Likelihood)]Results were first normalized by prediction of ASTM E900-15.

Therefore, the E900 results are 1, by definition, for all data subsets. Shading indicates greater deviation from the ASTM E900-15 result, with increasing intensity up to a maximum intensity at a ratio of 1.5. Normalized Ln(L) values > 1 for 50.61a indicate that the 10 CFR 50.61a ETC predicts the data less accurately than the E900-15 ETC, with accuracy decreasing as the values increase. Conversely, normalized Ln(L) values < 1 for 10 CFR 50.61a indicate that the 10 CFR 50.61a ETC predicts the data more accurately than the E900-15 ETC, with accuracy increasing as the values decrease.

The results in Table 3-7, Table 3-8, and Table 3-9 point out that both the E900-15 and 10 CFR 50.61a ETCs perform significantly better than RG 1.99, with or without the degree-for-degree modification. In particular, it can be seen that RG 1.99 significantly underpredicts the data for the high fluence data subset, based on the bias results, and RG 1.99 also has a large RMSD for the high fluence bin, indicating increased scatter.

When comparing U.S. results, the E900-15 and 10 CFR 50.61a ETCs perform similarly; when comparing international results, the E900-15 ETC predicts the surveillance data results more accurately than 10 CFR 50.61a. Overall, the E900-15 ETC performs the best with the lowest bias, better high fluence bias, and superior performance with international data (which include, among other things, a higher percentage of low Cu materials similar to more recently constructed nuclear power plants).

19

Table 3-4 RMSD, Bias, and Ln(L) Results for ResidualsAll Baseline Data b Ln(L)

[°F] Bias [°F] RMSD normalized 10 10 CFR 10 CFR E900-15 E900-15 E900 CFR 50.61a 50.61a 50.61a All 1878 -0.143 -1.566 23.981 30.918 1 1.113 Base 1212 -0.037 -1.505 22.445 24.883 1 1.050 Welds 666 -0.337 -1.678 26.548 39.608 1 1.224 BWR 342 -2.682 -0.107 22.076 22.792 1 1.017 PWR 1536 0.422 -1.891 24.384 32.452 1 1.134 Low Cu (0.08) 852 1.044 -6.254 20.749 33.117 1 1.198 High Cu

(>0.08) 1026 -1.130 2.327 26.365 28.966 1 1.046 Low F (3E19) 1512 -0.253 -0.924 22.843 23.518 1 0.994 High F (>3E19) 366 -1.251 -10.925 28.195 49.707 1 1.019 b Low/High Cu is cut at 0.08 weight %; Low/High Fluence is cut at 3x1019 n/cm2 (E > 1 MeV).

20

Table 3-5 RMSD, Bias and Ln(L) Results for ResidualsU.S. Data Only c Ln(L)

[°F] Bias [°F] RMSD normalized 10 10 CFR 10 CFR E900-15 E900-15 E900 CFR 50.61a 50.61a 50.61a All 1040 1.169 0.944 23.645 22.822 1 0.993 Base 692 0.960 0.448 21.197 20.207 1 0.992 Welds 348 1.584 1.932 27.882 27.287 1 0.994 BWR 170 -2.015 -0.483 20.954 22.761 1 1.012 PWR 870 1.791 1.223 24.136 22.834 1 0.989 Low Cu (0.08) 331 5.155 0.575 20.220 19.609 1 0.995 High Cu

(>0.08) 709 -0.692 1.117 25.085 24.176 1 0.992 Low F (3E19) 921 0.763 0.103 23.126 22.111 1 0.992 High F (>3E19) 119 0.592 -5.510 27.441 27.670 1 0.983 c

Low/High Cu is cut at 0.08 weight %; Low/High Fluence is cut at 3x1019 n/cm2 (E > 1 MeV).

21

Table 3-6 RMSD, Bias and Ln(L) Results for ResidualsInternational Data Only d Ln(L)

[°F] Bias [°F] RMSD normalized 10 10 CFR 10 CFR E900-15 E900-15 E900 CFR 50.61a 50.61a 50.61a All 838 -1.772 -4.682 24.390 38.677 1 1.262 Base 520 -1.365 -4.104 24.005 29.997 1 1.125 Welds 318 -2.439 -5.627 25.007 49.707 1 1.487 BWR 172 -3.343 0.264 23.133 22.823 1 1.022 PWR 666 -1.367 -5.959 24.705 41.806 1 1.323 Low Cu (0.08) 521 -1.567 -10.593 21.079 39.361 1 1.327 High Cu

(>0.08) 317 -2.110 5.033 29.023 37.528 1 1.165 Low F (3E19) 591 -1.836 -2.526 22.395 25.556 1 0.998 High F (>3E19) 247 -2.138 -13.534 28.550 57.378 1 1.036 d Low/High Cu is cut at 0.08 weight %; Low/High Fluence is cut at 3x1019 n/cm2 (E > 1 MeV).

22

Table 3-7 Bias, RMSD, and Ln(L) Results for Residuals for BASELINE (U.S. plus International Data)

Ln(L) normalized to E900 Bias, °F RMSD, °F (higher => worse)

RG 10 RG 10 RG RG E900- 10 CFR RG E900- RG 1.99R2 CFR 1.99R2 E900 CFR 1.99R2 1.99R2 15 50.61a 1.99R2 15 1.99R2 Subset No. (D/D) 50.61a (D/D) 50.61a (D/D)

All 1878 0.012 -0.143 -1.566 6.095 39.328 23.981 30.918 37.404 1.614 1 1.113 11.174 Base 1212 -4.476 -0.037 -1.505 1.906 35.622 22.445 24.883 32.147 1.654 1 1.050 16.758 Welds 666 8.180 -0.337 -1.678 13.717 45.300 26.548 39.608 45.436 1.543 1 1.224 1.403 BWR 342 -3.250 -2.682 -0.107 15.392 25.873 22.076 22.792 29.699 2.608 1 1.017 1.259 PWR 1536 0.738 0.422 -1.891 4.024 41.737 24.384 32.452 38.913 1.398 1 1.134 13.327 Low Cu (0.08) 852 -1.545 1.044 -6.254 0.848 34.117 20.749 33.117 34.516 1.539 1 1.198 23.722 High Cu (>0.08) 1026 1.305 -1.130 2.327 10.452 43.179 26.365 28.966 39.643 1.672 1 1.046 1.349 Low F (3E19) 1512 6.812 -0.253 -0.924 28.727 22.843 23.518 0.967 1 0.994 High F (>3E19) 366 -27.715 -1.251 -10.925 67.357 28.195 49.707 1.956 1 1.019 23

Table 3-8 Bias, RMSD, and Ln(L) Results for Residuals for U.S. Data Only Ln(L) normalized to E900 (higher =>

Bias, °F RMSD, °F worse)

RG 10 RG 10 RG RG E900- 10 CFR RG E900- RG 1.99R2 CFR 1.99R2 E900 CFR 1.99R2 1.99R2 15 50.61a 1.99R2 15 1.99R2 Subset No. (D/D) 50.61a (D/D) 50.61a (D/D)

All 1040 5.546 1.169 0.944 10.483 28.391 23.645 22.822 28.378 1.406 1 0.993 1.753 Base 692 3.906 0.960 0.448 8.991 25.702 21.197 20.207 26.138 1.344 1 0.992 2.082 Welds 348 8.808 1.584 1.932 13.449 33.096 27.882 27.287 32.376 1.520 1 0.994 1.142 BWR 170 -5.112 -2.015 -0.483 10.895 26.685 20.954 22.761 27.473 2.889 1 1.012 1.312 PWR 870 7.629 1.791 1.223 10.403 28.713 24.136 22.834 28.552 1.123 1 0.989 1.837 Low Cu (0.08) 331 6.135 5.155 0.575 6.144 23.974 20.220 19.609 23.975 1.358 1 0.995 3.092 High Cu (>0.08) 709 5.271 -0.692 1.117 12.509 30.234 25.085 24.176 30.215 1.427 1 0.992 1.158 Low F (3E19) 921 7.388 0.763 0.103 27.538 23.126 22.111 0.000 0.955 1 0.992 High F (>3E19) 119 -9.221 0.592 -5.510 34.646 27.441 27.670 0.000 1.111 1 0.983 24

Table 3-9 Bias, RMSD, and Ln(L) Results for Residuals for International Data Only Ln(L) normalized to E900 (higher =>

Bias, °F RMSD, °F worse)

RG 10 RG 10 RG RG E900- 10 CFR RG E900- RG 1.99R2 CFR 1.99R2 E900 CFR 1.99R2 1.99R2 15 50.61a 1.99R2 15 1.99R2 Subset No. (D/D) 50.61a (D/D) 50.61a (D/D)

All 838 -6.856 -1.772 -4.682 0.648 49.656 24.390 38.677 46.216 1.872 1 1.262 22.840 Base 520 -15.631 -1.365 -4.104 -7.522 45.591 24.005 29.997 38.725 2.056 1 1.125 35.788 Welds 318 7.493 -2.439 -5.627 14.009 55.669 25.007 49.707 56.361 1.570 1 1.487 1.701 BWR 172 -1.409 -3.343 0.264 19.837 25.044 23.133 22.823 31.747 2.330 1 1.022 1.207 PWR 666 -8.263 -1.367 -5.959 -4.307 54.227 24.705 41.806 49.268 1.757 1 1.323 28.275 Low Cu (0.08) 521 -6.424 -1.567 -10.593 -2.517 39.221 21.079 39.361 39.787 1.654 1 1.327 36.752 High Cu (>0.08) 317 -7.566 -2.110 5.033 5.850 63.167 29.023 37.528 55.179 2.201 1 1.165 1.760 Low F (3E19) 591 5.915 -1.836 -2.526 30.489 22.395 25.556 0.986 1 0.998 High F (>3E19) 247 -36.625 -2.138 -13.534 78.387 28.550 57.378 2.361 1 1.036 25

T-Test on Slope Results Table 3-10, Table 3-11, and Table 3-12 present the t-test results for the E900-15 ETC and the 10 CFR 50.61a ETC for the BASELINE data set, U.S. data only, and international data only.

Table 3-13 provides a key defining the data subsets and variables in the t-test results tables.

The t-test on slope results are presented here organized by data subset (listed in the second-to-left column) and variables (listed in the top row). The values reported in the tables are the Tslope values for each data subset and variable. The variables evaluated are the chemistry values (weight %) of Cu, Ni, P, Mn, temperature, neutron fluence, and neutron flux.

Results for RG 1.99 are not presented because they are poor in most data subsets and provide little additional insight. The results have been overlaid with conditional formatting as follows:

• Results < 2, representing statistically acceptable residuals in slope, have been marked in blue with increasing intensity for lower values.

• Results > 2, representing statistically significant (95 percent) residuals in slope, have been marked in red with increasing intensity for higher values.

The following examples illustrate how to interpret results in these tables. For example, in Table 3-11, for 10 CFR 50.61a, in the High F (fluence) data subset, the value of 3.610 in the Temp (temperature) column indicates that the 10 CFR 50.61a ETC does not model temperature very accurately for materials in the high fluence (> 3x1019 n/cm2) category. Also, in Table 3-11, for E900 in the low Cu data subset, the value of 3.675 in the Log(f) (log flux) column indicates that the E900-15 ETC does not model the effect of flux very accurately for materials in the low Cu (< 0.08 weight %) data subset. All results in red indicate likely inaccuracies in modeling.

These results do not indicate whether the necessary data to improve these inaccuracies exist.

It should be noted that extremely low t-test results do not, by themselves indicate model sufficiency, as such results may also indicate overfitting.

When comparing U.S. results, 10 CFR 50.61a performs better than E900-15; when comparing international results, 10 CFR 50.61a fares poorly in comparison to E900-15. Overall, E900-15 performs the best when compared to all data and subsets. It is particularly significant that 10 CFR 50.61a has a t-test slope > 2 for U.S.-only base materials and high fluence subsets, as this was of particular concern as a motivation for this work.

Although both E900-15 and 10 CFR 50.61a appear to contain statistically significant modeling residuals, 10 CFR 50.61a has considerably more and in a broader range of data subsets. This indicates that E900-15 performs better over a broader range of data subsets than 10 CFR 50.61a.

26

Table 3-10 T-Test on Residuals Results for E900-15 and 10 CFR 50.61aBASELINE E900 Major Set Subset n Cu Log(F) Ni Temp P Mn Log(f)

BASELINE All 1878 0.287 0.653 0.783 1.100 0.605 1.347 2.535 BASELINE Base 1212 1.290 0.482 1.284 0.679 1.521 0.657 2.521 BASELINE Welds 666 1.164 1.443 1.539 0.958 0.781 3.487 0.977 BASELINE BWR 342 0.566 0.405 0.828 1.445 1.223 0.097 1.346 BASELINE PWR 1536 0.187 1.164 0.450 0.296 0.045 1.133 0.682 BASELINE Low Cu 852 1.808 4.148 1.813 0.796 0.867 1.648 3.699 BASELINE High Cu 1026 2.192 2.074 0.034 0.209 0.147 0.421 0.558 BASELINE Low F 1512 1.400 1.950 0.290 1.290 0.930 2.310 3.220 BASELINE High F 366 2.620 3.400 1.380 0.010 0.290 1.430 0.120 10 CFR 50.61a Major Set Subset n Cu Log(F) Ni Temp P Mn Log(f)

BASELINE All 1878 2.864 2.959 4.003 7.249 6.085 0.496 0.248 BASELINE Base 1212 2.461 2.468 5.064 8.937 4.761 1.618 1.328 BASELINE Welds 666 1.665 1.781 5.815 1.867 4.093 2.590 1.380 BASELINE BWR 342 2.200 2.410 0.603 1.986 1.773 0.109 1.589 BASELINE PWR 1536 3.796 2.511 4.327 7.241 7.131 0.607 2.322 BASELINE Low Cu 852 4.601 4.377 10.704 3.067 1.045 1.209 2.406 BASELINE High Cu 1026 1.560 0.566 6.532 5.219 4.115 0.475 2.220 BASELINE Low F 1512 1.610 0.080 1.700 4.910 1.780 1.730 0.240 BASELINE High F 366 2.010 1.300 6.550 5.200 6.420 0.960 3.520 27

Table 3-11 T-Test on Residuals Results for E900-15 and 10 CFR 50.61aU.S. Data E900 Major Set Subset n Cu Log(F) Ni Temp P Mn Log(f)

US All 1040 0.592 1.046 2.306 2.700 2.338 0.744 2.808 US Base 692 0.755 0.273 0.689 1.936 2.920 1.345 2.523 US Welds 348 0.355 1.761 1.961 1.865 0.870 0.726 1.457 US BWR 170 0.376 0.130 0.907 1.023 1.544 0.544 1.385 US PWR 870 0.597 0.319 2.735 1.817 1.796 1.041 1.911 US Low Cu 331 1.394 4.336 0.644 1.931 0.320 0.087 3.675 US High Cu 709 2.687 1.164 2.845 0.848 1.063 1.765 1.204 US Low F 921 0.120 1.370 2.460 2.980 1.560 0.930 2.800 US High F 119 2.060 1.100 0.180 0.110 2.400 0.400 0.620 10 CFR 50.61a Major Set Subset n Cu Log(F) Ni Temp P Mn Log(f)

US All 1040 0.068 0.797 0.414 2.349 0.403 0.895 0.249 US Base 692 1.908 2.868 1.233 2.954 1.668 2.243 0.626 US Welds 348 1.336 1.731 0.033 0.352 0.380 1.961 1.001 US BWR 170 1.333 1.621 2.173 1.867 1.046 0.210 0.802 US PWR 870 0.558 0.868 0.447 2.517 0.059 1.200 0.408 US Low Cu 331 0.929 0.237 0.639 1.730 1.476 0.068 0.237 US High Cu 709 0.319 0.714 0.944 1.725 0.113 1.018 0.202 US Low F 921 0.310 0.510 0.350 1.070 0.060 0.920 0.760 US High F 119 0.040 0.850 0.140 3.610 1.360 0.250 0.620 28

Table 3-12 T-Test on Residuals Results for E900-15 and 10 CFR 50.61aInternational Data E900 Major Set Subset n Cu Log(F) Ni Temp P Mn Log(f)

INTERNATIONAL All 838 0.343 0.189 2.098 1.166 0.773 2.685 1.119 INTERNATIONAL Base 520 2.009 0.155 3.349 0.814 0.025 0.562 1.388 INTERNATIONAL Welds 318 1.574 0.375 0.504 0.847 1.626 4.350 0.032 INTERNATIONAL BWR 172 0.164 1.003 2.026 1.188 0.589 0.458 0.354 INTERNATIONAL PWR 666 0.447 0.460 2.894 1.994 1.208 2.614 0.498 INTERNATIONAL Low Cu 521 1.227 2.330 0.590 1.313 0.278 0.975 2.271 INTERNATIONAL High Cu 317 0.603 1.653 3.313 0.656 1.206 2.635 0.333 INTERNATIONAL Low F 591 1.070 1.100 3.910 1.560 0.490 4.940 1.570 INTERNATIONAL High F 247 2.190 3.110 1.220 0.070 1.600 2.090 0.170 10 CFR 50.61a Major Set Subset n Cu Log(F) Ni Temp P Mn Log(f)

INTERNATIONAL All 838 2.607 2.519 3.757 7.052 6.630 1.017 0.164 INTERNATIONAL Base 520 0.531 0.614 7.175 8.732 5.854 0.156 2.190 INTERNATIONAL Welds 318 2.657 2.737 6.083 2.146 4.324 1.485 1.963 INTERNATIONAL BWR 172 1.881 1.869 1.674 0.725 1.398 0.118 1.576 INTERNATIONAL PWR 666 3.624 1.172 3.375 6.491 7.653 1.125 3.540 INTERNATIONAL Low Cu 521 4.590 4.374 10.856 3.579 0.786 0.175 2.447 INTERNATIONAL High Cu 317 1.517 1.032 7.309 4.222 5.607 0.078 2.304 INTERNATIONAL Low F 591 1.600 0.970 3.180 6.040 1.670 3.390 0.580 INTERNATIONAL High F 247 2.310 0.880 6.770 4.330 7.000 1.210 3.930 Table 3-13 Key for T-Test Tables Low Cu 0.08 weight % Cu High Cu > 0.08 weight % Cu Low F Fluence 3x1019 n/cm2 (E > 1 MeV)

High F Fluence > 3x1019 n/cm2 (E > 1 MeV) n number of data in bin Cu copper content, weight %

Ni nickel content, weight %

F neutron fluence f neutron flux P phosphorus content, weight %

Mn manganese content, weight %

Temp Temperature, °C 29

3.1.6 Statistical Comparison Conclusions Several ETCs were compared using standard statistical methods consistent with the 2019 RG 1.99 assessment. The E900-15 and 10 CFR 50.61a ETCs perform roughly equivalently when compared using the U.S. data major set. The E900-15 ETC has significantly better performance when compared to the international data major set. Overall, E900-15 performs the best with the lowest bias, better high fluence bias, and better performance with international data (which include, among other things, a higher percentage of low Cu materials). In the t-test, E900-15 performs the best overall when compared to all data and subsets. Both ETCs retain some modeling residuals, but 10 CFR 50.61a has considerably more, and in a broad array of data subsets, indicating that E900-15 performs better over a broader range of inputs than 10 CFR 50.61a. Both E900-15 and 10 CFR 50.61a perform significantly better than RG 1.99 and RG 1.99 D/D over all tests and all data major sets (i.e., BASELINE, U.S., and international),

particularly with regard to the bias and RMSD in the high fluence data subset.

3.1.7 Subjective Factors Considered in Embrittlement Trend Correlation Selection Several important aspects of the E900-15 and 10 CFR 50.61a ETCs are not fully apparent through a direct statistical comparison. This section elucidates several subjective factors considered in arriving at E900-15 as the preferred ETC.

First, while both ETCs represent findings from considerably larger datasets than available for the development of RG 1.99, the E900-15 dataset included a larger quantity of data in the high fluence range. While the predominant source of this high fluence data is international, it represents the preponderance of available data in this regime. Consequently, while there may be some variation in process and measurement for the international data, the staff determined that this would be of less significance than the relative improvement in the ETC due to its inclusion (i.e., that the uncertainty of prediction would increase more by lack of data than by a potential difference in data acquisition between countries).

Second, researchers considered the utility of weighing performance of the 10 CFR 50.61a ETC against that of the E900-15 ETC for the international data. The international data comes from a somewhat more diverse group of designs, which are predominantly U.S. or U.S.-derived technologies. In addition, the international fleet is somewhat newer than the U.S. fleet and consequently contains material characteristics that would be more representative of any potential domestic new reactors (especially low Cu materials). Finally, as mentioned above, the international data constitute the bulk of high fluence data, providing the best estimate basis for curve-fitting in that regime.

Third, both ETCs required additional inputs relative to RG 1.99. This constituted an implementation concern. For the E900-15 ETC, the additional inputs are temperature, Mn, and P, of which only temperature was a strong term. For the 10 CFR 50.61a ETC, the additional inputs include temperature, flux, vessel manufacturer, and weld flux type, of which several have measurable impacts. The larger number of input variables for 10 CFR 50.61a (especially flux, for which no satisfying broad-range expression has yet been demonstrated) was found to increase the likelihood of overfitting while providing minimal improvements in overall ETC performance but would be worse when considering additional international data. Therefore, it would be more difficult for a licensee to implement the 10 CFR 50.61a ETC, as the additional required data may be difficult to ascertain for multiple reactor locations.

30

3.1.8 Rationale for Selection of the E900-15 Embrittlement Trend Correlation The NRC staff found that E900-15 was a better alternative ETC for the following primary reasons:

• for high fluence materials (i.e., > 3 x1019 n/cm2), E900-15:

- produces more accurate predictions of U.S. surveillance data; E900-15 has a small, conservative bias for the U.S. High Fluence subset, while 10 CFR 50.61a underpredicts the same subset

- produces more accurate predictions of the international data

• for new reactor applications, E900-15:

- performs better relative to the international data for the Low Cu category for the statistical measures (RMSD, bias, and Ln(L))

- performs better relative to t-test results for the Low Cu subset, as well as the input variables Ni, P, and temperature; this is particularly pertinent to new reactors, which will have low Cu, and consequently will be (relatively) more sensitive to other input variables (e.g., Ni, P, and temperature)

Additionally, the E900-15 ETC is based on a larger database, including additional U.S. surveillance data for 2004-2012 not included in the 10 CFR 50.61a database. Also, the 10 CFR 50.61a ETC may overfit due to the large number of input variables. Finally, the E900-15 ETC is expected to provide more accurate predictions of embrittlement in a broader band of temperatures than the 10 CFR 50.61a ETC, as indicated by the lower average t-test results for temperature for the E900-15 ETC.

31

3.2 Use of Surveillance Data 3.2.1 Background It has long been standard practice that plant-specific surveillance data be used in conjunction with the ETC. The procedure by which this has been historically completed in RG 1.99 was assessed as limited and, in certain cases, counterproductive in the RG assessment (Ref. 2).

Specifically, the likelihood that plant-specific surveillance data will be deemed noncredible when the RG 1.99 credibility criteria are applied increases as the number of surveillance data increases. Also, the default assumption of the RG 1.99 credibility criteria is that the RG 1.99 trend curve shape is correct, so the criteria do not effectively identify materials that have trend curve shapes not conforming the RG 1.99 trend curve shape. As a result, the staff investigated the potential for a more flexible and defensible methodology for the use of surveillance data.

3.2.2 Methodology and Results The staff used the statistical tests from 10 CFR 50.61a to investigate the quality of E900-15 ETC predictions and the plant-specific surveillance data, and then, depending on the outcome of these tests, to construct a refit procedure to provide a statistically justifiable adjustment to the E900-15 criteria that attains superior accuracy through use of the plant-specific data while not jeopardizing or overwhelming the statistical confidence gained by using a trend curve.

The statistical tests in 10 CFR 50.61a consist of four generic tests on the residuals between the E900-15 prediction and a series of plant-specific measured values for each material.

NUREG-2163, Technical Basis for Regulatory Guidance on the Alternate Pressurized Thermal Shock Rule, issued September 2018 (Ref. 12), describes the basis and significance of these tests. Figure 3-1 illustrates the function of these tests. The Type A test represents a bias test; Type B, a slope test; Type C, a scatter test; and Type D an outlier test. Note that 10 CFR 50.61a requires only the Type A, B, and D tests.

Figure 3-1 Explanatory diagram of Type A through D errors 32

Section 5.4 of NUREG-2163 (Ref. 12) defines the procedures for the Type A, B, and D tests, except for the Type C test procedure, which is performed as follows:

Determine the residual r for each datum using the following formula:

= 30() 30()

For each heat of material if:

2 then there is no Type C error.

Where is the number of data and is the standard deviation for the highest fluence datum as defined in Section 3.3 of this report.

The staff investigated the utility of these tests for the purpose described above. To do so, the staff applied the tests, using the E900-15 standard deviation formula (as U.S. specific standard deviations described in Section 3.3 were not yet available), to all domestic materials in the BASELINE dataset with sufficient data per material to apply the tests (any material with measurements at 3 or more fluence values). Use of the E900-15 SD should have resulted in needing to refit slightly more often compared to basing the SD on the U.S. data only (described in Section 3.3.), because the SD based on U.S. data tends to be larger than the E900-15 SD developed for the entire BASELINE dataset. Therefore, the number of materials requiring refit determined by this evaluation should be conservative. A 1-percent criterion (i.e., 2.33 SDs) was applied on the basis that a high degree of assurance would be desired that a genuine trend existed, contrary to the E900-15 results. A 1-percent criterion was also used for the surveillance checks required by 10 CFR 50.61a described in NUREG-2163. Use of a higher criterion (e.g., 5 percent) would give more weight to the plant-specific surveillance data. This was deemed appropriate, as E900-15 was generated using a large dataset to minimize the effects of errors in measurements of the individual materials. Consequently, the ETC is considered generically to have the greatest likelihood of approximating a true property without a strong material-specific contraindication (i.e., a 2.33 SD occurrence). Table 3-14 shows the results.

Table 3-14 Preliminary Type Testing Results with Unmodified BASELINE Data No Type Type Type Type Any Multiple Failures A B C D Failures Failures 100 29 3 44 35 47 41 Of the materials investigated, fully two-thirds exhibit acceptable behaviors according to the tests. Of the 47 that failed the Type tests, 41 exhibited multiple failures, suggesting a correlation. Two methods were proposed to refit data failing the tests. The first was to adjust the ETC by a bias adjustment (i.e., the Type A test result for that material, a scalar modifier),

33

while the second was to refit a modifying term to the ETC (equivalent to the CF (chemistry factor) refit in RG 1.99, a linear multiplying term). The objective was to manipulate the E900-15 ETC to the smallest degree (and thus retain the error-cancelling advantages of a broadly based ETC).

The results of the two refit procedures were virtually identical, and consequently the bias-based refit was selected as the most appropriate. Multiple reasonable scenarios exist for bias errors (e.g., variation in unirradiated property estimation, temperature effects). This provides a satisfying basis to both use a bias adjustment and potentially identify further sources of error with a numerically satisfying basis should such an exercise prove warranted. The slope adjustment lacked a high-confidence basis for overriding the curve-shape of E900-15 without a generic statistically justifiable numerical basis, while at the same time producing virtually identical results. Table 3-15 shows the results of bias-adjusted data.

Table 3-15 Preliminary Type Testing Results with Bias-Refit BASELINE Data No Type Type Type Type Any Multiple Failures A B C D Failures Failures 133 0 3 10 7 14 6 The results showed a large improvement in statistical performance with a minimal adjustment; specifically, the pass rate improves from 2/3 to 9/10. Several aspects of the conversion stand out. First, while 41 of the 47 failures in the unmodified sample exhibited multiple failures in testing, only 6 of 14 refit failures are correlated to multiple failure modes. This gives some confidence that the mean adjustment was the likeliest single cause of test failures in the overall population.

The Type B failures were identical for the unmodified and refit ETC. Each material data set for which a Type B failure occurred was manually examined. No common trend was evident among the three Type B failure materials. Consequently, the rarity of Type B failures and the lack of consistency among the Type B material data suggested that using Type B testing would be unproductive for the proposed refit framework.

As for the remaining postrefit Type C and D failures, a manual examination revealed no consistent trend in failure cause. Generally, these data sets exhibited odd scatter, large outliers, or other effects that would require a more indepth analysis to refit and consequently were not good candidates for constructing a generic methodology.

Figure 3-2 (top plot) shows an example of the E900-15 ETC for a plate material from a BWR plant, with the actual surveillance data also plotted. This material initially failed the Type A, C, and D tests. The bottom plot shows the residuals versus the ETC. Figure 3-3 shows the data for the same plant after the refit procedure. After refit, the material passed all four tests.

Figure 3-4 shows an example of the E900-15 ETC and actual surveillance data for a PWR plant forging material. This material passed all four type tests. Figure 3-5 shows the ETC for the same material after the refit procedure. While the refit curve does fit the data better, refit would not be allowed for this material in accordance with the procedure described below and shown in Figure 3-6.

34

Figure 3-2 Data for BWR Plant A before refit Top PlotE900-15 ETC for BWR Plant A with actual surveillance data superimposed Bottom PlotResiduals for the data versus the ETC 35

Figure 3-3 Data for BWR Plant A after refit Top plotE900-15 ETC with surveillance data superimposed Bottom PlotResiduals for the data versus the ETC 36

Figure 3-4 Data for PWR Plant B before refit Top PlotE900-15 ETC for PWR Plant B forging with actual surveillance data superimposed Bottom PlotResiduals for the data versus the ETC 37

Figure 3-5 Data for PWR Plant B forging after refit Top plotE900-15 ETC with surveillance data superimposed Bottom PlotResiduals for the data versus the ETC 38

3.2.3 Recommended Procedure Based on the above, the staff proposed a methodology to incorporate surveillance data as follows:

A. The material of interest must meet the limitations of this alternative as described in Section 3.5.

B. If three or more surveillance data points are available for the material of interest then the users should apply the Types A, C, and D tests to their data using a 2.33 criterion (do not add margin for comparison). Use as defined in Section 3.3 of this report.

a. If the data pass, use the ETC with a 2 margin term.

b. If the data do not pass one or more of the Type A, C, or D tests, the user may attempt to refit the data by adding the bias adjustment from the Type A result to the ETC and then performing the Type A, C, and D tests again.

i. If the data now pass, use the refit ETC with a 2 margin term.

ii. If the refit data do not pass, use the more conservative of the refit ETC or the initial E900-15 ETC (or present an acceptable alternative ETC) and add a 2.33 margin.

C. If two or fewer surveillance data points are available for the material of interest, use the E900-15 ETC with a 2 margin term.

Figure 3-6 depicts the process above in flowchart form.

The staff notes that the philosophy of use of surveillance data in the procedure outlined above differs from RG 1.99 in that RG 1.99 defaults to the use of the plant-specific surveillance data rather than the generic RG 1.99 ETC, provided the surveillance data meet certain criteria, while the proposed alternative defaults to the use of the generic E900-15 ETC, as long as the surveillance data pass certain tests. The refit procedure proposed above for materials that do not pass the tests also maintains the shape function of the E900-15 by a simple bias adjustment that simply moves the curve up or down.

39

Figure 3-6 Flowchart of surveillance data refit process 40

3.3 Margins 3.3.1 Structure of Margin Term The margin term is structured identically to that of RG 1.99.

M is the margin term.

= 2 + 2 (3-2) i is the SD for the initial RTNDT. In accordance with RG 1.99, if a measured value of initial RTNDT for the material in question is available, i is to be estimated from the precision of the test method. The NRC staff has typically allowed the value of i to be zero when a heat-specific measured value is available, although this is not explicitly stated in RG 1.99. Under this alternative, i may be set to zero when a heat-specific measured value is available, consistent with the precedent established by the staff in applying RG 1.99.

A is the number of SDs, normally A = 2. However, if the material has surveillance data, which failed one or more of the surveillance data consistency checks, even after refit, A = 2.33.

If generic mean values for that class of material are used, i is the SD obtained from the set of data used to establish the mean.

For plates and welds,

= x ( ) (3-3) where

= (° ) 1 900 15 and C and D are constants from Table 3-16.

3.3.2 Basis In RG 1.99, is defined as the standard deviation of RTNDT. The SD term determined using ASTM E900-15 Equation (9) is functionally identical to the term of RG 1.99. ASTM E900-15 does not address the uncertainty in initial RTNDT represented by the i term in RG 1.99.

Therefore, it is reasonable to use a margin term with the same structure as the RG 1.99 margin term, allowing for the inclusion of a nonzero i value, if appropriate.

RG 1.99 specifies a product form dependent form of the value, which is 17 degrees Fahrenheit (F) (9.44 degrees Celsius (C)) for plates and forgings, and 28 degrees F (15.56 degrees C) for welds. The RG 1.99 assessment (Ref. 2) noted that these values were too small when compared to the SD suggested by the BASELINE data set.

The C and D values were determined by the same procedure described in the E900 adjunct, Appendix D, except using the U.S. data only rather than all the baseline data for a given product form. Using the RMSD for the U.S. surveillance data for the SD of the TTS shift () is more appropriate than using the E900-15 SD equation because it is representative of the scatter to be 41

expected in the U.S. fleet materials. In addition, the SDs based on the U.S. data are somewhat higher that the E900-15 SDs, which is conservative. Also, the E900-15 SD for plate material is based in part on the data for standard reference materials (SRMs), which generally had less scatter than the plate, resulting in a lower SD. The staff considered use of an SD equation based partly on data from SRMs to be inappropriate because SRMs are not required to be tested and have no regulatory use in the United States. Further discussion of the effect of SRMs appears below under the heading Plate vs. Plate + SRM.

The data were sorted with respect to the predicted TTS e from E900-15, in ascending order. The data were then grouped in bins of 40 materials. The mean TTS for each bin, and the RMSD for each bin, were calculated. Table 3-16 provides the recommended C and D values.

( + )2

= (3-4)

Where n is the number of data in the bin.

The RMSD values were then plotted against TTS and a fit equation was determined using Excel. Since the fit equation for forgings had an essentially flat trend, it was determined to use a constant value for forgings. This was determined based on the RMSD of all 143 U.S. forging data, which was 21.49 degrees F (11.9 degrees C).

Table 3-16 Recommended C and D Values for Calculation Product Form C, ° F(°C) D Plate 5.11 (3.48) 0.35 Weld 14.94 (9.02) 0.14 Forging 21.49 0 Plate vs. Plate + SRM E900-15 uses a common SD term for plate and SRMs, which was determined based on the plate and SRM data combined (i.e., the BASELINE dataset). The staff found that when an SD equation was determined based on plate data only, the SD values as a function of TTS increased. This was true for both the U.S. data only and all the plate data in BASELINE. The effect of the SRM materials was to reduce the mean RMSD. SRMs are included in some but not all U.S. surveillance capsules, and there is no requirement to test SRMs (i.e., there is no regulatory use for them). Therefore, it seems appropriate to determine an SD based on plate data only. Figure 3-7 shows the fit of the equation to the U.S. plate data. Figure 3-8 shows a comparison of the SD values that would be determined from the U.S. plate data only, a fit for U.S. plate data plus SRM materials, a fit equation determined from both U.S. and international plate data only (labeled BASELINE plate), the E900-15 Plate + SRM equation, and finally the RG 1.99 base metal SD of 17 degrees F (9.44 degrees C), which applies to both plate and forgings. Figure 3-9 includes the same curves as Figure 3-8, plus showing the curves for SRMs e The TTS from E900-15 is equivalent to the T41J. TTS values were converted to degrees F for the purpose of determining the fit equation for degrees F.

42

only (BASELINE and U.S. only) that demonstrate why the inclusion of SRMs with plate reduces the SDs.

Figure 3-7 RMSD vs. TTS fit for U.S. Plate only Figure 3-8 Comparison of SD for Plate and Plate + SRM 43

Figure 3-9 Comparison of SD based on various data sets for Plate and Plate + SRM, and SRMs only Welds Figure 3-10 shows the fit to the U.S. weld data. Figure 3-11 shows a comparison of the TTS that would be calculated using both the E900-15 SD equation for welds, the U.S. weld-only equation from Figure 3-10, and the RG 1.99 constant SD value of 28 degrees F (15.56 degrees C). The SD values are similar for both, with the U.S. data predicting a slightly lower SD at higher TTS values, and the opposite at lower TTS values.

44

Figure 3-10 RMSD versus TTS fit to U.S. weld data Figure 3-11 Comparison of TTS determined using E900-15 and fit to U.S. welds only 45

Forgings Figure 3-12 shows the determination of the fit equation for U.S. forgings. Figure 3-13 shows a comparison of the SDs predicted as a function of TTS for the E900-15 equation (blue), the fit to the U.S. data (red), and a constant value determined from the RMSD of all the U.S. forging data (orange). The constant value of 21.49 degrees F was determined by combining all the U.S. forging data (144 pieces of data) into a single bin and calculating the average RMSD for the single bin. In Figure 3-13, the line for the constant value for all U.S. forgings and the values from the fit equation in Figure 3-12 lie almost on top of one another.

46

Figure 3-12 RMSD versus TTS fit for U.S. forging data Figure 3-13 Comparison of forging fits for U.S. only, E900-15, and RG 1.99 base metals 47

3.4 Default Values Default values are needed for the RTNDT calculation when certain data are missing. This includes chemistry composition values in weight % of Cu, Ni, Mn, P, and irradiation temperature. Default values should be conservative such that their use is biased toward calculating a higher RTNDT. Therefore, default chemistry values should be on the high side of the possible range and irradiation temperature values should be on the low side of the possible range.

3.4.1 Need for Default Values It is expected that missing chemistry values will be rare for beltline materials in U.S. commercial nuclear power plants, particularly Cu and Ni. For most plants, the irradiation temperature should also be known, since it is considered equivalent to the reactor inlet or cold-leg temperature for PWRs and equivalent to the reactor recirculation loop temperature in BWRs.

3.4.2 Approach To determine the default values, the staff examined the distribution of values for each variable in BASELINE. Histograms were created for each chemistry variable, and the quartiles of the distribution were determined. Figure 3-14 shows an example of a histogram for Cu for BWR forgings from BASELINE. The objective was to determine whether the values for each variable conformed to a uniform, normal, or other identifiable distribution that could be used to define percentiles, such that the default values could be defined in terms of a certain percentile (e.g., 95th). A similar approach was used for SA-508, Class 2 nozzle forgings as described in BWRVIP-173NP-A, BWR Vessel and Internals Project, Evaluation of Chemistry Data for BWR Vessel Nozzle Forging Materials, dated July 31, 2011 (Ref. 13), in which a +2 value, corresponding to a 97.8 percent confidence interval, determined from available industry data, was used as the default value for Cu, Ni, Mn, and P when actual values were missing.

48

Figure 3-14 Distribution of Cu (weight %) values for BWR forgings from BASELINE 3.4.3 Results/Recommendations The distributions did not consistently conform to any recognized distribution such as normal or uniform.

3.4.4 RecommendationChemistry For the default chemistry values, it is recommended to use the maximum values from the database for Cu, Ni, Mn, and P. The staff considered using an upper 95 or 75 percentile value; however, since values do not appear to be normally distributed, the staff decided to use maximum values from distribution. These values are conservative, and missing chemistry values are expected to be a rare case. Table 3-17 gives the recommended default chemistry values.

49

Another option considered was the use of a specification maximum; however, specifications do not contain ranges for all elements. For example, SA-533, a commonly used specification for RPV plates, does not specify a range for Cu.

Table 3-17 Recommended Default Chemistry Values (PWR and BWR)

Product Form Cu Ni Mn P Forgings 0.16 0.86 1.41 0.020 Plate 0.25 0.68 1.65 0.021 Welds 0.41 1.20 1.96 0.024 3.4.5 RecommendationTemperature PWRs The reactor inlet or cold-leg temperature should be used as the irradiation temperature for PWRs. A weighted average should be used if the temperature changed for different cycles, such as due to power uprates. The default value for PWRs is 523 degrees F (272.8 degrees C) based on the U.S. fleet minimum from BASELINE.

BWRs The recirculation loop temperature should be used as the irradiation temperature of BWRs. A time-weighted average should be used if the temperature changed for different cycles, such as due to power uprates. The default value for BWRs is 530 degrees F (276.7 degrees C) based on the minimum value from BASELINE.

50

3.5 Limitations 3.5.1 Background ASTM E900-15, Section 1.1.2.1, lists A533 Type B Class 1 and 2, A302 Grade B, A302 Grade B (modified), and A508 Class 2 and 3, and European and Japanese steel grades that are equivalent to these ASTM grades, as the applicable grades. These grades are essentially equivalent to those listed in RG 1.99 except that RG 1.99 also lists SA-336. Therefore, this alternative is considered to be applicable to the material grades listed in RG 1.99 and ASTM E900-15 and grades other than those listed should be justified.

ASTM E900-15, Section 1.1, provides the range of material and irradiation conditions in the database for variables used in the embrittlement correlation. These maxima and minima do not restrict the use of E900-15 within these bounds; however, ASTM E900-15, Section 1.2, recommends caution when using the E900-15 ETC near these maxima and minima, and requires the user to ensure that the ETC is appropriate for the conditions. Table 3-18 provides these maxima and minima.

Table 3-18 Chemistry, Temperature, and Fluence Limits of the E900-15 Database (BASELINE)

Parameter Minimum Maximum Cu, weight % None 0.4 Ni, weight % None 1.7 Mn, weight % 0.55 2.0 P, weight % None 0.03 Irradiation Temperature 491 °F (255 °C) 572 °F (300 °C)

Neutron Fluence (n/cm2) 1x1017 1x1020 3.5.2 Methodology Comments during the ASTM voting process for ASTM E900-15 expressed concerns about the range of applicability, with one commenter recommending more restrictive limitations based on

+/- 3, and warning levels based on +/- 2 (see comments related to Negative Vote by Tim Williams on E900-14 in Appendix E to the E900 adjunct, Ref. 8). The staff evaluated the need for similar limits. The approach for each individual variable was to divide the BASELINE database into two populations based on the percentile of all data (for example, upper 5th percentile versus the entire data set). Then the staff performed the surveillance data consistency checks (Type A, B, C, D) on both populations to determine whether there was a statistically significant difference in the proportion of the data passing and failing the consistency checks, for the different percentiles. Fishers exact test for count data and Pearsons Chi-square test with a simulated p-value (based on 2,000 replicates) were performed on the upper 5th, 4th, 3rd, 2nd, and 1st percentiles of RPV chemistry and lower 5th, 4th, 3rd, 2nd, and 1st percentiles of irradiation temperature. A 95-percent confidence level was used, meaning a p-value < 0.05 would indicate a statistically significant difference. The resulting p-values for either test were never below 0.249, demonstrating that there was no statistically significant difference in performance for the ETC between the entire population and any of the other percentiles for any of the variables. Therefore, the staff concluded, based on the statistical 51

tests, that it was not necessary to impose limitations more restrictive than the limitations described in ASTM E900-15.

However, a review of the irradiation temperature data in BASELINE for all U.S. plants (Figure 3-15) shows that data are very sparse below 523 degrees F (272.8 degrees C), which is relatively consistent with the lower limit of 525 degrees F (273.8 degrees C) in RG 1.99.

Therefore, the minimum temperature limit for this alternative is 523 degrees F (272.8 degrees C). ASTM E900-15 has an upper temperature limit of 572 degrees F (300 degrees C), which is slightly lower than the upper limit in RG 1.99 of 590 degrees F (310 degrees C). Since embrittlement should be less as temperature increases, the staff finds it acceptable to use the alternative up to the RG 1.99 upper limit of 590 degrees F (310 degrees C). Correction factors for temperatures outside of these limitations should be justified.

The limitations on chemistry and neutron fluence specified by ASTM E900-15 are less restrictive than those of RG 1.99, at least with respect to Ni content. The procedures of RG 1.99 are described as being applicable to the neutron fluence levels, Cu content, and Ni content within the ranges given in Figure 1 and Tables 1 and 2 of RG 1.99, respectively. These limitations are a maximum Cu content of 0.4 percent, maximum Ni content of 1.20 percent, and a maximum fluence of 1x1020 n/cm2. As these chemistry limits are well within the limitation of E900-15, all materials acceptable for use with RG 1.99 are acceptable for use with ASTM E900-15.

3.5.3 Conclusions and Recommendations The staffs evaluation of limitations concludes that it is acceptable to use the potential alternative described in this report within the following limitations:

• The alternative may be used with A533 Type B Class 1 and 2, A302 Grade B, A302 Grade B (modified), and A508 Class 2 and 3, European and Japanese steel grades that are equivalent to these ASTM grades, and SA-336.

• The range of Cu, Ni, Mn, and P, and neutron fluence values must be within the maxima and minima listed in Table 3-18.

• The maximum irradiation temperature of 590 degrees F (310 degrees C) is consistent with RG 1.99.

• The minimum irradiation temperature is 523 degrees F (272.8 degrees C), which is more restrictive than the maxima and minima of the E900-15 database.

Correction factors for temperatures outside of these limitations should be justified. This alternative may also be used with other material grades if justification is provided of equivalency to the listed grades.

52

Figure 3-15 Distribution of U.S. reactor temperature data from the BASELINE database versus Ni content 53

4. Fleet Impact Evaluation 4.1 Methodology of Fleet Impact Evaluation The NRC staff recognized that using an alternative RG based on the E900-15 ETC could impact the operating fleet by resulting in increased ARTs. To better understand the extent to which the ARTs would change, the staff performed a fleet impact study on a smart sample of 21 reactors to determine the change in ART and RTPTS resulting from a change from RG 1.99 to an alternative ETC. This change in ART is designated the embrittlement shift delta (ESD). The equation for ESD is:

= 90015 1.99 (4-1)

Where:

90015 = () + 41 +

where T41J is determined using the E900 15 ETC, M = margin determined in accordance with Section 3.3

()

1.99 = 1.99 The fleet impact study was conducted for a hypothetical change from RG 1.99 to the E900-15 ETC. The number of materials experiencing increases or decreases in ART and the amount of these increases and decreases were not used to inform the decision on which ETC should be chosen. However, this information was used to qualitatively assess the impact that would be expected with adopting the E900-15 ETC.

Another important purpose of the fleet impact study was to determine the range of the changes in ART resulting from switching from the RG 1.99 ETC to the E900-15 ETC. This range of ESDs was used as an input to the PFM evaluation licensing basis ARTs from the plant data searches used to calculate the ESD. Licensing basis ARTs are generally based on RG 1.99, Position 2.1, for materials having credible surveillance data, and RG 1.99, Position 1.1, for materials without credible surveillance data.

E900-15

= () + + (4-2)

The TTS is calculated using Equation 1 of E900-15, as modified using the refit procedure described in Section 3.2. The refit procedure used the available surveillance data in the Reactor Embrittlement Archive Project (REAP). The surveillance data consistency checks used a program written in the R computer language for those materials with available and sufficient surveillance data in REAP (three or more surveillance data points). The refit term thus determined was added to the TTS calculated as described above. Only three materials in the fleet impact smart sample actually required a refit.

54

For determining the ARTs, the 1/4T fluence was used; that is, the fluence estimated at 1/4 of the thickness of the vessel from the inner surface. The 1/4T fluence was calculated using the attenuation formula of RG 1.99, based on the end-of-life RPV inner surface fluence for the material of interest. For calculating RTPTS, the RPV inner surface fluence was used without attenuation. The 1/4T location was chosen since the ART at this location usually supports regulatory criteria related to the pressure-temperature limits for normal cooldowns.

The margin, M, was previously determined as described in Section 3.3.

Calculations of the ARTs were executed in an Excel spreadsheet.

4.2 Results Figure 4-1 shows the distribution of ESDs for all materials in the smart sample. At the 1/4T location, the median ESD is in the bin for 10-25 degrees F, with few materials having ESDs greater than 70 degrees F. For the inner diameter (ID) location, the values are somewhat higher, as expected. For the ID location, the highest ESD for any material was 123 degrees F, while at the 1/4T location, the highest ESD was 102 degrees F. In Figure 4-1 through Figure 4-9, the numbers on the X-axis represent the highest value for the bin. For example, in Figure 4-1, the bars adjacent to the number 0 on the X-axis (on the left side) represent the numbers of ESDs having values > -25 and 0.

55

Figure 4-1 Distribution of ESDs, all materials in fleet impact smart sample Figure 4-2 shows the distribution of ESDs for the limiting materials only for the 1/4T location; in other words, those materials with the highest ART or RTPTS for a given reactor at the 1/4T location. In Figure 4-2, the light blue bars represent the ESDs for materials that are limiting when the same material remains limiting, whether RG 1.99 or ASTM E900-15 is used, while the dark blue bars show the ESDs for those materials that have a change in limiting material when the E900-15 ETC is used. In Figure 4-2, the ESD for those materials that had a change in limiting material is calculated on the difference in ARTs for old and new limiting materials:

= 90015( ) 1.99( ) )

This situation occurred for 5 of 21 reactors in the smart sample. Thirteen reactors had positive ESDs for the limiting materials, and nine reactors had negative ESDs, for the 1/4T location. f For those reactors with positive ESDs at the 1/4T location for limiting materials, only two reactors had increases in ESD at the 1/4T location of 50 degrees F (10 degrees C) or greater.

Figure 4-3 shows the distribution of ESDs for the limiting materials only for the ID location; in other words, those materials with the highest ART or RTPTS for a given reactor at the ID location.

f One reactor had two limiting materials identified in the plant data searches. One material is a longitudinal weld and one material is a circumferential weld. Therefore, the total number of limiting materials in Figures 4-2 and 4-3 equals 22 rather than 21.

56

In Figure 4-3, the light orange bars represent the ESDs for materials that are limiting when the same material remains limiting whether RG 1.99 or ASTM E900-15 is used, while the dark orange bars show the ESDs for those materials that have a change in limiting material when the E900-15 ETC is used. This situation occurred for 5 of 21 reactors in the smart sample.

Thirteen reactors had positive ESDs for the limiting materials, and nine reactors had negative ESDs for the ID location. For those reactors with positive ESDs at the ID location for limiting materials, only three reactors had increases in ESD at the ID location of 50 degrees F (10 degrees C) or greater.

For limiting materials, the maximum ESD was 60 degrees F at the ID and 46 degrees F at the 1/4T location. Figure 4-2 and Figure 4-3 do not show the ESD for the original limiting material if the limiting material changed, since the ESD for the former limiting material would no longer be relevant.

Figure 4-2 Distribution of ESDs for limiting materials only, at 1/4T location 57

Figure 4-3 Distribution of ESDs for limiting materials only at ID location Figure 4-4 shows the distribution of ESDs as a function of neutron fluence for both the ID and 1/4T locations. A trend toward higher ESDs occurs as fluence increases. The ID location tends to have higher ESDs, which is not surprising, since neutron fluences are higher at the ID, and the RG 1.99 ETC is known to be nonconservative at higher fluences. Figure 4-5 shows the distribution of ESDs for limiting materials only. For both limiting and nonlimiting materials, a similar trend is observed with an increase in ESDs as fluence increases.

For base materials (plates and forgings), Figure 4-6 and Figure 4-7 show the distribution of ESDs versus fluence for all materials and limiting materials only, respectively. Both figures show a trend of increasing ESDs as neutron fluence increases.

For weld materials, Figure 4-8 and Figure 4-9 show the distribution of ESDs versus fluence for all materials and limiting materials only, respectively. The weld materials show a less pronounced trend of increasing ESDs with neutron fluence than the base materials, and approximately equal numbers of materials have positive and negative ESDs. For limiting weld materials, more materials actually have negative ESDs than positive ESDs.

The results of the fleet impact study showed the following, if the potential alternative RG framework were implemented:

• There is a tendency for material reference temperatures to increase, particularly for base metals.

• ID reference temperatures tend to increase more than the 1/4T reference temperature (ART).

• Base materials are more likely to see increases in reference temperatures.

• Many weld materials see reductions in reference temperatures at fluences

< 4x1019 n/cm2.

58

• Based on the smart sample, only a handful of plant limiting materials will have increases in reference temperatures > 50 degrees F (30 degrees C), and these tend to be at fluences ~6x1019 n/cm2.

• Approximately 20 percent of plants would experience a change in the plant limiting material.

59

Figure 4-4 Distribution of ESDs versus fluence for all materials Figure 4-5 Distribution of ESDs versus fluence for limiting materials only 60

Figure 4-6 Distribution of ESDs versus fluence for all base materials Figure 4-7 Distribution of ESDs versus fluence for limiting base materials only 61

Figure 4-8 Distribution of ESDs versus fluence for all weld materials Figure 4-9 Distribution of ESDs versus fluence for limiting weld materials only 62

5. Conclusions As a result of the 2019 evaluation of the adequacy of RG 1.99 (Ref. 2), the staff initiated an effort to evaluate a potential alternative to RG 1.99, and whether formal implementation of such an alternative was necessary, based on both technical adequacy and PFM considerations. This report documents the technical basis of a potential alternative to RG 1.99 that was developed to address the issues for further consideration identified in Reference 2. This report also documents the results of a study of the fleet impact if the potential alternative were implemented. This report does not address whether implementation of a revision or alternative to RG 1.99 is necessary from a safety or risk perspective. The results of the related risk assessment contained in TLR-RES-DE-CIB-2020-09RG-1.99R2 Update FAVOR Scoping Study, dated October 26, 2020 (Ref. 14), supported the decision by the staff not to pursue implementation of a potential alternative to RG 1.99, Revision 2, at this time.

5.1 Elements of Alternative This report documents the technical basis for the elements of a potential alternative or revision to RG 1.99. This alternative consists of a methodology for estimating the ART or RTPTS based on the E900-15 ETC. The potential alternative has the following elements:

• ETCSection 3.1 of this report documents the staffs evaluation of two candidate alternative ETCs to RG 1.99 ETC: the E900-15 and 10 CFR 50.61a ETCs. A statistical evaluation of the performance of the two candidate ETCs against surveillance data (consisting of both U.S. and international LWR surveillance data) in the BASELINE database aided in selecting the ETC. The staff considered the statistical evaluation results in addition to nonquantitative factors in selecting the E900-15 ETC as the basis for this alternative framework.

• Use of surveillance dataSection 3.2 of this report describes the method for using plant-specific surveillance data:

- Four surveillance data consistency checks are evaluated, known as Type A (bias test), Type B (slope test), Type C (scatter test), and Type D (outlier test). The staff recommended only Types A, C, and D for the proposed alternative.

- If the Type A, C, and D tests are passed, then the E900-15 ETC is used without adjustment; if one or more tests failed, a refit procedure is performed based on a bias adjustment. The checks are then performed on the refit curve. If the refit passes, the refit curve is used with the same margins as the nonrefit curve (2 SDs). If the refit curve fails any checks, the more conservative results between the refit and nonrefit curve are used, with an increased margin of 2.33 SDs.

- The philosophy of use of surveillance data differs from RG 1.99 in that RG 1.99 defaults to the use of the plant-specific surveillance data rather than the generic RG 1.99 ETC, provided the surveillance data meet certain criteria, while the proposed alternative defaults to the use of the generic E900-15 ETC, as long as the surveillance data pass certain tests.

• MarginsSection 3.3 of this report describes the determination of the margins to be added to the E900-15 ETC to account for uncertainty. The structure of the margin term 63

is similar to RG 1.99; however, the SD of the RTNDT term () is derived from U.S. data in the BASELINE database and varies with the magnitude of RTNDT. This results in somewhat larger margins than are currently employed in RG 1.99, which addresses a finding in the RG 1.99 assessment report that found the margins were too small at higher neutron fluences.

• Default ValuesSection 3.4 of this report describes the default values for the input parameters to the ETC (chemistry values and irradiation temperature). These are to be used if the user cannot determine certain input parameters. The default values are generally based on the highest values in the database for chemistry values (which is conservative), and low values for temperature (which is conservative).

• LimitationsThe E900-15 standard defined the limits of applicability of the standard with respect to chemistry values, irradiation temperature, and neutron fluence.

Section 3.5 of the report describes the staffs evaluation of whether more restrictive limits are needed than those of ASTM E900-15, based on a comparison of the Type A, C, and D test results for the population versus certain more restrictive percentiles. The staff determined the ASTM E900-15 limitations are adequate.

5.2 Fleet Impact Study Section 4 of the report documents the fleet impact evaluation of a smart sample of 21 plants.

The evaluation used licensing basis material inputs to determine the change in ART and RTPTS resulting from a change from RG 1.99 to an alternative ETC. This change in ART associated with switching from RG 1.99 to an alternate ETC is designated the ESD.

The fleet impact study found the following, if the proposed alternative framework based on E900-15 were implemented:

• There is a tendency for material reference temperatures to increase, particularly for base metals.

• Many weld materials see reductions in reference temperatures at fluences

< 4x1019 n/cm2.

• Based on the smart sample, only a handful of plant limiting materials will have increases in reference temperatures greater than 50 degrees F (30 degrees C), and these tend to be at fluences ~6x1019 n/cm2.

• Approximately 20 percent of plants would experience a change in the plant limiting material.

The results of the fleet impact study, with respect to the range of ESDs to be expected, was used to inform a PFM analysis in TLR-RES-DE-CIB-2020-09RG-1.99R2 Update FAVOR Scoping Study, dated October 26, 2020 (Ref. 14).

64

Acknowledgements The authors would like to acknowledge the support of the RG 1.99 Oversight Group, Hipo Gonzalez, Raj Iyengar, Dave Rudland, Allen Hiser, and Rob Tregoning, for their oversight and technical support; Patrick Raynaud for developing the companion PFM analysis to this document; Brian Harris (Office of the General Counsel) for legal advice; Louise Lund, Jeremy Bowen, Anna Bradford, Bob Caldwell, Joe Donoghue, and Meena Khanna for management oversight and support; Ganesh Cheruvenki, Joel Jenkins, and Pat Purtscher for support with the fleet impact data searches; and finally, Matt Mitchell and Dave Alley for support as former Oversight Group members.

65

6. References

1. Regulatory Guide 1.99 (Task ME 305-4), Revision 2, Radiation Embrittlement of Reactor Vessel Materials, May 31, 1988 (ADAMS Accession No. ML003740284)

2. Assessment of the Continued Adequacy of Revision 2 of Regulatory Guide 1.99 Technical Letter Report, July 31, 2019 (ADAMS Accession No. ML19203A089)

3. Transcript of the Advisory Committee on Reactor Safeguards, Metallurgy and Reactor Fuels Subcommittee Meeting on Regulatory Guide 1.99, August 22, 2019

4. Transcript of the Advisory Committee on Reactor Safeguards 668th Full Committee Meeting, November 6, 2019 (Open), pp. 1-89 (ADAMS Accession No. ML20009C415)

5. Letter from P. Ricardella to M.M. Doane, Assessment of the Continued Adequacy of Revision 2 of Regulatory Guide 1.99, November 27, 2019 (ADAMS Accession No. ML19331A231)

6. Letter from R.V. Furstenau to P.C. Ricardella, Response to the Advisory Committee on Reactor Safeguards Letter Regarding the Staff's Assessment of the Continued Adequacy of Revision 2 of Regulatory Guide 1.99, December 23, 2019 (ADAMS Accession No. ML19352E588)

7. Summary of the May 19, 2020, Category 2 Public Meeting on Regulatory Guide 1.99, Revision 2, and Reactor Vessel Material Surveillance Program, June 15, 2020 (ADAMS Accession No. ML20168A009)

8. Adjunct for E900-15 Technical Basis for the Equation Used to Predict Radiation-Induced Transition Temperature Shift in Reactor Vessel Materials, ASTM International, West Conshohocken, PA, A Product of ASTM Subcommittee E10.02, on the Behavior and Use of Nuclear Structural Materials Work Package WK34875, September 18, 2015

9. ASTM E900-15, Standard Guide for Predicting Radiation-Induced Transition Temperature Shift in Reactor Vessel Materials, American Society for Testing and Materials, West Conshohocken, PA, April 2015 (editorial correction 1 approved in April 2017)

10. Title 10 of the Code of Federal Regulations, Section 50.61a, Alternate fracture toughness requirements for protection against pressurized thermal shock events, https://www.nrc.gov/reading-rm/doc-collections/cfr/part050/part050-0061a.html

11. Numerical Recipes, The Art of Scientific Computing, 3rd Edition by William H. Press, Saul A. Teukolsky, William T. Vetterling and, Brian P. Flannery, Cambridge University Press, 2007 (ISBND10: 0521880688, or ISBND13: 978D 0521880688)

12. NUREG-2163, Technical Basis for Regulatory Guidance on the Alternate Pressurized Thermal Shock RuleFinal Report, September 30, 2018 (ADAMS Accession No. ML18255A118) 66

13. BWRVIP-173NP-A, BWR Vessel and Internals Project, Evaluation of Chemistry Data for BWR Vessel Nozzle Forging Materials, July 31, 2011 (ADAMS Accession No. ML12083A268)

14. TLR-RES/DE/CIB-2020-09, RG 1.99 Revision 2 Update FAVOR Scoping Study, October 26, 2020 (ADAMS Accession No. ML20300A551) 67

Appendix A Comparison of BASELINE T41J Values to REAP T41J The transition shift temperature is determined by fitting a four-parameter tanh curve to each surveillance capsule data set. The coefficients for Equation A-1 were determined by fitting U.S. surveillance data from the Reactor Embrittlement Archive Project (REAP) surveillance database:

= + ((( ) )) (A-1)

The resulting reference temperature, nil ductility transition (RTNDT) values were matched and compared with the RTNDT values from the BASELINE subset of American Society for Testing and Materials (ASTM) E900-15, Standard Guide for Predicting Radiation-Induced Transition Temperature Shift in Reactor Vessel Materials. Records were excluded from analysis where no match between the data in REAP and BASELINE was identified. Unmatched records between the databases were caused by subtle differences in nomenclature. The resulting histogram in Figure A-1 showed that the calculated RTNDT from the REAP surveillance data matched closely with values from BASELINE. The y-values in Figure A-1 represent RTNDT(BASELINE) - RTNDT(REAP).

Figure A-2 shows an example of a significant difference between the T41J in BASELINE versus the T41J calculated from surveillance data in REAP. In the example of Figure A-2, there is no pronounced lower shelf energy. As a result, the algorithm used to generate the characteristic S-curve for the Charpy impact energy data was unable to converge on a mathematical solution and determine T41J, although a T41J can be estimated visually. In such cases, significant differences resulted between the T41J calculated from REAP data and the T41J for the same surveillance data in BASELINE.

68

Figure A-1 Comparison of T41J between BASELINE and the REAP surveillance database 69

Figure A-2 Example of a lack of mathematical convergence using Equation 1 with the surveillance capsule data for a pressurized-water reactor plant 70