ML20217F469

SER on Nuclear Criticality Safety Scale Code Validation Rept for Avlis Facility

ML20217F469
Person / Time
Site:	07003089
Issue date:	10/14/1999
From:	NRC OFFICE OF NUCLEAR MATERIAL SAFETY & SAFEGUARDS (NMSS)
To:
Shared Package
ML20217F462	List: ML20217F458 ML20217F469
References
NUDOCS 9910200329
Download: ML20217F469 (55)

Text

-n. ,

]

,- l

)

~

l l

l l

l SAFETY EVALUATION REPORT l

ON THE NUCLEAR CRITICALITY SAFETY SCALE CODE VALIDATION REPORT FOR THE ATOMIC VAPOR LASER ISOTOPE SEPARATION (AVLIS) FACILITY

) 5 Attachment 1

9910200329 991014 PDR ADOCK 07003089 POR m.

r~~-~ .

L i,v r

(

l 1

TABLE OF CONTENTS List of Acronym s . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ii List of Open items . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . iii List of Figu res . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . iv List of Tables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . iv 1.0 . BAC KG RO U N D . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 2.0 DI S C U S S I O N . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 2.1 NRC Expectations and Available Guidance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 2.2 Determination of Jalculational Bias . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5 2.3 Margin of Subenticality . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9 2.4 Area of Applicability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11 2.4.1 Generic Area of Applicability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .12 2.4.2 Specific Applicability to AVLIS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15 2.5 Documentation and Programmatic issues . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23 3.0 C O N C LU S I O N S . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24 REFERENCES ....... ..............................................28 i

l Appendix A: Sensitivity / Uncertainty Analysis of Benchmark Applicability . . . . . . . . . . . . . . . A-1 ]

Appendix B: Independent Trending Analysis of LEU Benchmarks . . . . . . . . . . . . . . . . . . . . B-1 '

l 1

l l

i I

j

y ..

1 f ,4 ic

ii List of Acronyms

- AOA area of applicability

. AVLIS atomic vapor laser isotope separation ECALCF energy corresponding to average lethargy causing fission GLLSM generalized linear least squares method HSE' hypothetical suberiticality evaluations H/X - hydrogen-to-fissile ratio LEU low enriched uranium LLNL- Lawrence Livermore National Laboratory l

LTL ' lower tolerance limit

! NRC Nuclear Regulatory Commission

-ORNL Oak Ridge National Laboratory

' RAI request for additionalinformation l RSL recommended safety limit  ;

SCALE Standardized Computer Analyses for Licensing Evaluation 3 i

SER - safety evaluation report ,

USEC- United States Enrichment Corporation USL upper . safety limit / upper subcritical limits t

e a

• W i

iii List of Open items item Rescription Section  !

01-1 Lack of AVLIS-specific process information 2.1 01-2 Unjustified extrapolation over gaps in parameter data 2.2 l 01-3 Findings of no significant trend contradicted by data 2.2 l 01-4 Use of a single pooled bias over two distinct AOAs 2.2 01-5 Inadequate methodology for determination of bias trend 2.2 2

01-6 Failure to consider bias trends for low-correlation (R ) 2.2 01-7. Failure to determine bias for different subsets of data 2.3 01-8 Minimum margin of subcriticality of 0.02 unjustified 2.3 l

01-9 inadequate validation in intermediate energy range 2.4.2 01-10 Inadequate extension of AOA to intermediate enrichment range 2.4.2 01-11 ' Failure to address anticipated credible process conditions 2.4.2 l

01-12 Inadequate justification of data normality 2.5 01-13 Models and code options not adequately described 2.5 l-u I

p .

lv I \

l I

i

i iv i

List of Figures l

l (Figs.1 through 23 are a correlation between AVLIS benchmarks and the cases listed.) i l Fig.1 U(11 )O, cases at H/X=0 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . A-4 l Fig. 2 U(11 )O2cases at H/X=3 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . A-4 Fig. 3 U(11 )O2cases at H/X=5 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . A-5 Fig. 4 U(11)O, cases at H/X=10 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . A-5 Fig. 5 U(11 )O2cases at H/X=20 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . A-6 Fig. 6 U(11)Og cases at H/X=40 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . A-6 Fig. 7 U(11)O, cases at H/X=80 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . A-7 Fig. 8 U(11)O 2cases at H/X=200 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . A-7 Fig. 9 U(11 )O2cases at H/X=300 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . A-8 Fig.10 U(11)O, cases at H/X=400 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . A-8 l Fig.11 U(11)O, cases at H/X=500 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . A-9 i ' Fig.12 U(11)O, cases at H/X=600 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . A-E Fig.13 U(11 )O2 cases at H/X=800 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . A-10 l- Fig.14 U(11)O, cases at H/X=1000 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . A-10 Fig.15 U(10)-SiO2cases at H/X=219 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . A-11 Fig.16 U(10)-SiO, cases at H/X=1140 . . . . . . . . . . . . . . . . 4 . . . . . . . . . . . . . . . . . . . . . A- 1 1 Fig.17 U(10)-SiO2 cases at H/X= 1350 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . A-12 l Fig.18 U(10)-SiO, cases at H/X= 0 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . A-12 .

l Fig.19 U(10)-SiO2 cases at H/X=21.6 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . A-13 i Fig.20 U(10)-SiO, cases at H/X=34.2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . A-13 '

Fig.21 U(10)-SiO, cases at H/X=0 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . A-14 Fig.22 U(10)-SiO, cases at H/X=10.9 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . A-14 l

Fig.23 U(10)-SiO2cases at H/X= 17.2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . A-15 Fig.24 Trend of LEU critical benchmarks as a function of logw ECALCF . . . . . . . . . . . . . . A 15 i List of Tables i

(

Table i Comparison of Stated AOA with Range Covered by the AVLIS Benchmarks .17 l Table A-1 Table of Benchmarks Omitted from ORNL S/U Analysis . . . . . . . . . . . . . . . . A-1 l

Table A-2 Summary of Applicability of Benchmark Data to Selected Test Applications . A-3 Table B-1 Summary of LEU Critical Benchmark Experiments used in ORNL Analysis . . B-2 Table B-2 Output of the USLSTATS Program . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . B-3 L

I

'9 E

' SAFETY EVALUATION REPORT ON THE NUCLEAR CRITICALITY SAFETY SCALE CODE VALIDATION REPORT  !

I FOR THE ATOMIC VAPOR LASER ISOTOPF SEPARATION (AVLIS) FACILITY

1.0 BACKGROUND

)

The Atomic Vapor Laser Isotope Separation (AVLIS) process is a technology for enriching l 8

uranium in the '5U isotope,' which uses tunable dye lasers to selectively ionize the desired j fissile isotope of uranium in a beam of vaporized uranium atoms. The uranium metal feed is l vaporized in a crucible at the bottom of the separation pod. The ionized atoms are then separated from the unionized atoms by means of a magnetic field and collected in the form of

' uranium metal buttons on a plate in the pod. The pilot program for AVLIS had been under development by the United States Enrichment Corporation (USEC, or the applicant) at the Lawrence Livermore National Laboratory (LLNL) in Pleasanton, Califomia, prior to its suspension. Because this process would involve more than a safe mass of fissionable material, j the Nuclear Regulatory Commission (NRC) reviewed the criticality code validation report as part j of the preliminary licensing work prior to submittal of the actual license application. j

- Historically, the nuclear criticality safety community has performed criticality computer code i

- verifications' and validation checks2.s for evaluations of normal and abnormal operating )

conditions to demonstrate nuclear subcriticality for safety. These historic criticality computer code verifications and validations have generally provided a suite of critical experiment ~

benchmarks with alleged relevance to intended suberiticality evaluations for safety. The quality, {

and identification of deficiencies, of validations have evolved through the availability of computational diagnostics and the criticality safety community familiarity with and usage of the diagnostics. Such diagnostics include statistical analyses tools4. s. The definition of the area of applicability (AOA) or the relevance of the validation effort to subcritical evaluations for safety i has likewise improved through this evolutionary process. One of the more recent descriptions -

of a thorough validation process has been reported by Oak Ridge National Laboratory (ORNL)*.

NRC considered the substance of that report during the limited review of the AVLIS Criticality

~ Code Validation Report. The verification and validation of a criticality computer code must document the demonstration of either the specific methods or the specific subcritical evaluation example (s) whereby acceptance criteria for the AOAs, statistical margins of subcriticality, and administrative margins of suberiticality are determined and applied for safety. To date, subjective professional judgement, knowledge, and technical expertise and analyses have been required to assess the of benchmarks relative to subcriticality evaluations for safety.

The SCALE 7 (Standardized Computer Analyses for Licensing Evaluation) code package is a modular neutron transport modeling code, based on the Monte Carlo method, for performing radiation shielding and criticality safety calculations in support of NRC-licensed activities. The portion of the SCALE-4.3 package used for criticality safety is denoted KENO-V.a. This safety evaluation report (SER) will focus on use of the SCALE-4.3 code as applied to the AVLIS process for the determination of subcritical limits. The applicant submitted an initial code

validation report by letter L-98-005, "AVLIS Criticality Code Validation Report," dated April 22,1998. The NRC's review of this report resulted in two requests for additional information (RAls) on June 10,1998, and December 4,1998. USEC submitted responses to

r ,-

C

.i 2

the first RAI by letter L-98-011 dated October 22,1998, after oral presentation in a meeting on July 21,1998. USEC submitted responses to the second RAI by letter L-99-003 dated April 12,1999.

The first set of RAI questions focused on the following issues:

e Justification for extending the area of applicability beyond the benchmark data to the intermediate neutron energy spectrum, without the use of alternate codes or application of additionalmargin.

• . A more detailed description of the range of parameters encompassed by the individual benchmark cases to permit independent analysis.

e Justification for the 0.02 minimum margin of subcriticality._ ,

The second set of RAI questions focused on the following issues:

• A detailed description of the process conditions expected in the intermediate energy range and a discussion of the adequacy of the current experimental evidence.

• Application of appropriate bias and margin of safety for extension to enrichments in the range of 5 to 10wt% masU.

-e Review of multiple parameter interdependence and perturbation for AVLIS-specific

parameters.

• Adequacy of cross-sections for high temperature metal and other systems, or supporting rationale for why such elevated temperatures are not possible.

The responses to these RAI questions will be discussed throughout the following sections. l The purpose of this SER was initially to provide reasonable assurance that the SCALE criticality 1 code could be used with an adequate margin of safety to determine the criticality safety basis for the proposed AVLIS process. However, in light of the cancellation of the AVLIS project in mne 1999, this SER now documents the technical basis for NRC findings regarding the adequacy of the submitted validation report, including identifying outstanding technical issues (open items). Although AVLIS has been suspended and a completed application was never received, experience during the review of this validation report has generic applicability to future licensing actions, especially for proposed operations where the area of applicability is being

- extended beyond that normally encountered in traditional fuel cycle facilities. Because of the open items identified, the NRC has concluded that the AVLIS validation report does not provide

- an adequate generic or specific validation.

W' i.

L l 3 2.0 DISCUSSION 12.1. NRC EXPECTATIONS AND AVAILABLE GUIDANCE e

i There are several guidance documents available that present acceptance criteria for performing l an adequate criticality code validation. NRC has endorsed the American National Standards ,

L: Institute (ANSI) standard ANSI /ANS-8.1-1983,8 " Nuclear Criticality Safety in Operations with i'

'l Fissionable Materials Outside Reactors," which contains code validation criteria generally I acceptable to the NRC. (Note: Quotes herein are from the 1983 version, which was the applicable version at the time of the AVLIS submittal. The most recent NRC-endorsed version .

-is ANSI /ANS-8.1-1998.) The specific technical practices prescribed therein are:

e 4.3.1 Bias shall be established by correlating the results of criticality experiments with results obtained for these same systems by the methods being validated.... The bias I serves to normalize a method over its area (s) of applicability so that it will predict critical I conditions within the limits of the uncertainty in the bias. Generally neither the bias nor  !

l its uncertainty is constant; both should be expected to be functions of composition and i = other variables.

i

~

e 4.3.2 The area (s) of applicacility of a calculational method may be extended beyond the l ,

l range of, experimental conditions over which the bias is established by making use of i L trends in the bias. When the extension is large, the method should be supplemented by

[ . other calculational methods to provide a better estimate of the bias in the extended area (s).

e 4.3.3 A margin ... shall be prescribed that is sufficient to ensure subcriticality. This l

margin of subcriticality shall include allowances for the uncertainty in the bias and for uncertainties due to any extensions' of the area (s) of applicability. ,

! l l These relate to the' technical practices in performing the validation. The documentation of the  !

L code validation is prescribed as follows: i I

.eL 4.3.6 A written report of the validation shall be prepared. This report shall:

l (1) Describo the method with sufficient detail, clarity, and lack of ambiguity to allow L Independent duplication of results. 1 l' (2) State computer programs used, the options, ... the cross-section sets, and any i numerical parameters necessary to describe the input.

. (3) Identify experimental data and list parameters derived therefrom for use in the validation of the method.

(4) State the area (s) of applicability.

1 , (5) State the bias and prescribed margin of subcriticality over the area (s) of applicability.

j. ' State the basis for the margin.

L

,y, 7, ,

4 )

L . I l

+

4-i L

' These are the standards applied by NRC during this review to determine adequacy of the '

'j validation process and its documentation. The area of applicability is defined in l

< - ANSI /ANS-8.1 1983, paragraph 3.3 as "The ranges of material compositions and geometric L arrangements within which the bias of a calculational method is established." In addition, there f are several NUREGs covering computer code validation, including NUREG/CR-6361,8 L" Criticality Benchmark Guide for Light Water-Reactor Fuel in Transportation and Storage -

Packages," referenced below.

The essential elements in' performing an adequate criticality code validation are summarized

. here and in the following sections: (i) establishing the calculational bias by comparing calculated results with accepted critical benchmark experiments; (ii) determining an adequate margin of subcriticality to ensure that'results calculated as being suberitical are actually

!' subcritical; (iii) determining the AOA of the code (including methods for extending the area of applicability where there is insufficient benchmark data); and (iv) documenting the validation in sufficient detail to permit independent duplication of results, including the statistical -

1 methodology; the computer code, cross-section libraries, and hardware platform used; the experimental data used; and the basis for the bias and margin of subcriticality over the code's -

area of applicability.' The ultimate goal is to provide adequate assurance that the calculational l methodology used to' determine subcritical limits for the applicant's process is reliable. l

. - Therefore, in addition to the above criteria, it is necessary to ensure that the applicant's process conditions fall within the code's AOA. Only then may a finding be made concerning the appropriateness of using the code for determining subcritical limits in the specific process for

, which licensing approval is being sought. This additional information is in essence the distinction between generic and specific code validation and requires the submittal of detailed .

j process-specific information.

. ' Generic vs. Soecific Validation .

1 There is a distinction between the generic and specific validation of a computer code or other calculational method. The SCALE code developers at ORNL have extensively revisod the code and tested it for a wide variety of applications since the original development of KENO in 1969.7 The code is well-established in the criticality safety community and is recognized to generate ,

, reliable results under most normally encountered conditions. The purpose of such a generic l

- validation is to ensure that the code is functioning properly on a particular platform at a specific time (at the time of installation, or periodically throughout its use). There remains, however, the l- . issue of validation of the code for the specific uses to which an applicant or licensee wishes to

, apply the code.' Without process-specific information regarding the physical and chemical forms, moderation levels, geometry, absorbing and reflecting materials, and other parameters, f it is not possible to draw any conclusions about the suitability of the code for a specific  :

application.

NRC could not tell from the AVLIS validation report whether it was intended as a generic or specific code validation, and thus the desired regulatory outcome was in question. NRC's

-_ assessment of the submitted validation report would depend strongly on the validation report's intended purpose (generic or specific code validation). Several statements in the validation

. report made the purpose of the validation report unclear. Specifically, Sections 2.0,4.0,5.0, 7.0, and 13.0 through 15.0 describe an attempt at an AVLIS-specific validation study, but the  ;

report did not present any detailed description of conditions likely to be encountered in AVLIS )

j

y u w .

5 processes and did not present a compelling argument why these processes would be within the code's area of applicability. The following excerpts are provided as examples, without describing in detail all of the statements contributing to this ambiguity:

This report documents the validation of the SCALE 4.3 code system with the Hansen-Roach,27-Group ENDF/B-IV, and 238-Group ENDF/B-V cross-section libraries for the AVLIS enrichment facility. The selection of appropriate experiments is sufficient to validate the code system for the entire range of normal and off-normal operating

- conditions expected at the AVLIS plant. [taken from the Abst'act]

...the 238-group library is the recommended library for criticality safety and analysis of

- AVLIS system. [taken from Section 1.0)

The range of applicability... identifies the range of important parameters and/or characteristics for which the code was validated. This range of applicability is sufficient ,

to encompass the expected normal and off-normal AVLIS plant operating conditions.

[taken from Section 13.0) ,

The SCALE 4.3 Computer Code System is validated for Atomic Vapor Laser Isotope a, [ Separation) (AVLIS) applications. The selection of appropriate experiments is sufficient to validate the code system for the entire range of normal and off-normal operating

. conditions expected at the AVLIS plant. (taken from Section 14.0]

These and similar claims were made throughout the validation report, but in general were not ]

substantiated. Section 5.0 of the report,"AVLIS Criticality Safety Computational i Requirements," describes in broad qualitative terms the range of geometries, enrichments, i chemical forms, other materials, moderation levels, and the energy spectra to be encountered  !

in the AVLIS processes. Section 13.0,." Area of Applicability," describes the ranges of several d of these parameters that are claimed to be adequately validated, it would therefore appear that the intended purpose of the report is to provide an AVLIS-specific validation. However, this

. type of report is entirely unsuited to Just such a specific validation, in the absence of any hard ,

. data regarding AVLIS process conditions. NRC finds the above assertions quoted from the report to be unsupported (Open item 1).

~

The appropriate question is not whether the code is valid or not, but howvalid (more precisely,  !'

how reliable) is the code for a specific application or AOA7 Statistical equations may be used to establish an upper subcritical limit over a certain range of parameters, but the validation may )

not be useful if the chosen cases are dissimilar to those for which the code is to be used.  !

Because detailed AVLIS-specific process information was not submitted to the NRC, it is l unknown whether or not this would have been the case. Particular attention must, therefore, b

• paid to the specific validation of the code with respect to a certain AOA.

2.2 DETERMINATION OF CALCULATIONAL BIAS l

The calculational bias is defined as the difference between the calculated k,, and experimental k,, of a critical benchmark experiment (nominally k,, = 1). Because actual fiss:le configurations are in general different from experimental benchmark configurations, the bias must be

~

estimated by comparing calculated and experimental results for a statistically significant number i

j

6 L of. experiments similar to the cases being modeled. Thus, the bias is determined for a range of

. experiments that should be similar in material composition and other physical and neutron

- characteristics to the cases for which the code is being validated. The bias is defined as: ,

= k, - 1. (Eq.1) 9 For conservatism, the bias is traditionally taken as zero if positive. Generally, the bias is a function of those physical and neutronic parameters of the system to which 4 is sensitive and is not constant over the range of parameters; the functional form must be determined by i trending the bias as a function of these parameters. In addition, there is uncertainty in the bias i due to the statistical nature of the Monte Carlo method and the experimental uncertainties.

Linear Reoression -

Section 11.0 of the validation report bases its treatment of statistical trends in the bias in

. performing a linear regression fit, with the three independent parameters being enrichment, .

. neutron energy,' and moderation. These parameters are not necessarily the only parameters L that can affect the bias, and there is no demonstration presented as to why trending the bias

' with respect to these parameters should account for all significant variation in the bias. The moderation level has a strong and direct influence on the neutron energy spectrum, so even considering moderation and neutron energy to be independent, as the validation report does, is a questionable assumption. Moreover, a cursory review of Figures 1,2, and 3 demonstrates that the data set is only poorly correlated to a linear fit, that is, the data do not typically lie on a straight line. This implies that much of the variation-perhaps most-is due to factors other than the three (actually two) independent parameters. Since there are no error bars on the data, it is not possible to determine to what extent the variation is statistical and to what extent it is systematic. There is so much scatter in the data (Figurea 1,2, and 3) that the NRC cannot positively evaluate whether there is a trend in the bias, i

One of the weaknesses in the AVLIS methodology for determining the bias is that a linear form of the bias is assumed without adequate justification. As stated in the previous paragraph, a linear fit to the bias is a poor representation of the data. The bias, as stated above, is in general a multidimensional function of the composition and configuration of the system. One of the main sources of the bias is the experimental uncertainty in the cross-section data, particularly in

. the poorly-resolved resonance region of many isotopes. This is why the intermediate energy I range is of particular concern to NRC. These intermediate cross-sections are measured with a  !

finite accuracy, and there are portions of the neutron spectra for many different isotopes that  !

are not even measured experimentally; cross-section data must be extended over these ranges using theoretical models. (This theoretical treatment of the data set is inherent in the cross- ,

I section data set that is provided by the code developers and is not part of the validation

~

process.) The magnitude of absorption and scattering cross-sections in the resonance energy range can vary by several orders of magnitude over an interval of a few eV, so that uncertainties can have.a large effect on 4. In addition, the use of neutron energy groups (the

- AVLIS validation used 16 ,27 , and 238-group libraries) introduces inherent gralniness in the j

. data and is a source of inaccuracy. Flux-weighted cross-sections are produced using a j particular flux distribution for each cross-section library. The use of cross-section libraries for

.istems with different flux distributions (than those used to determine the flux-weighted cross-sections) thus can cause systematic biases in the results. These are all reasons why the code I

i i

t .:.

7

cannot be trusted to yield accurate results for all cases without a rigorous and thorough validation.

The use of any parametric fit for the bias is, therefore, a phenomenological approximation to very complex neutron behavior; there is no fundamental reason why a linear fit should describe

- the trends in the bias.- A linear least-squares approximation is only valid when the following set of condit;ons is met: (1) the variation in the value of 4 is statistical in nature, (2) the variation ofk,n about the mean is normally distributed, and (3) the means are reasonably well-represented as lying on a straight line-that is, a linear model is a good estimate of the behavior of the curve as a function of the trending parameter.' The third condition may be expressed mathematically by the equation:

k,n(x) = ko( xo ) + a( x-xo) + O((x-x o)2), (Eq.2) where O((x-xo)2) is the sum of all higher order terms in x. This equation may only be used reasonably to represent the functional form of the bias when we expect, either on the basis of first principles or that the set of data has been shown to be linear over an extended range, that the third term in the ' equation above is_small. An example would be interpolating between data points where the gap over which the interpolation is performed is small compared with the total range over which the behavior is essentially linear. The AVLIS methodology implicitly assumed the linearity of the bias, without demonstrating the validity of the assumption over the entire range of applicability. There were extensive gaps in enrichment, moderation (H/X), and neutron energy (or energy corresponding to average lethargy causing fission, ECALCF) (Open item 2).

Despite these gaps in the data, a linear bias model was used even though there was no a priori

. knowledge of the functional form of the bias over the extensive intermediate range.

Multiole Parameter interdsp3ndence -

Making the assumption that the behavior of the data is linear removes the possibility of '

' detecting higher-order effects in the bias, or. correlations involving multiple parameters. The bias in the regions where there are significant holes in the data in terms of enrichment,9/X, l and neutron energy must be extrapolated from the existing data, which will involve a rr.cre {

careful analysis of the contributions to the bias as a function of the underlying parameters. In c general this will also mean the application of an additional margin. This margin is a function of

. the statistical treatment employed and will tend to increase as the bias is extrapolated further q from the bounds of the experimental data. Question 3 of the second RAI (dated December 4, j 1998) concemed the issue of "... multiple parameter interdependence and perturbation for those parameters specific to the AVLIS pro 6ess." Only Response 3(c) is relevant to a discussion of the possible multiparameter interdependence. The applicant states in its response that "There are insufficient benchmark data to support evaluating the effect of multiparameter inter-depenconce by evaluating 4 versus all combinations of values for all NCS parameters." This can be seen to be an accurate statement by examining the data in Figures )

14-49 as the second RAI response in detail. l Review of the results (discussed at length in SER Section 2.4.2) contradicts the assertions made in the second RAI response that "Each sub-group yielded 4 values reasonably distributed about the overall average 4 value" and "Each sub-group yielded k,, values that were well above the AVLIS failure limit" (Open item 3). In some cases, the data are l

m y, i

8 significantly below the bias (Figures 37,47, and thermal portions of other figures in the RAI response) or displays apparent trends (Figures 37-39 and 47-48), and should be carefully

. examined to determine whether a more conservative local bias is warranted. In other cases,

= there are insufficient statistics to determine whether the bias is sufficiently conseivative (Figures 26,27,30,31,33,36,45,46, and 49). Recall that 25 cases is the minimum needed-according

. to NUREG/CR-6361-to demonstrate that the data set is normally distributed, which is a key assumption in the statistical treatment. NRC review bas shown that a cluster of benchmark experiments (Figures 15,17,18,19, 20, 21, 23, 25, 28, l52, 35, 37, 43, 44, and 47) with a significantly lower bias than the global average; there should be an attempt to determine why the bias is lower in these cases and whether there is some commonality between these cases.

Rather than pooling these cases with the remainder of the data, there may be some substantially different neutron physics responsible for the trend, in which case a lower margin shouki be ascribed to cases similar to these. Particularly when a minimum subcritical margin of l 0.02 is desired, there should be a careful and rigorous study of trends in the bias and a l thorough attempt to understand the contributing factors to any observed trends, in order to support the adequacy of a margin which is lower than that normally considered acceptable. .

The NRC believes that the submitted analysis does not meet this standard. I The response also stated'that ". . .no extensions to the range of applicability beyond the limiting values of the critical experiments are made in the analysis of AVLIS processes." The NRC

- disagrees strongly with this statement, based on comparison of the data presented in validation report Section 10.0, "Results," and Section 11.0, " Analysis of Results," with Section 13.0, " Area

! of Applicability." This statement ignores the adverse effect of large holes in the data, which should result in additional uncertainty in these regions. Furthermore, staff finds that the low-enriched and high-enriched cases comprise two distinct areas of applicability and that an attempt to determine a global bias in the area between them is inappropriate (Open item 4).

Trends in the Bias The statistical method used to determine whether there is a trend in the bias is also flawed (Open item 5). The criterion utilized was that the difference in the maximum and minimum values of k,, was no more than three times the standard deviation of the data. Mathematically,  ;

there was assumed to be no trend if lkm - kmd s 30. A trend (non-zero slope) in the bias may be discemed when the variation is less than 30,- but despite this the use of this criterion would have no adverse effect as long as statements about the trend are confined to the range spanned by the data. That is, the data set extends over the entire range or a vast majority of

~

. the range and statements about the existence of a trend are confined to the range covered by  !

the data. Outside this range, no statement about the existence of a trend can be made with any degree of certainty. - Any validation method that is accurate-and not extremely conservative-will necessarily result in larger margins in areas where there is little or no

. experimental data. If the converse were true, there would be no need to benchmark the code

'at all.? A more rigorous criterion of whether there are trends in the bias would be to calculate a linear least squares fit (only where there are data over the entire range) and calculate the slope and its uncertainty. If the slope m am straddles zero, the claim that there is no slope can be  !

made with statistical backing.

A further technical issue is the dismissal of the possible need for linear regressinn bias trending having correlation coefficients less than 0.5 (R8) thereby requiring no statistical determination of V w ,

n 9

the impact of probable AOA extrapolations for bias and needed margins of subcriticality for.

subcriticality and safety.

The response to Question' 2 of the first RAI (RAI dated June 10,1998) merely reiterated that the validation results were examined for trends in the bias, and no trends were discovered.

There was still no rigorous statistical analysis of the trending provided to support this assertion, however.

Because~of the inadequate treatment of blaa trending, ORNL staff performed an independent statistical analysis of the 31 low-enriched uranium (LEU) benchmarks. The results demonstrate that trending may need to be taken into account even when there is a low linear correlation coefficient R2(Open item 6). The results of this analysis are presented in Appendix B.

2.3 MARGIN OF SUBCRITICALITY The margin of subcriticality is determined by a statistical analysis of the critical benchmarks

[ over the area of applicability. One of the main purposes of a validation report is to establish

' sufficient margin to provide assurance at a pre-determined confidence level that results calculated below a specified k,, (the safety limit) are actually subcritical. The criterion for L subcriticality is defined as:

k,, + 2Ak , s k - Ak, - Akm.,, or (Eq.3) k,, + 2Ak,, s (1 + )- A - Akm .,,

where k,,is the calculated result for a given case, Ak,,is the standard deviation o in ke, and AQ is an " administrative" margin to account for unknown uncertainties. Since the bias is a function of the physical and neutronic parameters of the system, so is the suberitical margin.

Statistical Methodoloav

- The NRC examined the applicant's statistical method for determining the subcritical margin, and L determined that it was technically sound, although concerns are discussed throughout this SER with the application of that method. The formulae for determining the lower tolerance limit (LTL) were the standard ones employed in most licensee validation reports. This LTL was found by

pooling all of the data together and then computing a constant limit. This simplification of the determination of the .blas can only be defended if it is proven there are no trends in the bias, and thus the validity of this technique is undermined by the inadequate treatment of trends in

[: the bias, as discussed in the previous section.

A casual observation of the data in Figure 10 of the validation report-covering Hansen-Roach l cross-sections as a function of enrichment-shows an apparent trend in the bias at around Swt%

8 6U. Because the data were pooled together without any attempt to explain the bias, it is unknown why the. bias in this region is significantly lower than the average bias, for certain I

cases. The nature of these cases is not described in sufficient detail to ascertain the cause of this apparent trend. This graph does not support the assertion that there are no significant trends in the bias, or that 99% of all future calculations below the LTL will be suberitical with a

~ 95% confidence (0.95/0.99 criterion.)

l

7 n f . ,

l: 10 f These same observations also demonstrate the danger of interpreting non-statistical behavior

- as statistical in nature. Without error bars, the NRC cannot determine to what extent the variation is statistical, but it is'at least oossible there is a systematic reason for the lower bias, if we assume, without any documented justification, that the variation in bias is due to statistical

- effects alone, then it would be true that any case within the area of applicability selected at random would meet the 0.95/0.99 criterion. If, however, a configuration is chosen that is similar

! in a significant way (the significance cannot be ascertained without a more detailed analysis) to one of the benchmarks showing a lower bias, then the LTL derived on pooled data would

' probably not be sufficient. Therefore, the NRC finds that the derivation of a constant LTL and the assertion that there are no significant trends in.the bias are unsupported in the report (see

._Open items 4 and 5).

Global (Pooled) Bias The danger in using a global bias is that it glosses over real, systematic effects. For instance, the upper subcritical limit (USL) is defined using a 0.95/0.99 confidence (95% certainty that 99% of future calculations below the USL are subcritical), and a 0.02 " administrative" margin.

Suppose this is correct and only 1 out of 100 cases will be calculated as below the USL when in

' fact it is critical. The USL is computed globally, that is, on the basis that all the benchmarks are

essentially the same, and that any variation about the mean is statistical in nature. However suppose there are.two distinct types of benchmarks, with the secord type having a substantially lower bias than the first type. The 0.95/0.99 criterion only holds if one does not differentiate

' between the two groups. If, however, an application similar to one of the second type of benchmarks is selected, then this criterion is no longer valid. The margin of safety becomes j

' reduced because a global bias was calculated without consideration that the factors I contributing to variations in the bias were systematic rather than statistical in nature (Open item 7).

Minimum Suberitical Marain The applicant stated, without adequate demonstration, that a minimum subcritical margin of O.02 was sufficient. Given the extensive region within the declared range of applicability over 4

- which little if any data exist (such as most of the intermediate enrichment range), and the lack of a rigo.ous methodology to determine trends in the bias, the NRC finds that this margin of subcriticality is insufficient.' The range of possible operating parameters of material form, enrichment, neutron energy, and moderation, is much broader than that encountered by other fuel facility operations. A significantly smaller minimum subcritical margin than that imposed on other fuel cycle facilities is only justified when trends in the bias and the neutronic behavior are well-understood and quantified, and the systems for which the code is to be used are unusually close to the benchmark experiments.

The minimum subcritical margin is essentially to compensate for those uncertainties and sources of error that are not included in the bias. The calculations are normalized with respect

to the various sources of uncertainty by adding a bias correction, and the uncertainties will be l exactly compensated for provided the cases run are identical to the benchmark experiments.

p ' There are however other sources of uncertainty not included in the bias. These include unquantified uncertainties in the bias due to differences between the particular case and the set of benchmark experiments,' approximations due to the simplified geometric modeling, and 4

I 1

v 7

A.  :

t.

L 11 L

l uncertainties connected with convergence and sampling. These are not part of the statistical uncertainty calculated by the code; additional margin must be allowed to ensure that they are

~ accounted for.? The applicant did not provide any justification of the proposed subcritical margin

(. and did not estimate the magnitude of the residual errors.(l.e., uncertainties which are not

-c pensated for by the' addition of the blas).

Quhstion 3 of the first RAI (dated June'10,1998) concerned the suitability of the minimum

. subcritical margin of 0.02. Rather than provide a cogent technicaljustification for this unusually

.. small margin-most fuel facility licensees with much more restricted ranges of operating .  ;

parameters use a more conservative margin, typically about 0.05-the response presented an L

. argument concoming the fact that with a margin of subcriticality of 0.02, the actual configuration i is still adequately subcritical (fraction of critical mass -0.90 at 4 = 0.98 and -0.75 at a 4 =

L 0.95.) The NRC acknowledges this interesting fact but considers this argument to be irrelevant to the issue at hand. The fraction of critical mass concerns the issue of how accurately one can control the mass and how sensitive 4 is to changes in the mass. Sensitivities of this type should be taken into consideration when setting operating limits; the minimum subcritical margin relates to our ability to calculate 4 accurately. Therefore, the behavior of 4 as a E

. function of mass is not germane. The uncertainty in calculating 4 translates into a corresponding uncertainty in calculating the critical mass, and thus the fraction of critical mass.

if we knew the critical mass precisely, then a subcritical mass limit just below the critical would

- be acceptable. However, this is not the case.

Given the scarcity in the data above 10wt% assU,' NRC finds that a margin of 0.02 is unsupported (Open item 8).

2.4 AREA OF APPLICABILITY The AOA of the code is the range covered by the physical and neutronic parameters of the l . benchmark experiments used to determine the bias. Since the magnitude and functional form of the bias must be determined by comparing calculated results to experiments, the uncertainty in the bias grows dramatically for cases outside the AOA defined by the span of the experimental data.

Where there are no experimental data available over a certain range, compensating measures

may be used to minimize the risk associated with uncertainties in the functional form of the bias. Where the extension in the AOA is significant, the calculation should then be supplemented by other calculational methods or the subcritical margin increased to allow for uncertainty in the behavior of the bias.. Increasing the suberitical margin can be achieved by

. performing a sensitivity study of the bias as a function of the parameters, and extrapolating any trends in the bias, taking into vcount uncertainties in the magnitude and functional form of the bias. When a supplemental criticality code is used as part of providing the additional assurance, this code should be sufficiently independent to ensure that the two codes do not

' have a common systematic trend in the bias. This is a very stringent condition and one not trivial to demonstrate.' These codes should utilize different cross-section libraries and different numerical algorithms to the extent possible and should be compared over the ranges where there are known benchmark cases to determine whether there are systematic trends in the bias common to both codes.1 The use of a second, independent code only verifies that the two

v.

7

?: '

1 l

l' 12 l

. codes compute the same results, but does not demonstrate they compute results correctly.

Thus this approach cannot completely compensate for the lack of experimental data.

I l 2.4.1 Generic Area of Acolicability i I

l The NRC conducted a~ careful examination of the data submitted in the AVLIS validation report, and categorized in the RAI responses dated October 22,1998, and April 12,1999, to determine f

{

l the ectual AOA covered by the experimental benchmark experiments. Neither the initial l validstion nor the subsequent submittals provide for extensions to the AOA by supplemental methods or a statistical study of the trends in the bias. The ANSl/ANS-8.1-1983 definition of j i AOA is "... the ranges of material compositions and geometric arrangements within which the j bias of a calculational method is established." Therefore, this SER considers the AOA to be i bounded by the range of parameters corresponding to and covered by the selected benchmark data. j I

NRC Determination of the AOA l I

Based on a review of the data, the NRC believes the actual supportable AOA is restricted to the

following range

• H/X limited to fully fast or thermal systems with H/X > 500.*

• Absorbers and other materials limited to the following: boron (excluding fast), graphite (excluding fast), water (excluding thermal), paraffin, oxygen, stainless steel, iron, and l aluminum (except fast intermediate).

l

• Material forms limited to metals (fast only) and solutions (thermal).

l 1 l l l i l l t

I *For the purposes of this SER, thermal fisslori fractions 20.7 are defined as " thermal" and cases 50.2 are defined as " fast."

t I

l

v 13 e Enrichment limited to low-enriched (<5wt%) at thermal energies, intermediate (5-10wt%) at all but fast intermediate energies, and high-enriched (>60wt%) at fast i energies.

e Geometry limited to simple geometries (spherical, cylindrical, cuboidal) and water-moderated arrays. Large interacting arrays in air are outside the AOA.

.. Infinite homogeneous cell treatment not well-represented in the intermediate energy range, or latticecell treatment in the fast energy range.

l e Reflectors limited to low Z reflectors (water, graphite, borated water, paraffin) at thermal and intermediate energies, metal reflectors at fast energies, and unreflected systems at thermal energies.

Only the bounds of the presented data were considered in evaluating the above list; ttw mMes no finding with regard to the adequacy of the calculated margin. The NRC also noted several apparent trends, including an apparent cluster with a lower-than-normal bias for certain

! absorbers at fast neutron energies, an apparent increasing negative bias with increasing mass,

.a trend with lattice pitch, and results for the 16-group Hansen-Roach cross-section library at low H/X. Therefore, because these apparent trends were not adequately treated (as discussed throughout this SER), the actual AOA supported by the validation would be even more limited

than is indicated by the above list.

Sensitivitv/Uncertaintv Methodoloav

_ Adequate coverage of the AOA is a partly subjective determination. Where the benchmark l data are continuously distributed with respect to a particular parameter, the domain of that parameter bounded by the data should be considered the AOA. Where data occur at discreto locations, such as a 2,3,5, and 7.5wt% 2"U, the coverage is more difficult to ascertain. The sensitivity / uncertainty (S/U) methods developed at ORNL can be used to determine on a more rigorous basis the AOA of the code and the applicability of a set of benchmarks to a particular case. This S/U methodology is described in draft NUREG/CR-5593, " Sensitivity and l Uncertainty Analysis Applied to Criticality Validation," Volume 1: " Methods Development."'

l Draft NUREG/CR-5593, Volume 2: " Illustrative Applications and initial Guidance," discusses l the potential application of S/U methods to systems at greater than Swt% enrichment in 2"U."

Draft NUREG/CR-5624," Evaluation of Critical Experiment Parameters and Uncertainties with

First-Order Sensitivity Techniques," discusses one- and two-dimensional modeling of newly l available experiments recently obtained from the Russian Federation.i2 The approach of sensitivity analysis is to use first-order perturbation theory to express changes in k,, in terms of ,

l changes in the underlying cross section data. Uncertainty analysis can be used to determine j the AOA of the code or to compare a particular case to the established AOA. 1 The particular tools which have been developed (discussed in the above references) are summarized below:

e Sensitivityparameters allow the analyst to compare different fissile systems with regard -

. to their response to changes in the cross-section data. Systems which produce a similar Ak,n when there is a given Ao in the cross-section data mav be considered -

l l'

a

7' 14 similar in some regard with respect to the development of the bias as a function of the trending parameters.

e Correlation coefficients indicate the degree of shared cross-section variance between sets of benchmark experiments as well as the total variance produced by uncertainties in the cross-section data.' Benchmarks which are highly correlated have similar cross-section uncertainties that contribute to their biases; thus the bias determined for one set of benchmarks may be applied to another with which it is highly correlated. This indicates they belong to the same area of applicabiiity.

e The Generalized Linear Least Squares Method (GLLSM) is used to decompose the bias in terms of its dominant cross-section uncertainties and to correct the cross-section data to eliminate trends in the bias. Once the behavior of the bias has been characterized, the GLLSM method can be used to precict the bias in areas outside the original AOA; thus, it is a quantitative and rigorous method of extending the code's AOA.

e D-Values are averaged parameters which indicate the similarity between systems based on the sum of differences in their sensitivity coefficients, with respect to fission, neutron capture, and scattering behavior. These are more compressed than the sensitiuty data and provide a single quantity that indicates similarity between systems.

Extensions to the AOA Cuestion 1 of the first RAI (dated June 10,1998) requested a description of how the range of applicability v>as extended to cover moist oxide powders and nitrate solutions in an intermediate

- energy spectrum. The NRC stated that where there was no available data, the ANSI standard

' called for additional margin or the use of an attemate code. It should be stressed that the use of an alternate code will not by itself resolve the concern over lack of available benchmark data 1

' becat.se inis second code would be similarly unvalidated. Using a second code can merely provice a more complete understanding of the behavior of the bias as a function of the parameter being extrapolated, by comparing the results of the two codes. Since many codes share the same general techniques and same or similar cross section data, the fact that two codes generate consistent results is good, but not sufficient to show that either code is functioning correctly.' The response, rather than addressing the concern, provided the NRC with a fuller understanding of the range covered by the benchmarks. Table B in the first RAI response showed that there were no cases involving low-enriched metals, that there were only three cases involving medium-enriched metals, that there was only one metal case with <

H/X > 100, and that there were only six cases involving high-enriched solutions (all from the I same set of experiments, which is less than desirable). Rather than adequately answering the question, this demonstrates that there are in fact significant holes in the data and that the code's AOA should be defined more narrowly.

l

7 l$.

15

[. ' 2.4.2 Soecific Aoolicability to AVLIS -

l l Demonstration that the proposed processes are covered by the submitted validation report can

!- . only be done by comparing the range of parameters covered by the set of AVLIS processes with the range of parameters corresponding to the AOA established in the validation. Since

- Section 2.4.1 demonstrates that the code has been validated for only certain ranges of the applicable physical and neutronic parameters, process-specific information detailing the range of parameters covered by the AVLIS processes is essential before a finding about the applicability of the code to those processes can be made.

For instance, a validation study could be a completely rigorous and thorough determination of l the bias over a well-defined AOA. However, it may not b9 correct to apply this validation and its

! derived bias to specific cases. Those cases may fall outside the defined AOA of the code. For L each case for which the code is used to establish subcritical limits, there must be a careful l determination of whether the case falls within the defined area of applicability. This may be done by thoroughly describing the AOA in the validation report as a set of criteria, in sufficient l detail to permit an analyst to easily compare the desired case to those criteria. This may also

' be done in the individual criticality safety analyses, although it is seldom done in practice. No conclusions may be drawn about the specific validation of the report to AVLIS processes because: (1) the AVLIS validation has not described the AOA in sufficient detail given the gaps l ' in the data and (2) the AVLIS process has not been described in sufficient detail to permit comparison of the AOA to the range of normal and upset operating parameters.

l l' ' Despite the above shortcomings, the validation report apparently attempted to provide a L specific validation for use of the SCALE 4.3 code for modeling AVLIS processes. The AVLIS l validation report and responses to RAls implicitly or explicitly contend that the validation is l applicable to all subcritical evaluations for safety that will or may be encountered in the AVLIS l process / production, storage, and transport. However, there are no specific descriptions of L ' typical or future demonstrated suberitical evaluations aside from a general assemblage of noted benchmark parameter ranges and general process descriptions. Within the validation . report,

.USEC provides a table of " Systems and Components with NCS Controls" that includes the name of each tabu;ated:

l

! * . " System" (e.g., " Product Handling," ' Uranium Recovery," etc.), identified

• - "Component" within the " System" (e.g., " Product Blending Fumace" within " Product l Handling,"" Enriched Feed Bar Stub Handling and Storage" within " Uranium Recovery,"

etc.) and the j

• . Coverage" within the " Component". (e.g., " Covered in product down-blend furnace i 2

analyses,""Similar to the analyses of the product and tails withdrawal canisters," etc.).

H - Thers are no definitive material / geometric descriptions of the process subcritical evaluations for safety for comparisons to the selected benchmarks. Therefore, there are no means to determine or to observe the adequacy of the validation process and of the establishment of subcriticallimits and administrative margins of suberiticality for safety. There are no demonstrations as to how the selected benchmarks' AOA are appropriate for subcriticality

. evaluations of the specifically identified " Systems," " Components," and ' Coverage."

As recognized by USEC in_ reference 4, the AOA of a validation is necessarily dependent upon the neutronic characteristics of the selected critical benchmarks and the subcritical evaluation being performed as influenced by the operational / environmental parameters of:

1

7 n.

0 le.

bl  ;

! 3 L.

v 16 -

l L ,  ?* ' ' isotopic. enrichment,-

? r . chemical composition, .

l' *: . material density, L

• . neutron moderation,'-

.>*1 neutron absorber,

.

• i interaction of uniform and dissimilar units / systems (heterogeneity)

L L* mass, and er - geometry..

Each of these parameters influence the . energy spectrum of the neutron flux carrying the fission -

! < chain to some degree thereby impacting the AOA, computational biases, and uncertainties of computed results related among the benchmark experiments and subcriticality evaluations for

.  : safety. The intermingling of these parameters in specific subcriticality evaluations, taken l' together in differing proportions than the benchmark experiments, can be expected to have "non-linear effects on computed results, their biases, and uncertainties of biases.

It is possible to compare the broad qualitative description in Section 5.0 of the validation report,

,"AVLIS Criticality Safety Computational Requirements," with the AOA described in Section 13.0,

" Area of Applicability." in making this comparison, the NRC noted several apparent .

discrepancies. 'These discrepancies include apparently contradictory statements about the range of enrichments and the list of other materials present. Also, geometry for the AVLIS processes was briefly described, but this parameter was dropped from the description of the AOA in Section 13.0. More significantly, the list of parameters used in analyzing the results

(Section 11.0," Analysis of Results") differs from the set of parameters discussed in Section 5.0.-

Most significantly, the range covered by the benchmark data, as presented in various figures.

and tables throughout the report, does not agree with the claimed AOA in Section 13.0. Based 4 on information in Sections 13.0 and 11.0, the NRC has constructed the following table

' contrasting the claimed AOA and actual range of the data:

. zTable i Comparison of Stated AOA with the Range Covered by AVLIS Benchmarks Parameter- . Claimed AOA - Range of Data Geometry: no restrictions not sufficiently described Enrichment 2 to 20wt% rasU 2 to 10wt% 85U-Uranium Forms metals, oxides, nitrates metals, compound, solution Other Materials ' B, C, C2 H,, H2 0, paraffin,0 2, not sufficiently described Stainless Steel, Fe, U n w Moderation H/X = 0 to 1500 H/X = 500 to 1200 Energy Spectrum thermal (<1 ev) to fast (-1 Mev) thermal (<10 ev) for LEU &

fast (>1 Mev) for HEU 2 f ,

I~ l u

u

..w 17 Intermediate Mass'Nuclide Compositions The following example illustrates how an unexpected anomaly over a sensitive parameter range can lead to seriously erroneous results:

An unreviewed safety question determination'8" resulting from the misunderstanding and

. potential misapplication of a criticality computer code validation involving aluminum recently highlighted the concern about piece wise interpretation of benchmark AOA from validations." '8 The following, extract from reference 14 generalizes the potential problem:

As stated in the referenced occurrence report, nuclear criticality computational evaluations that were used to support the authorization basis for operations, l were questioned regarding ". . . the validity of the measured values for the {

- cross-section of aluminum. . . ." As further stated, " Preliminary discussions I indicated that the resultant impact on the computed value of K-effective may be as high as 10 percent." The bias of the computed value of k. or k.n is highly j

problem dependent and may be negative or positive, depending upon the many circumstances of the problem.

If aluminum is the predominant scattering and absorbing media [ sic]in a fissile material system having an intermediate to-fast neutron spectra [ sic] energy of l average lethargy causing fission, the computed multiplication factor using ENDF j will be in error irrespective of the code or cross-section-processing that is used, l

. If there is a substantial amount of hydrogenous moderating material, such that I the flux has a substantial thermal energy contribution, then the ENDF aluminum cross-sections should be adequate. Numerous industry code and data i validations for nuclear criticality safety applications have shown the adequacy of

' the cross-section for thermal neutron systems, e.g. criticality experiments involving material test reactor fuels comprised of aluminum and high-enriched

' (93wt% 8 'U) or low-enriched (spproximately 20 and Swt% 2asU) uranium.

This concem for aluminum, is as mentioned in the referenced occurrence report, .

is likely true for any other materials which have few, if any, integral benchmarks with neutron energy spectra over the intermediate-to-fast energies of average lethargy causing fission.

l The substance'of the above issue has been historically recognized" 8 and discussed in various i forums. This issue is of likely concern for any of the intermediate mass elements (e.g., iron, manganese, chromium, etc.) that may dominate the neutron transport in the fissile material. In j

the above particular instance, it was assumed that the criticality computer code and cross- l section validation were adequate for application to poorly neutron-moderated systems when, in i fact, the validation was performed in a piece meal fashion for aluminum in well neutron- I

' moderated systems such as light-water-moderated material testing reactor environments.

With this caveat, specific areas of observed deficiencies are discussed below.

y i

V V L

18 Intermediate Enerav Ranae Question 1 'of the'second RAI (dated December 4,' 1998) asked for "... analysis and review of process conditions occurring in the intermediate energy range and the adequacy of current (experimental evidence to support these cases." The response was that the benchmarks chosen contained very little data in the (slow) intermediate energy range. USEC presented a

_ graph showing that the maximum fission fraction in each energy group for the actual AVLIS -

. cases closely matched the maximum fission fraction per energy group for the benchmarks.

This was presented as evidence that the benchmark cases closely matched the AVLIS cases with respect to neutron energy. The presentation of and discussion about Figure 1 promotes the notion of neutron-energy group-wise piece-meal validation for undefined subcritical evaluations by comparing maximum energy-group-wise fission fractions from dissimilar-benchmarks, "Criticals," and dissimilar suberiticality evaluations fcr safety, " Cases." Despite the fact the NRC has not been provided any AVLIS cases to compare against the benchmarks la the validation report, the results appear to be an important and significant demonstration of the

. fact that in the aggregate, the two sets of calculations (that is, the benchmarks and actual AVLIS cases) give rise to qualitatively similar neutron spectra. This does not, however, adequately substantiate the conclusion in the RAI response that "The excellent agreement in maximum fission fraction per energy group between the critical experiments and the AVLIS analyses shows that AVLIS processes are well covered by the critical experiments." It is not

- evident that the ' maximum fission fraction in energy group 11 for the " Critical" is induced by the same material / geometry influences causing the maximum fission fraction in energy group 11 for the undefined " Case."-

The NRC rejects the assertion that this information shows that there is very little contribution from the intermediate energy region (Open item 9). In fact, the response to the first RAI contains the statement that "... the range of applicability established by (the] validation data set

- includes (the) intermediate energy spectrum and damp oxide systems." What was presented was not a neutron spectrum, but rather the maximum fission fraction in each energy group, when compared over a large set of calculations. Approximately 8% of the' integrated area under this curve occurred in the slow intermediate energy range (considered to be groups 14 -

22 using the 27-group library), which the NRC considers significant. The unnormalized sum under the curve is -0.25, meaning that in the aggregate for all benchmark and actual cases, about 8% of fissions occurred in this range, but that for any particular case the proportion cc uld be as high as 25%.. Moreover, this graph does not contain any information regarding the statistics associated with the maximum fission fraction. This bame graph could have been

. generated from a set of 100 critical experiments, with 99 of these in the thermal range and only 1 in the fast range. This would obviously not havs provided sufficient coverage, so ths

= assertion that the agreement between the two data sets shows adequate coverage is nu necessarily true.

Intermediate Enrichment Ranoe -

' Question 2 of the second RAI (dated December 4,1998) was similar, requesting "... application of appropriate bias and margins of safety for the extension of enrichment values in the range of 5'- 10 weight percent." Draft NUREG/CR-5593,"" Sensitivity and Uncertainty Analyses Applied to Criticality Validation," Volume 2: "lliustrative Applications and initial Guidance," and draft J

.w 19 l

NUREG/CR-5624,'8 " Evaluation of Critical Experiment Parameters and Uncertainties with First- 1 Order Sensitivity Techniques," have explored the extension of code validation by evaluating l critical benchmarks at >5wt% enrichments. The response describes the statistical methodology used in deriving what is referred to as a Recommended Safety Limi! (RSL). NUREG/CR-6361 uses a different statistical methodology to derive what it calls an Upper Safety Limit (USL). The  ;

response goes on to state: "Since the Recommended Safety Limit is less than the USL, the j AVLIS method of validation is more conservative than the method specified in NUREG/CR-6361." This is the evidence presented that adequate margin has been applied.

This statement is not necessarily true for the reasons stated below.

. The statistical method is not just the application of mathematical formulae to an arbitrary set of

, benchmark experiments, but a complete self-consistent method. The complete NUREG/CR-6361 method should be compared to the method followed in the AVLIS validation report to determine which is the more conservative, not just the numerical results. The j

. methodology that was followed in NUREG/CR-6361 contains several stringent requirements  !

conceming the selection of benchmark experiments and determination of the area of I applicability, that provide much of the conservatism of the method. Selected excepts from Chapter 4 of the NUREG follow (italics added):

The value ke and thus the bias S are not necessarily constant over the range of a parameter of interest. -if trends exist which cause the ben;hmark values of k,, to vary with one or more parameters...then f can be determined from a best fit for the l calculated k,a values as a function of each of the paratreters upon which it is  ;

dependent. Trends must be taken'into account if extrapolation outside the range of -

validation is to be performed. [p.157] . ,

i The set of critical experiments used as benchmarks in the computation of p should be representative of the composition, configuration, and nuclear characteristics of the system for which ksis to be determined. [p.157] ') l The range of applicability may be extended beyond the range of conditions represented by the benchmark experiments by extrapolating the trends established for the bias.

When the extrapolation is large relative to the range of data...the calculational method .

I applied should be supplemented by other methods in order to better estimate the extrapolated bias. [p.158).-

Because both Ap and p can vary with a given parameter, the USL is typically expressed as a function of the parameter, within an appropriate range of applicability derived from the parameter bounds. [p.158) .

2 A minimum of 25 data points (i.e., calculated k,, values) are [ sic] required to verify normality, which is assumed in these methods...// there is an insufficient number of observations to ensure normality with this test, other statistical methods should be employed to determine normality or additional uncertainties to account for the paucity of data shouldbe considered. [p.158)

The ab6ve requirements are all vital components of the NUREG/CR-6361 statistical method for determining the bias. The validation report and responses to the RAI did not address how the I

l 4

g ,

4.

l- .

i-20

< considerations regarding trends in the blas, extensions to the range of applicability, or statistical adequacy of the data were met. The response also did not address the issue of how the bias in the extended enrichment range was derived (Open item 10). The method applied in the . .

validation report was applied over a broad range that contained significant gaps in the data. I The NRC therefore disagrees that the method in the validation report is more conservative than the NUREG/CR-6361 method, despite the favorable comparison between the calculated RSL and USL Hiah-Temoerature Cross-Sections Question 4 of the second RAI (dated December 4,1998) concemed the adequacy of high-temperature cross-sections for high-temperature systems anticipated as part of the. AVLIS -

process. The response indicated that under normal operating conditions the standard cross-section libraries could be used and that upset conditions resulting in highly elevated temperatures would result in the exclusion of liquid water, thus causing the modeled configuration to remain highly subcritical. The NRC examined this response and agreed that high-temperature cross-section data were not required to evaluate normal and credible abnormal. conditions that could occur in the AVLIS process. 1 l

Although this specific concern was resolved o' n the b' asis of designed process conditions, the lack of suitable benchmark data over broad ranges of important neutron parameters has been discussed at length above and remains a concern. This raises a global issue with regard to the adequate testing of the cross-section data accuracy or correctness, which is discussed in the i following section.

Accuraev of Cross-Section Libraries There is a global issue is related to data accuracy or correctness. Without experimental benchmark data to test processed neutron cross-section files _used for computer applications, the user of the cross-section data is limited to his/her understanding of the strengths and weaknesses of the theoretical models used for cross-section processing. A specific example

• of such an issue was observed in the use of the SCALE 123-group cross-section library as compared with the SCALE 44-group cross-section library. Greater than a 5.5% bias was -

~ observed between these cross-section libraries for highly enriched uranium systems with -

intermediate neutron energies causing fission (i.e.,100 ev). A concluding statement in SER Reference 19 is that:

A fixed resonance self-shielding value, corresponding apparently to an infinite dilute system, is provided by the original GAM-THERMOS data. This situation can cause unacceptable, non conservative discrepancies between k, values calculated with the 123-group library and libraries that provide for a more rigorous treatment of the resonance region. Based on this study, the i nonconservative bias is insignificant in low-enriched systems but can be as large as 5.5% for highly enriched systems with intermediate... values between thermal

- and fast.

t

. A caveat.must be added to the above conclusion: because of the paucity of experimental

' benchmarks in the intermediate neutron energy range, the accuracy of the 44-group cross-o n

+ ,  ;

1.; .

j l

/'

section library is also untested-the 44-group library merely has a rigorous resonance self-

! shielding model used for processing the point data to a group' structure for8 "U.

L Another specific example of the impact of data accuracy, with potential relevance to the AVLIS

, process, is provided in the Intemational Handbook of Evaluated Criticality Safety Benchmark  !

Experiments,88 specifically HEU-MET-FAST-035. As quoted from the handbook,1 l  : This assembly provides a useful benchmark for testing criticality calculations. As l Just noted, two core constituents,8"U and iron, dominated the neutronics behavior in the core region. This offers an extraordinary opportunity to test neutronics prcdictions involving these two materials. The8 "U/Fe ratio is auch i

that the bulk of the. neutron spectrum is in the 1-kev to' 3-Mev energy range and i

the peak is at about 200 kev.'

' Notable from this particular evaluation is that the KENO, MCNP, VlM, and MONK criticality l computer codes calculated this particular critical experiment, using various neutron cross-section libraries, with highly variable biases-0.988 to 1.0944. These extreme disparities are attributed to the " resonance-like" structure of the elastic scattering and the total cross-sections f for iron throughout the 100 kev to a few MeV energy regime. It is judged that inclusion of this, l; and similar, experiments into the suite of AVLIS benchmarks would potentially impact the uncertainty of validated areas of applicability in the AVLIS validation report.

. This particular global conceptual issue liksly has relevance to proportional combinations of materials with' limited benchmarks for validating neutron-energy spectra applicable to AVLIS

' processes and equipment. Sensitivity and uncertainty analysis methods can reveal the

'importance of the cross-section values and uncertainties to actual suberitical evaluations but only to the degree that the neutron cross-section data set is accurate, j I

Errors in tne cross-sections of specific nuclides would be' expected to have a significant impact I L

, . on the resulting bias! Thus, great care should be taken to ensure that all materials modeled in )

- the applications be adequately represented in the validation benchmark suite. Trending the

{

= bias with respect to the concentrations of specific nuclides-especially those expected to display ]

the most significant reactivity worth, such as "B-is important to understanding any deficiencies J in the underlying cross-section data.

)

l- LMiscellaneous Areas of Concern 9

Additional specific technical issues include the lack of relevant benchmarks related to defined '

. subcritical evaluations for: 1

- laFge, air' spaced, interacting systems of fissile material with interstitial neutron moderating and absorbing materials and' equipment,

• - l intervening neutron reflecting or moderating materials (e.g., concrete, steel, etc.) likely

~to be encountered in process facility environments (e.g, walls, floors, containment

. equipment, etc.),'

I L m i-

p l..

22 material forms and compositions (e.g., damp-to-dry uranium compounds and metal blended with AVLIS-specific materials) causing intermediate-to-high neutron-energy flux spectra.

Again, while it is not possible to comment directly on the applicability without specific suberitical l applications, these represent conditions that the NRC considers credible to occur based on its limited AVLIS-specific process knowledge (Open item 11).

Conclusion (Soecific Acolicability to AVLIS)

A criticality computer code is validated for specific subcriticality evaluations. In the purest sense, a criticality computer code validation can not be performed without an identified evaluation AOA unless it is validated for the calculation of the benchmarks only. In the case of the AVLIS Criticality Code Validation Report,it is not possible to. determine the adequacy of the validated area of applicability to the AVLIS suberitical evaluations without examination of the specific evaluations of interest. Validations for suberiticality evaluations for safety necessarily require the application of knowledge, expertise and professional judgement. These applications are necessary to define and interpret the areas of applicability related to the extrapolations and interpolations of the validation benchmark computational biases and uncertainties as they relate to each safety evaluation. The adequacy of the applied knowledge, expertise and professional judgement in the performance of a validation for subcritical evaluations cannot be assessed l without demonstration of the intended characteristic system evaluations.

Despite the lack of proces-specific applications, NRC requested that ORNL apply the sensitivity / uncertainty methdology (Section 2.4.1) to the applicant's benchmark suite. In lieu of specific applications against which to compare the benchmarks, certain intermediate enrichment and neutron energy ases, familiar to ORNL from previous S/U studies, were substituted. This should be sufficie't to provide a qualitative estimate of the applicability of the applicant's benchmark suite to cases n, the above-identified regimes of concern. The result of this analysis is presented in Appendix A.

In addition, the NRC has determined, based on the above considerations, that there is insufficient coverage of the AOA by the chosen benchmark experiments. Specifically, the cata set does not cover the entire range of H/X from 0 - 1500, does not cover the range of enrichment from 2 - 20wt%, and does not cover intermediate neutron energies. The neutron energy range covered is actually mainly thermal energies for low-enriched, water-moderated systems, and fast neutron energies for high-enriched uranium metal systems. Thus, the fast range at low enrichments and the thermal range for high enrichments is also under-l represented. The AOA is also too broad because it does not place any restrictions on the l geometrical forms and/or code options for which the code has been validated. Regarding the comparison between AVLIS conditions and the AOA, the scant information presented shows apparent inconsistencies that render this report unsuitable as a specific validation for AVLIS processes.

1 L

't L

l 23' t

\

2.5 DOCUMENTATION AND PROGRAMMATIC ISSUES - I

! According to Section 4.3.6 of ANSI /ANS-8.1-1983, the validation report should be documented

.in sufficient detail, clarity, and lack of ambiguity to allow independent duplication of results. The

specific areas where the NRC has determined that this standard was not met are:

e1 The purpose (generic vs. specific) of the validation report was ambiguous. Statements in the abstract and in Sections 1.0, 5.0,13.0,' ar:d 14.0 that the code was validated for c

! use with AVLIS processes were unsupported.

e Section 12.0 claims that the data set is normally distributed about the mean. However, L - this reflects a fundamental misunderstanding of probability theory: with finite data, it is >

~

l not possible to determine that the population is normally distributed with absolute certainty. The' method determining normality and the degree of confidence were not described (Open item 12).

<e The statement that there were no trends in the bias was repeatedly asserted, but was L

not rigorously justified. Thus, the bases for using a pooled approach to determine the bias and treatments of variations in the bias were not documented. The results cannot l be evaluated to determine which set of experiments is responsible for the observed apparent trends.

e The basis for the statistical methodology used was not provided. The use of a linearized

. bias model, the choice of trending parameters, the pooled / global bias approach, and the l' ' demonstration that the lower tolerance limit (LTL) is adequate over the entire AOA were not documented.-

~

e The derivation of the AOA in terms of the actual extent of the benchmark data is not l . supported, o There is in'sufficient description of how the benchmarks were selected and treated. The  ;

basis for excluding certain benchmarks, that could have been selected, from inclusion.in )

the blas', is not described.- The effect that the inclusion of these other benchmarks has l

on the bias is not known and the. selection of benchmarks appears arbitrary. Moreover, there was no discussion of the following practices:

. e- The difference between using different sets of experiments, such as the ICSBEP data sets, and using (as was done) individual experiments as separate entities was not addressed. The individual experiments in a single data set are closely related and may be biased by the same experimental systematics, reducing the amount of usefulinformation that can be culled.

ce - Also regarding the ICSBEP data, there is insufficient discussion as to why the ICSBEP Input decks were not used and whether the results were compared with ICSBEP results as a quality assurance check.

l

+

i<

24

-a The statement is made that because two of the three computers were the same

' workstation model, the results on the two machines match exactly, It is not i stated whether the code was run on both machines or whether the outcome was merely assumed.

e. There is insufficient discusCon of what is learned from Figures 1 through 6. The steps L

i used to make the unqualified conclusion that the code is validated over the entire claimed AOA were not documented.

e Sections 5.0, "AVLIS Criticality Safety Computational Requirements," and 8.0, " Selection

' of Critical Benchmark Experiments," do not discuss the criteria for choosing a short list of "other materials" for which the code was determined to be validated. The statement that the validation was performed only for "other materials" which most affected the

. neutron energy spectra and reactivity, and not other paramsters, was not explained.

e The minimum subcritical margin of 0.02 is not adequately justified.

e Models and code options used were not adequately described to permit independent duplication of results (Open item 14).

Thus, the validation report did not in general contain enough information to allow the NRC to verify the conclusions made in the report, or to duplicate results, as required by ANSl/ANS-8.1-1983. The NRC did not have sufficient information available to draw a conclusion that there l was reasonable assurance the code had been adequately validated for the AVLIS process.

3.0 CONCLUSION

S

~

L Based on the review of the validation report and the associated RAI responses, NRC does not find reasonable assurance that the SCALE-4.3 code is validated for use with the AVLIS processes over the entire claimed AOA. In particular, the report did not substantiate that (1) the

- benchmarks were sufficiently similar to expected AVLIS process conditions so as to constitute a specific validation for AVLIS; (2) the data were sufficient to cover the entire claimed AOA; (3)

. the bias, margin of subcriticality, and minimum margin of 0.02 were sufficient over the entire AOA; and (4) there were no significant trends in the bias justifying separate calculations of the bias. The lack of benchmark experiments in the intermediate enrichment and neutron energy range and the lack of AVLIS-specific information were the main deficiencies.

i NRC review of the AVLIS validation report leads to several generic conclusions regarding the

(. conduct of a criticality code validation in the absence of benchmark experiments. Where there l are no benchmark data available, there must be other methods of ensuring there is an j' adequate margin of subcriticality. These methods can include the use of additional codes l l(although the independence of the codes and the underlying cross-section data may continue to I-  : be an issue) or the use of additional suberitical margin. This margin should be sufficient to account for the lack of knowledge of the bias and of the uncertainty in the bias. The development of S/U methodology provided a rigorous technical basis to understand the behavior of the bias in the extended region.

L

I 25 J

The entire philosophy associated with performing code validation appeared to be fundamentally misguided. The purpose of code validation is to determine whether the code is valid (and how valid) for applications closely represented by a given set of critical experiments and to

_ determine a prudent margin of subcriticality to use for those particular cases. Because of the extensive and successful past experience with the use of the SCALE-4.3 code for a wide variety of. systems, the starting assumption is often a preconceived conclusion that the code is valid for essentially any system. This prejudiced thought-process has the unwanted effect of preventing j rigorous application of the scientific method. Code validation should be a vigorous attempt to  ;

' disprove that the code can accurately predict k,,, and a search for possible anomalous behavior

- and areas where there is insufficient margin. Only when the code has been tested as thoroughly and rigorously as practical can the conclusion be drawn that the code can be used safely to determine an adequate margin of subcriticality for real world applications. The particular points on which the code validation did not provide reasonable and prudent assurance of adequate margin are:

-e :The AOA discussed in the validation report was too broad to be supported by the range 1 of the benchmark data. -There was no justification provided for drawing conclusions i

about the bias where there were significant gaps in the available benchmark data. The particular areas where validation was unsupported were the intermediate enrichment _  ;

(between 10wt% and 93wt% 8 5U) and intermediate (epithermal and resonance regime) neutron energies.

e' ' _ The AOA was extended to the above areas y Mout adequate justification for the use of )

linear extrapolation methods or justification f( - dy additional margin or supplemental j calculational techniques were not required. l 1

e. . There was no rigorous statistical treatment of systematic trends or local variations in the bias. The validation report repeatedly asserted that there were no significant trends but did not demonstrate this adequately.. Multiple parameter interdependence and possible trends in parameters other than enrichment, moderation, and neutron energy were not -

treated in this report.-

*- The validation report lacked specific process information concerning the detailed material and physical characteristics of systems encountered in AVLIS, despite the apparent attempt of the report to demonstrate AVLIS-specific validation. Too little L information was presented in the original USEC submittal, and subsequent responses to NRC requests for additional information, to determine that the benchmark areas of l applicability coincide with the intended or actual AVLIS safety evaluation applications.

Without the submittal of.this information, it was not possible to evaluate the applicability of the benchmarks in the validation report to the cases modeled in support of the AVLIS

. process.

^

• : The minimum.subcritical margin of 0.02 was not adequately substantisted. There was

~no detailed analysis of the contributions to the bias or other uncertainties.

e.  ! The data tabulating the calculated k,n results in Section 8.0, Table 2, and in the first RAI response, Tables A and B, did not cross reference the results with the parameters that were deemed to be important. Therefore, neither the graphs presenting this data set

p  :-

4; 9

26

-n nor the statistical analysis of the data (as a function of those parameters) could be duplicated or analyzed.

Of particular concem is the inference from the USEC responses that. piece-wise validation of systems having aluminum or steel present in critical benchmarks (where the fission chain

- reaction is carried on by thermal neutrons) is applicable to subcritical evaluations where the fission chain reaction is cas.ied on by fast- or intermediate-energy neutrons. This inference is reinforced by the presentation of Figure 1 in the validation report which offers the notion that

. because there is a strong correspondence between the " Maximum Fission Fraction for Each Energy Group, Cases and Criticals" the benchmarks have strong areas of applicability to subcritical evaluations of systems composed of mixed-and-matched materials and geometries, it is clear that USEC has exerted substantial effort in extracting useful spectral infonnation from benchmarks regarding their "NCS Parameter" and ." Parameter Sub-groups." It is apparent from Figure 6 of the validation report that numerous benchmarks were calculated that spanned a broad range of hydrogen-to 2"U atom ratios. However, it is apparent from this review that the commingling of rather innocuous elements (e.g., silicon and oxygen) with mildly water-moderated systems do not lie within the benchmark areas of applicability examined by ORNL (see Figures 18,21,22, and 23 in Appendix A). Benchmarks omitted from the ORNL computations are not likely to significantly alter the conclusions from the sensitivity and uncertainty analyses.

The proposed practice of pooling data for determining Recommended Subcritical Limits needs to be justified with more than numerous applications. It is apparent that the demonstrations provided in the validation report assure that substantial conservatism is imposed by the RSL

, relative to the USL as reported by USEC in the validation report. However,in the absence of specific justifications for pooling data, it is appropriate to require the caveat that the applicant test the impact of a trend before the linear-fit-correlation-coefficient-(R )-less-than-0.5 threshold is used nonconservatively for safety.

The above identified issues are not unusual nor new to the nuclear criticality safety community.

- These issues have been both addressed and avoided by the community. The elaborations of the global conceptual and technical issues are merely provided for demonstration purposes relative to the AVLIS Criticality Code Validation Report.

' The NRC believes that it is in principle possible to validate the criticality code in areas where

. there are no applicable benchmark experiments, including the greater than Swt% 2"U range.

This will generally necessitate the use of additional margin to ensure subcriticality. This margin should be very conservative in the absence of a fundamental understanding of the physical reasons for the behavior of the bias. The tools d3veloped by ORNL (as the S/U methodology) are designed to provide this rigorous basis for the behavior of the bias. The NRC further believes that the data in the validation report are adequate to validate the code generically for

- cases within the AOAs of the code. The covered ranges include: highly thermalized solutions and oxides in the range of 0 - 10wt%8 "U, high-enriched (>93wt% "U) metal systems with fast neutron spectra, and certain selected fast neutron configurations in the range of 36wt% and 93wt% 2"U. These constitute separate areas of applicability, and separate biases should be computed for each of these systems. The neutron physics is sufficiently different between

- these groups that any attempt to fit the bias between them is improper. The above statement should not be construed to imply adequate validation for any particular system; specific 7

a g

• s. o l

, l 27 l ' validation should still be conducted to ensure that the application cases are sufficiently similar i- ' to the benchmarks modeled in the validation with regard to chemical and physical form, geometry, absorbing and reflecting materials present, and code options used. The NRC believes that with the available data it is possible to perform and document an adequate generic code validation for use in the intermediate enrichment and neutron energy ranges, but that the current validation report is not adequate substantiation of the proposed margins.

Princloal Contributor

• Christopher S. Tripp, i

l

)

1 l

l I

!. l b

l

• Significant input was also prepared by Calvin Hooper (ORNL) and Harry Felsher (NRC).

y; i

28 REFERENCES-

?1. Department of Energy (U.S.) (DOE). Y-KC-96,"The Reliability of Criticality Safety Calculations using the New Codes CSC and KENO." DOE: Washington, D.C.1967.

. 2. - Department of Energy (U.S.) (DOE). " Validation Checks of the "ANISN" and " KENO" ,

Codes by Correlation with Experimental Data." Oak Ridge Y-12 Plant, Y-1858. j l DOE: . Washington, D.C. November 20,1972.

i

3. Department of Energy (U.S.) (DOE). " Validation of the " KENO" Code for Nuclear i- Criticality Safety Calculations of Moderated, Low-Enriched Uranium Systems."

l Oak Ridge Y-12 Plant. Y-1948. DOE: Washington, D.C. June 13,1974. l l

l 4. SAS/ STAT Softwarec. Cary, North Carolina: SAS Institute, Inc.1999.

5. Cain, V.R. A Computer Code to Perform Analyses of Criticality Validation Results, Y/DD-574, Oak Ridge Y-12 Plant. Oak, Ridge, Tennesee. Martin Marietta Energy Systems, Inc., September 1995 (update available at http1/www. cad.ornl. gov / cad _nea/ text / download.html).

6. Nuclear Regulatory Commission (U.S.) (NRC). NUREG/CR-6361, " Criticality Benchmark Guide for Light-Water-Reactor Fuel in Transportation and Storage Packages." NRC: Washington, D.C. March 1997.

J

7. . NUREG-0200, Rev.4.," SCALE - A Modular Code System for Performing Standardized Computer Analysis for Licensing Evaluation." NRC: Washington, D.C.

March 1995.

8. American Nuclear Standards Institute /American Nuclear Society (ANS!/ANS).

l ANSI /ANS-8.1-1983, " Nuclear Criticality Safety in Operations with Fissionable Materials

'Outside of Reactors." ANS: 1983.

9. _ Nuclear Regulatory Commission _ (U.S.) (NRC). NUREG-1475, " Applying Statistics."

NRC: Washington, D.C. February 1994.

10. . NUREG/CR-5593, Draft " Sensitivity and Uncertainty Analysis Applied to Criticality Safety Validation," Volume 1

Methods Development." NRC: Washington, l D.C. October 1998.

11 Nuclear Regulatory Commission (U.S.) (NRC). NUREG/CR 5593, Draft, " Sensitivity and

Uncertainty Analyses Applied to Criticality Validation," Volume 2

lllustrative Applications and initial Guidance." NRC: Washington, D.C. November,1998.

L. 12. . NUREG/CR-5624, Draft," Evaluation of Critical Experiment Parameters and Uncertainties with First-Order Sensitivity Techniques." NRC: Washington, D.C.

December 1998.

l 1

l l

l 1

oi 'l

v-r_

H. .

29 l 13. Department of Energy (U.S.) (DOE). Unreviewed Safety Question Determination No. DSO-98-USOD-011, " Aluminum-Uranium X-Section Evaluation," Oak Ridge 1 l

l .

Y-12 Plant. DOE: Washington, D.C. (see http // tis.eh. doe. gov / criticality for USOD.)

l I J 14. .

. DOE Unusual OccurrencA Report No ORO-LMES-Y12 Nuclear-1998-0095 (12/17/98), Oak Ridge Y-12 Plant. (see http // tis.eh. doe. gov / criticality for information).

l j .15. . DOE EH-34 Internet Web Site Article, " Operation Guidance for Resolving Criticality Safety Issues involving the Aluminum Cross-Section Discrepancy."

16. . DOE EH-34 Internet Web Site Article," Validation of Computational Methods and Data for Nuclear Criticality Safety Applications. (see http 1/ tis.eh. doe. gov / criticality / temp /aLxsec.html.)(February 17,1999).

2

17. Lawrence Livermore National Laboratory. "k for Certain Metals Mixed with 3sU, Criticality Safety Quarterly." Pleasanton, California.1993,

18. Transactions of the American Nuclear Society. (C.V. Parks, W.C. Jordan, L.M. Petrie, R.O. Wright). "Use of Metal Mixtures to Explore Data Uncertainties." 73,217-218.

1995.

19. . NUREG/CR-6328," Adequacy of the 123-Group Cross Section Library for

Criticality Analyses of Water-Moderated Uranium Systems." NRC

Washington, D.C.

August 1995.

20. Nuclear Energy Agency. "The Uranium / Iron Benchmark Assembly: A 23sU(93%)/ Iron Cylinder Reflected by Stainless Steel," HEU-MET-FAST-035, International Handbook of Evaluated Criticality Safety Benchmark Experiments, OECD, NEA/NSC/ DOC (95)03, l NEA: September 30,1998.

l k-t

s, i

A-1 Appendix A Sensitivity / Uncertainty Analysis of Benchmark Applicability Developing analytic and diagnostic tools, b8 ' not available currently to the community, provide l further insights regarding the anticipated areas of applicability of the benchmarks selected for  !

the AVLIS validation. Because USEC did not provide specific examples of AVLIS suberitical l safety evaluations to demonstrate the very general validity of benchmark selections or code / cross-section bias determinations, ORNL selected 23 hypothetical evaluations to compare with 97 of the benchmarks USEC provided in its validation report. Some benchmarks were omitted because of incomplete diagnostic-tool software development. However, the following comparisons of the AVLIS benchmarks for the general validation of hypothetical safety evaluation calculations are demonstrative of potential areas of applicability. The omitted benchmarks are:

TABLE A-1 Table of Benchmarks Omitted from ORNL S/U Analysis bawl 484c08 bawl 844cl2 bawl 484cl8 leu-met fast- leu-comp- leu-comp. zpr6z 005 therm-019c3 therm-022cl bawl 484cl0 bawl 484cl3a bawl 484c19 ieu-met-fast- leu-comp- leu-comp-005b therm-020cl therm-024cl l 1

bawl 484cil bawl 484cl4 bawl 484c21 leu-comp- leu-comp- zpr6c j therm-019cl therm-020c6 l baw!484c11f bawl 484cl6 leu-met-fast- leu-comp- leu-comp- zpr6r 004 - therm-019c2 therm-021c6 .

I 1

~The selected hypothetical suberiticality evaluations (HSE) were taken from previous studies  !

performed by ORNL The HSEs are fourteen variably water-moderated unreflected spheres of l U(11)O2 -H 20' and nine variably water-moderated spheres of U(10)-SiO2 -H2O reflected with 1 4m thick SiO2-H2 O 5. These systems were selected to be generally innocuous from the standpoint of potentially complex resonance neutron capture / scattering overlaps, aside from hydrogen, masU and 238U capture. Additionally, these HSEs were selected to provide broad neutron-energy spectra. The substitution of unconstrained general and variable proportions of AVLIS identified fissionable materials (e.g.,23sU and 2 8U homogeneously and/or heterogeneously blended with boron, oxygen, stainless steel, iron, aluminum, water, graphite, polyethylene, paraffin) and reflector conditions (e.g., borated water, water, paraffin, po'/ ethylene, steel, nickel, natural uranium, and void) would compound and expand the invalidity of the AVLIS selected suite of benchmarks for systems that may be relevant to AVLIS l

processes. f l

Sensitivity and uncertainty analyses were performed for each HSE as compared with each of the 97 benchmarks using the SCALE 44-group cross-section library. Correlation coefficients were generated for each benchmark as compared with each HSE. Additionally, total calculated k,, variances were determined from computed sensitivities and cross-section data uncertainties l l

m -

t.

'.m A-2 for each benchmark and each HSE. These correlation coefficients relate the degree of shared cross-section variance between each benchmark and a selected HSE. Benchmarks which are highly correlated to an HSE have similar cross-section sensitivities that contribute to their calculated k,, biases and therefore identify common areas of applicability among the

' benchmarks and the HSE..

, Plots are provided in Figures 1 through 23 of each of the 97 benchmark k,,s, and their i ' associated standard deviation due to neutron cross-section uncertainties, versus the calculated correlation coefficient for each benchmark relative to a particular HSE. Figures 1 through 14 L provide correlation coefficients of the 97 benchmarks relative to each of the 14 HSEs for bare U(11)O,-H,0' spheres. Figures 15 through 23 provide correlation coefficients of the 97 l - benchmarks relative to each of_the 9 HSEs for reflected U(10)-SiO 2 -H2 O spheres. As

! discussed in reference 19, benchmarks having correlation coefficients on the order of 0.8 or i . larger are judged to be of a similar neutronic nature to the correlated HSE relative to calculated k , bias. Though these sensitivity and uncertainty analyses lack 25 of the benchmarks considered by USEC, the plots of information provide indications of the benchmark areas of l applicability, at least for the considered systems of U(11)O 2 -H 2 O and U(10)-SiO 2 -H 2 0.

L Each of the Figures 1 through 23 provide the l .e . basic material description on the first title line,

f. .*: linear relationship of k,, to the correlation coefficient, C(k), on the second title line,

-e statistical linear regression parameters on the third title line, and .

et intercept and slope of the linear regression.

Each of the plotted points is the AVLIS benchmark k,, calculated by ORNL with error bars

^

representing one standard ' deviation of the calculated k,, due to cross-section data uncertainties only, typically a value between 0.008 Ak,,'for relatively thermal systems and 0.023 ak,, for relatively fast systems. The SAMS* Monte Carlo calculated k,, standard deviations were

, relatively small by comparison (e.g.,0.001 - 0.002 ak .). The error bar values were determined I by the application of the SEN18 code using the energy-group-wise forward and adjoint sensitivities that are generated by the SAMS code and the 44-group cross-section uncertainty  ;

cross correlation files. The central graph line of a' plotted figure is the linear fit to each calculated kJ as weighted by the standard deviation due to cross-section data uncertainties g lonly. The two centrally located curvilinear plots provide the 99% confidence interval bounding j: the linear fit. . The two externally located curvilinear plots provide the 99% confidence interval n bounding the next calculated k , for a benchmark relative to its correlation coefficient with the l - HSE..

L l An interpretation.of these plots is that the extension of the linear fit to a correlation coefficient of L

1.0 demonstrates the similarity, or area of applicability, of the suite of benchmarks to the HSE as well as the. calculated bias and uncertainty for the HSE. Examination of Figures 1 through

~_14 demonstrates that there is little relevance among the 97 AVLIS benchmarks to simple

systems of UO, and H2 O at hydrogen-to SU atom ratios (H/X) between 0 and 10. However,

- there is' substantial relevance among the benchmarks and UO2 and H2 O systems having H/X

~ equal to or greater than 40.

Examination'of Figures 15 through 23 demonstrates that for spherical systems of II , i f f

['

A-3 L ..U(10)-SiO2-H 2 O reflected with 4m of SiO2-H2 0, having the same Si to H atom ratios, the 97 L AVLIS benchmarks also demonstrate variable areas of applicability. The observed benchmark

! areas of applicability for these systems are classified in the following table.

l TABLE A-2 Summary of Applicability of Benchmark Data to Selected Test Applications f

p- Acolicability Figure H/**5U ratio SI/23sU ratio U!U atoms / barn-cm -

Zero. 18 0 29 7.71 x 10 4 l

. Exceedingly poor to zero 21 0 11.8 1.54 x 104 Exceedingly poor 22' 10.9 7.51 1.54 x 10 3 Poor 23 17.2 5.01 1.54 x 10 4 Poor- 19 21.6 20.4 7.71 x 104 Moderately good 20 34.2 15.4 7.71 x 10 4

' Good , 17 1350 817 1.95 x 10 5 Good 16 1140 1350 1.46 x 10-5 Excellent 15 219 2080 1.22 x 104 The above classifications seem to reinforce the notion that the benchmarks are relevant to

- U(10) systems with H/23sU atom ratios greater than about 40. However, such a conclusion is

' likely not appropriate for systems having combinations of Intermediate mass elements that dominate the neutron transport and/or absorption in the fissile materials.

The above observations have limited application to other systems with multiple material compositions and proportions (e.g., organic or inorganic compounds or pure elements of iron, aluminum, stainless steel, copper, carbon, zirconium, boron, yttrium, organics, inorganics, lithium, nickel, cobalt, manganese, etc.) and compound geometries (e.g., layers of intervening

- neutron absorbers and moderators such as used in packaging, storage, or transportation, etc.)

that will alter the energy of neutrons carrying the fission chair. reaction.

This analysis demonstrates that for certain hypothetical applications with intermediate

, enrichments and neutron energies, the AVLIS benchmarks show varying degrees of applicability. Much of the AVLIS benchmark data has very low applicability to typical cases in the ranges of enrichment and neutron energy discussed in Section 2.4.2. This analysis also illustrates that cases which may appear qualitatively similar may not be applicable to each other when examined with modern statistical techniques, e

1 a

9 i

1 A-4 U(11)O2 0 H/X = 0 ketf = a + b .C(k)

W=0.031589961 DF Adi d=0.010985492 FitStdEfr=0.005703306 Fstat=3.0989417 a=1.002148 be 0.0078650066 1.04 1.04 1.03 . . 1.03 1.02 ,, i, . , t .02 1L -

y ,

1.01 o ,, 0i 1

% 5 ll

, _ h- -

t 4,

j 1

l

'q; -.nu.

. .n .

~

0.99 .. -

0.gg 0.98 q .

2 1 -

. m

,0.ge 0.97 0.97 0 0.~2 0.4 0:e 0:e C(k)

Figure 1 U(11)O2 @ H/X = 3 koff = a + b e C(k) e2=0.041943076 DF A4 e2=0.021568886 FN.0056727375 Fotst=4.1590349 o=1.0038981 b= 0.0002233103

. 1.04 1.04 1.03 _

1.03 1.02 -% n 1.02 7' --

-T" 1; -u .i "

~~

1.01 . h"- '

. [. f .{_ 1.01

]- N

.J

. T l,t'.'

g m._ .

f* ,c ri

.it i

's

[ EI T p pby.,;.

1 '1

< T N_.

t; __

, ~

-a L , -%

0.00 ,

_- p (o N~ 0.90 N--

g' % _.ip

~,; __, .

i

%._a..- -

i 0.08 ~~~~% 0.ps 0.97 0.g7 i 0 0.2 0.4 0.8 0.8 i C(k)

Figure 2 r

7 j.

P A-5 U(11)O2 O H/X = 5 ke# = a + b

• C(k) r*24036171626 DF Asq f2=0.015684641 FileklErr=0.0060007986 Fatst=3.5652081 a=1.0030061 ,

b= 0.0084979042 1.04 1.04 1.03 1.03 1.02 -~ _

. 1.02

~~'

, ;- m-7 - -

~ ~ ~ '

T 1.01 j 1.01

{l;l 5[I.j' T ]

] i 9 , .j t 1

]', '

L.

1 l i J, i t -

,4 i: l -

t.il-

.} ,J.,

i N~%

O.90 .-c -  !

r .i 0.00

"- - ~... . . ,_. ,

.. _'j-' 4 .r s  %;

0.08  %, 4 96

, 0.97 _ .

0.97 O 0.2 0.4 0.8 0.8 1 C(k)

Figure 3 U(11)O2 @ H/X = 10 ke# = a + b + C(k) f2=0.027710037 DF Asq r a2=0.0070322083 FitSedEn=0.0067146033 Fate >2.7083423 a=1.0030963 b=-0.0074027184 1.04 1.04 l

1.03 1.C3 1.02 -~. ,

1.02 u-~- - .

7.y '

1.01 . 1.01 1 _

- .a. .. <

c

]

1 ,

s 4.sp=-

j y .:r=_,

. . .y --_

1

,="1

~

~~

pC.:st'[l

] l [ l" .

09

)d,.-.- 0 ee i -- ..

5- ..

__W'-~. -

, ' O.97 0.97 l 0 0.2 0.4 0.8 0.8 1 C(k)

Figure 4 L ..

c. m.-

+.

A-6 U(11)O2 @ H/X = 20 )

i koff = a + b = C(k) a a r 2=0.01940lL93 DFhCr 2=0 Fits 91 Err =0.0067390831 Fetat=1.8790883 a=1.0030421 1 M .000108340 1.04 1.04 1.03 -

. 1.03 1.02 _.; ,_, , ,

1.02 m _ _. m.

m:

1.01 -~~N _ - - - - -g- - - - .--[. 4b

.;t 1.01 g

H - -

_. H 47, , ,

-o m

l J . '

.)"3 i

1 -

['.

v

_ u

-q i j, .

l. .

"1 I < l'%,_

d--

0.99 ,

0.99

~

~

~

)-.l. l __

0.00 - .

0.96 0.97 ' . . .

0.97 0 0.2 0.4 0.6 0.8 1 C(k)

Figure 5 U(11)O2 @ H/X = 40 koff = a + b

• C(k) a r*2a0.0098167494 DF Adl r 2=0 Fh8tdErr=0.00576706d7 Fate >0.94183897 a=1.0030F06 t 4 004052117 1.04 " 1.04 1.03 . 1.03 l

1.02 - - - _ . __ , _ _ .

< LO2

-7_ _ -- ; ~T 1.01 -'%4. ~

'j n :__ f -' _

1.01

~%, ,

N y, jJ l '

d' _ .

/ s -m- -- #

p

__Y :[,

I

- . - l

.((

O.99

'Z. 0.99 Jli

-.- _. _{_y

. __- ~

0.98 0.97 h 0.97 0 0.2 0.4 0.8 0.8 i C(k)

Figure 6

o.

. A-7 U(11)O2 @ H/X = 80 kaff = a + b + C(k)

. e2=0.0023171722 DF A4 e2-0 FN8tdErr=0.0057888831 Fatm>0.22084282

- a=1.0018779 b=-0.0017375378 1.04

1.04 1.03 . .1.03 1.02 - -

-1.02

- 1

~ , .

~

1.01 -._. '

~

-1.01 f-- '%. .. _ ,

ni'_$ 'J-

.h, N----__."l ,, j '

p ,

[._ . . _ ___

i

[ iJ t ,

p ,

?

0.90 -

3 ,

l ,

._y >

0.99 1.

a.l... _

^ ~

0.98 -

.0.98 0.97 0.97 0 0.2 0.4 0.8 0.8 i C(k)

Figure 7 U(11)O2 @ H/X = 200 kaff = a + b + C(k) r*2=3.7288284+45 DF A4 r*2=0 FR8tdErf=0.0067964737 FatsNO.0036404280 e=1.0001782 b=0.00010101515 1.04 VJ 1.03 l

l 1.02 -

1.02

~'

~

---?. , ,,. ,_.

< s'

^ '

y 4 y -~~ .

-- a._ ,

i -

y-

=

i A 1 ,a ;4L.-.

, . . l 'w#  : 2_ 1

[

~

_.. .. d i l[_ f 0.98 i E$_ 0.99 m 4. ..

0.08 -

0.98 0.37 . .

0.97 0 0.2 0.4 0.8 0.8 1 C(k)

Figure 8 l

l 1

7 u-5 I

l' l

A-8 U(11)O2 @ H/X = 300 keff a a + b + C(k) f*240004029001 DF Ad4 r*2=0 FtStdErr=0.0067944130 Fala>4.030290041 e 4 90082817 b=0. - 166

! 1.04 1.04 1.03 -

1.03 1.02 - ~ , 1.02

. 3 1.01

) 1 1.01 I

~~~-

--~ -

_,, _ . _ ., I h ,

T m

[]

t -

! i , l +p .r- -

1 --

, , g 1 i  : M i

.[y!

m. _._

~ ~'- u l

.; 6, 0.00

-1

i"$_ -

7 1'

1 O.90 1 ,e1.__..,_

j __

~ ~ -

0.98 0 to 0.97 . .

0.97 0 0.2 0.4 0.6 0.8 1 C(k)

Figure 9 i

U(11)O2 @ H/X = 400 keff = a + b + C(k) r*2=0.00006046300 DF Adq r*2=0 FilStdErr=0.0067930414 Feia>C.083840733 a=0.00000167 b=0.00078002422 1 1.04 1.04 1.03 - 1.03 1.02  ;-- 1.02

. , y_

d* -

? ,"

~

1.01 - . .T 1.01 l g- E

}

~_ fy, ,

.a 1

a i d '

fi lf ,7* C1

---u"- _ . _.__

.,  ? .

e M g, g.g ;

O. -

. a 0.

0.98 0, 0.97 . _

0.97 l- 0 0.2 0.4 0.8 0.6 1 C(k)

Figure 10 l l l

[

y _

s .

I r

J' A-9 i'

t.

U(11)O2 @ H/X = 500 l koff = a + b + C(k) e2=0.00077340003 DF Ad4 r"b0 FIISWErmo.0067933300 Fole>0.073634434 sp0. Seas 4ee4 tp0.00081510820 1.04 1.04 l -

i

1.03 -

1.03 W . .

- 1.02 1.01 1 1.01 3

x_ q. .( b r[,-- y

, ,p,7 1.. ., -

i "-m ',

' ~

_ _ _ , .' ] 4

] y_ ._

1

'1 ,,y d' d 7 ,

0.00 '

.' 4 .N_ 0.90 j.j ;..

. s.

~

! 0.98 0.98 0.97 .

0.97 0 0.2 0.4 0.8 0.8 1 C(k)

Figure 11 U(11)O2 @ H/X = 600 koff = 6 + b + C(k) r"do.00078329808 DF Ad4 f2=0 FIISWErr=0.0067933004 Fate >0.072560619 a=0.9ese660 170.00000325437 1.04 1.04 1.03 1.03 1.02 - 1.02

-- >,, ? ._

.{ .!

~

l 1.01 1.01 7.

]

-~ - -

~_. ,,_ --

c

] -

't

=

.d.

,i s 1 ._ , -. -..

j i t. - -

1

>jj^

i < ,

i 0.90

=~

T L i 7 0.90

4} ,,

sue

~_ . .

.c. ~

m 1

0.98 -

0.98 l

0.97 _

0.97 l 0 0.2 0.4 0.0 0.8 1 l C(k) l Figure 12 i

7

.p -

A-10 i

U(11)O2 @ H/X = 800 koff = a + b + C(k) e2=0.00064301401 DFM F2=0 F08tdErr=0 0067940078 Falm>0.061819300

.=0.See7eas3 b=0.00067100181 I04 l 1.04 143 -

1.03 1.02 1.02 j-141 lbc- . i.o,

--[ [ If ; ,]

]

1 f --j . ~ . _ _ . _

p h a t.t

. ,__ . ~~- .._..

,, -Jit' uj 0.90 0: i [, i 0.90

.-- m e:-;;._

~ ~

0.90 0.M 0.97 . 0.97 0 0.2 0.4 0.6 0.8 1 C(k)

Figure 13

'1 1

U(11)O2 @ H/X = 1000 keff = a + b = C(k) e2 0.00023100385 DFM e2=0 FitSidErr=0.005794012 Fals>0.02195800 e=0.posee022 b=0.00043722667 104 1.04  ;

143 -

1.03 1

1,02 .

1.02 Q ~

- ay  ;[

.a p imi 33, J

_]

~ -

1

. _ _ _ _ _ y o

,< g [

a u,

] ]

, -] ,

% j-

,v_ ---

j a

___ - is ,, ,

y y

0.n -

_4

a.,,

0.08 ee.

.l.

0.98 1

i 0.97 0.97 0 0.2 0.4 0.8 0.8 1 C(x)

Figure 14 4 1

! l c 1

b.

,+

l A-11 U(10) 602. H2O O Hr(236)U = 219, Sv4236)U = 2000,1.22 x 10*(4) a(236)U artwwm koff = a + b

• C(k) r*2=0.034200636 DF A4 f*2=0.01373944 FitSWEn=0.f 8** Fatat=3.3720647 a=1.006122 ba4.0067172010 1.04 1.04 1.03 1.03 T% I 1.01 1.01

~-.._.

h 1 -

(._.__ !4 7 1 a 't ,

~l 0.99 ,. O.99

~

~

l +._ ..

l 0.9e ' 1 0.9e 0.97 0.97 0 02 0.4 0.0 0.6 1 C(k)

Figure 15 U(10) SO2. H2O O Hr(236)U = 1140,8vN235)U = 1360,1.46 x 10a(4) a(235)U at/bewm koff = a + b = C(k) r*240065535002 DF A4 r*2=0 FitSWEn=0.0067794064 Fatat=0.63052950 a=1.0022646 l b=-0.0022734796 1.04 1.04 1.03 1.03 1.02 - -

1.02 1.01 '

1.01 1 m-~ _ _. L 1

+}. F 0.99

. [ 0.99 0.96 0.98 0.97 _ _ . .

0.97 0 0.2 04 0.0 0.8 1 C(k) j l

Figure 16 l L

g --

p.

I A-12 i U(10) . 802. H2O O Hr*(236)U = 1360, 8&a(235)U = 818,1.06 x 10a(-6) *(236)U at/tncm ken = a + b

• C(k) r*2=0.nnuum7 DF Ad4 r*2=0 FitSWErr=0.0067863283 Fatat=0.337039 e=1.0018613 tM.0010067106 1.04 1.04 1.03 1.03 1.ca 1.02 1.01 .

1.01 A (

g -m-- y l

7 7 l _- ,

i . ..

. I-0.00 Q.;L 0.90 7

.- i,__1 _

7" i 0.00 0.98 0.97 . 0.97 0 0.2 0.4 0.8 0.8 i C(k)

Figure 17 U(10)- 802. H2O O Hra(236)" = 0, Sva(235)U = 29,7.71 x 10a(-4) *(236)U st/tnctn ken = a + b

• C(k) r a2=0.0( 112507 DF Adj r*2=0.04624256 FitSWErr=0.0066007256 Fatat=6.7?S3156 a=1.0046476 t M .012606483 j 1.04 1.04

{

. 1.03 1.03 l

[

1.02 l

% 1.02 J

.f ..

,,.__.q '" 4

,, . 3.. e fj 7+- -, .

e 'N :lr '

- l ' g i S k w .& j .'

1 y g '

1 l,} ' a '%

0.es

~ a.~

=

~y __

1 m N ~s N 0.se 0.ee 1 0.ee

~~N.

0.97 . . . .

0.97 0 0.2 0.4 0.6 0.8 1 C(k)

Figure 18

f a~ ]

y, q 4

A-13 U(10) . 8402 - H2O O HF(236)U = 21.6. SVa(236)U = 20.4,7.71 x 1014) *(236)U aWbn cm koff = a + b e C(k) a r*2=0.026714841 DF M r 2=0.004e8637 Fu8tdErr=0.0067206802 Fatat=2.607387 a=1.0046708 b=4.0007annaan 1.04 1.04 1.03 -

1.03 1.02 '~ % _ .

1.02

~~ ~ ~ ~ -

j

.g N i, :..J i.01 .

f _,

i.0i 3 ---

T'-~ I I- : di .:T; 1

,c_a'

. 4 i

1 1 ,

1

,y .,,'.i " [-. .

N 0.se j < L l &

0.se

,-. -~. .

.c 0.98 0.08 0.07 _ . . .

0.97 0 0.2 0.4 0.8 0.8 1 C(k)

Figure 19 U(10) 8302 H2O O Hr(236)U = 34.2,8va(236)U = 15.4, 7.71 x 10a(4) *(236)U at/bn cm koff = a + b

• C(k) a r*2=0.018627198 DF M r 2=0 FleStdErr=0.0067474006 Fatat=1.6064887 a=1.0030406 b=-0.0061011482 1.04 1.04 1.03 1.03 1.02 -~ %. -.

% 1.02

'~- - - = _ _ _,_,

.P .- ..

N 1.01  %.

( 'I ia

$l1 '

1.01 w

2 j' ~w, _,

"w- '

J'ks l

i t

,d lL _. F u

JE g 't4; i,  :.,10 h,r. '

t'T 3 L.. - -

e

[ r, tt--* -

- ~.

[. i 2. [I'

- 0.00 .: " 0.90

- _=: _, ___ ,

0.98 1 0.98 0 97 . . _ .

0.07 0 0.2 0.4 0.8 0.8 1 C(k)

Figure 20

7.... g. -

b.

i.4 A-14 U(10) . 802 H2O O W(236)U = 0, SW236)U = 11.8.1.54 x 10^( 3) *(236)U allbn cm haff = a + b

• C(k) a r 2=0.0470837 DFM r*2=0.020000006 F1tsedErr=0.006067408 Fatst=4.0030006 a=1.0031784 1 4 .010307006 1.04 g.o4 1.03 1.03 i

i 1.02 1.02 v'- -

~

I i.01 -

9,og N

n e N ., .

_.. r- ,,i l 1

'[;cI'Z[--

It t .. g 1 7

. i d'  % N 0.00 I N-0.00 )

'N -- .

.. nL. '%  %~

1 i

-].~(" I 0.06 ~

0.08 i

~

0.07 . 0.07 \

0 0.2 04 0.8 0.0 1 l i

c(k) J

\ l Figure 21 i U(10)- 802 H2O 3 Hla(236)U = 10.0, Stra(236)U = 7.61,1.54 x 10a(4) *(236)U atttww:m keff = a + b

• C(k) a r*2=0.03382076 DF M r 2=0.013087060 FR8tdErr=0.0050072003 Fatat=3.3060047 e=1.0046324 t 4 0000082203  ;

1.04 1,04 1.03 1.03 i

1.02 '~ ~ 1.02

~

~]?~. T -.

,_..~-'

jr ,

1.01 .-'- 1.01

.j I 1 i .

]

}e:i .,,a := e

~

+, .,

m h +

F i -

i'-~_' t u..~

s.

l n , ,

[ , d I.  ; 'l' O.00 <

,- ' 0.00

=~-.- . .

=- [ .

l 0.08 O.06 i ,

1 0.07 . _ _ _

0.07 0 0.2 0.4 0.0 0.8 1 c(k)

Figure 22 r

I r

M L4 A-15 U(10) 8102 H2O O HfM236)U = 17.2. 8FTJ36)U = 6.01,1.54 x 1014) *(236)U atttnan koff = a + b + C(x) '

r*2=0.020644100 DF Aq r a2=0.0068374406 FilStdErr=0.0067181302 Felat=2.600 eses a=1.0044 ass b=40070002006 1.04 1.04 1.03 1.03 182 '-' ~ ~ ~ .

1.02 1.01 ,, 1.01 1 -

% a;j ]

~ -

1 L-~ 1

, ty i -~.. .

~

g., ; "N_

0 se

(l. 0.se J~

0.00 0.98 0.97 . . .

0.97 0 0.2 0.4 0.0 0.8 1 CN Figure 23 31 LEU Critical Benchmarks j koff = a + b + log (ECALCF) 1 r^2=0.33164802 DFMJ r*2=0.2830006 FitSedErr=0.017408604 Fatat=14.300311 .

a=1.0084421 ,

b= 0.0000448008 1.07 - ~ 1.07 1.C6 N %,'N 1.06 . I 1.03 NT -

1.03 g m  %.

N I 1.01 L rI 1.01

~

g L.__

% 1 h.00  % 0.90 h 0.97 -% 0.97 N

0.96 I 0.96 i l

0.03 N NI 0.93 N

0,91 0.91 i I

-2 0~ 2 4 6 log 10 (ECALCF) l Figure 24 l l

l l

1 1

y ,

0

e.

.A A-17 APPEND!X A REFERENCES

1. Nuclear Regulatory Commission (U.S.) (NRC). NUREG/CR-5719,"SEN1: A One-Dimensional Cross-Section Sensitivity and Uncertainty Module for Criticality Safety Analysis." NRC: Washington, D.C. July 1999.

2. . Draft NUREG/CR-5593," Sensitivity and Uncertainty Analysis Applied to Criticality Safety Validation," Volume 1: " Methods Development." NRC: Washington, D.C. (August 1999).

3. . Draft NUREG/CR-5593," Sensitivity and Uncertainty Analysis Applied to Criticality Safety Validation," Volume 2: "lliustrative Applications and initial Guidance."

NRC: Washington, D.C. August 1999.

4. B.T. Rearden, Development of SAMS: A Sensitivity Analysis Module for the SCALE

Code System Using KENO V.a in the CSAS25 Sequence, Ph.D. Dissertation, Texas A&M University. (1999).

5. Nuclear Regulatory Commission (U.S.) (NRC). NUREG/CR-6505, "The Potential for Criticality Following Disposal of Uranium at Low-Level Waste Facilities," Volume 2:

" Containerized Disposal." NRC: Washington, D.C. August 1999.

i

)

i 1

i

rg3.LO; y

p l l L B-l' l

I.

l Appendix B l- Independent Trending Analysis of Low-Enriched Uranium Benchmarks l l

L Repeated examples of the conservative values of the "AVLIS Recommended Safety Limit"

. (RSL) are provided in the validation report, however it is not clear that there is a technical justification for assuming that no bias trend exists for a linear-least-square fit having a ". . ,

l correlation coefficient (R8)" less than 0.5. Further, it is not clear that the RSL would necessarily i be conservative for some circumstances meeting the correlation coefficient (R 2) less than 0.5, l relative to upper subcritical limits (USL-2) as recommended by the model used in reference 10

j. -which is representative of the scatter of data about the linear regression.

The following information in Table B 1 was examined for trends and the USLSTATS program

(see Reference 5 of SER) was executed to determine the USL-2 for the information. Table B-2 1 l provides the output of the USLSTATS program.

Because the correlation coefficient (R 2) of a linear regression was determined to be less than

. 0.5 (i.e.,0.3), the AVLIS Recommended Safety Limit, RSL, derived from a 0.95/0.99 percent .  ;

confidence level of a Lower Tolerance Limit was estimated from the statistical information )

tabulated in Tables B-1 and B-2 as: l Average k, 0.99925 j

<. Pooled variance from USLSTATS 0.00036569 i

! Estimated experimental variances 0.00002500 l Total pooled variance 0.00039069 Total pooled standard deviation 0.01977 .

(Administrative) margin of sub-crit 0.02

]

! One sided Tolerance Limit Factor i l

for Normal Distributions l for sample size 31, resulting -

E in 95% Conf./99% Coverage 3 l RSL = (0.99925) - (3) - (0.01977) - 0.02

= 0.91994 i-Though the 1.5% difference between the lowest USLSTATS USL-2 value (0.9058) and the RSL L value (0.9199) is not profound, the difference demonstrates the need to more rigorously evaluate possible bias trends beyond a simple correlation coeffi:: lent (R 2) threshold of 0.5.

[

l i

l l

[

l' B-2

-TABLE B-1 Summary of LEU Critical Benchmark Experiments Used in ORNL Analysis 31 LEU Critical Benchmarks ECALCF(ev) logi,(ECALCF) k,,,  %

50,667.44 4.704729 0.93165 0.001823 40.20 1.604217 1.015965 0.002214 28.34 1.452389 0.995297 0.00154 60.46 1.781464 1.008211 0.002427 5,799.27 3.763373 1.00719 0.001878 0.14 -0.85673 1.029901 0.001524 1,075.50 3.031611 1.00298 0.002012 1,284.04 3.108578 4.994239 0.001586 0.03 -1.54512 1.v23451 0.00211 222.90 2.348108 1.037076 0.002203 47.30 1.674845 0.984172 0.001736 10,568.87 4.024029 1.007926 0.00167 318.15 2.50263 0.991866 0.001089 12,609.08 4.100683 0.989784 0.001707 7,892.25 3.897201 0.950528 0.001785 0.81 -0.09264 0.99279 0.001416 26.63 1.425335 0.995089 0.001976 0.02 -1.73508 1.016825 0.001712 0.33 -0.48747 1.004462 0.001048 35.08 1.545038 0.973808 0.002415 2,465.47 3.3919 0.986385 0.002182 1.78 0.25025 0.999711 0.002276 98.62 1.993948 1.022477 0.002093 0.04 -1.40512 1.02405 0.002197 10,761.20 4.031861 1.002848 0.001809 22,902.13 4.359876 0.998058 0.002472 l 0.67 -0.17323 0.993334 0.002156 73.18 1.864386 1.001178 0.001756

-1.50044 1.022605 0.03 0.001416 5,081.81 3.706019 0.967639 0.001981 2.90 0.462073 1.005298 0.00161 l

l l

l l l

'1

p.

m. y if b

B-3 L

TABLE B-2 Output of the USLSTATS Program -

usistats: a utility to calculate upper subcritical

. . . . . . . . . . . . . . . . . . . . . . . . . . . .l. i.m. i.t.s. .f.o.r. .c.r. i. t. i.c.a. l. i.t.y

.. s.a.

. ..f e....

t.y a.ppl i.c.a. t.. i.on.s. . . . . . .

Version 1.3.6, December 15, 1998

. . . . . . .O.a. k. .R. i.d.g e. .N.a. t.i.o.n.a.. l. .T.ab.o.r.a. t.o. r.y

! Input to. statistical treatment from file: data.'.n

Title:

31. LEU Cr.itical Benchmarks i

Proportion of the population = .990 Confidence of fit = .950 l Confidence on proportion = .950 i Number of observations = 31 Minimum value of closed band = 0.00 Maximum value of closed band = 0.00 Administrative margin =- 0.02 independent dependent deviation independent dependent deviation variable - x variable - y in y variable - x variable - y in y 4.70473E+00 9.31650E-01 1.82319E-03 1.42534E+00 9.95089E-01 1.97583E-03 1.60422E+00 1.01596E+00 2.21368E-03 -1.73508E+00 1.01682E+00 1.71211E-03 1.45239E+00 9.95297E 1.54031E-03 -4.87468E-01 1.00446E+00 1.04830E-03 1.78146E+00 '1.00821E+00 2.42702E-03 1.54504E+00 9.73808E-01 2.41536E-03 l 3.76337E+00 1.00719E+00 1.87849E-03 3.39190E+00 9.86385E-01 2.18197E-03

l. ' -8,56728E-01 1.02990E+00 1.52409E-03 2.50250E-01 9.99711E-01 2.27557E-03 3.03161E+00 1,00298E+00 2.01204E-03 1.99395E+00 1.02248E+00 2.09260E-03 3.10858E+00. 9.94239E-01 1.58581E-03 -1.40512E+00 1 32405E+00 2.19721E-03

-1.54512E+00 1.02345E+00 2.11010E-03 4.03186E+00 1.00285E+00 1.80882E-03 l 2.34811E+00 1.03708E+00 2.20347E-03 4.35988E+00 9.98058E-01 2.47210E-03 1.67495E+00 9.84172E-01 1.73642E-03 -1.73234E-01 9.93334E-01 2.15570E-03 4.02403E+00 1.00793E+00 1.67024E-03 1.86439E+00 1.00118E+00 1.75612E-03

! 2.50263E+00 9.91866E-01 1.08893E-03 -1.50044E+00 1.02261E*00 1.41625E-03 4.10068E+00 -9.89784E-01 1.70683E-03 3.70602E+00 9.67639E-01 1.98102E-03  !

! 3.89720E+00 9.50528E-01 1.78494E-03 4.62073E-01 1.00530E+00 1.60963E-03 l .-9.26376E-02 9.92790E-01 1.41640E-03 WARNING *** the test.for normal may be unreliable due to insufficient data, chi = 2.0645 (upper bound = 9.49). The data tests normal.

Output from statistical treatment

'31. LEU Critical Benchmarks

!. Number of data points (n) 31 Linear regression, k(X) .1.0097 + (-6.0593E-03)*X Confidence on fit (1-gamma) [ input] 95.0%

Confidence on proportion (alpha) [ input) 95.0%

Proportion of population f alling above lower tolerance interval (rho) [ input]' 99.0%

Minimum value of X -1.7351 Maximum value of X 4.7047 Average value of X 1.71706 Average value of k 0.99925 Minimum value of k 0.93165

. Variance of fit, s(k,X)^2 3.6209E-04 l- Within variance, s(w)*2 3.6082E-06 Pooled variance, s(p)^2 3.6569E-04 Pooled std. deviation, s(p) 1.9123E-02 C(alpha, rho)*s(p) 7.5409E-02 student-t 9 (n-2,1-gamma) 1.69900E+00

' Confidence band width, W 3.4591E-02 Minimum margin of suberiticality, C*s(p)-W 4.0817E-02

--Upper subcri tical . lim..i.ts.: ..(...1 73 <= 4.7047 )

. .. .51 .. ..<.=.X.

,, u

.4 B-4 TABLE B-2 (Cont.)

USL Method 1 (Confidence Band with Administrative Margin) USL1 = 0.9551 + (-6.0593E-03)*X (X > 1.5935 )

= 0.9454 (X <= 1.594)

USL Method 2 (Single-Sided Uniform Width closed Interval Approach) USL2 = 0.9342 + (-6.0593E-03)*X (X > 1.5935 )

= 0.9246 (X <= 1.594)

U.3.L.s. E.v.a.

. lu.a.t.e.d. O.v.e.r.

• • • . Range. o..f P.a.r.ame

..... ter. X. .:

X: -1.7 -0.82 0.10 1.0 1.9 2.9 3.8 4.7 USL-1: 0.9454 0.9454 0.9454 0.9454 0.9433 0.9377 0.9322 0.9266 USL-23 0.9246 0.9246 0.9246 0.9246 0.9225 0.9169 0.9113 0.9058 Thus spake USLSTATS Finis.

I l

f 1

I I

i i

i 1

1 I

f- October 14, 1999 R. W. Woolley, USEC 2 adequacy of the submitted validation report, including identifying outstanding technical issues (open items). Although AVLIS has been suspended and a completed application was never received, experience during the review of this validation report has generic applicability to future licensing actions, especially for proposed operations where the area of applicability is being i extended beyond that normally encountered in traditional fuel cycle facilities. Because of the )

open items identified, the NRC has concluded that the AVLIS validation report does not provide j an adequate generic or specific validation.

If you have any questions regarding this matter, please contact Dr. Christopher Tripp of my staff at (301) 415-7733.

1 Sincerely, g signed by M. Galloway Robert C. Pierson, Chief Special Projects Branch Division of Fuel Cycle Safety l and Safeguards, NMSS )

l Docket: 70-3089

Enclosure:

Safety Evaluation Report i i Distribution:

Docket: 70-3089 PUBLIC NMSS r/f FCSS r/f T. Sher (rPjRCfile Centerl.,

C. Emeigh P. Ting SPB r/f ,

i D. Persinko J. Davis D. Damon , i G:\SPB\ CST \avlis-sr-coveritr.wpd b &RF-6 4l D606- ~ [/78Mk OFC SPB 8

SPB (' t$ SPB b SPEL b )

l NAME CTripp:h DH M Ni rh DATE 10/ #9 /99 10{l7//99 10/ l f /99 10/ /t/ /99 C = COVER E = COVER &' ENCLOSURE N = NO COPY OFFICIAL RECORD COPY

! l 1

l l

l i

_ . .