ML19318F214

ORNL-SPR-2015-74 Superbadger Uncertainty Assessment During Zion SFP Measurements - Revision 1 - Review Copy

ML19318F214
Person / Time
Site:	Zion  File:ZionSolutions icon.png
Issue date:	12/31/2019
From:	Chapman J, Croft S, Eric Focht Oak Ridge, US Dept of Energy (DOE), UT-Battelle
To:	NRC/RES/DE
Eric Focht
Shared Package
ML19318F213	List: ML19318F214 ML19318F216
References
DE-AC05-00OR22725, NRC-HQ-25-14-D-0004, NRC-HQ-60-15-D-0001 ORNL-SPR-2015-74, Rev 1
Download: ML19318F214 (67)

Text

ORNL/SPR-2015/74 Rev. 1 Reactor and Nuclear Systems Division (94)

Addressing Uncertainties in SuperBADGER Measurements at the ZION Nuclear Power Plant (ZNPP) Spent Fuel Pool on December 5-8, 2014 Revision 1 Jeff A. Chapman Stephen Croft December 2019 (with foreword by Eric M. Focht, U.S. NRC)

Prepared by Oak Ridge National Laboratory Oak Ridge, TN 37831-6283 managed by UT-BATTELLE, LLC for the U.S. Department of Energy under contract DE-AC05-00OR22725 Prepared for U.S. Nuclear Regulatory Commission under Agreement Nos. NRC-HQ-60-15-D-0001 and NRC-HQ-25-14-D-0004/Task Order NRC-HQ-60-16-T-0010

DOCUMENT AVAILABILITY Reports produced after January 1, 1996, are generally available free via US Department of Energy (DOE) SciTech Connect.

Website http://www.osti.gov/scitech/

Reports produced before January 1, 1996, may be purchased by members of the public from the following source:

National Technical Information Service 5285 Port Royal Road Springfield, VA 22161 Telephone 703-605-6000 (1-800-553-6847)

TDD 703-487-4639 Fax 703-605-6900 E-mail info@ntis.gov Website http://www.ntis.gov/help/ordermethods.aspx Reports are available to DOE employees, DOE contractors, Energy Technology Data Exchange representatives, and International Nuclear Information System representatives from the following source:

Office of Scientific and Technical Information PO Box 62 Oak Ridge, TN 37831 Telephone 865-576-8401 Fax 865-576-5728 E-mail reports@osti.gov Website http://www.osti.gov/contact.html This report was prepared as an account of work sponsored by an agency of the United States Government. Neither the United States Government nor any agency thereof, nor any of their employees, makes any warranty, express or implied, or assumes any legal liability or responsibility for the accuracy, completeness, or usefulness of any information, apparatus, product, or process disclosed, or represents that its use would not infringe privately owned rights. Reference herein to any specific commercial product, process, or service by trade name, trademark, manufacturer, or otherwise, does not necessarily constitute or imply its endorsement, recommendation, or favoring by the United States Government or any agency thereof. The views and opinions of authors expressed herein do not necessarily state or reflect those of the United States Government or any agency thereof.

i FORWARD TO REVISION 1 In 2012, Oak Ridge National Laboratory (ORNL) performed an assessment of the measurement uncertainty of a neutron attenuation measurement system called BADGER. BADGER stands for Boron Areal Density Gauge for Evaluating Racks and is used to measure the 10B areal density of neutron absorbing materials in spent fuel pools. The NRC published ORNLs assessment in a 2012 technical letter report entitled, Initial Assessment of Uncertainties Associated with BADGER Methodology, which can be found in the Nuclear Regulatory Commissions (NRC)

Agencywide Documents Access and Management System (ADAMS) accession number ML12254A064 [1]. BADGER was designed and is operated by Northeast Technologies Company (NETCO), which is now a subsidiary of Curtiss Wright. At the time of its assessment in 2012, ORNL did not have access to the BADGER system design or performance documentation, which limited their uncertainty evaluation to a Type B uncertainty analysis (defined by the National Institute for Standards and Testing for performing uncertainty analyses where measurement data is not available). The Type B uncertainty analysis identified over 40 different factors that influence the measurement uncertainty, many of which could not be assessed due to the lack of information. However, several factors were identified that had the potential for combined uncertainty greater than 40 percent. The uncertainty factors and ORNLs initial uncertainty estimates were documented in Table 6-1 of the 2012 technical letter report [1].

Subsequent to the publication of the ORNL report on the Type B uncertainty analysis, NETCO made several improvements to the BADGER hardware to help improve its performance.

Through an addendum to a memorandum of understanding (MOU), the NRC and the Electric Power Research Institute (EPRI) conducted a cooperative research project to assess the newly improved system. The Zion Plant spent fuel pool was being decommissioned which provided a unique opportunity to perform in-pool (or in situ) measurements on the neutron absorbing panels and then remove the panels for confirmatory testing. The first phase of the project consisted of the NRC and ORNL witnessing the measurement campaign at the Zion spent fuel pool and making preliminary observations of the improved system. The second phase evaluated the data collected during the Zion campaign and additional design, calibration and performance data provided by NETCO. ORNL provided a technical letter report for the first phase entitled, Addressing Uncertainties in SuperBADGER Measurements at the ZION Nuclear Power Plant (ZNPP) Spent Fuel Pool, December 5-8, 2014, ORNL/SPR-2015/74 (ML15238A462) when, at the time, the improved system was called SuperBADGER. NETCO now calls the system BADGER ver.2.

In the second phase of the project, ORNL performed a Type A uncertainty analysis for certain factors using the data obtained during the Zion campaign and other data provided by NETCO.

Improved expert opinions regarding other factors were made, based on further information provided by NETCO and more in-depth discussions with NETCO personnel, but are still considered Type B. For completeness, the results of the second phase are reported in Sections 3 and 4 of this version (Revision 1) of the ORNL technical letter report.

In the original Type B uncertainty analysis, ORNL identified factors leading to uncertainty, such as instrument error due to the gamma field, mismatch between the neutron absorbing material in the spent fuel pool and materials used for calibrating the BADGER system prior to making

ii measurements, and misalignment of the source and detector heads. In this report, ORNL addressed these factors and others based on the data obtained during the Zion campaign, and in-depth discussions with NETCO, and reported their findings in an updated version of Table 6-1 in Appendix A. ORNL performed a top-down analysis for factors for which data was available.

For instance, ORNL assessed the uncertainties associated with calibration, model bias, and panel-averaged measurement error. One of the largest sources of measurement error identified in the original analysis was head misalignment, where errors up to 40 percent were postulated.

However, through discussions with NETCO regarding the improvements made in the hardware and observing the Zion measurements, ORNL modified its estimated error for head misalignment to be less than 10 percent.

In addition to ORNLs assessment of the BADGER ver. 2 system and analysis of the Zion campaign measurement data, ORNL and NETCO performed independent calculations of 10B areal density using identical representative input data (i.e. a code-to-code comparison) and their results agreed.

Although the total measurement uncertainty based on all the factors listed in Appendix A could not be determined, the analysis performed by ORNL determined that the errors postulated for several of the factors are lower than originally estimated and that NETCOs algorithms appear to calculate 10B areal density and propagate the uncertainties associated with measurement data correctly.

Forward by:

Eric M. Focht Office of Nuclear Regulatory Research U.S. Nuclear Regulatory Commission

iii CONTENTS FORWARD TO REVISION 1......................................................................................................... i

SUMMARY

.....................................................................................................................................v 1

INTRODUCTION....................................................................................................................1 2

TEST AND EVALUATION PLAN AND MEASUREMENTS AT ZNPP............................3 2.1 Day 1 at ZNPP, Friday, December 5, 2014.....................................................................3 2.1.1 The BADGER ver. 2 Test Plan (December 5-8, 2014).......................................3 2.1.2 The Overall Project Test and Evaluation Plan (Through November 2015)........3 2.1.3 ORNLs Proposed Contribution to Overall Project (Through November, 2015)....................................................................................................................4 2.2 Days 2-4 at ZNPP, Saturday-Monday, December 6-8, 2014..........................................4 3

ANALYSIS OF BADGER ver. 2 DATA FROM THE ZION CAMPAIGN..........................5 3.1 Introduction.....................................................................................................................5 3.1.1 Model Bias..........................................................................................................5 3.1.2 Areal Density Precision for Panels......................................................................6 3.1.3 Calibration Standards..........................................................................................6 3.1.4 BADGER ver. 2 AD Versus PSU Beam Lab Results.........................................7 3.2 10B AD Measurement Algorithm Testing: Addressing Item 5.6.12 in the Table 6.1 Addendum.................................................................................................................7 3.2.1 Introduction.........................................................................................................7 3.2.2 Outline of the Transmission Method...................................................................7 4

SUMMARY

AND CONCLUSIONS.....................................................................................19 4.1 Zion Site Visit...............................................................................................................19 4.2 BADGER Data Uncertainty Evaluation........................................................................19 4.3 BADGER 10B Areal Density Calculation Algorithm Testing.......................................19 5

REFERENCES.......................................................................................................................19 APPENDIX A............................................................................................................................. A-1 APPENDIX B..............................................................................................................................B-1

iv FIGURES Figure 1 Example of calibration data at ZNPP................................................................................6 TABLES Table 1 List of parameters needed to run Croft - ALGORITHM TEST.xlsx...........................16 Table 2 Example input for data for METHOD-I...........................................................................17 Table 3 Example input for data for METHOD-IIa........................................................................17 Table 4 Example input data for METHOD-IIb..............................................................................17

v

SUMMARY

This report is a revision to a 2015 ORNL technical letter report entitled, Addressing Uncertainties in SuperBADGER Measurements at the ZION Nuclear Power Plant (ZNPP)

Spent Fuel Pool, December 5-8, 2014 (ML15238A462) [2] prepared for the NRC under a previous project which described insights gained from the observation of BADGER ver. 2 (SuperBADGER was an interim name for the system in 2014) calibration, test, and evaluation measurements conducted at the Zion Nuclear Power Plant (ZNPP) spent nuclear fuel pool. The 2015 report also included revisions to the Type B uncertainty analysis originally reported in Table 6-1 of an NRC technical letter report prepared by ORNL entitled, Initial Assessment of Uncertainties Associated with BADGER Methodology, published in 2012 (ML12254A064).

This revision of the 2015 ORNL technical letter report adds ORNLs analysis of the BADGER ver. 2 data from the ZNPP campaign and further revisions to the Type B uncertainty analysis.

Witnessing the measurements at ZNPP, analyzing the BADGER ver. 2 data obtained during the ZNPP campaign, and assessing information provided by NETCO helped eliminate several influence parameters as sources of significant uncertainty and reduced the initial uncertainty estimates for several others. Based on areal density measurements using the BADGER data collected during the Zion measurement campaign, neutron attenuation data performed on the panels at Penn State and chemical analyses performed on calibration panels, the following relative standard deviations were calculated: calibration, 2% (systematic; precision can be reduced by averaging); calibration bias against chemical analysis, 3%; under reporting due to interpolation, 5% (but dependent on the number of standards); statistical precision for a sector or full panel, 3-6%; reproducibility following disassembly of all equipment and return to the job-site sometime later, 10%. The actual uncertainties are case specific and are generated from the actual data collected by NETCO and reported as per client needs. Finally, using the same set of inputs, ORNL and NETCO performed independent 10B areal density calculations and obtained good agreement. Thus, it was concluded that NETCOs algorithms are producing reasonable results.

vi

1 1

INTRODUCTION In September 2012, Chapman and Scaglione published a U.S. Nuclear Regulatory Commission (NRC) report entitled, Initial Assessment of Uncertainties Associated with BADGER Methodology. [1] At that time, only limited information was available regarding calibration, testing, evaluation, and implementation of the system against specific design objectives for system precision, bias, and total measurement uncertainty. The NRC posed questions to Oak Ridge National Laboratory (ORNL): (1) How well does the system function? (2) Are the results of good quality? (3) What is the total uncertainty in the estimate of 10B areal density for each transit, and for each panel? (4) What is the sampling plan for panel selection? (5) How do BADGER results compare to analyses of the coupons, in estimating 10B areal density?

Most of the BADGER design and approach is proprietary. In addition, ORNL did not have access to either preliminary test or evaluation data (1997-1999) or to raw data from actual panel measurements. As a result, the Initial Assessment of Uncertainty employed a qualitative, expert-elicitation/engineering-judgment approach to identify sources of error and to estimate the relative magnitudes of error that could be encountered. In Table 6.1 of the initial assessment [1],

all influence parameters that could affect the measurement error were identified and presented in eight broad categories: (1) Neutron Source, (2) Detector Characteristics, (3) Interferences, (4)

Electronics, (5) Measurement Geometry, (6) Calibration, (7) Data Processing, and (8) Statistical extrapolation and surveillance frequency. Each physical influence parameter was identified within each category. Alongside this parameter were four additional columns: (1) information used to support the discussion, (2) comments about relative uncertainty, (3) primary reference that supports the uncertainty statement, and (4) the uncertainty range of the given parameter.

Chapter 6 of the report describes the hierarchy that was established, as well as the basis for several terms used in the table, including one of the more often used notations, INQ, which indicates that insufficient data were available for review to support quantifying an uncertainty range for a given parameter. Hence, a quantitative roll-up (or root mean sum of squares) analysis could not be performed for overall measurement uncertainty.

When the initial assessment report [1] was issued, discussions with the NRC continued regarding the need to conduct a phase 2 of the project, with scope including (1) onsite observations of the BADGER system deployed, (2) review of calibration procedures and calibration data, (3) review and analysis of statistical test and evaluation data from the test and evaluation (T&E) procedure that would allow an independent investigation for the total measurement uncertainty (TMU) analysis, and (4) review of implementing procedures and analysis of raw data from actual panel measurements to confirm the bounded TMU analysis from T&E. By the middle of 2013, the US NRC notified ORNL that NETCO had begun a redesign of the original BADGER system and had intended to continue development and testing into 2014. Once a location for T&E could be secured, testing would commence, with ORNL present for observation. At the time, NETCO called the new system SuperBADGER, but it is now called BADGER ver.2. The design/build of BADGER ver. 2 is far superior to that of BADGER, which was designed and fabricated in the mid-1990s. Many of the influence parameters identified in Table 6.1 of the initial assessment report [1] with bases for design have been reduced significantly or removed altogether.

This report describes the insight gained from the observation of BADGER ver. 2 calibration, test, and evaluation measurements conducted at the ZNPP spent nuclear fuel pool on December 5-8, 2014. The measurements were conducted by NETCO engineers Matt Harris and Spencer

2 Feuerstein. Observing the measurements were Hatice Akkurt from EPRI, Eric Focht and Matt Rossi from the US NRC, and Jeff Chapman from ORNL.

3 2

TEST AND EVALUATION PLAN AND MEASUREMENTS AT ZNPP 2.1 Day 1 at ZNPP, Friday, December 5, 2014 Matt Harris, Spencer Feuerstein, Hatice Akkurt, and Jeff Chapman met at the auxiliary operations facility to plan the day and weekends measurements.

Harris provided Chapman a proprietary (draft) document on the upgrades for BADGER ver. 2.

The team reviewed and discussed elements of the Chapman-Scaglione report [1] against the new design upgrades, focusing specifically on how the redesign addressed many of the influence parameters identified in the report.

The new, as-built system was reviewed by physical inspection against the document that Harris provided, and the T&E plan, written to cover three-and-a-half days of measurements, was reviewed. NETCO developed, maintained, and executed the test plan. It was also conveyed that the draft design report was proprietary and not to be distributed, that the software was not to be photographed in any way, and that all measurement data were the property of NETCO. NETCO would consider releasing some of the data as requested for analysis of uncertainties once the coupon data results were analyzed in mid-summer of 2015.

2.1.1 The BADGER ver. 2 Test Plan (December 5-8, 2014)

The basic test plan was as follows:

1. Calibrate the system in the standard calibration cell (that is built by NETCO and moved from site to site, including ZNPP).

2. Identify the cells in the ZNPP SNF racks that are available to measure.

3. Measure those cell locations.

4. Conduct quality control/calibration/verification checks as needed on a daily basis, according to the NETCO procedure.

2.1.2 The Overall Project Test and Evaluation Plan (Through November 2015)

During the first day on site, the project team discussed the overall goals of the ZNPP NETCO effort to calibrate/test/evaluate BADGER ver. 2 and to inter-compare the 10B areal density (and degradation) BADGER ver. 2 measurements against measurements on coupons planned to be pulled from the pool racks by mid-summer, 2015.

The general concept for the overall project T&E effort was laid out by NETCO and EPRI in the following processes:

1. Evaluate the Super-BADGER tests collected December 5-8, 2014.

2. Reduce the raw data according to the new algorithms developed.

3. Report the results of the measurements in the measured cells.

4. Report the results of the measurements in the calibration standard cell.

5. Evaluate system performance and determine whether adjustments to method, algorithm, and/or design should be made going forward.

6. Evaluate the coupon results

a. In Spring 2015, pull coupon samples from ZNPP.

4

b. Measure by neutron transmission, according to ASTM C1671-2007 [3]

c. Analyze coupon measurements.

d. Inter-compare BADGER ver. 2 measurements with coupon measurements for normalization of the data and development of correction factors for algorithms.

7. NETCO intended to issue a report on the BADGER ver. 2 T&E measurements and inter-comparison of results with the coupon sample results, but as of this writing, the report has not been published.

2.1.3 ORNLs Proposed Contribution to Overall Project (Through November, 2015)

ORNL (Chapman) proposed to continue to work with the US NRC to establish a framework to review raw data such that an independent evaluation of TMU could be made. Harris and Chapman believed that by working together over the course of one or two group meetings, they can examine data, review algorithms, and prepare an error propagation model so that the Type A uncertainty analysis could be completed. However, as will be discussed later, not all the data needed to complete a full Type A uncertainty analysis was available, but significant progress was made regarding reducing the estimated uncertainties via an improved Type B uncertainty analysis based on the information obtained during this project.

2.2 Days 2-4 at ZNPP, Saturday-Monday, December 6-8, 2014 Conduct of the BADGER ver. 2 measurements went extremely well. The plan was executed without incident or problem. The ZION-Solutions operations personnel (Security, RadCon, Fuel Operations Manager) were exceptional. The team had unfettered access to the spent fuel pool and racks, as well as access to the overhead crane, the bridge crane, handling tools for the pool, cameras and video, and receipt/handling/transfer of the relatively intense 252Cf source for BADGER ver. 2. Many logistics and operational issues had the potential to impact the teams schedule, but this was simply not the case at ZNPP. Credit is given to all operations staff that supported the project. Also to be commended in particular was NETCOs engineer, Spencer Feuerstein. Feuerstein was involved with the design, fabrication, and subcomponent testing of BADGER ver. 2. He knew all details of the system and had worked with the system extensively, so that any issue that arose was easily and quickly ameliorated. The team was on site each day by 6:00 to 6:30 a.m. and departed by 5:00 to 5:30 p.m. On Monday evening, December 8, 2014, Jeff Chapman of ORNL departed for Los Alamos.

5 3

ANALYSIS OF BADGER ver. 2 DATA FROM THE ZION CAMPAIGN 3.1 Introduction After the first version of this document was prepared and issued in March 2015, dialog and exchange of information on performance information for BADGER ver. 2 continued between NETCO and ORNL. Outstanding questions concerning the impact of design options and how to conduct a realistic bottom-up uncertainty quantification were addressed by top-down considerations founded on operational experience. Experimental investigation is generally the preferred approach when possible and is further encouraged here as future questions arise. For example, the impact of panel imperfections (for instance verification that gaps circa 25 mm in size can be reliably identified) can be quantified by measurements on mock-ups. Appendix A from the 2015 ORNL technical letter report [2] was augmented with addenda highlighted in red to capture the outcome of these follow-up interactions. In this section some additional remarks are also provided. Numerical values are given for guidance only and are not intended to be used in place of campaign-specific client information reported by NETCO. The ZNPP campaign is just one study with limited scope. Here we draw on the limited data generated during the ZNPP campaign and supplemental information provided by NETCO to describe the performance achieved by BADGER ver. 2 in broad terms. However, NETCO has far more information available to them and are adding to that knowledge base over time. We expect application specific questions will be better addressed by harvesting that resource more fully.

In addition to the analysis of the BADGER ver. 2 data from the ZNPP campaign, ORNL also developed its own algorithm for estimating 10B areal density and propagating measurement uncertainty as an independent code-to-code comparison with NETCOs algorithm. The details are provided below.

3.1.1 Model Bias To illustrate the model bias (potential for under/over reporting of 10B areal density), Figure 1 shows a typical calibration shape (note this is a linear-linear plot, not transformed onto ln-linear space). This is based on the 2014 measurement campaign performed at the ZNPP [2]. Only three calibration standards were used. NETCO provided raw data from the ZNPP campaign, including calibration data. For illustrative purposes, we combined the count rates from Detectors 2 and 3 for Cal 3 (Cal 3 indicates a specific set of calibration data) which was used for 18 of the Region 2 panels. The counting precision on each point is less than 1% (and are barely visible in the figure compared to the size of the plotting symbol) while the nominal uncertainty on the reference values are approximately 2.9 %, 2.0 % and 1.5 % going from lowest to highest 10B areal density (AD), respectively. For a given measurement panel, measured AD values varied between 0.026 and 0.030 g/cm2, thus, spanning two AD calibration zones. Values below 0.0279 g/cm2 fall into the first AD-zone where the slope is steepening. The count rate corresponding to an AD value of 0.026 g/cm2 in the first zone would yield a value of only 0.022 g/cm2 had the calibration parameters of the second zone been used. Of course, the direction would usually be in the other direction (in this example we are extrapolating rather than interpolating). It is clear there is an argument in this case for including an additional calibration point at 0.022 g/cm2, for example, to reduce the lever arm effect. The use of five standards, where practical, would be preferred so that the spacing between points improves the quality of the interpolation. However, this is mainly a procedural matter, not an inherent instrument limitation. The choice of standards

6 to bracket and interpolate between should be made in relation to the data quality objectives set at the on-set, what is known about the panels to be scanned, and the local constraints. As things currently stand in Figure 1, observed rates in the first zone (roughly 2300 to 5500 counts per sec) will tend to over report the true AD value. Adding an additional calibration standard in this range will lower the reported values. This is a consequence of the non-exponential behavior of the instrument.

Figure 1 Example of calibration data at ZNPP.

3.1.2 Areal Density Precision for Panels BADGER ver. 2 is typically used to estimate the average panel AD. Based on repeat data provided by NETCO, ORNL estimated that the statistical precision on the AD for such a determination is circa 4-6 %. Replicate measurements performed by NETCO, where the crew disassembled all equipment returning 10 days later, are suggestive that results for the same panels may differ by circa 10 % compared to the initial determination.

3.1.3 Calibration Standards The 10B AD of the standards used for the ZNPP measurements were determined by chemical analysis with a reported accuracy of about 2 % at 1-sigma. Results of comparable overall reported accuracy are reported using neutron attenuation measurements at the Penn State University Radiation Science and Engineering Center Neutron Beam Laboratory. Using paired data (i.e. chemical and transmission results on the same material) provided by NETCO and analyzed by ORNL, there is some evidence that, on average, the neutron attenuation method returns values that are slightly lower than the chemical analysis by about 3 +/- 1 %. The reason for this not clear if the calibration items used at the neutron beam facility are reasonably representative (e.g. boron carbide grain size) and certified based on destructive methods. A

7 possible contributory factor is manufacturing heterogeneity and it has been speculated that B4C self-shielding may be involved. The chemical and neutron beam methods are both susceptible to sampling error and across a 12-inch panel section the neutron beam results can vary by 3-6 %.

3.1.4 BADGER ver. 2 AD Versus PSU Beam Lab Results Comparison between the BADGER ver. 2 AD results measured in-situ at ZNPP by height to the PSU beam results on the corresponding section are in good agreement. For each panel there are six 12-inch sectors for direct comparison. Overall, the statistical variation in the difference between the two results is about 4 % 1-sigma per segment. On an absolute basis, the BADGER ver. 2 results had a slight tendency to under report AD relative to the beam results. The magnitude of this effect was of the order of 3 %.

3.2 10B AD Measurement Algorithm Testing: Addressing Item 5.6.12 in the Table 6.1 Addendum 3.2.1 Introduction Neutron transmission measurements provide a nondestructive way to monitor the reactivity control provided by neutron absorbing materials used at nuclear spent fuel pools. The measurement equation used to estimate the value of the 10B areal density at a single location of a BORAL panel, based on the deadtime and background corrected in-situ counting rate observed in one of the individual BF3 detectors of the BADGER ver. 2 tool, depends in detail on how the data is collected and interpreted. Here, we outline two general approaches and three implementations. These have been coded into an Excel spreadsheet to enable test calculations to be performed against NETCOs independent implementation, and to assess the influence of background (because NETCO does not make deadtime or background corrections). By working in terms of independent random variables (IRV), the evaluation of the estimated uncertainty by propagation of variance (PoV) or by forward-backward finite difference is simplified. The input parameters are shown in Table 1.

3.2.2 Outline of the Transmission Method For a steady-state pencil-beam of monoenergetic neutrons, the measured transmission factor,,

defined as the ratio of the monoenergetic ray-intensity (that is the neutron current in n/s) recorded with and without the absorber-item in place, may be expressed in the following way:

=

0

= exp ()

8 where is the mass attenuation coefficient of the medium at the neutron energy, and is the areal density of the medium along the direction of the in-coming beam.

From this relationship we obtain:

= ln () = ln 0

=

where we have replaced

0 by

0 the ratio of the measured and straight-through deadtime corrected net (i.e. also background corrected) counting rates since is directly proportional to.

In practice we can expect deviations from this ideal linearized mathematical form as a function of. For instance, actual neutron beams have both a finite energy and finite spatial distributions and the distribution of neutron-absorber may not be uniform over the item. However, what the analysis of the ideal case provides is the suggestion that, provided a truly representative calibration can be performed, linear interpolation over a narrow dynamic range is perhaps a reasonable pragmatic approach. This is borne out by empirical calibration and calibration verification measurements. The concept has been accepted as a standard ASTM test method [4].

For thermalized neutrons, the 10B content of BORAL utterly dominates the removal of neutrons and it is therefore both reasonable and convenient to associate with the 10B areal density. The purpose of the calibration is to determine the effective value of the model parameter over some range of.

In Method-I we treat the straight-through (unattenuated) counting rate as the fiducial or reference rate. Let and be the 10B ADs for two of our calibration panels where >, and our unknown BORAL panel is bounded by them; that is experimentally we observe.

The interpolation scheme follows by fitting a straight line through the two calibration points in our transformed space:

=

() +

()

Note that the assay value is expressed directly in terms of the calibration data; we did not need to expose the slope and intercept of the fitted line explicitly. This is possible in such a simple way because the line through just two experimental points does not require a minimization; it is an exactly determined problem. Note that in principle the value of and values of & could be measured at different times, that is with different high-voltage and threshold settings, and so will need separate straight-through estimates, or if not, all three will be correlated through the use of the same unattenuated counting rate. In practice, establishing the ambient background rate and selecting a suitable place to measure the unattenuated or straight-through rate for the unknown panel are challenging because of the practical constraints present at the spent fuel pool.

The exact experimental choices influence both the explicit form of the measurement equation and the propagated uncertainty when expressed in terms of the IRV of detected number of counts.

In Method-II we take the fiducial or reference intensity to be that observed with one of the calibration panels in place. This has the advantage of not requiring the unattenuated straight-

9 through intensity, which also narrows the operational range minimizing rate effects, but does impose the requirement for the calibration and assay to be performed close in time so that the system efficiency is the same. Taking as our reference case, the Method II model can be expressed as follows:

= ()

and

= ()

Solving for gives:

= +

ln

()

In evaluating the deadtime and background corrected rates we are fortunate that the deadtime losses are small to begin with (i.e. the product of the rate and deadtime is small) and to a good approximation get included in the calibration and so do not need to be considered explicitly (provided the calibration and measurements are not widely separated in time so that the neutron emission rate from the 252Cf-source remains similar throughout - something readily achieved).

However, we do need to make a choice as to whether the background can be neglected or if a common background is adequate or if a separate background is needed for each assay location (a single background applies to the calibration cell since it is located in a quiet region and the spacing between the different calibration plates in relatively small). Again, for evaluation and propagation of variance (PoV) we work in terms of independent random variables (number of observed counts) so that any correlations are properly accounted for.

In addition to algebraic PoV, as a check, we also implement a numerical scheme based on forward-backward finite difference. Let (1, 2 ) be the measurement equation expressed in terms of the IRV. Then:

= 1

1

+

2 evaluated with all of the set to their nominal values, and the estimated fractional standard uncertainty becomes:

10

2

2

=1 We shall now develop the uncertainty treatment.

Method-I When we use the unattenuated (boron free) straight-through counting rate as the fiducial we have, in terms of the independent random variables:

= +

0 0

()

where and are the areal densities of the low areal density and high areal density calibration standards bounding the unknown value 0 is the deadtime corrected straight through or unattenuated counting rate measured in-situ in a region where there is no B4C in the panel, and 0 is the corresponding ambient background rate measured without the 252Cf source present is the deadtime corrected counting rate measured in-situ at the elevation of interest along the unknown measurement panel, and is the corresponding ambient background rate

, and are the deadtime corrected rates obtained during calibration for the low, high and background cases. We are using to denote calibration rates to distinguish them from assay rates which are denoted by, because in principle they could be performed at different times and with the detector settings somewhat different (a proportionate change in the efficiency) -

although the water chemistry and temperature of the pool water must be the same and the calibration cell representative of the condition of the measurements of the unknown.

Note, we are explicitly assuming that 0 and are independent because they relate to different positions along the panel and that for the calibration a single value for the background rate,,

applies.

If the rates are derived from a single Poisson count and the deadtime is only modest (so that the counting distribution is not perturbed significantly) then:

+/-

+/-

11 where is the number of counts recorded in time period and the uncertainty is stated at approximately the 1 (standard deviation) level provided the number of counts is not too small (e.g. greater than 25 or so). Similar expressions apply for each of the seven rates. There is no necessity or reason to assume Poisson counting statistics from the on-set. Experimentally it is better to split a counting period into a number of shorter cycles so that the counting uncertainty can be evaluated statistically. Furthermore, the random positional uncertainty should be included which requires repeat measurement data. It is prohibitive on throughput to do this every measurement position and so, instead, positional uncertainty can reasonably be assessed separately as part of building the total measurement uncertainty. The fractional variation due to repositioning is likely similar for all cases for well-designed equipment, properly adjusted and operated. For each campaign this is something that should be tracked using control charts as part of the measurement control program.

For test purposes only, the METHOD-I measurement equation was coded so that the overall measurement uncertainty could be propagated assuming the model is correct. The partial derivatives with respect to the independent random variables were estimated in two ways as a check. The first way was by numerical forward-backward finite difference with steps of plus and minus one standard deviations, and the second was algebraically. Agreement between the two approaches provides confidence that the uncertainty treatment has been implemented correctly.

The default analysis assumes simple Poisson counting and this is adequate to enable meaningful code to code comparison.

It seems reasonable to assume that and are correlated because they are subject to a common systematic error associated with the analytical method used to certify the 10B areal density. To include this in the analysis a multiplicative model is assumed:

=

and

=

where has a numerical value of unity and has an assigned standard deviation of, so that the fractional standard deviation is =

, and & are independent with random standard deviations of & respectively. The form of the measurement equation then becomes:

= +

0 0

()

To treat and as independent (as NETCO do) then set to zero.

12 Since the experimental approach leading to Method-I is no longer used by NETCO we shall not go into the details of uncertainty analysis here. Instead we refer to Croft - Algorithm Test notes.pdf and Croft - ALGORITHM TEST.xls [available on request] for the necessary expressions and implementation of the measurement uncertainties. Example input values are shown in Table 2.

Method-IIa: Comparison to NETCOs Algorithm The second general method is like the first but instead of referencing the rates to the straight through (or unattenuated) deadtime corrected net counting rate, the low areal density calibration standard is used as the reference case.

= +

()

Similar comments apply for this method as for Method-I, but note that we are assuming all of the counting rates are determined with the detector under the same operating conditions (e.g. same high-voltage and neutron detection efficiency) and that the same background rate,, applies to all of the net rate determinations.

For test purposes only, the METHOD-IIa measurement equation has been coded into ALGORITHM TEST.xlsx with the overall uncertainty propagated using both finite difference and PoV. Agreement between the finite difference and PoV estimates lends confidence to the coding of both approaches and, in particular, the algebraic expressions used for PoV. This is important because the NETCO software uses only the analytical approach. Also, the NETCO algorithm document is preliminary and the NETCO analysis spreadsheet is difficult to deconstruct. Thus, the present top-down analysis was conceived as a complementary way to confirm the basic correctness of NETCOs results. A set of input parameters was run through the present implementation and NETCOs routine analysis engine and good agreement was obtained for both the estimated measured value and the associated uncertainty (neglecting certain factors which may best be described as model bias and which are not amenable to PoV; these include how representative the calibration cell is etc.) [Matt Harris of NETCO undertook these comparisons using representative experimental inputs, Priv. Comm.]. For illustrative purposes only, an example of the input data, invented simply to test spreadsheet implementation, is provided in Table 3.

Method-IIb Method-IIb is a variant of Method-IIa. The difference is that we acknowledge that the calibration is performed in a different radiation environment than the assay and so we assign a separate independent background rate,, to the assay position. Example input values are shown in Table 4.

13

= +

()= +

()

where we have introduced

=

= [] [

]

and

=

= [] []

For use in the PoV method we form the partial derivatives as follows:

=

= 1

=

= ()

1

(1

)

= ()

1

= ()

1 1

14

= ()

0 1

1

1

= +()

1

For completeness we note that the partial derivatives for Method-IIa are identical except that

= 0 (because there is no dependence in that formulation), and:

= ()

1

1

1

1

()

The statistical variance in the 10B areal density according to PoV is given in all cases by:

2 =

2

2 where the sum extends over all independent input variables.

Model Bias Representing the (BADGER ver 2) instrument response as a straight line in ln-linear space between a pair of calibration points allows us to use the inherent simplicity of a two-point fit. It permits a straightforward uncertainty analysis. However, empirically it is observed from extended range multi-point calibration data that (). is not a straight line but has curvature with positive second derivative. This means that the use of the stick interpolation model will tend to over-report the value of. The bias will depend on the spacing of the calibration points and where in the spread the measurement panel falls. The best way to assess the magnitude of the bias is through direct experimental performance demonstration. The number and spacing of calibration points is a tradeoff that should be made as part of a wider set of performance management criteria. It is not an inherent limitation. It is not part of the Algorithm Testing.

15 Calibration Bias The quality of the BADGER ver 2 calibration rests on the quality of the calibration standards and how closely they represent actual measurement conditions. For instance, actual cells may not be as geometrically perfect as the calibration cell or have the same B-content in all four sides.

It is also difficult to know whether the BORAL sheets used to construct the calibration cells have the same microstructure (neutron self-absorption) properties. That is, two panels with the same certified 10B areal density, as determined by analytical chemistry methods, may exhibit different neutron transmission behaviors. This issue relates to the consistency of the production process over time. The impact should to be assessed experimentally although the magnitude of the effect is not expected to be large. It was not part of this Algorithm Testing.

16 Table 1 List of parameters needed to run Croft - ALGORITHM TEST.xlsx Parameters for METHOD-I Calibration:

xH =

10B areal density in g/cm2 of the high areal density calibration standard NH =

corresponding gross number of counts observed lH =

and data associated collection time xL =

ditto low NL =

lL =

Nb =

and for the background estimation at the time of calibration lb =

fc =

fractional common (scale) systematic uncertainty in the areal desnity of the calibration items Assay:

C =

gross counts collected for the unknown item t =

measurement time Cb =

ditto background tb =

Co =

straight through (or unattenuated) case to =

Cbo =

ditto background at straight through position tbo =

Parameters for METHOD-IIa Calibration:

2 =

10B areal density in g/cm2 of the high areal density calibration standard C2 =

corresponding gross number of counts observed t2 =

and data associated collection time 1 =

ditto low C1 =

t1 =

Cb =

gross counts collected for during the backlground determation tb =

background measurement time c =

multiplicative scale factor (equal to unity) but with uncertinty to represent calibration item systematic uncertainty Assay:

C =

gross counts collected for the unknown assay-item t =

assay measurement time Parameters for METHOD-IIb as above but for the Assay we add:

B =

gross counts collected for the background determination at the location (height in the particular cell) for the assay-item tB =

corresponding measurement time

17 Table 2 Example input for data for METHOD-I Table 3 Example input for data for METHOD-IIa Table 4 Example input data for METHOD-IIb Input Parameters:

Calibration:

xH =

0.03

+/-

0.00015 g10B.cm-2 Random uncertainty contribution only is enterd here NH =

1000

+/-

31.6227766 counts lH =

30 sec xL =

0.02

+/-

0.0001 g10B.cm-2 Random uncertainty contribution only is entered here NL =

2000

+/-

44.72135955 counts lL =

30 sec Nb =

30

+/-

5.477225575 counts lb =

30 sec fc =

0.001 Fractional standard uncertainty in the common absolute scale-factor on the certificates for the areal density of the calibration panels Assay:

C =

1500

+/-

38.72983346 counts t =

30 sec Cb =

90

+/-

9.486832981 counts tb =

30 sec Co =

2700

+/-

51.96152423 counts to =

30 sec Cbo =

60

+/-

7.745966692 counts tbo =

30 sec Input Parameters:

Calibration:

2 =

0.03

+/-

0.0001 g10B.cm-2 Calibration item value with random uncertainty contribution only C2 =

1000

+/-

31.6227766 counts t2 =

30 sec 1 =

0.02

+/-

0.0001 g10B.cm-2 Calibration item value with random uncertainty contribution only C1 =

2000

+/-

44.72135955 counts t1 =

30 sec Cb =

30

+/-

5.477225575 counts Provided the number of counts is greater than about 12 we are fine using sqrt for our purposes.

tb =

30 sec c =

1

+/-

0.015 Calibration standard items scale factor Assay:

C =

1500

+/-

38.72983346 counts t =

30 sec Calibration:

2 =

0.03

+/-

0.0001 g10B.cm-2 Calibration item value with random uncertainty contribution only C2 =

1000

+/-

31.6227766 counts t2 =

30 sec 1 =

0.02

+/-

0.0001 g10B.cm-2 Calibration item value with random uncertainty contribution only C1 =

2000

+/-

44.72135955 counts t1 =

30 sec Cb =

30

+/-

5.477225575 counts Provided the number of counts is greater than about 12 we are fine using sqrt for our purposes.

tb =

30 sec c =

1

+/-

0.015 Calibration standard items scale factor Assay:

C =

1500

+/-

38.72983346 counts t =

30 sec B =

60

+/-

7.745966692 counts Background at place of assay tB =

30 sec

18

19 4

SUMMARY

AND CONCLUSIONS 4.1 Zion Site Visit Observing the BADGER ver. 2 calibration, test, and evaluation measurements at ZNPP was extremely productive. Critical measurement data were acquired to assist with a Type-A uncertainty analysis, if needed, in the future. Error propagation models can be developed and implemented. Quality control charts can and should also be developed to ensure that the system is in statistical control. Appendix A provides an update to Table 6.1 of the initial analysis [1].

Appendix B provides a few photos from the many photos collected during the measurements of December 5-8, 2014. All parties agreed that the influence parameters in Table 6.1 had been identified correctly and that no additional parameters could be envisaged before or after the ZNPP T&E. The questions regarding uncertainty range, or magnitude of error, are the core questions to resolve. Witnessing the measurements at ZNPP, analyzing the BADGER ver. 2 data obtained during the ZNPP campaign, and assessing information provided by NETCO helped eliminate several influence parameters as sources of significant uncertainty and reduced the initial uncertainty estimates for several others.

4.2 BADGER Data Uncertainty Evaluation To summarize the main contributions to measurement uncertainty based on operational experience, the following indicative relative standard deviations were calculated: calibration, 2%

(systematic; precision can be reduced by averaging); calibration bias against chemical analysis, 3%; under reporting due to interpolation, 5% (but dependent on the number of standards);

statistical precision for a sector or full panel, 3-6%; reproducibility following disassembly all equipment and return to the job-site sometime later, 10%. The actual uncertainties are case specific and are generated from the actual data collected by NETCO and reported as per client needs.

4.3 BADGER 10B Areal Density Calculation Algorithm Testing ORNL developed an independent calculational approach to determine the 10B areal density and propagate measurement uncertainty for the BADGER ver. 2 data to verify NETCOs algorithms.

ORNL outlined three approaches above of which Method IIa is the method used by NETCO.

Using the same set of inputs, ORNL and NETCO performed 10B AD calculations and obtained good agreement. Thus, it was concluded that NETCOs algorithms are producing reasonable results.

5 REFERENCES

1. Chapman J.A., Scaglione J.M. Initial Assessment of Uncertainties Associated with BADGER Methodology, September 2012. (ADAMS Accession No. ML12254A064).

2. Chapman, J., SuperBADGER Measurements at the Zion Nuclear Power Plant (ZNPP)

Spent Fuel Pool, December 5-8, 2014, ORNL/SPR-2012/74, March 2015, (ADAMS Accession No. ML15238A462).

20

3. ASTM C1671-15, Standard Practice for Qualification and Acceptance of Boron Based Metallic Neutron Absorbers for Nuclear Criticality Control for Dry Cask Storage Systems and Transportation Packaging, ASTM International, 100 Barr Harbor Drive, PO Box C700, West Conshohocken, PA 19428-2959, USA

4. ASTM E2971-16, Standard Test Method for Determination of Effective Boron-10 Areal Density on Aluminum Neutron Absorbers using Neutron Attenuation Measurements, ASTM International, 100 Barr Harbor Drive, PO Box C700, West Conshohocken, PA 19428-2959, USA

APPENDIX A REVISIONS TO BADGER UNCERTAINTY FACTORS

A-1 Table 6. 1 from Ref. 1 Addendum (Updated with Remarks in Last Column based on Assessment of BADGER Data and Information)

(See: COMMENT POST ZNPP and Addendum in last column for Remarks/Qualitative Comment)

Sec-tion Report subsection Physical influence parameter Information used to support discussion Comments regarding relative uncertainty Primary reference source Uncertainty range 5.1 Neutron Source 5.1.1 Count rate Publicly available No experimental data provided to estimate impact of parameter on overall uncertainty on implementation of BADGER. With reported count times per axial measurement of 10 seconds, the net count rate of 1,000 net counts per count interval gives a relative error of roughly 3 to 4%, neglecting contribution from background. Relative error of Poisson counting statistics given by the relative error in subtracting background count rate from gross count rate, adjusted for any effect associated with pulse pile-up, or variability in the in-scatter from neutrons not originating from the 252Cf source and transmitted directly through the Boraflex. From looking at traces of count rate data published in Reference 8, count rates appeared to be ~200 neutron counts per 10-sec interval (sometimes less), in which case the Poisson error is on the order of 7-8% and will increase based on other neutron count rate effects mentioned above. These effects require additional evaluation.

Reference 8 (Figures 3-16 and 4-

10) 8% on count rate [Neutron count rate uncertainty should be documented in system test and all panel measurement results.]

COMMENT POST ZNPP Intensity of 252Cf source combined with new head design produced nominal count rates with Poisson error less than 1%.

ORNL would need final raw data to calculate and confirm.

Addendum: Counting precision depends on the neutron emission rate of the source at the time of the measurement but, between source replacements, can be managed by choosing the counting time.

The software reports the precision on a case-by-case basis. Counting precision is far less for the cell average than for an individual point, because many data points are combined. This is evident in the scan records collected by NETCO.

A-2 Sec-tion Report subsection Physical influence parameter Information used to support discussion Comments regarding relative uncertainty Primary reference source Uncertainty range 5.1.2 Source strength and decay No information available No information available on source strength. The half-life of 252Cf = 2.65 years, which yields ~0.7% change per 10 days. Corrections and adjustments for source decay should be accounted for in the calibration and on-site QC measurements with normalization correction made accordingly Reference 23 INQ

[Decay-corrected source strength should be presented on calibration and measurement data records]

COMMENT POST ZNPP A decay-corrected source curve was identified by NETCO. ORNL would need final raw data to calculate and confirm.

The data would need to be provided over at least a 6-month period to confirm correct.

Addendum: Calibration and quality control data are collected close in time to the scan data and so source decay is not a significant influence.

A-3 Sec-tion Report subsection Physical influence parameter Information used to support discussion Comments regarding relative uncertainty Primary reference source Uncertainty range 5.1.3 Neutron moderation by covers on heads No information available No experimental data provided to estimate impact of parameter on overall uncertainty on implementation of BADGER. This is likely to be a large contribution to uncertainty to invalidate results for Region I racks, in particular, where the system is over-moderated and the irradiating neutron flux on the second panel of two is thermalized to such an extent that the 10B areal density is infinitely thick to the flux.

N/A INQ

[Neutron moderation and down-scatter effects and the influence on system accuracy and precision should be documented in the system test plan.]

COMMENT POST ZNPP For the ZNPP Rack design, the new SuperBADGER design appears appropriate. As-built drawings need to be reviewed to confirm. MCNP or MAVRIC calculations can also benchmark this effect.

Addendum: Any concern is removed by using a representative calibration configuration. The system is calibrated and used as build and so moderation effects are included and do not need to be treated separately.

A-4 Sec-tion Report subsection Physical influence parameter Information used to support discussion Comments regarding relative uncertainty Primary reference source Uncertainty range 5.2 Detector characteristics 5.2.1 Size No information available Performance Specification reflected in count rate (Section 5.1.1) and panel coverage (Section 5.5.5)

N/A INQ

[Neutron detector specifications should be provided in technical design basis document of system, which captures the trade-offs on count time, count rate, spatial resolution, wall effects, and any other specific measurement parameter that impacts data quality objectives.]

COMMENT POST ZNPP New detectors were purchased. Claims regarding spatial resolution and variation in response across the width of panel need to be verified by data.

Addendum: This is a design question and not relevant when considering the BADGER ver.2 as-built instrument.

5.2.2 Efficiency No information available Performance Specification reflected in count rate (Section 5.1.1)

N/A INQ

[Same as previous]

COMMENT POST ZNPP Count rate appeared nominal for ZNPP design. Design spec met.

Addendum: As 5.2.1

A-5 Sec-tion Report subsection Physical influence parameter Information used to support discussion Comments regarding relative uncertainty Primary reference source Uncertainty range 5.2.3 Fill gas pressure No information available Performance Specification reflected in count rate (Section 5.1.1)

N/A INQ

[Same as previous]

COMMENT POST ZNPP Count rate appeared nominal for ZNPP design. Design spec met.

Addendum: As 5.2.1 5.2.4 10B enrichment of the gas No information available Performance Specification reflected in count rate (Section 5.1.1)

N/A INQ

[Same as previous]

COMMENT POST ZNPP Count rate appeared nominal for ZNPP design. Design spec met.

Addendum: As 5.2.1

A-6 Sec-tion Report subsection Physical influence parameter Information used to support discussion Comments regarding relative uncertainty Primary reference source Uncertainty range 5.2.5 Aging Expert judgment QC to maintain within design limits. If unaccounted for, results are unreliable.

Reference 8 and Reference 2 4

INQ

[Neutron detector aging is accounted for by a QC measurement program, to ensure that the detector produces a consistent and reliable performance, with time. When out of control, the detector is repaired or replaced (but normally replaced)].

COMMENT POST ZNPP Count rate appeared nominal, but these were new (non-aged) detectors. QC program still under development during ZNPP tests to confirm and verify that neutron detectors respond consistently over time. Final QC needs to be verified.

Addendum: Because the calibration is performed close in time to the measurements (i.e. on the same day) long-term aging is not a significant factor.

Also, short term stability is excellent and readily demonstrated and maintained within measurement control by periodic checks. The comparison of rates between the four BF3 detectors is one such check.

A-7 Sec-tion Report subsection Physical influence parameter Information used to support discussion Comments regarding relative uncertainty Primary reference source Uncertainty range 5.2.6 Wall material No information available Different wall materials will result in different scattering and absorption rates, and can also contribute to background counts.

N/A INQ

[Neutron detector specifications should be provided in technical design basis document of system, which captures the trade-offs on count time, count rate, spatial resolution, wall effects, and any other specific measurement parameter that impacts data quality objectives.]

COMMENT POST ZNPP Count rate appeared nominal for ZNPP design. Design spec met.

Addendum: As 5.2.1

A-8 Sec-tion Report subsection Physical influence parameter Information used to support discussion Comments regarding relative uncertainty Primary reference source Uncertainty range 5.3 Interference 5.3.1

Background

neutron flux from surrounding assemblies No information available There is no evidence that either engineering controls and/or administrative controls are used to understand, measure, and evaluate background neutron count rates.

Background in this case refers to neutrons entering the detector(s) from other adjacent spent fuel assemblies and not to "background" neutrons reflected or inscattered from the 252Cf transmission source.

N/A INQ

[Neutron count rate uncertainty should be documented in system test and all panel measurement results. Neutron background rate (real or poor n/

discrimination) should be documented separately.]

COMMENT POST ZNPP High neutron output assemblies not in vicinity of tests. Thus, this still requires testing and development for proper neutron background subtraction (when applicable).

Addendum: Calibration is performed in a low background environment. An elevated background when measuring an unknown cell will therefore result in an underreporting of 10B areal density. This is a conservative approach. In principle a passive scan using a head without a Cf-source could be performed to estimate the background.

A-9 5.3.2 Gamma interference Expert judgment Contingent on operator and on the manner in which the nuclear electronics are adjusted, in situ for gamma-ray pileup pulse that exceeds the lower-level discriminator setting on the Single Channel Analyzer (SCA). Low (~2%) if properly accounted for, or results could be unreliable if not properly accounted for.

N/A Low to high

[Same as previous. In addition, n/

discrimination measurements should be documented in the measurement records.]

COMMENT POST ZNPP Not tested. Needs testing (when applicable).

Addendum: Off-line testing has demonstrated adequate gamma-to-neutron rejection and absence of gamma-breakthrough is confirmed by observing the signal trace on an oscilloscope during data collection. High-voltage plateau measurements (e.g. at a chosen high does rate position) can also reveal if there is a problem. An even greater gamma rejection factor can be obtained by running at lower voltage off plateau.

However, normal operating procedures are expected to ensure that gamma breakthrough is not a causing a bias. Note also that gamma-breakthrough would result in an elevated count which would be interpreted as a loss of 10B. It would therefore lead to an under-reporting of 10B areal density.

A-10 Sec-tion Report subsection Physical influence parameter Information used to support discussion Comments regarding relative uncertainty Primary reference source Uncertainty range 5.4 Electronics 5.4.1 EMI susceptibility No information available Instrumentation technical performance specifications not provided.

N/A INQ

[Parameter effect should be documented in test plan (with results) that evaluates performance over range of expected conditions.]

COMMENT POST ZNPP Not tested at ZNPP, but no local problems were easily identified. Should be tested in future.

Addendum: NETCO experience shows that this is not a cause for concern.

Counts are routinely compared to expectations and scrutiny of the pulse train using an oscilloscope are used to confirm correct operation.

5.4.2 RF pickup No information available Instrumentation technical performance specifications not provided N/A INQ

[Same as previous]

COMMENT POST ZNPP Not tested at ZNPP, but no local problems were easily identified. Should be tested in future.

Addendum: As 5.4.1

A-11 Sec-tion Report subsection Physical influence parameter Information used to support discussion Comments regarding relative uncertainty Primary reference source Uncertainty range 5.4.3 Amplifiers, discriminators

, power

supply, acquisition board No information available Instrumentation technical performance specifications not provided N/A INQ

[Same as previous. In addition, the selection of these components should be described in the system technical basis design document, tested, evaluated, and documented, accordingly.]

COMMENT POST ZNPP New electronics are in SuperBADGER.

When possible, NETCO should conduct system tests to ensure that electronics matched and that any environmental or pulse-train effects are identified. From measurements at ZNPP, the system appeared stable and solid.

Addendum: As 5.4.1 5.4.4 Signal processors No information available Instrumentation technical performance specifications not provided N/A INQ

[Same as previous]

COMMENT POST ZNPP Same as 5.4.3 above.

Addendum: As 5.4.1

A-12 Sec-tion Report subsection Physical influence parameter Information used to support discussion Comments regarding relative uncertainty Primary reference source Uncertainty range 5.4.5 Discriminator s for pile-up rejection, wall effect No information available Instrumentation technical performance specifications not provided N/A INQ

[Same as previous]

COMMENT POST ZNPP Same as 5.4.3 above.

Addendum: As 5.4.1 5.4.6 Dead time No information available Instrumentation technical performance specifications not provided N/A INQ

[Same as previous]

COMMENT POST ZNPP Same as 5.4.3 above.

Addendum: Firstly, deadtime losses are a few hundred counts per sec with a deadtime of a few micro-sec per event correspond to a fractional loss of at most a fraction of 1 % and, secondly, rate losses are effectively included as part of the empirical calibration so that it becomes a non-issue.

A-13 Sec-tion Report subsection Physical influence parameter Information used to support discussion Comments regarding relative uncertainty Primary reference source Uncertainty range 5.5 Apparatus geometry 5.5.1 Head misalignment First-principle computation Because the proximity between the source head, panel, and detector is close

(< 6 in. [15.2 cm]), and it has been reported that BADGER sticks (or jambs) in some warped panels, the measurement geometry is likely not consistent throughout a single panel (see other parameters below)

Monte Carlo calcula-tions conducted by authors in Appendix A

> 40% (on average areal density)

[Logsheets should contain records for the difficulty in easily moving the heads up the rack. A look-up table should be prepared, based on experimental data, to determine the magnitude and direction of bias introduced at that measurement point.]

COMMENT POST ZNPP The electro-mechanical design of SuperBADGER is far superior to that of BADGER. When the ZNPP data are analyzed (including the calibration cell),

this geometric effect uncertainty can be calculated and most likely can be significantly reduced. The ball-springs were very effective at keeping the measurement geometry constant.

Addendum: This is a legitimate design consideration which is now superseded by experimental performance data, especially repeatability data collected by NETCO. The point variation due to head misalignment for the BADGER ver 2 design seems to be less than 10% based on discussions with NETCO.

A-14 Sec-tion Report subsection Physical influence parameter Information used to support discussion Comments regarding relative uncertainty Primary reference source Uncertainty range 5.5.2 Rack cell fit No information available Mitigated for with Section 5.5.1 and 5.5.3 Design element reflected in Section 5.5.5 N/A INQ

[Same as previous]

COMMENT POST ZNPP Same as 5.5.1.

Addendum: As 5.5.1 5.5.3 Compression

springs, shims, vertical offsets No information available Uncertainty reflected in Section 5.5.1.

N/A INQ

[Mechanical stops, guidance, and shims should be described in the system technical basis document, and then tested to ensure the measurement is conducted in a stable geometry.]

COMMENT POST ZNPP Same as 5.5.1.

Addendum: As 5.5.1

A-15 Sec-tion Report subsection Physical influence parameter Information used to support discussion Comments regarding relative uncertainty Primary reference source Uncertainty range 5.5.4 Effect of rack cell deformation:

panel cladding

bulges, collisions with assemblies Expert judgment Uncertainty similar to Section 5.5.1 across an axial segment due to (1) broad beam irradiation of point 252Cf source across panel surface 2-3 in. (5-7.6 cm) from source; (2) detectors ~0.5 in. (1.3 cm) from panel; and (3) bulges reported with dimensions of 1/4 in. (0.6 cm) or more. Neutron transmission measurements are very sensitive to these changes in measurement geometry.

N/A

[See Section 5.5.1 and 5.5.3]

COMMENT POST ZNPP Same as 5.5.1.

However, it is clearly unavoidable to keep the heads from not sticking/resting/catching on the wrapper.

In the laboratory, this effect can be studied and analyzed to determine the contribution to error at or near the seam or where the cell is bulged or deformed.

For ZNPP, this did not appear to be an issue.

Addendum: As 5.5.1

A-16 Sec-tion Report subsection Physical influence parameter Information used to support discussion Comments regarding relative uncertainty Primary reference source Uncertainty range 5.5.5 Determination of detector coverage of panel area No information available Design element to meet performance specification. A broad-beam point source projection across a panel surface 2-3 in. (5-7.6 cm) from source is subject to specific measurement controls placed on the uniformity of the irradiation.

Uncertainty can be quite high, as opposed to a uniform, "pencil-beam" irradiating flux used in a laboratory-grade coupon-sample analysis (ex situ).

N/A INQ

[Design information not contained in open BADGER literature. This should be provided in the technical basis document, showing the trade-offs in spatial resolution, interference with indirectly scattered neutrons, etc.]

COMMENT POST ZNPP Same as 5.5.1.

Analysis of the raw data will yield a direct calculation of this uncertainty.

Addendum: As 5.5.1. Also, the different clients have different needs, and this results in contract specific data quality objectives which are outside the current scope. However, for cell average assessments the current spatial resolution seems quite satisfactory.

A-17 Sec-tion Report subsection Physical influence parameter Information used to support discussion Comments regarding relative uncertainty Primary reference source Uncertainty range 5.6 Calibration 5.6.1

Number, range, and precision of 10B areal density of standard panels No information available Not known whether the BADGER calibration function extrapolates beyond the smallest 10B areal density provided in the calibration panel. Since a logarithmic exponential function is fit to the calibration data, it is recommended that at least three calibration densities be run above the required minimum areal density, and two below.

N/A INQ

[The calibration report should include a certificate from the manufacturer of the various panels. The certificate should report the fabrication tolerance, with bias and precision listed separately.]

COMMENT POST ZNPP Not examined on site. Need to conduct a post review of the method, procedure, and analysis of the data.

Addendum: The number and length of the standards is dictated in part by physical constraints in the pool. The 10B areal density of the standards should bound the measurement range which may call for some judgement because it might not be well known ahead of time. At ZNPP quite often only three standards were used. Our suggestion is that when practical five standards should be used; 3 above and 2 below the required minimum 10B areal density. The dynamic range of calibration is extended and the spacing between interpolation points is reduced.

A-18 Sec-tion Report subsection Physical influence parameter Information used to support discussion Comments regarding relative uncertainty Primary reference source Uncertainty range 5.6.2 Standard panel degradation No information available Low if specific administrative controls are in place to ensure that the standard panel does not suffer degradation. If necessary administrative controls are not in place, the bias in the BADGER measurement is directly proportional to the amount of loss in the standard panel.

N/A INQ

[Acceptable tolerances should be documented in a QC procedure/plan]

COMMENT POST ZNPP Not examined on site. Need to conduct a post review of the method, procedure, and analysis of the data.

Addendum: The storage and inspection routine seem appropriate. The chemical and radiation exposure is minimal in comparison.

5.6.3 Adjustment of calibration procedure for specific pool characteristics No information available Unknown effects of local pool characteristics on measurement uncertainty, relative to both the baseline calibration and to the site-specific calibration.

N/A INQ

[Site-specific adjustments to calibration and measurement protocols should be provided in measurement report. Estimate of uncertainty should be derived]

COMMENT POST ZNPP Not examined on site and beyond scope. Need to conduct a post review of the method, procedure, and analysis of the data.

Addendum: In-situ calibration is used to mitigate this.

A-19 Sec-tion Report subsection Physical influence parameter Information used to support discussion Comments regarding relative uncertainty Primary reference source Uncertainty range 5.6.4 Relevance of standard panel material to rack panel material, e.g.

using a Boraflex calibration assembly to measure Carborundum or Boral.

Expert judgment Neutron channeling effects diminish as the thickness of the neutron absorber increases. During the calibration scan of panel segments of different thicknesses, the rate of change in areal density as a function of neutron transmission will be different for different materials.

References.

38 and 40

+/-30% (on areal density)

[Any specific variability introduced between the standard panel and the reference panel should be described in the measurement report]

COMMENT POST ZNPP Not examined on site and beyond scope. Need to conduct a post review of the method, procedure, and analysis of the data, although this is likely not applicable. However, when it is applicable, the contribution to uncertainty could be as high as 30%, as originally stated. Need to discuss with NETCO.

Addendum: In-situ calibration with representative items is used to mitigate this.

A-20 Sec-tion Report subsection Physical influence parameter Information used to support discussion Comments regarding relative uncertainty Primary reference source Uncertainty range 5.6.5 Location and acclimatizatio n of calibration cell in pool No information available Unknown. However, system response will likely vary with temperature.

N/A INQ

[Should be described in measurement reports]

COMMENT POST ZNPP No problem.

Addendum: Temperature and pressure vary with measurement location, but one would expect such effects to be very small (perhaps no more than 0.1%, for such a detection system based on general experience of moderated proportional neutron counters) and indeed NETCOs operational experience suggests it is not a concern.

A-21 Sec-tion Report subsection Physical influence parameter Information used to support discussion Comments regarding relative uncertainty Primary reference source Uncertainty range 5.6.6 Choice of zero-loss panel and uncertainty of actual 10B areal density No information available The reference panel, also referred to as zero-loss panel, is used as the y-intercept index to all other panels. Whether from a manufacturing tolerance on the reference panel or the misrepresentation of zero-loss, if the reference panel value is biased, all examined panel biases will be of the same sign and magnitude.

N/A INQ

[Should be described in measurement reports]

COMMENT POST ZNPP Not examined on site and beyond scope. Need to conduct a post review of the method, procedure, and analysis of the data, although, this is likely not applicable.

Addendum: The assay value is based on where the observed count rate falls between a pair of reference panel values, so the comment applies equally to the both bounding panels. The remark is therefore a general one, emphasizing that the quality of an assay cant be better that the quality of the calibration.

A-22 5.6.7 Non-linearity of calibration

curve, especially as applies to flux-trap racks No information available Actual BADGER calibration data is not provided. Available reports use, for illustration purposes only, the exponential curve fit of detector count rate versus the known characteristic, 10B areal density, from the calibration panel.

The 10B areal density range of calibration panels should extend beyond all likely panel densities, with at least three calibration densities above and two calibration densities below the minimum 10B areal density credited in the rack.

Sample exponential fits (and associated fit error) are shown in the text of Section 5.6.8.

N/A INQ [A calibration methodology report should provide the results of all empirical models and include actual data to support the curve fit results and methodology that would allow one to calculate MSE]

COMMENT POST ZNPP Not examined on-site, and beyond scope. Need to conduct a post review of the method, procedure and analysis of the data.

Although, this is likely not applicable.

Perhaps the inter-comparison between these measurements and the planned coupons will reveal features necessary to reduce uncertainty].

Addendum: See also 5.6.1. The 10B areal density (AD) is based on a ln-lin interpolation between two standards that bound the observed count rate. This is a simple numerical scheme which allows a straightforward and robust propagation of counting and calibration uncertainty.

However, because the transmission curve vs AD is not an ideal single exponential a model bias (over-reporting of AD) is possible. By using five calibration standards the magnitude of this effect is limited. More elaborate fitting functions could be used but they would have no better physical basis and would require a more complicated error treatment. With only three calibration points the bounding model-error can be estimated by comparing the result to that which would

A-23 Sec-tion Report subsection Physical influence parameter Information used to support discussion Comments regarding relative uncertainty Primary reference source Uncertainty range be obtained by interpolating between only the low and high standards. With five points the model-error estimate can be refined. This is an effect NETCO know about and expressed a desire to explore further.

A-24 Sec-tion Report subsection Physical influence parameter Information used to support discussion Comments regarding relative uncertainty Primary reference source Uncertainty range 5.6.8 Uncertainties in calibration

slope, especially as applies to flux-trap racks Expert judgment The Watt-fission spectrum of neutrons emitted from the 252Cf source is down-scattered by the time it reaches the first panel in a Region I (flux-trap) design.

Neutrons that penetrate the first panel, with sufficient energy to do so, are subsequently down-scattered prior to reaching the face of the second panel.

The down-scattered flux is of such low energy that the second panel is under-sampled and thereby overmoderated.

Without knowing the degradation in the first panel, there is no way to estimate degradation (or areal density) in the second panel of a flux-trap design.

N/A

+/-50% or more (on areal density)

[Same as previous]

COMMENT POST ZNPP Not examined on site. Need to conduct a post review of the method, procedure, and analysis of the data. In addition, during the tests, NETCO was still developing the algorithms. It is likely that this error propagation had not yet been complete.

Addendum: The comment is that two layers of BORAL separated by a water gap will behave differently than a single layer with the same 10B AD. This is a general comment. It is good practice not to make measurements outside of the demonstrate range of calibration and applicability. Said differently, flux trap racks require an appropriate calibration set up and AD range. So managed, the speculative observation of 50%

uncertainty is moot replaced by case-by-case demonstrated performance estimates.

The uncertainty is then not expected to be uniquely affected or substantially different.

A-25 Sec-tion Report subsection Physical influence parameter Information used to support discussion Comments regarding relative uncertainty Primary reference source Uncertainty range 5.6.9 Frequency of calibration during a BADGER campaign No information available The QC and recalibration plan used for ensuring measurement quality is unknown.

N/A INQ

[Requirements should be provided in the QC plan, which implements in part, the quality assurance requirements]

COMMENT POST ZNPP OK.

Performed each morning before measurements.

Addendum: At the end of a campaign statistical analysis of the daily calibrations also serve as quality control check. In principle the calibration data may be aggregated to obtain a more precise (counting and replacement) calibration that can be applied to the data collected during the campaign.

A-26 Sec-tion Report subsection Physical influence parameter Information used to support discussion Comments regarding relative uncertainty Primary reference source Uncertainty range 5.6.10 Confirmatory analysis with destructive examination No information available Unknown in the literature how many confirmatory, validation measurements are conducted between the BADGER in situ measurements, and the collection of coupon samples, for subsequent laboratory analysis.

N/A INQ

[Confirmation and validation requirements and methods should be provided in the quality assurance plan]

COMMENT POST ZNPP This will be identified with the coupon measurements, 2015.

Addendum: Independent comparison to another technique is the basis of the top down approach to uncertainty quantification. In the case of the ZNPP measurements chemical dissolution of panel sections was used as the other method.

5.6.11 Panel-specific use and interpretation of unattenuated region data No information available Reports cited discuss a "drift" correction (see Reference 10), but it is unclear how this drift correction is conducted.

N/A INQ

[Same as 5.6.7]

COMMENT POST ZNPP Need to discuss with NETCO in the overall interpretation approach of the measurement data.

Addendum: No longer relevant.

Calibrations are made close in time.

A-27 Sec-tion Report subsection Physical influence parameter Information used to support discussion Comments regarding relative uncertainty Primary reference source Uncertainty range 5.6.12 Algorithms to convert BADGER trace data into input for calibration curve No information available Details of the calibration curves used and how the contributing parameter uncertainties are propagated forward were not available for review, so no estimate on uncertainty can be made.

N/A INQ

[Same as 5.6.7]

COMMENT POST ZNPP Need to discuss with NETCO in the overall interpretation approach of the measurement data.

Addendum: Code-to-code comparison shows that the 10B areal density results are being extracted as expected. Algorithms to extract AD and associated uncertainty were developed independently and implemented in a spreadsheet. Results to the same input data showed that the NETCO analysis agreed with expectations.

A-28 Sec-tion Report subsection Physical influence parameter Information used to support discussion Comments regarding relative uncertainty Primary reference source Uncertainty range 5.7 Data processing 5.7.1 Reliance on operator experience to detect and characterize heterogeneous degradation Expert judgment Current experience is predominantly based on Boraflex and reliance on RACKLIFE as an aid to identifying what is being observed. Until an equal experience base can be developed for recognizing the degradation features of neutron absorbers other than Boraflex, the uncertainty in scan interpretation will be high.

Reference 10 High enough to invalidate scans COMMENT POST ZNPP Need to discuss with NETCO in the overall interpretation approach of the measurement data. This issue may not be as significant as originally thought.

Nevertheless, needs to be examined.

Addendum: The use of suitably qualified and experienced human personnel to make judgements based on observed data is common practice in the nondestructive assay community where algorithms are difficult to implement using a machine. In this example visual review of scan data to identify anomalies or trends is an important and valuable data evaluation step. It is good practice to involve more than one person and insert blind checks of skill into the process. However, Boraflex was not part of these follow-on-on discussions with NETCO.

A-29 Sec-tion Report subsection Physical influence parameter Information used to support discussion Comments regarding relative uncertainty Primary reference source Uncertainty range 5.7.2 Applicability of Boraflex-based algorithms and experience to non-Boraflex material such as Boral or

Metamic, which experience different degradation mechanisms Expert judgment The formation of localized scallops and blisters in the localized geometry of the measurement will significantly impact the count rate of neutrons transmitted through the neutron absorber.

N/A High COMMENT POST ZNPP Needs to be examined. Was not studied and was not applicable at ZNPP. Need to discuss with NETCO, as insufficient time was available to get into the details of this type of effect.

(Again, may not be applicable in most cases.)

Addendum: This is no more than a reminder that it is a generally good measurement practice not to step outside the range of direct calibration and demonstrated performance without due consideration. At ZNPP Boral standards were used to assay Boral panels. The use of Boral standards to assay non-Boral material was not discussed and we did not consider what the impact on uncertainty might be of doing that.

A-30 Sec-tion Report subsection Physical influence parameter Information used to support discussion Comments regarding relative uncertainty Primary reference source Uncertainty range 5.7.3 Feedback procedure No information available Process control issue N/A INQ

[Documented in operating and QC procedures primarily to identify pass/fail criteria of result]

COMMENT POST ZNPP Process control was still in development at this time. Need to verify feedback and measurement control procedure prior to implementation.

Addendum: Measurement control and reporting is part of an overall contract specific quality program which is outside the present scope.

A-31 Sec-tion Report subsection Physical influence parameter Information used to support discussion Comments regarding relative uncertainty Primary reference source Uncertainty range 5.8 Statistical extrapolation and surveillance frequency 5.8.1 Choice of test panels to survey No information available The statistical basis for selecting a subpopulation of panels to inspect by BADGER is not known. To some degree, RACKLIFE is used to estimate the most degraded panels, and then this estimate is used to estimate the nominal and worst case conditions for neutron absorber degradation.

N/A INQ

[Panel selection process should be documented in an inspection and test program plan type of report]

COMMENT POST ZNPP Was not part of T&E plan. Need to discuss with NETCO at later date.

Addendum: This is not an instrument question and falls outside the present scope. The sampling requirements for inspection population is covered in guidance documents for surveillance programs and should be referred to.

A-32 Sec-tion Report subsection Physical influence parameter Information used to support discussion Comments regarding relative uncertainty Primary reference source Uncertainty range 5.8.2 Lack of duplicate scans of the same test panel No information available Unknown. However, duplicate scans provide a means to increase confidence in the estimated uncertainty range.

N/A INQ

[Requirements should be provided in the QC plan, which implements in part, the quality assurance requirements to ensure that the system is reliable and stable (via duplicate measurements)]

COMMENT POST ZNPP Final procedure not yet developed.

Confirmatory, QC-based duplicates were suggested onsite.

Addendum: Duplicate scans have been taken and used for this purpose as discussed in the main text. When time onsite allows, duplicate scans can be used as a good-practice measure to support the overarching quality program. Data quality objectives are set on a case by case basis.

A-33 Sec-tion Report subsection Physical influence parameter Information used to support discussion Comments regarding relative uncertainty Primary reference source Uncertainty range 5.8.3 Application of results of selected panels to entire pool No information available Not an artifact of BADGER but a process issue.

N/A INQ

[Model validation report not contained in open BADGER literature]

COMMENT POST ZNPP Beyond the scope at ZNPP. For racks where sampling error requires complete error propagation to the entire pool on the basis of a subset of n measurements with SuperBADGER, this error needs to be understood. Discuss with NETCO.

Addendum: This is not an instrument performance issue and falls beyond the current scope.

5.8.4 Frequency of BADGER campaigns, especially for panel materials where intermediate calculational methods such as RACKLIFE are not available No information available Unknown. Should start with high frequency until adequate experience base is developed to reduce frequency.

N/A INQ

[Interval frequency should be documented in an inspection and test program plan type of report]

COMMENT POST ZNPP Beyond the scope at ZNPP. Discuss with NETCO.

Addendum: As 5.8.3

A-34 Sec-tion Report subsection Physical influence parameter Information used to support discussion Comments regarding relative uncertainty Primary reference source Uncertainty range 5.8.5 Application of final output to criticality analyses of record No information available N/A: Site specific N/A INQ

[Model validation report not contained in open BADGER literature]

COMMENT POST ZNPP Beyond the scope at ZNPP. Discuss with NETCO or the organization responsible for integrating the measurement results with the overall analysis of keff.

Addendum: As 5.8.3 5.8.6 Use of final output in abnormal or accident sequence criticality analyses No information available N/A: Site specific N/A INQ

[Model validation report not contained in open BADGER literature]

COMMENT POST ZNPP This is beyond the scope at ZNPP. Discuss with NETCO.

Addendum: As 5.8.3 Note the following limitations of the comments POST ZNPP. ORNL did not cite the draft and proprietary document prepared by NETCO because there were algorithms that NETCO considered proprietary and preliminary. ORNL did not collect raw data. ORNL did observe raw neutron count rates in each of the four detectors on the PC screens but did not record or analyze the data.

APPENDIX B PHOTOS

B-1 Figure B-1 ZNPP spent fuel pool. BADGER measurements made from the spent fuel bridge. December, 5, 2014 Figure B-2 BADGER heads moving from one cell to another from the spent fuel pool bridge.